Interactive Designer and Developer
Collider Exhibit Series

36 View of a Bridge

interactive video playlist

Alex and Andy both filmed various time-lapse and high-speed videos to be projected on the main screen. But Alex didn't want to just be playing videos; he wanted to add in some type of interactive element. By using the multi-touch table and/or an iPhone, the audience could learn more about the individual videos and were able to vote on which video would be played next. It just went to show what amazing results can stem from a collaboration of art and technology as well as artists and audience.

The ArtistsThe Application Specs and Development

Early on, I knew that Alex wanted to run all the videos in HD. I knew Flash ran h.246 encoding, I just never had a display to test it on. My home monitors run 1680x1050, just a little short, but I tested some of his videos to see how they’d look. At first view, things ran a little choppy when there was fast motion covering most of the screen. So I switched the window to full screen mode. Voila! No more breaks in the video or choppiness.

Last test for the video playing: Continuous Play. Things would work for a while and then one of two things would happen.

  1. The screen would freeze, and taking it out of full screen would crash Flash Player
  2. The screen would freeze, but if you take the Flash out of full-screen it would still be playing as if nothing was wrong. Then you could go back to full-screen for a while.

Actually, that 2nd happening was also happening to the multi-touch table's application. The screen would freeze until I took it out of full-screen mode.

After searching Google with numerous terms including 'freezing,' 'full-screen,' 'flash display,' 'stand alone player' 'projector,' 'WTF?!' I finally came across something worth trying. There is a possible conflict with hardware acceleration in Windows XP with Nvidia cards for Flash Projectors running full-screen for long periods of time. That was what was happening to me. Turning off all hardware acceleration was the only solution that worked.

Now both displays finally worked for an all day exhibit. But I have two separate computers running two separate pieces of flash. How do I create active communication between them?

Solution: a PHP Socket. I wrote a previous post all about it.

The Design

mapcompareThere were some considerations we took about how to make the application work. First and foremost, there needed to be a map of Cleveland. That was rather simple. We could integrate Google Maps Flash API. People could have fun with the table moving around the map and exploring the world. But we wanted the audience to be reading information on the table about the videos. So we needed markers and little window displays to remain stationary on the table. Considering we wanted to create this is a multi-user based interface, we wouldn't want one person reading a detail display then it randomly move because another user moves the map on them. So Google maps was out, but using the map image was still in. We opted to use the Yahoo Map image of Lake Erie, rather than the Google map image of Lake Erie, because the colors are more balanced and overall it just looks nicer.

multitouch video playlist and descriptions As to not take away from map exploration, I added in a little magnifier app that people could use to take a little closer look at Cleveland that gave them 3 times magnification. At first, it was setup to create a new magnifier when the map itself was touched. But as I was testing it, it was easy to recognize that there would be many MANY miss-touches happening and that would create too much unwanted clutter on the screen. So it was changed so that there would be just one trigger in the lower right corner of the table for people to use to bring up the magnifying glass.

Then there were the markers and display. Simple in and out animation made for a very dynamic display of the information. Each detail view was customized based on the video played. And the marker that triggered the animation was designed to look like a button to help enforce that it was intended to be touched. When the display was closed, the button was green. When it was opened, the button turned red. And the button could then be used to close the display without voting. It seemed 80% of the users I witnessed picked up on this right away without needing explanation.

The display information could also be rotated and scaled with one touch. We didn't want to make the displays completely free-roaming because they did pertain to specific locations in Cleveland. I used the button marker as the anchor point of the scaling and rotation. There were quite a bit of people who continued to use 2 fingers to try and scale the display information, as though they were using an iPhone. Even in a brand new environment, it seems there will always be users that will have to still “unlearn” even relatively new preconceptions about a multi-touch interface.

On the left side of the interface was the playlist of all the videos. As the videos were voted on, the most popular ones slid to the top. Users could also touch the playlist item to open and close the additional information display about that video. There was a little trouble with light balance on the table, making half of the triggers more difficult to use than they needed to be. I will be addressing issues such as this in my next prototype build, which will be beginning soon. Making the playlist items slide, rather than just swap locations, was definitely a key visual that helped users understand how the most voted videos moved to the top of the list.

The final piece of the app was the integration of FluidSolver. If at any time all the videos in the queue had been voted on, the app would switch over to FluidSolver to give the video playlist a chance to finish showing 2-3 videos without additional changes from the map. Clearing the palette per se. Credit was given to Jos Stam, Mehmet (aka Memo) Akten, Eugene of astatic notes and Seth Sandler for making FluidSolver available for multi-touch Flash use.

One last consideration I took into account was visual feedback that you were indeed touching the table. I watched a video a while back about the MS Surface table app upgrades and how they added in a subtle visual and trail to help users realize their touches were being collected by the interface. So for this app, instead of completely turning off the debug mode in TUIO, I merely took off the squares in the upper left corner and the array display in the upper right; leaving the rings at the active touch-points on the table. I made no mention of this feedback to the user, nor was any made from the user to me throughout the exhibition, so it is hard to tell if users found that useful or not.

User Observations

To me, the best part of this exhibit was being able to finally see first-hand how a member of the general public would interact with this multi-touch interface. I was curious to see if what was created had too steep a learning curve? Was it intuitive enough? Did it give enough feedback for continued use? This was my moment of truth.

From everything I observed, and in agreement with my fellow artists, the application wasn't difficult to use at all. People had no problem approaching the table and pressing on the buttons right away (except for the skiddish ones that looked like they were worried they may break it). There were a couple kids, however, that starting POUNDING on the surface…luckily there was no breakage or damage done. 3/8" Acrylic is just that awesome.

Now, aside from the simplicity of opening and closing the display windows, the entire exhibit was quite in-depth compared to other exhibits being shown. The most common question I received was "what does this 'Vote' mean?"  When someone did ask, one of the other moderators or myself would explain that the “vote” button ultimately puts the user in control of the audience playlist control. Once the explanation was done you could, more times than none, see the exact moment when that person (or people) had their “AHA” moment and completely understood what the exhibit was all about.

Different Numbers of Users

There looked to be a general breakdown of how involved people would be with the multitouch table.

One (1) User (Adult):  The individual was very involved in reading and making an informed decision of which video they’d like to move up in the play list. Decisions seemed to be based on the idea that they would not have to vote numerous times for their video because they were the only one voting. However, once he or she stepped away, the voting was free to be changed before the intended video was loaded to play.

Two to Four (2-4) Users (Adult): This interaction seemed to have evenly split results; The group would either read the displays together and agree on what videos they wanted to watch and in what order they’d be played in OR they would have conflicting interests and begin to “stuff the ballot” with votes and compete against each other to see which video would be played before the others.

Five (5) or More Users (Adult):  It seemed that when there were five or more people at the multi-touch, no one was comfortable being in control of the playlist. Some users would open and close a few displays or play with the magnifying glass, but no distinct voting patterns were formed.

One to three (1-3) Users (Children): Kids just loved opening and closing the display views, over and over and over. The ones who were old enough to understand that they had the ability to completely control the playlist would vote to see one specific video, go watch the video, and when it was over return to the table to re-stuff the votes for that same video to watch once again. There were a few times we had to step in and ask for other videos to get some playing time.

Four (4) or More Users (Children): Free for all! It would usually turn into a competition to see who could open and close the most amounts of display boxes the fastest. Due to the fact that their fingers were smaller and less likely to trigger the sensors, many would become frustrated and strike the surface in an attempt to get it to register.

And when FluidSolver was turned on, everyone instantly became entranced. I defy you not to be.

Quotes and Comments

  • "Who would think I'd live to see stuff like that" An older woman upon reluctantly using the multi-touch table
  • "I've never seen cleveland look this good" multiple people
  • "the views of the bridge application/installation was one of the best things at Ingenuity"
  • "I grew up in Cleveland and have been away for a long time. I just moved back, and I'm getting reacquanted with it. This film helped so much. It got me excited to be back"
  • "Is that this year's parade in the circle? Why would I want to vote for that? I was there"
  • "Booooorrrrrriiiiinnnnggg." Small boy tugs mom's arm. "Mom, It's creeepy. Lets go" (most kids loved it, though)
  • "What do all of these have to do with the bridge?"
  • "Mind-Opening"
  • "Your work is awesome"
  • "Very Cool. Yay Cleveland Bridges!"
  • "Awesome Idea and execution!!"
  • "Amazing"
  • "Wow!"
  • "Rocking and rolling!"
  • "<3 it!"


What do you all think?

What developer tool[s] do you use to compile your SWF files?

View Results

Loading ... Loading ...

What kind of tutorial do you prefer?

View Results

Loading ... Loading ...

What is your prefered language for developing multi-touch apps?

View Results

Loading ... Loading ...

Shameless Attempts to Pay the Bills