Let's get physical: Gesture technology engages consumers

By Vincent John Vincent GestureTek

Applications using intelligence to transform industries always draw our attention. This article describes a new application where the consumer becomes part of the display in applications such as advertising in large venues. This new system projects dynamic visual content onto virtually any floor or wall space and creates a full-body interactive experience, literally allowing users to control an ad or game with their body motions.

Going to a sports competition, movie, live-entertainment show, or retail shopping center just got significantly more engaging.

Gesture technology is enabling advertisers to entertain and inform as an individual walks across a floor space or interacts with a wall and moves their head, hands, legs, or feet – wiping away one colorful picture or dynamic image and revealing another. GroundFX from GestureTek, for example, enables people to walk over a floor space and turn over interactive tile components, step on animated buttons to call up information and run videos, add color to black and white images they step over, and even kick balls and other objects that appear on a floor or wall (see Figure 1).

Figure 1

The cutting-edge GroundFX technology enables consumers to take control of advertising as they view it. For instance, with a quick wave of a hand or foot, consumers can manipulate car colors and styles in automobile ads. Some advertisers have elected to turn their ads into games, prompting customers to putt golf balls or race around a car track while observing an advertiser’s brand.

Interaction in a Flash
The GroundFX system is based on a Pentium PC and an integrated video projector. The key to the system is a FireWire camera and proprietary software applied to define and capture the interactive area, tracking the user and detecting their interactions with the projected images (see Figure 2).

Figure 2

The system supports content created in the Macromedia Flash environment by providing a custom fscommand() interface that allows any Flash programmer to develop projected interactive content. In addition to the Flash interface, custom effects are coded in a proprietary engine in C/C++. This engine utilizes a DirectX9-based 3D architecture that provides access to current high-end graphics cards. A variety of off-the-shelf effects packages help users get started.

“The product is limited only by one’s creativity,” says John Payne, president of Monster Media, the US company best known for placing brand advertising on stadium turnstiles. “For instance, travel clients can literally bring their destinations to life by adding the soothing sound of ocean waves that ripple as consumers stroll through the visual display. In one case, tropical fish dart around the feet of unsuspecting consumers.”

GroundFX is a unique product that integrates sound to enhance the interactive feature and further immerse consumers into the advertising environment. In addition, GroundFX advertisers have the unique ability to monitor customer reaction to their ads at any time utilizing a proprietary built-in monitoring feature.

The monitoring is handled by adding a webcam to the mix. This webcam is pointed at both the projected display area and its immediate surroundings. Custom software is used to take an image snapshot every five seconds. These images are made available live on the Web and are additionally stored on servers for verification and later analysis if desired.

A short movie on Monster Media’s use of GroundFX (02:48 - WMV)

Monster Media has installed dozens of GestureTek systems across the United States and Puerto Rico, in such well known venues as the American Airlines Center in Dallas, Texas; Angels Stadium in Anaheim, Calif.; and the Wachovia Center in Philadelphia. “Stadiums and theatres across North America are not just where America’s greatest athletes and entertainers play, but the world’s top brands compete for attention,” Payne says. “GestureTek’s system is smart, engaging, easy to install, and highly reliable. And we can monitor the system remotely for performance, as well as to gauge traffic participating in the interactive advertising.”

Scripting applications
The technology and business model for gesture technology embedded in digital signage seems to have a universal appeal. Says Juan Benet, director of operations at Dgnet, a Mexico-based company that helps large brands like Unilever advertise in retail malls, “We have successfully deployed a large number of GestureTek’s GroundFX systems in shopping malls in Mexico City. Because of our success in Mexico, we are now planning a major expansion into other countries in South and Central America with this technology.”

Benets says that in order to create Dgnet’s own content for clients such as Unilever, Dgnet has also purchased the GroundFX Software Development Kit (SDK) from Gesturetek. “We have found the GroundFX Software Development Kit to be a wonderful tool that gives us a fast and inexpensive way to develop new effects for our advertising clients,” Benet says.

The GroundFX SDK runs on Windows XP or Windows 2000, and provides three layers of access to interactive experiences. At the simplest level, users can modify the graphics of existing content to customize it for a specific client. The next level allows custom applications to be created in Flash as described earlier. Finally, the specialized effects and transitions can be layered and customized through an XML-based script file that also controls the sequence and timing of playback for an installation.

Transforming advertising
While some interactive advertising systems designed by GestureTek are employed by one single brand or company, typically up to 10 advertisers have multiple 30-second slots to interact with individuals passing by. Advertisers and corporate sponsors at stadiums and live-entertainment venues and theatres can sign up to reserve space on an annual basis.

In December, GestureTek continues its rollout of the GroundFX technology. Working in collaboration with Epson, GestureTek will be officially announcing the GroundFX application for Virgin Megastores in two locations – the flagship Manhattan store in Times Square and the store in Hollywood, Calif.

. . . . .

Vincent John Vincent is cofounder and president of GestureTek and is recognized as a leader in the world of multimedia. Along with GestureTek cofounder Francis MacDougall, Vincent has been overseeing the enhancement of GestureTek’s robust library of human video gesture capture, analysis, control, and real-time 3D animation technologies for the last 15 years. In 1994 he co-created and was one of the executive producers of The International Digital Media Awards. He is on the advisory board for the Toronto Design Exchange VR Visualization Center, and the Art Institute of Toronto. In 2003, he was honored by the Canadian Digital Media Association with a Lifetime Achievement Award.

To learn more, contact Vincent at:

GestureTek
317 Adelaide St. West, Suite 302
Toronto, ON M5V 1P9
Canada
Tel: 416-340-9290
Fax: 416-348-9809
E-mail: vincent@gesturetek.com
Website: www.gesturetek.com

GroundFX is a trademark of GestureTek. Principals of GestureTek patented the “video image-based control system” under U.S. Patent 5,534,917.