Multi-touch
displays are the natural evolution of the old touch screens as they are
capable of recognising multiple touches at the same time. This way of
dealing with the hardware (and applications) opens up a host of new
opportunities for human-computer interaction, enabling the objects on a
screen to be directly manipulated by the user. For this reason these
techniques are also known as natural user interfaces.
There are different ways to create multitouch screens, depending on
the context of use and the size required for the display. The techniques
that allow the realization of multi-touch hardware, are based on
different solutions such as capacitive, resistive, motion, orientation,
pressure and so on. The one we chose to implement our table is a
computer vision based technique and is called Laser Light Plane (LLP).
This vision based solutions are the most known and used when you need
to interact with large screens because the scalability, the low cost and
the easy to build feature; in addition to this there are other
four major (computer vision based) techniques that allow the creation
of a stable multitouch hardware system: Frustrated Total Internal
Reflaction (FTIR), Rear Diffused Illumination (Rear DI), LED-Light Plane
(LED-LP) and Diffused Surface Illumination (DSI).
How LLP works.
Each vision based techniques consist of a
camera, an infrared light source and a display. The infrared light
source, in the LLP solution, is generated by IR-laser and this infrared
light is shined just above the surface to create a laser plane of light
that is about 1mm thick and is positioned just above the surface. When
the finger touches the display, the laser plane hits the tip of the
finger and the system registers an IR blob so infrared lasers and LLP
method are an easy and usually inexpensive way to create a MT setup.
Most setups go with 2-4 lasers, positioned on the corners of the
touch surface. The laser wattage power rating (mW) is related to the
brightness of the laser, so the more powerful the laser, the greater the
brightness of the IR plane. The common light wavelengths used are 780nm
and 940nm. Laser modules need to have a line lense generator on them to
transform the beam into a light plane; the 89 and 120 degree line lens
are most commonly used to reduce the number of lasers necessary to cover
the entire touch surface. The advantages of this technique are that no
compliant surface is needed, so you can use any transparent material for
the surface like glass and not only acrylic; no LED frame and no closed
box needed and could be cheaper than other solutions. The disadvantages
are that traditional objects and fiducials are not trackable and is not
pressure sensitive (is not able to detect the force with which someone
presses on the surface but only the contact).
Laser Light Plane
For more detailed information and reference, please visit to this
NUI Group Wiki page about LLP technology.
CSP Mutlti-touch table.
In order to test multitouch applications at the development stage,
CSP equipped itself with a dedicated hardware solution. When it comes to
large screens (non-portable devices), there are various commercial
platforms available – e.g. Microsoft Surface. However these are
closed solutions that do not integrate with other software programmes
and do not allow modifications to the configuration of the hardware, as
well as representing a considerable expense. For these reasons CSP’s
applied research has focused on creating and integrating an “open
source” hardware. For multitouch applications a development
framework based on Adobe Flash was used, chosen for its versatility and
compatibility with the vast majority of operating systems. The
applications developed are also based on interaction patterns typical of
multitouch environments, such as pinch to zoom, tap to open and drag to
move. In its research work CSP has developed a series of applications
designed to respond to the needs of a range of different settings and
sectors, from library services and public information, to teaching tools
for learners with special needs, from the aviation industry to
tourism, to entertainment, with multitouch games. Our prototype (built
in 2009) is a table with a surface composed by a retro projection film
sandwiched between two tempered glasses. On each corner of the surface
there is an infrared laser with a line lens generator. A short throw
projector, placed under the table, display the images and a modified
infrared camera captures the users generated blobs. A mirror assists the
projector in redirecting the image on the multitouch surface.
The surface has been produced in the form of a horizontal table, but
the versatility of LLP technology means it can be used to construct
vertical interactive surfaces in larger sizes, while keeping costs
low. Indeed 2010 saw the creation of an interactive wall measuring 100”
(around 2 metres wide and 1.5 metres high), with costs comparable to
those of producing a table, despite being four times larger.
Multitouch table hardware specs:
- Surface Size: 100 x 75 cm (48″ @ 4:3)
- 4 IR Lasers:
- Power: 10mW
- Wavelength: 780 nm
- Supply: 3.2V DC
- Laser class: IIIB
- Line Lens Generator: 89°
- USB Camera:
- Up to 60 frames per second
- IR Interference band pass filter: wavelength 780nm (+/- 25nm)
- Projector BENQ MP772:
- Resolution: VGA (640×480) – XGA (1024×768) – SXGA (1280×1024) interpolated
- Brightness: 2500 ANSI Lumen
- Contrast Ratio: 2400:1
- Throw Ratio: 0.61:1
- Lens: F=2.6, f=6.9mm
- Mirror:
- First surface mirror – 330×220 mm
CSP Multi-Touch Table
Lasers Safety
Safety when using lasers of any power is important. Infrared lasers
are used to achieve the LLP effect, and these lasers carry some inherent
risk. Infrared lasers are imperceptible to the human eye and therefore
activate no link response, allowing for greater damage to the retina. A
Class IIIB laser is hazardous if the eye is exposed directly but a line
lens is used to expand a laser’s line into a plane. This line lens
reduces the intensity of the laser from IIIB to Class I. Exercise anyway
common sense and be mindful of where the laser beams are travelling. If
extreme caution is exercised, LLP setups are quite safe.
Software
To implement multitouch applications, you can choose between
different programming languages. Each technology has to know when and
where the fingers are on the screen, so a protocol is needed in order to
translate blob information into touch point messages. We used the TUIO
(
http://www.tuio.org)
protocol as a middleware between the multitouch hardware and the user
application. TUIO is an open framework that defines a protocol (and API)
to standardize an interface that makes it possible to exchange touch
events between a detecting device and an application or a user
interface. TUIO is built on top of OSC (Open Sound Control) that is
a protocol, defined in 1983, used to interconnect PC to multimedia
devices. It represents an Internet based alternative to the MIDI
interface. As OSC, the TUIO protocol is based on the exchange of UDP
messages between the device and the client application. The TUIO
protocol encodes control data from a tracker application (e.g. based
on computer vision) and sends it to any client application that is
capable of decoding the protocol.
The TUIO protocol defines two main classes of messages: set messages
and alive messages. Set messages are used to communicate information
about an object’s state such as position, orientation, and other
recognized states. Alive messages indicate the current set of objects
present on the surface using a list of unique session IDs. Since TUIO is
implemented using OSC, it follows its general syntax:
/tuio/[profileName] [messageType] [Parameters]
The
profileName is used to indicate the sensing capabilities of the devices:
- 2D Interactive Surface (2Dobj or 2Dcur)
- 2.5D Interactive Surface (2.5Dobj or 2.5Dcur)
- 3D Interactive Surface (3Dobj or 3Dcur)
- Custom Profile (in this case the name is a format string that gives the ability to correctly parse the following parameters)
The “obj” profile is used to recognize objects, the “cur” profile only to obtain cursor information.
The following tables describes the basic messages used by the TUIO protocol when using a 2D Interactive Surface.
| messageType and parameters |
Description |
| source name |
Identify the message source |
| set id x y X Y m |
States the update for an object/finger providing the numeric
identification, the x,y position, the X,Y motion vector and the
acceleration (m) |
| alive id1 id2 … idn |
This is an alive message sent to indicate what are the
objects/cursors that still are on the surface |
| set id x y X Y m |
States the update for an object/finger providing the numeric
identification, the x,y position, the X,Y motion vector and the
acceleration (m) |
| fseq num |
This message sends a periodic sequence number. This is provided
because actually UDP does not guarantee that the messages are received
in the same order in which are sent. |
The TUIO implementations are available for most common programming languages and media environment:
- AS3 / Flash (ActionScript)
- Objective C
- Python
- WPF (Windows Presentation Foundation)
