Qt, as a framework, makes extensive use of touch - particularly on mobile and embedded devices. Unfortunately, Qt isn’t readily able to magically make touch input appear on hardware that doesn’t support it, which can make exact testing of touch-related code difficult in typical developer environments – not a very desirable situation for a cross-platform development framework.

Thankfully, someone else has thought of a solution that helps to fill this gap, in the form of the TUIO protocol. TUIO is a specification for a protocol to encode touch events to be sent over a remote transport (such as a network).

In practice, this means that one can create a server that synthesizes touch events and sends them to an application, even if it is running on a completely different physical machine.

For example, there are a number of mobile clients such as TUIOPad and TUIODroid which can be used to take touch events from a mobile device (such as a phone or tablet) and send them to a remote TUIO listener.

I came across this last year, and thought the idea was pretty neat, so I went off and built something to bring this to Qt users - the result was an input plugin for Qt which listens for touch events coming in from a TUIO source.

This was made easy in that in the Qt 5 world, things such as input and display are abstracted away from the core of Qt itself, so that one can easily provide an additional input source.

show me!

If you’d like to check this out yourself, it’s easy! Grab a copy of Qt 5.5 (the alpha just came out, and that will do nicely), and build it as normal. When running your application, use the command line argument “-plugin” to tell Qt to load the TUIO input plugin as follows:

mytestapplication -plugin TuioTouch:udp=40001

The udp=40001 tells the plugin to bind to the UDP port 40001 and listen for incoming TUIO traffic.

known limitations

There’s a few caveats, here.

  • Only UDP sources are supported, although other transports would not be impossible to add
  • Sources are bound on QHostAddress::Any, meaning that your network is considered “trusted”
  • Only a single TUIO server is supported. Multiple senders on a network will not intrinsically break, but it won’t work well.
  • Only the 2dcur TUIO profile is supported at this time.

Patches are more than welcome! (to src/plugins/generic/tuiotouch in qtbase).

technical details

TUIO at a high level is a specification of events. The actual wire protocol used by TUIO is a seperate specification called “OSC”

  • Open Sound Control.

The plugin contains a simple OSC parse (QOscBundle and QOscMessage) for handling the wire protocol parsing, and on top of that, a handler for TUIO messages (QTuioHandler).

QTuioHandler is responsible for receiving TUIO messages, using the OSC classes to parse them, and creating touch messages to send to QWindowSystemInterface, to notify Qt about the events. QWindowSystemInterface subsequently takes care of letting the application itself know about the touches through the regular and well known QTouchEvent interface.

The application can then consume the QTouchEvent without having to know about the origin :)