Wednesday, 23 November 2011

Fast UI with Qt 4 on mobile

For device manufacturers, and those targeting device manufacturers like us in the Mer¹ and Nemo Mobile².communities, we need a performant base, and Qt's default configuration on Linux is ..not really that performant. It uses what is known as the 'native' graphics system, which uses X (and XRender) to do a lot of the grunt work. Unfortunately, XRender isn't exactly what you'd call speedy in many cases, and making loads of round trips to ask X to draw things probably doesn't help either.

There's another option in the 'raster' graphics system, which does all rendering client-side in your application using (as you'd expect) Qt's software rasterizer, which is adequate enough for performance on most desktops, but still not quite optimal on desktops: hardware acceleration is the missing goodie.

Qt as of 4.8 also includes what's known as the MeeGo graphics system - as the name implies, it's used on the Nokia N9. It uses hardware acceleration (plus some additional EGL extensions) to perform absolute magic and make your pixels (especially QML :)) fly, so if you're working on getting a device together using Qt 4, I'd highly recommend you look at it.

Because no blog post is complete without a video, here's one:
(comparing the performance of raster and MeeGo graphics systems on a Lenovo S10-3t)

This sounds great, but there's one caveat. If you're on certain types of graphics hardware (SGX in particular), then you'll probably not to just want to enable the MeeGo graphics system for everything, because you'll end up with a lot of GL contexts allocated, which is not good for two reasons; they're scarce resources, and they take up a large chunk of RAM (something in the order of 5-10mb, depending on your PVR configuration file. I never actually looked).

There is good news at hand, though! Qt has *another* graphics system, specifically designed to proxy everything through another system, and allow for runtime switching back to raster. In our case, obviously, we want it to use MeeGo's graphics system by default, but fall back to raster to avoid taking extra resources when not needed.

There's also some bad news: it didn't work out of the box with Qt 4.8 RC1.

But some more good news: I fixed it (and yes, I upstreamed the patches)!
are the two commits you'll want.

After that, configure Qt with -graphicssystem runtime, and -runtimegraphicssystem meego (or set QT_DEFAULT_RUNTIME_SYSTEM=meego), and it'll work beautifully, provided your system has the required GL extensions.

Alternatively, you can use Mer, which already includes these patches right now, and, as of this week, has this beautiful magic enabled by default on the Nokia N900/N950/N9 hardware ports, where this is really needed (and works well). I'm working on getting it enabled by default on other systems that can support it, too, like the Lenovo S10-3t, as time permits.


  1. Great post and thanks for the patches! The runtime graphics system was originally written for Symbian to toggle between raster and OpenVG to workaround the 32 MB of graphics memory on the N8 and friends. It was never taken into use there though because knowing when to do the switch turned out to be a pretty hard problem to solve reliably. The other problem was the expense of uploading everything to the GPU again which was slow on the N8. On Harmattan, they switch to raster when the app is backgrounded. This has the benefit of not using any GL contexts, but still giving you the ability to have a "live" task switcher.

  2. So why can't our desktops use hardware acceleration? Games can, KWin does as far as i know.

  3. @Mase: if your desktop supports egl, and all the required extensions, you can use this too. (If you build your own Qt, or somehow talk your distro into enabling this just for you - it's not completely a straightforward matter of switching it on as I tried to show in the post). Otherwise, you're out of luck until Qt 5 / QML2.

    There is also the opengl graphicssystem, but I'd not recommend it. It's really not that great on any of the desktop platforms (it's still marked experimental for a reason) and it can (and often does) give quite drastically different visual results.

    As for KWin, it *can* use opengl, but it is a compositor in such a mode. Meaning, it doesn't render your applications themselves. It just takes whatever they render and position them as it needs to. This post is about actual applications drawing themselves with hardware acceleration.

  4. Would you happen to know *which* EGL extensions that are used?