Qt 5 and Android

Astute observers of the Qt 5 repositories may have noticed that for quite a while, patches have been trickling in from me allowing Qt 5 to compile on Android.

The goal in mind was to allow use of Qt on Android primarily in order to work at the system level (not using the regular Android display stack, but using Wayland on Android) - tying in with Collabora's other work on Android, but this work also doesn't preclude someone from e.g. implementing a platform plugin to allow Android applications to run natively on unhacked devices, similar to Necessitas on Qt 4 - and I'd very much like to see that happen upstream.

In terms of compilation, there is one approach currently upstreamed that involves using the NDK, see this wiki page for more information. You'll note it's quite easy to do a build yourself, something that was quite intentional, since I figure that the only way it's going to improve easily is if it is easy to hack on it. I'm sure the build & installation instructions can be more optimal still (like installing to /system/lib, etc) but it's a start. Contributions welcome. I should also take a moment to thank the Necessitas guys, their mkspecs provided a nice starting point.

I had started an alternative route of integrating Qt with Android image builds (so, check out the Android tree, repo sync, drop Qt in place, run 'make' and have it built & deployed for you), but unfortunately, my sponsored time to work on this ran out, and so I wasn't able to finish it. It's still an interesting area of work, and so, I do plan to try continue it in my spare time.

In terms of actually using it, one area which is a bit of pain still, is that there's a bug in the way bionic's linker handles R_ARM_COPY relocations - instead of looking up the symbol to copy in the shared libraries the binary depends on, it finds the binary's symbol instead, meaning it doesn't really do any actual relocation.

The symptom of this is that your binary will crash on start due to things being zero'd out that really shouldn't be (like QObject::staticMetaObject in my case), depending on how it's been built. Thanks to Thiago for helping me nut that very difficult problem out. There is a patch pending on Android's gerrit instance, but I need to find the time to go rebase the patch and retest it to make sure it still works, although the code changes in the area look quite trivial.

For those of you who are visually oriented: I'm sorry, but there's not much to show here, because - as of yet - I don't have anything graphical running. Though in theory, it might be already possible to easily shoehorn Wayland libraries into the NDK using Pekka's work, and build QtWayland that way. But if anyone wants to talk Qt on Android, or better still, contribute, I'm all ears.

Massive kudos to Collabora for sponsoring my work on this!

writing a layout in QML

Sometimes, for whatever reason, the layouts provided "out of the box" in QML just don't cut it. lately, I've been doing a few rather different things for experimentation and learning purposes that have meant I've run into quite a lot of these cases.

when this happens, the first instinct is to fall into despair - but there's really no need for that. writing your own layout really isn't that hard. Here's a small, fairly self contained example doing just that.

MyLayout.qml:

import QtQuick 2.0

Item {
    id: layout

    property bool ready: false

    onChildrenChanged: performLayout()
    onWidthChanged: performLayout()
    onHeightChanged: performLayout()

    /* the meat of the layout */
    function performLayout() {
        /* nothing to layout? don't bother then */
        if (layout.children.length == 0)
            return

        var currentX = 0

        console.log("DOING LAYOUT FOR " + layout.children.length + " ITEMS")

        /* first real step of doing anything: go over all the children */
        for (var i = 0; i < layout.children.length; ++i) {
            var obj = layout.children[i]

            /* in the real world, we'd probably do something a lot more complex,
             * but let's just position our children along a row.
             */
            console.log("Positioning at " + currentX + " to " + (currentX + obj.width))
            obj.x = currentX
            currentX += obj.width
        }

        console.log("LAYOUT DONE")
    }
}

MyItem.qml:

import QtQuick 2.0

Rectangle {
    width: 100
    height: 50
    color: "black"

    Rectangle {
       width: 90
       height: 40
       anchors.centerIn: parent

       color: "red"
    }
}

main.qml

import QtQuick 2.0

Rectangle {
    width: 1000
    height: 100

    MyLayout {
        MyItem { }
        MyItem { }
        MyItem { }
        MyItem { }
        MyItem { }
        MyItem { }
        MyItem { }
    }
}

This is obviously very simplified for demonstration purposes, to name a few things that it doesn't do:

• it omits things like margins, wrapping
• it will break if MyItem is ever anchored
• it doesn't use anchoring (which might make for a more optimal implementation in this particular case - at the least, it wouldn't have to relayout if the width/height of the layout changed)
• it doesn't relayout if the geometry of the children change
• it relayouts whenever properties changes, which isn't optimal if e.g. the layout is animating a change, instead, it should delay a relayout using a Timer

All of these are left to the reader, but hopefully it's of some help in getting started.

san francisco, may 2012

I recently attended the Tizen Conference. I thought it might be good to write up a little about that visit (belated though it is, because I wrote half of this up, and then forgot).

I arrived on Saturday 5th, a little early, which gave me some time to recover from the flight (and the massive, massive queue through SFO immigration control - I think Carsten and I were waiting there some 1.5-2 hours). Once through, though, we met up with Thiago, and it was a swift, and generally pleasant trip on the BART to the Hyatt Regency, where the ghosts of MeeGo past were certainly felt by me at least. The rest of the day wasn't all that memorable for me, as I was quite tired.

Sunday was also mostly non-eventful, thanks to being really really tired, although we had a very nice (and amusing) breakfast in the Hyatt with Ash and others. Then spent a while wandering around with Ash and Carsten while waiting for an old friend from IRC to turn up, and picked up a new laptop for Kamilla.

I spent most of Monday touring around San Francisco and tinkering with cable cars with David and Denise. I even managed to stumble across the San Francisco Sjømannskirken by accident, which was a nice touch. We had a nice lunch in Chinatown, after wandering around looking for places offering decent food for a while. We stumbled back to the Hyatt later that afternoon, got ourselves registered for the conference, and started talking to folks, which continued on for quite some time. I even stumbled across Rob from Collabora, even though neither of us knew the other was coming, which was a pleasant surprise.

The evening concluded with a keynote from Jim Zemlin. I've spoken with Jim in person a bit before, and had the pleasure of appearing on stage with him at last year's MeeGo Conference keynote, but I have to admit that I wasn't a huge fan of the content of his speech, which didn't really seem to offer too much real content relevant to Tizen, but I heard later on that he had to parachute in at the last minute due to scheduling conflicts with some of the other planned content for the keynote, which would certainly explain that.

The following morning, I stumbled down, chatted to various people, and wandered in for the keynotes, which thankfully were of a much better quality. I especially enjoyed the presentation by Imad and JD, from Intel OTC and Samsung respectively. They quite clearly care a lot about their work, and managed to fit a few demos in too, which was great to avoid that "vaporware" taint that (at least for me) MeeGo had, until Nokia actually shipped a product after killing the ecosystem off.

Some discussion about Tizen governance also happened through some more introduction about the Tizen Association. I'm still not sure precisely how they fit in with Tizen as an open source project, something which I expect will become clearer over time, along with whether or not Tizen will be meritocratically governed or not.

I'll wrap up, for now, as I'm getting quite long - but suffice to say, after the keynotes, I went to loads of great sessions, met & talked with lots of interesting folks - some old friends, made some new ones - and in general had a great time. I'm keeping one eye on Tizen to see what develops. Especially as it already runs Qt.

on the importance of doing nothing

I've been meaning to write about this for a while, but I've only just now been driven over the edge by having to go and basically run sed over code again for no good reason.

When you're programming, always make sure you question *why* things are done. Qt provides three functions, helpfully named qMalloc/qRealloc/qFree. Despite the 'q' in front of their names, these functions do absolutely nothing useful, they just wrap around their stdlib friends. This was originally done to enable replacement of the allocator inside Qt (but there are better ways to do that, without getting sidetracked from my central point), but in reality, doesn't have much use. That's why I'm trying to deprecate them.

Now, you might ask, "what impact could a simple function call have, anyway"? I'm glad you asked. Benchmark time (spoiler for the lazy: ~10% extra overhead for small allocation sizes, ~0-5% for larger allocation sizes).
virgin:~/mallocbench% cat main.cpp

#include <QtCore>
#include <qtest.h>
#include <qcoreapplication.h>
#include <qdatetime.h>

class MallocBenchmark : public QObject
{
Q_OBJECT
private slots:
    void qtMalloc();
    void qtMalloc_data();
    void regularMalloc();
    void regularMalloc_data();
};

void MallocBenchmark::qtMalloc_data()
{
    QTest::addColumn<int>("size");
    QTest::newRow("1") << 1;
    QTest::newRow("10") << 1;
    QTest::newRow("100") << 100;
    QTest::newRow("10000") << 10000;
    QTest::newRow("1000000") << 1000000;
    QTest::newRow("10000000") << 10000000;
}

void MallocBenchmark::qtMalloc()
{
    QFETCH(int, size);

    QBENCHMARK {
        void *p = ::qMalloc(size);
        ::qFree(p);
    }
}

void MallocBenchmark::regularMalloc_data()
{
    qtMalloc_data();
}

void MallocBenchmark::regularMalloc()
{
    QFETCH(int, size);

    QBENCHMARK {
        void *p = malloc(size);
        free(p);
    }
}

QTEST_MAIN(MallocBenchmark)

#include "main.moc"

And now, the results on my machine:

********* Start testing of MallocBenchmark *********
Config: Using QTest library 5.0.0, Qt 5.0.0
PASS   : MallocBenchmark::initTestCase()
RESULT : MallocBenchmark::qtMalloc():"1":
     0.000059 msecs per iteration (total: 62, iterations: 1048576)
RESULT : MallocBenchmark::qtMalloc():"10":
     0.000062 msecs per iteration (total: 66, iterations: 1048576)
RESULT : MallocBenchmark::qtMalloc():"100":
     0.000087 msecs per iteration (total: 92, iterations: 1048576)
RESULT : MallocBenchmark::qtMalloc():"10000":
     0.000083 msecs per iteration (total: 88, iterations: 1048576)
RESULT : MallocBenchmark::qtMalloc():"1000000":
     0.0043 msecs per iteration (total: 72, iterations: 16384)
RESULT : MallocBenchmark::qtMalloc():"10000000":
     0.0063 msecs per iteration (total: 52, iterations: 8192)
PASS   : MallocBenchmark::qtMalloc()
RESULT : MallocBenchmark::regularMalloc():"1":
     0.000053 msecs per iteration (total: 56, iterations: 1048576)
RESULT : MallocBenchmark::regularMalloc():"10":
     0.000051 msecs per iteration (total: 54, iterations: 1048576)
RESULT : MallocBenchmark::regularMalloc():"100":
     0.000082 msecs per iteration (total: 86, iterations: 1048576)
RESULT : MallocBenchmark::regularMalloc():"10000":
     0.000076 msecs per iteration (total: 80, iterations: 1048576)
RESULT : MallocBenchmark::regularMalloc():"1000000":
     0.0043 msecs per iteration (total: 71, iterations: 16384)
RESULT : MallocBenchmark::regularMalloc():"10000000":
     0.0060 msecs per iteration (total: 99, iterations: 16384)
PASS   : MallocBenchmark::regularMalloc()
PASS   : MallocBenchmark::cleanupTestCase()
Totals: 4 passed, 0 failed, 0 skipped
********* Finished testing of MallocBenchmark *********

Around 10% extra time per iteration on smaller allocation sizes, 0-5% on larger sizes (most likely explained by glibc falling back to using mmap for larger allocations, which is going to take an awful long time compared to a single function call). These, obviously, aren't huge numbers. But remember: this is overhead you're taking for no reason at all. Don't do it. Your CPU cycles will thank me.

Qt 5.1, aka: when QFileSystemWatcher might not be so useless

With 5.0 now being feature frozen, I thought I'd turn myself towards something I've been meaning to do (and talking about doing) for a very long time. Back before the Qt Project launched, even before Qt Contributors Summit 2011, in fact. I thought I'd make QFileSystemWatcher more useful.

Let's review what QFileSystemWatcher offers: a way to add and remove paths for monitoring directories and files, a directoryChanged(path) and a fileChanged(path) signal. That's it. If you think about this for a moment, you realise just how broken the semantics of the signals are: what _is_ a 'change' anyway? I guess this is one of the reasons that QFileSystemWatcher has been called 'deprecated' (I fixed one of the others, a performance issue, a while back).

So I've been working on making the signals a bit less useless. In the longer run, I plan to call fileChanged and directoryChanged deprecated. They'll still be emitted (of course) to keep existing code working, but in addition, you'll have (subject to review):

pathCreated(path) - emitted when something is created inside a directory you are monitoring, or if something that didn't exist that you were monitoring for is created (more on this later)
pathDeleted(path) - emitted when something is deleted inside a directory you were monitoring, or something that you _were_ monitoring was deleted
fileModified(path) - emitted when a file you were monitoring is modified (attributes or contents)

I also have early plans to introduce a pathMoved(oldLocation, newLocation), but that one has a lot of caveats: it might only work on certain platforms, in certain phases of the moon, and only if you're very lucky - on many platforms, it will likely continue to be synthesised as a pathDeleted(oldPath) and pathCreated(newPath) (if you're watching the new location).

I also have vague ideas about introducing more syntactically friendly API over the top of this, something like:

QPathMonitor pathMonitor(myPath);
connect(&pathMonitor, SIGNAL(deleted()), SLOT(watchedPathDeleted()));
... etc ...

but that's a bit farther away in that I haven't really thought it through, yet.

In working on the new signals on Windows, I stumbled across QTBUG-2331, which sort of proves just how useless the existing signals were: deleting a watched directory on Windows essentially never removed the watch, because it didn't notice the deletion. Hopefully, it will be fixed soon, because it's going to block the work on the new signals: otherwise, pathDeleted will never be emitted on a watched path on Windows, and that makes my unit tests sad.

I noted earlier that I have plans to emit pathCreated when you monitor a non-existent path, well, that's correct - I intend to allow just that. None of the native APIs (that I know of) allow for recieving events when you monitor a non-existent path, which is a bit sad, as it means we'll need to fall back to polling on a timer for those cases, but it's certainly much better than nothing.

Oh, and if you're curious, it seems like Linux has the best filesystem monitoring API, in the form of inotify. OS X/BSD comes in second with kqueue (though having to open file descriptors to do the actual monitoring is a bit crap, and not being able to get any sort of real fine-grained notifications on _what_ was added/removed in a directory is also painful). Windows is also incredibly painful due to running into many stupid limitations (like only some ~60 paths being able to be monitored per thread, so the backend spawns loads of threads if you monitor a lot of paths), and not being able to get even remotely useful signals without a lot of extra legwork. Not to mention the above bug, where signals are emitted before deletion.

OS X may get more love in the future, as there is talk about another contributor revisiting the state of the FSEvents backend (which has been disabled for a very long time, and the code removed in Qt 5, due to being massively buggy and unmaintained).

For Windows, ReadDirectoryChangesW might improve the situation, somewhat, but I certainly don't have the time to investigate it for 5.1, and I also lack the motivation, not being a Windows user myself. Contributions welcome?