i'm special

software is still awful

Sat, 07 Jan 2017 00:00:00 +0100

In my year of absense from writing anything here at all, it wasn’t necessarily only because the year was awful - although it kind of was, not just for the world at large, but also for more personal reasons – a subject for another post, or perhaps not.

Regardless, when I did eventually turn my attention back here, I managed to come back to find that Jekyll could no longer generate the same blog I left - even without any alterations or new content.

I first found out that jekyll couldn’t tell the time. In the less destructive cases, I had posts that were being timestamped an hour or two off where they should have been. In worse cases, they were a full day off the correct times. Sigh. I searched around a bit and I can now confirm, in case you were wondering, that dates and times are still awful in software.

I don’t know if that is precisely the issue I hit (because again, I just wanted to move on past all this and start actually doing things), so my workaround was to just remove dates off everything, because really, who needs to know what time of day I write things anyway. One s/([0-9]{4})-([0-9]{2})-([0-9]{2})T.+Z\'/\1-\2-\3'/ later, and my posts were pretty much as they should be. I have to ask, though, why in the actual fuck does a static content generator like jekyll care about timezones? The dates and times I give you are the dates I want displayed.

Oh, and for whatever reason, before I could even try to generate things, I had to gem install jekyll-paginate and add gems: [jekyll-paginate] to my _config.yaml - I don’t know what caused this (I didn’t bother to do my homework, as the solution was easy enough). I’m sure this was a change made with the best of intentions, but it certainly wasn’t helpful when I’ve already been struggling with the motivation to write anything at all.

Once I got past all that, though, it wasn’t all bad. I finally fixed the Makefile I use to write here to automatically generate title metadata for Jekyll (one less piece of friction involved in writing a post):

post:
    $(eval POSTTITLE ?= $(shell read -p "Post title: "; echo $$REPLY))
    $(eval POSTTITLE_ESCAPED ?= $(shell echo "${POSTTITLE}" | sed 's/ /-/g'))
    $(eval FILENAME := $(shell date +%Y/%Y-%m-%d-${POSTTITLE_ESCAPED}).md)
    mkdir -p _posts/$(shell date +%Y)
    cp _drafts/.template.md _posts/${FILENAME}
    sed -i '' "s/\%TITLE/$(POSTTITLE)/" _posts/${FILENAME}
    sed -i '' 's/\%DATE/$(shell date "+%Y-%m-%d")/' _posts/${FILENAME}
    vim _posts/${FILENAME}

I’m sure this isn’t ideal somehow, but I didn’t want to deal with some kind of exotic language or system with dependencies back when I started, so I’ve just been gradually making this more arcane to (somehow) accomplish what I want. I don’t have a clue about Makefiles, so this is all trial-and-error. So sue me.

It’s still not perfect, because it means when I want to rename a post (which happens a lot after I start writing, as I did a few times on this post even), it means that I have to figure out how to do the escaping for the new filename, locate it in the awfully structured directory tree, and remember to change the title attribute. But I suppose this is the price I pay for not using a fancy web interface anymore.

Not that I miss the complete shit that blogger’s web interface generated instead of writing my posts in Markdown.

I think next, I want to figure out how to generate tag-based archives again – I did some initial research and it doesn’t look too difficult, just a matter of actually trying it out. And then I need to start actually using tags again, and maybe making the Makefile nightmare even worse, to help me tag a post.

∞ Permalink

that was the year, that was: 2016

Fri, 06 Jan 2017 00:00:00 +0100

I didn’t write anything about that year here so far. Let’s just say I figure that enough already has been said about that one. Here’s to 2017.

∞ Permalink

Qt ♥ TUIO

Wed, 18 Mar 2015 00:00:00 +0100

Qt, as a framework, makes extensive use of touch - particularly on mobile and embedded devices. Unfortunately, Qt isn’t readily able to magically make touch input appear on hardware that doesn’t support it, which can make exact testing of touch-related code difficult in typical developer environments – not a very desirable situation for a cross-platform development framework.

Thankfully, someone else has thought of a solution that helps to fill this gap, in the form of the TUIO protocol. TUIO is a specification for a protocol to encode touch events to be sent over a remote transport (such as a network).

In practice, this means that one can create a server that synthesizes touch events and sends them to an application, even if it is running on a completely different physical machine.

For example, there are a number of mobile clients such as TUIOPad and TUIODroid which can be used to take touch events from a mobile device (such as a phone or tablet) and send them to a remote TUIO listener.

I came across this last year, and thought the idea was pretty neat, so I went off and built something to bring this to Qt users - the result was an input plugin for Qt which listens for touch events coming in from a TUIO source.

This was made easy in that in the Qt 5 world, things such as input and display are abstracted away from the core of Qt itself, so that one can easily provide an additional input source.

show me!

If you’d like to check this out yourself, it’s easy! Grab a copy of Qt 5.5 (the alpha just came out, and that will do nicely), and build it as normal. When running your application, use the command line argument “-plugin” to tell Qt to load the TUIO input plugin as follows:

mytestapplication -plugin TuioTouch:udp=40001

The udp=40001 tells the plugin to bind to the UDP port 40001 and listen for incoming TUIO traffic.

known limitations

There’s a few caveats, here.

Only UDP sources are supported, although other transports would not be impossible to add
Sources are bound on QHostAddress::Any, meaning that your network is considered “trusted”
Only a single TUIO server is supported. Multiple senders on a network will not intrinsically break, but it won’t work well.
Only the 2dcur TUIO profile is supported at this time.

Patches are more than welcome! (to src/plugins/generic/tuiotouch in qtbase).

technical details

TUIO at a high level is a specification of events. The actual wire protocol used by TUIO is a seperate specification called “OSC”

Open Sound Control.

The plugin contains a simple OSC parse (QOscBundle and QOscMessage) for handling the wire protocol parsing, and on top of that, a handler for TUIO messages (QTuioHandler).

QTuioHandler is responsible for receiving TUIO messages, using the OSC classes to parse them, and creating touch messages to send to QWindowSystemInterface, to notify Qt about the events. QWindowSystemInterface subsequently takes care of letting the application itself know about the touches through the regular and well known QTouchEvent interface.

The application can then consume the QTouchEvent without having to know about the origin :)

∞ Permalink

ditching google

Thu, 12 Mar 2015 01:53:07 +0100

I’ve been working on moving away from Google for a while now. I have a lot of reasons, but the primary one is that I don’t like the feel of being a product rather than a consumer. I want to be in control of my own data, and how it is used and disseminated. With Google in control of it, I never really felt I had that, and it was only getting worse with time.

Anyway, 2015 has been a good year for making good on this resolution:

I moved to using DuckDuckGo a few weeks ago
About a week ago, I migrated my blog’s data to Jekyll (which you’re reading, right now).
Most recently, after getting annoyed at Google once again, I finally got annoyed enough to take the last, most drastic step: email, calendaring and contacts.

I was on my second email account with Google. My original one was a Gmail account, created back in 2005 or so (I don’t even remember anymore). I stopped using that one in fan of using my own GApps account with my own domain (shared amongst other people in my family) in 2011 or something like that.

My migration plan went something like this:

Sign up for FastMail (business tier, partly because we’re using it as a family, and partly because I actually have my own business, and partly because I want to support FastMail)
Start FastMail’s IMAP import
Clean up the imported stuff (my email has a long-going reputation for being a junkyard where important things go to die)
Export ICS/VCard for my calendar & contact data out of Google and import them into FastMail
Delete everything out of my GMail account

All of this went pretty painlessly, and so far, I’m very happy with the switch. I’ve found one pretty serious bug with FM’s iOS client which I need to report, but aside from that, smooth sailing. I look forward to writing my own JMAP client, when FastMail’s servers support it.

At this point, I’ve still got the Google account open (but now, dataless) so that I can keep G+. That (and occasional recreational YouTube video watching) are the only things I still have Google in my life for, and it feels pretty good.

∞ Permalink

bye, blogger

Wed, 11 Mar 2015 15:35:37 +0100

It’s been a good run, but when you read this, it won’t be hosted by blogger anymore.

I’ve had hideous experiences with blogger being full of bugs in the past, and none of these issues ever seem to get any better. That, in conjunction with my not being happy with other people being in full control over my data.

So I figure that it’s time for a change.

Getting the text of my posts into Jekyll was pretty easy (thanks to the importer plugin). Unfortunately, the source text is in unimaginably bad HTML (thanks, blogger), so I hope I don’t ever need to go back and adjust formatting or anything.

Images are also hosted on blogspot, but there don’t seem to be too many. I’m not sure I’ll bother trying to migrate them over (I spent far too long trying to write some scripts to do that, and not having much luck).

We’ll see.

∞ Permalink

So long, Gitorious

Wed, 04 Mar 2015 00:00:00 +0100

Yesterday, I read the news about GitLab acquiring Gitorious to shut it down gracefully (since Gitorious was apparently no longer profitable), and honestly, I'm not at all surprised.

When Gitorious first opened up, I jumped on it with open arms and pushed up quite a bit of stuff there. I migrated some of the larger group projects I worked on, such as InspIRCd there, contributed to others, and made my best to use the place, generally.

And after a while, I found myself increasingly getting frustrated with things. The UI was clunky in a lot of places, buggy in others (merge requests and the like in particular caused a lot of grief to me), and there was no real insight or visibility about the problems or work happening to fix any of that. So, I migrated my personal projects to Github, InspIRCd also moved to Github (once I noticed that it had been in a period of lull for quite a long time), and as far as I can tell, nobody really looked back.

I'm not sure what to make of any of this, if it's a case of complacency on the part of the people involved, or simply a "worst possible storm" of multiple factors, but I certainly can't say I'm surprised, one way or another.

The only thing that does surprise me, is that it took this long. ∞ Permalink

My experience with Chocolat (the text editor for OS X)

Sun, 19 Oct 2014 02:32:00 +0200

Or, "why I won't be buying it again".

Some of you might have heard of Chocolat. Some of you might not have, so here's a quick summary. It describes itself as a "Native text editor for Mac".

I found Chocolat earlier this year, and kind of liked it. It had pretty good JavaScript support, and as I write a reasonable amount of JavaScript on random projects usually involving QML, I found it worthwhile enough to buy it for the few bucks it cost. I was quite happy with it. The user interface was also neat. I bought it in May, and was pretty happy.

But soon after, OS X Yosemite bursts onto the scene.OS X Yosemite was announced and released to developers on June 2, 2014. The public beta was released on June 24, which I upgraded to on day one. I work on a lot of software, so it's kind of natural that I'm on the close-to-bleeding-edge, so I can make sure that things work.

Chocolat, unfortunately, didn't work on Yosemite. It crashed. As I don't use it all that much, I didn't mind, figuring that this would get sorted out sooner or later. Aside from that, my Yosemite story has been pretty painless and enjoyable.

This week, I finally decided to give Chocolat another try, figured out that version 3 supposedly doesn't crash on Yosemite, and got redirected to http://chocolatapp.com/3/. Right up there, in green font is this:

Why, I hear you ask, is the pricing different? Good question.

It costs money because, well, it's a major upgrade. Not too much visible on the user end, but apparently there was a lot of technical debt that needed to be paid down. I can respect that.

On the other hand, it's free for Mavericks users because 2.x crashes on Yosemite, and if it wasn't free for users before they upgrade, there would probably be a lot more pissed off people.

But where does that leave people like me: people who had the (misfortune?) of trying to live on the bleeding edge of software? Well, I'm stuck with that crashing 2.x, because I didn't upgrade when I still had Mavericks available to me. Now, I get to pay another $15 for software I already bought this year, and was able to actually use for two months.

Meanwhile, everyone who stuck with Mavericks gets it for free, because otherwise, there would be no "smooth upgrade path" (read: a lot more pissed off people). I'm sorry, but where the hell is my "smooth upgrade path"?

What am I going to do about this? Well, I'm definitely not going to buy Chocolat for the second time in a year. Apparently, it's pissed enough other people off that they had to write a "FAQ" page about it. But an FAQ page doesn't give me the working software that I paid for already.

Sad, really.

As an aside, I was originally not going to write about this, and just let it slide (it's not like it's expensive software). But then I actually talked to the author about it, and the answer I got could be tl;dr'd down to "well, but don't you think I should get paid for the work I did on v3?" to which my answer is "sure, but I already paid you two months before I stopped being able to use it". And that got me annoyed enough to write this post.

I'll be clear here: I have nothing wrong with the authors of software getting paid so they can keep writing software.

What I do have problems with is buying a product and not being able to use it two months later. And still not being able to use it once the admittedly beta operating system I was on at the time is out in public release. Especially when everyone else is able to upgrade without a second thought.

And that's why I'll be avoiding Chocolat in the future. ∞ Permalink

Wayland and Qt 5.4

Tue, 23 Sep 2014 10:39:00 +0200

Since nobody else has done the honors yet, I'm happy to announce that - as decided at the Qt Contributors Summit this year - support for running applications under a Wayland compositor will be seeing its initial release with Qt 5.4. That is, the QtWayland repository is finally going to stop sitting in the corner, sulking. :)

There's a few "buts", though.

Firstly, it should be noted that support for QWidget-Based applications (and other desktop-based usecases) may be far from ideal, and quality may not be great. This is a consequence of most development on QtWayland having been driven from mobile/embedded viewpoints to date, and is not, in general, an inherent limitation on the windowing system. It's also something of a reflection on Wayland itself, which is only now starting to mature for desktop use (through xdg-shell etc etc.)

tl;dr: Think of this as a technical preview, keep your expectations realistic, and if you want to use it, expect to roll up your sleeves a bit and get dirty from time to time.

Secondly, the QtCompositor API in the QtWayland module (allowing you to write your own Wayland compositor) will not be seeing a release at this time. The API is not frozen, and has not seen the usual polish/quality that you might expect from Qt APIs. As this API is only of use to a limited number of people (those looking to implement an embedded/mobile device, typically, or write their own DE) this should not impact too many people.

tl;dr: If you want to write a compositor, you get to keep both pieces if it breaks. If you want to use applications under an existing Wayland compositor, you're fine.

Future work to QtWayland is largely an open story, but some obvious candidates come to mind:

Continued work on xdg-shell support
~~Plugin based window decorations (to enable environment-specific look and feel)~~ this has now landed in the 5.4 branch :)
Integration with the rest of Qt's autotests (I spent a while getting tests fixed or at least runnable under window-compositor, but it would be nice to automate this)
"Official" subsurface protocol support

If there's something you would like to see happen, here or not, you're more than welcome to pitch in. If you'd like to talk to the other people hacking on QtWayland, please pop by on #qt-lighthouse on freenode, and talk to the folks there :-)

I'd also like to take a moment to thank everyone for their contributions to QtWayland. In particular, I'd like to say thanks to the following, in no particular order (and I'm extremely sorry if I've missed someone, please let me know and I'll happily add you to the list):

Kristian Høgsberg & Jesse Barnes, for their initial work on the port, sponsored by Intel,
Jørgen Lind, Samuel Rødal, Andy Nichols, Laszlo Agocs, and Paul Olav Tvete for continuing work on it excellently and admirably,
Nokia for sponsoring a good deal of the development up until their abrupt departure from the Qt world,
Digia for continuing to help out after Nokia left,
Andrew Knight, for ably shepherding problems encountered by Jolla for quite a long time,
Jolla for sponsoring a large chunk of work on QtWayland (past and present),
Gunnar Sletta for rewriting integration with rendering (especially QtQuick), removing a large number of bugs & improving performance,
Giulio Camuffo for numerous fixes, improvements and interaction with the wider Wayland community.

As a conclusion, I'd like to note that I'm really happy to see this finally happen - I've wanted it for a very long time now - and for Wayland to keep moving on for bigger and better things. Hopefully, this release will achieve its intended result (that more eyes/hands get exposed to the code, and start to use it, and help out with it). ∞ Permalink

profiling is not understanding

Fri, 12 Sep 2014 20:06:00 +0200

When software goes slow, generally, the first reaction is to profile. This might be done through system tools (like Instruments on OS X, perf/valgrind/etc on Linux, VTune, etc). This is fine and good, but just because you have the output of a tool does not necessarily correlate to understanding what is going on.

This might seem like an obvious distinction, but all too often, efforts at improving performance focus on the small picture ("this thing here is slow") and not the bigger picture ("why is this so slow"). At Jolla, I had the pleasure of running into one such instance of this, together with Gunnar Sletta, my esteemed colleague, and friend.

As those of you who are familiar with Jolla may know, we had been working on upgrading to a newer Qt release. This also involved quite a bit of work for us, both in properly upstreaming work we had done on the hurry to the late-2013 release, and in isolating problems and fixing them properly in newer code (the new scenegraph renderer, and the v4 javascript engine in particular have been an interesting ride to get both at once!).

As a part of this work, we noted that touch handling was quite slow (something which we had worked around for our initial release, but now wanted to solve properly). This was due to the touch driver on the Jolla introducing touchpoints faster than the display was updating, that is, while the display might be updating at 57 hz (yes, the Jolla is weird, it doesn't do 60 hz) - we might be getting input events a lot more frequently than that.

This was, in turn, causing QtQuick to run touch processing (involving costly item traversals, as well as the actual processing of touch handling) a lot more frequently than the display was updating. As these took so much time, this in turn slowed rendering down, meaning even more touch handling was going on per frame. A really ugly situation.

Figure 1: Event tracing inside the Sailfish OS Compositor

Figure 1 demonstrates this happening at the compositor level. The bottom slice (titled "QThread") is the event delivery thread, responsible for reading events from evdev The peaks there are - naturally - when events are being read in. The top thread is the GUI thread, and the high peaks there are touch events being processed and delivered to the right QtQuick item (in this case, a Wayland client, we'll get to that later). The middle slice is the compositor's scenegraph rendering (using QtQuick).

With the explanation out of the way, let's look at the details a bit more. It's obvious that the event thread is regularly delivering events at around-but-not-quite twice the display update. Our frame preparation on the GUI thread looks good, despite the too-frequent occurrence of event delivery, though, and the render thread is coping too.

But this isn't a major surprise - the compositor in this case is dead simple (just showing a fullscreen client). What about the client? Let's take a look at it over the same timeframe...

Figure 2: Event tracing for the client (Silica's component gallery, in this case)

Figure 2 focuses on two threads in the client: the render thread (top), and the GUI thread (bottom). Touch events are delivered on the GUI thread, QtQuick processes them there while preparing the next frame for the render thread.

Here, it's very clear that touch processing is happening way too often, and worse than that, it's taking a very long time (each touch event's processing is taking ~4ms), not leaving much time for rendering - and this was on a completely unloaded device. In a more complicated client still, this impact would be much, much worse, leading to frame skipping (which we saw, on some other applications).

Going back to my original introduction here, if we had used traditional profiling techniques, we'd have seen that touch handling/preparation to render was taking a really long time. And we might have focused on optimizing that. Instead, thanks to some out-of-the-box thinking, we looked at the overall structure of application flow, and were able to see the real problem: doing extra work that wasn't necessary.

As an aside to this, I'm happy to announce that we worked out a neat solution to this: QtQuick now doesn't immediately process touch events, instead, choosing to wait until it is about to prepare the next frame for display - as well as "compressing" them to only deal with the minimal number of sensible touch updates per frame. This should have no real impact on any hardware where touch delivery was occurring at a sensible rate, but for any hardware where touch was previously delivering too fast, this will no longer be a problem as of Qt 5.4.

(Thanks to Gunnar & myself for the fix, Carsten & Mikko for opening my eyes about performance tooling, and Jolla for sponsoring this work.

P.S. If you're looking for performance experts, Qt/QML/etc expertise or all round awesome, Gunnar and myself are currently interested in hearing from you.) ∞ Permalink

sailing in search of fresh waters

Wed, 13 Aug 2014 13:55:00 +0200

I've had a long, quiet time on this blog over the past few years while I've been frantically helping Jolla to launch their self-named product: the Jolla. I've enjoyed (almost) every day I've been there: they really are a great bunch of people and the work has been plentiful and challenging.

But as the saying goes, "this too shall pass". Nothing lasts forever, and it's time for a change: after this week, I will be taking a break from Jolla to get some fresh perspective.

On the bright side, maybe I'll have some more time for writing now :)

If anyone is interested in getting a hold of a C++/Qt/QML/Linux expert with a focus on performance, expertise on mobile, and a wide range of knowledge across other areas who loves open source, please let me know. ∞ Permalink