Qt: open testing infrastructure
As a result, I'd like to introduce you to a project I lovingly title the Qt Community Integration System. Its aim: to automatically build, test, and publish community merge requests.
This is an important step in order to help streamline the process of community contribution and get feedback to individual developers quicker. Qt internally use a similar system, but theirs is not (yet?) public.
How does it work? There are a number of components:
- A community Qt repository (why not the 'official' one? read on - this is not a fork)
- This repository is updated by an integration script, which pulls pristine Qt, applies designated merge requests over the top of it, and then pushes it to the master-proposed branch, if they all merge cleanly.
These lists of merge requests are currently maintained by ..me.
However, the intention is to give other trusted people, such as those in #qt-labs or who submit frequent merge requests, access to edit these lists and push to this community repository too, so it becomes possible to request integration of a change if you're experienced and trusted.
- A BuildBot which pulls Qt from the master-proposed branch of this repository, and then proceeds to try to configure, compile, build, and run Qt's extensive test suite, with the help of a custom tool I have purpose-written for this task, unimaginatively called qtestrunner.
The results of these tests are then published on a website, where they may be viewed by others.
- This last bit of this puzzle isn't quite finished, but, assuming all of the above steps succeed with no regressions from the previous test run, the master-community branch is updated with the latest contents of the master-proposed branch, as a working snapshot of Qt+community patches.
(Of course, 'master' will eventually be 'all current Qt branches'. I just wanted to pick one to start with.)
So, after I've described all of the above utopia, what's the current status? Well, I've got a lot of the pieces in place, and now I'm fighting problems with Xvfb causing some of the Qt GUI tests to fail *sometimes*, (even though they don't when I run them manually). Once that is solved, I hope to start adding people's access to integration, and start test-integrating merge requests. :)
Hopefully, all this work will help to accelerate the time it takes to send merge requests back for review if they break something, and ease the burden on getting requests merged in (due to requiring a little less testing). Time will tell. But at least I'm doing my bit. Patches welcome. :)
(thanks to John Brooks for copy-editing my appalling English as usual)