facebrick: end of the line for me

Hi,

As many of you might have noticed in recent times, facebrick has ceased to work. This is (obviously) unfortunate, but mostly out of my control. I have attempted to look into this, without any luck, and at this point, I've decided to make it public that I have no desire to further work on Facebrick.

Facebook seem to have changed something on their API, which has caused Facebrick to cease to function, with an error about invalid source IP on the client end (but this is actually misleading: the real problem is that the session proxy appears to no longer work, constantly returning error 100 - invalid parameter).

Now, the why of why I no longer want to work on Facebrick:

• I no longer (really) use Facebook as often as I did
  
  I consider other services like Twitter to be (mostly) better oriented towards the way I used Facebook, and they are also much less woeful at trying to make my private data public
  
• As a result of the above, I no longer really use Facebrick
  
  I have been helping brik to keep writing code under GSOC and afterwards, but I haven't really done any development of my own for quite some time.
  
  This has partly been a result of time constraints (so much to do, so little time) - but a lot of it is simply that I don't use Facebook any more, and hence, have no need or desire to write a Facebook client for mobile usage.
  
• Facebook's developer documentation is woeful
  
  This was bad when I initially started writing Facebrick, but now it's gotten rather critical. Facebrick needs to use what is called a 'session proxy' to talk to Facebook, yet Facebook's example doesn't actually work. (How they expect a third party to squeeze gold from a pile of crap is beyond me...) - see http://wiki.developers.facebook.com/index.php/Session_Proxy
  
  I (and many others) rolled our own session proxies (for example, http://www.n8gray.org/blog/2010/06/09/a-simple-facebook-session-proxy-for-google-app-engine/) - however, theirs, like mine, all seem to no longer function.
  
  This bad attitude towards documentation is also evident in things like occasional references to svn.facebook.com, which (helpfully!) stopped existing a long time ago, possibly replaced by github.com/facebook (but none of the documentation will mention that), and likewise, none of the documentation will tell you that the PHP SDK available there is nowhere near the original PHP Facebook API, etc.

I'd like to stress that this has nothing at all to do with my lack of desire to support MaemoMeeGo, it's purely related to the above (my frustrations with Facebook's closed mindedness and bad documentation, however, being the straw that has broken the camel's back). I still use my N900 and am as happy as ever with it.

I'm open to continuing to host the server end of Facebrick, if anyone can get it working again. If you'd like to help, your best bet is to join in the forums thread.

Another day, another itch.

(First, an apology in advance, this is a bit of a braindump, and as such, isn't very well structured.)

I've recently been diving into SQL support in Qt, provided by the QtSql module. Specifically, I've been looking into providing IPC on top of QSqlTableModel.

(For those of you who aren't too technically oriented, I'm trying to have multiple processes have the same basedata store, and know when another process modifies the data in some way, to keep all processes in sync).

While I won't go into details on that now, as it isn't complete, it has been an interesting journey, and I think I've found yet another few things I want to add to my todo list. You see, QSqlTableModel is a great class, but it has a number of shortcomings.

It can't be used in a tree

This one is pretty obvious. It is, after all, a table - not a tree. However, making it represent a tree wouldn't be all that difficult I think (a customised implementation of QAbstractItemModel::parent() method looking at e.g. the 'parent' column of a table and returning a QModelIndex to the appropriate row or similar).

I don't believe that this can be solved in the current implementation, though, as QSqlTableModel inherits QSqlQueryModel which inherits QAbstractTableModel - meaning there's a non-trivial amount of tableness about the current implementation.

For some data (think a task manager: you might have subtasks under a task), this is an annoying limitation.

Incremental fetching isn't always supported

I'm actually not positive on this point, but it looks like incremental fetching is left up to the database driver, meaning that on some database types, you won't have incremental fetching.

On a large database containing thousands of rows (or gigabytes of data) this could be a very bad thing if I am correct.

Partial updates aren't really supported

If you delete a row from a database table, what would you expect the client to do after deleting that row?

I know I certainly wouldn't expect it to re-run the query to refetch the data, yet that's exactly what QSqlTableModel seems to do. Not nice.

This also happens on any other conditions of alteration to the data, which strikes me as being rather inefficient.

Furthermore, it isn't really possible to update values on the fly; this proved to be quite an annoyance when implementing IPC notifications of updates. QSqlQueryModel internally contains a QSqlQuery. QSqlQuery has a ::value() method, to retrieve a QVariant containing the data in a given row of a result set. It doesn't, however, provide any method to change that data (even in memory), meaning that once again, a full reload is the only real solution to something else changing the data out from under you.

This isn't really so unexpected, as it's a bit of an odd use-case, but it has made my implementation trickier than I'd wish.

Conclusion?

I wish. I don't know what to do, yet, but I suspect it's going to involve writing code.

Thoughts welcome.

Strings and Qt

One thing which comes up quite often when I'm talking to developers new to Qt is the topic of strings, more specifically, character encoding: how to do it right, what options are available, and best practices.

Having written this a few times now, I thought that perhaps it was about time I write it up in a more permanent location (here) in the hopes that people will stumble across it and magically become enlightened, and end world hunger. ;)

Qt (and C++) have a number of different string types.

QString

Qt has a string type in QtCore called QString. QString, internally, stores data in utf16, and *does* have knowledge of character encoding.

Services across a network (like web services) often want data in utf8. QString, however, stores data in utf16. To get to utf8, you want QString::toUtf8(). To convert from utf8 back to utf16 QString (e.g. parsing input from a web service) see, QString::fromUtf8().

std::string

C++ also has std::string (although you won't find a lot of this in Qt applications). Simply put, it's a wrapper around a C string providing convenience operations and nicer syntax. It still doesn't have such (fairly essential) things like character encoding.

You probably want to avoid using this in an internationalized application or one requiring interaction with network services unless you find your own solution for encoding issues.

QString has ::toStdString() and ::fromStdString() methods if you must use them for whatever reason.

C strings (char*)

Finally, you have C strings (char*) which don't have any idea what encoding is, they are just a bunch of bytes.

Generally speaking, they're latin1 encoded (ASCII), to put them into a QString.. QString::fromLatin1(). If they aren't latin1, see QTextCodec::setCodecForCStrings().

QLatin1String class is also helpful - in particular, this will allow you to compile when using QT_NO_CAST_FROM_ASCII (which itself is helpful to make sure you explicitly give encodings for all of your strings).

achieving openness: it's about the journey, not the destination

I have written a lot of articles previously about project openness, and I've had this one cooking in drafts for a while without time to write much around the actual issues I'm presenting.

With some more thought, I realised that the best way to proceed is just to publish, and not point fingers and draw conclusions (though I certainly did write this with some projects in mind), so here goes.

• Do you reject contributions for non-technical reasons?
• Do you have development documentation (e.g. build instructions, architecture information) available for external contributors?
• Do you answer questions on design, architecture, etc. from external contributors?
• Do you work with contributors to polish their contributions and educate them as to best practices and your project?
• Do you let external contributors take part in design decisions?
• Do you hold external contributions to the same standards of review as internal contributions?
• Do you grant rights (such as commit access, ability to close bugs) to external contributors?
• Do you have a public means for (preferably real time) communication that you use?
• Do you have a public issue tracker?
• Do you use your public infrastructure wherever possible unless an issue is explicitly private?

Run your project against the list -- perhaps you'll find some things you can improve on.

If you've any suggestions to add to the list, why not write a comment? :)

Tracking QSharedPointer leaks

Smart pointers are a great thing. When used properly, they can really help make life easier, and simpler. But things can, and do, occasionally go wrong - and that is when the hurt comes in. Qt provides a number of smart pointer classes, some might say too many, but that's a topic for a whole different discussion, one of which is QSharedPointer.

From the documentation:
"The QSharedPointer class holds a strong reference to a shared pointer

The QSharedPointer is an automatic, shared pointer in C++. It behaves

exactly like a normal pointer for normal purposes, including respect

for constness.

QSharedPointer will delete the pointer it is holding when it goes out

of scope, provided no other QSharedPointer objects are referencing it.

A QSharedPointer object can be created from a normal pointer, another

QSharedPointer object or by promoting a QWeakPointer object to a strong

reference.

Essentially, QSharedPointer works through reference counting, which means if you somehow make a mistake with cleaning up your references, the object your shared pointer refers to won't be deleted, and you've got a hard to track memory leak on your hands. This is precisely what happened to me recently at work, amongst a jungle of a few different libraries, so tracing the problem by hand was really not going to happen, so I needed a miracle, or short of that, a reliable way to track reference count changes on a QSharedPointer instance.

Reading up on QSharedPointer's internals, it became obvious that the reference counting was stored in the dpointer of each QSharedPointer instance. The dpointer is shared amongst QSharedPointer instances referring to the same pointer. So, we should be able to set a watch in gdb to break whenever the refcount changes.

First, we need to find out the address of a QSharedPointer instance, so set a breakpoint just after we first create it:


 (gdb) break main.cpp:22
 Breakpoint 1 at 0x8048806: file main.cpp, line 22.
 (gdb) r
 Starting program: /home/burchr/qsharedpointer/qsharedpointer.
 [Thread debugging using libthread_db enabled]

 Breakpoint 1, main (argc=1, argv=0xbffff444) at main.cpp:22
 22>-    QSharedPointer<MyClass> copy(initial);
 (gdb) p initial
 $1 = {<QtSharedPointer::ExternalRefCount<MyClass>> = {<QtSharedPointer::Basic<MyClass>> = { value = 0x804c438}, d = 0x804c448}, <No data fields>}

Now we have the address, we can set a watch on the QBasicAtomicInt in the dpointer, we can watch the refcount for changes:
 (gdb) watch $1.d->weakref
 Hardware watchpoint 2: $1.d->weakref

Continue debugging, and gdb will break whenever the refcount changes, telling us the old and new values, like so:
 (gdb) c
 Continuing.
 Hardware watchpoint 2: $1.d->weakref

 Old value = {_q_value = 1}
 New value = {_q_value = 2}
 0x08048953 in QBasicAtomicInt::ref (this=0x804c44c)
     at /usr/include/QtCore/qatomic_i386.h:120
 120>                 : "memory");

Much thanks to:

• Arun Raghavan's comment on http://tecnocode.co.uk/2010/07/12/reference-count-debugging-with-gdb/
• Mathias Hasselman's stellar guidance and poking me to the above post