So, you may have heard that there was a WebKitGTK hackfest. I’d meant to post about it too, but hadn’t gotten around to it yet because of (a) a missed connection causing me to arrive home a day later than planned, (b) Christmas travel, (c) New Year’s Eve plans, (d) general blogging lameness.

Anyway…

“I passed through the seven levels of the Candy Cane Forest, past the Sea of Swirly Twirly Gum Drops, and then I walked through the Lincoln Tunnel.“ — Buddy

I did a bunch of network-level hacking at the hackfest, and made good progress in two areas.

GSocket TLS (aka SSL) support

Last year we got networking support into gio, but there are still a handful of important features missing, most notably support for TLS (Transport Layer Security, the protocol formerly known as SSL). I had been working on this on and off throughout the fall, but finally had a chance to sit down and work on it non-stop for a few days in A Coruña. It’s not done yet, but the outline is all there now. If you’re using GSocketClient, and want the default security options (“only accept valid certificates signed by a CA in the system CA list”), it will end up being pretty trivial. If you’re using GSocket directly or need a bit more control over the connection, then it’s a bit more work, but still nice. If you need to manipulate certificates in detail (eg, to pop up a dialog that the user doesn’t understand, telling him things he doesn’t care about) then you’ll probably have to take the raw certificate data that gio gives you and then use your own favorite TLS/SSL library to parse it.

The last bit is because we don’t want to directly expose the TLS library data structures for various reasons. The TLS support is being done via an extension point, to avoid depending directly on the TLS library, and there may end up being two implementations of the TLS extension point; one using GNU TLS (which is nice and small and doesn’t have a zillion dependencies) and one using NSS (which is more complicated and vaguely annoying to use outside the context of mozilla, but is FIPS 140 certified, which is important to Fedora). (There will also probably eventually be implementations using the native Windows and OS X TLS support as well.)

Anyway, the bugzilla bug has the details (which are still changing a bit).

SoupURILoader

The other thing I hacked on was the SoupURILoader/libsoup io rewrite work. This started as an idea to improve the non-http URI support in WebKit (eg, following ftp: links, like when a download page for a piece of software sends you to an FTP directory listing). The general idea was to provide an API in libsoup that was more like libcurl; taking just a URI and returning a stream of data, without the caller having to worry much about the specific details of how the protocol works. Later, people pointed out that this would also be useful as a way to plug custom URI protocols into WebKitGTK-using apps (eg, “about:” URIs in a web browser or “cid:” URIs in mail clients).

Gabriel Corvalan Cornejo picked this up as a Google Summer of Code project this past summer and made some great progress on it; he improved on my original API design and then implemented ftp file and directory fetching/parsing using that API; his final SoC report shows libgweather patched to use SoupURILoader and ftp to fetch the weather (which is a nice improvement because the http interface we’re currently using actually returns HTML and then we have to scrape the METAR report out of the HTML response, whereas the ftp interface just returns the plain text METAR data).

During the hackfest, Benjamin Otte and I took this and… well, turned it into something totally different. It’s unfortunate that not a whole lot of the SoC code survived into the final form, but I still feel like the the SoC project was a success. Our updated version of the API was added to SoupSession rather than creating a new SoupURILoader object as in the original. (This solves a few problems in the original API about how to integrate http and non-http activity; eg, when an HTTP response contains a redirect to an ftp URI.) And in the version currently in git, Benjamin changed it to use gvfs for ftp rather than having its own ftp code; there are good reasons for not wanting to do that (notably, that the gio APIs are really only intended for filesystem-like usage, not web-browser-like usage), but there are also very good reasons for doing it (especially, having only a single codebase, which is better for bugfixing and security, and it also makes things work better when talking to ftp servers with very low connection limits). And we think we can fix (or work around) the problems that make the gio/gvfs APIs not quite work for this case, so overall it seemed like the better plan.

Of course, to make the gio stream-based API work well for HTTP, we need to change libsoup to be more gio-based internally, which means making sure gio’s networking classes support all of the functionality libsoup needs, which is why the GSocket TLS work was particularly relevant to WebKit again.

So, thanks to Igalia for hosting the conference, Collabora for co-sponsoring it, the GNOME Foundation for paying for me to go, and Red Hat for letting me ditch work for a week :-). Now back to GNOME Shell hacking…

I have now joined the ranks of people who used to have a blog based on hand-written perl scripts (well, actually ruby) and rsync (well, actually unison), and who eventually got sick of that and upgraded to a sane blogging platform. Well, actually WordPress. I look forward to getting hacked into and spamming you.

Also, hello Planet Fedora desktop.

I’ve just merged the gresolver branch to the master branch of glib.

And there was much rejoicing:

	[08:44]  <alex___> YAY!

OK, there was some rejoicing anyway.

So, as discussed in the original bug, GResolver solves 4 problems for glib-based network apps:

1. It provides an asynchronous DNS API (and also a synchronous-but-at-least-cancellable DNS API), so your UI doesn’t block while resolving hostnames. It even lets you do asynchronous lookups without using threads, through the magic of Lennart Poettering’s libasyncns (which has been sucked into the glib source tree à la xdgmime rather than being a new external dependency). (On Windows, async lookups-without-threads are done by using the Windows thread pool APIs, which are always available even if you aren’t using glib-level threads.)
2. It handles IPv6 transparently, which is good for you
3. It handles Internationalized Domain Names, which are also becoming increasingly popular. GResolver itself just transparently accepts either ASCII or UTF-8 names, but you can also explicitly convert back and forth between them with g_hostname_to_ascii() (to get a network-safe version) and g_hostname_to_unicode() (to get a user-friendly one).
4. It does SRV record lookup, which is an important part of several protocols like XMPP (Jabber), but which is fairly annoying to do with the standard libc resolver APIs. The gresolver code includes an interface, GSocketConnectable, that abstracts around different ways of describing a network endpoint, so that it’s easy to write APIs that can accept either a hostname-and-port combo, or a SRV record description.

And of course, as suggested by the last point, GResolver is a building block of gnio, the gio-based network I/O library that Ryan Lortie and a few other people are working on, which will now also be making its way towards the glib mainline in time for GNOME 2.28, which will make your life even better than GResolver is already making it.

Now to finish porting libsoup to it…

	> svn rm -f foo
	svn: invalid option character: f
	Type 'svn help' for usage.
	> svn rm --force foo
	D         foo
	>

…time passes…

	> git rm --force foo
	error: unknown option `force'
	> git rm -f foo
	rm 'foo'
	>

Given that Clinton and Obama are allegedly neck-and-neck, I find it fascinating that the blogs I read included 10 or 15 “Vote Obama, OMG he’s so amazing” posts this morning, and not a single “Vote Hillary” (or, for that matter, “Vote Romney”/“Vote McCain”). Apparently Obama has a total lock on the tech/geek vote. (Either that, or Obama is the only person that anyone is actually excited about voting for.)