GSoC '10: Final Report

This is my final report on my Google Summer of Code 2010 project. During the last three months I've learned quite a lot about Psi's and Iris' codebase and implemented most of what's been planned.

I started off implementing a new SASL mechanism, SCRAM-SHA-1, in Psi which will be used if no external SASL library is available.
Using this mechanism users can login securely even over unencrypted connections and if they want Psi to remember their password, this can be done more securely if SCRAM-SHA-1 is available at the server.
More on this part here.

The second part of the project was implementing Stream Management, XEP-0198, in Psi. Luckily, Matthew Wild, one of Prosody's main developers, started to implement it around the same time so we could easily test each independent implementation against each other.
I've implemented the most important and interesting parts of XEP-0198: stanza acknowledgment and stream resumption. Together they make chatting, but basically everything in XMPP, more reliable.
Especially stream resumption is nice in case your connection is dropped. In this case you don't have to go through the whole roster retrieval and presence distribution steps again. The stream resumption part wasn't that easy to implement, because currently Psi destroys its complete XMPP stack state on disconnection with the server.

During the last couple of weeks I've added a new groupchat join dialog to Psi. This included reusable data models for browsing server room listings, bookmarks and history of joined rooms. Additionally the new dialog lets you choose more than one room to join. I've also modified the still existing old join dialog (It still exists because it also handles join logic.) to support bulk join. This means if you join multiple rooms on login, due to bookmarks, or via the new join dialog, the dialogs indicating join process are hidden as long as no error occurs.

It was quite interesting getting to know Psi's and Iris's codebase which are from quite varying design quality considering their parts but most of the time it was quite understandable and Justin, Psi's and Iris's original and main developer, was always able to answer my questions on the code and design.
Coding in the GSoC umbrella organisation XSF was quite fun and well organized. The weekly meetings helped to keep you on track and frequent reports from fellow student kept you up to date on their projects' progress.

All the developed code is available at my github account.

Posted with tags , , ,

GSoC '10: New Join Dialog Finished

I finally got the new join dialog for Psi finished.

New Join Dialog for Psi

When the dialog opens it shows you the rooms available at your server, if there are any, your recently joined rooms and your bookmarked ones. At the top you simply enter the nickname which you want to use to join the rooms.

There's little special about this dialog and it works how you'd expect it to work, hopefully. If you want to give it a try yourself the code is available here.

Posted with tags , , , , ,

GSoC '10: New MUC Join Dialog

Here another short update on my Google Summer of Code project.

I've designed a new dialog for Psi which should ease the process of joining a room. The new dialog's features include:

  • all sources of existing rooms in a single dialog
  • allow entering a JID of a room manually
  • select multiple rooms from server room list, history and bookmarks and join them at once
Currently I'm hammering out the last couple of models for the server room list and bookmarks/history views. To round up this small post, here a screenshot of the dialog: Screenshot of new join dialog.
Posted with tags , , ,

GSoC '10: Psi User Interface Improvements - The Current Situation

Psi is a powerful and popular XMPP client which runs on nearly any platform. It recently got features like Jingle Audio which enables peer-to-peer audio calls.

These new features aside, Psi's user interface is lacking in various places, including, but not limited to, the multi-user chat area. This is kind of sad since the project's mission statement includes ease of use.
The second part of my Google Summer of Code project is to improve this area.

Psi's known to be used more by power IM users than by people who are new to IM, and as power user you like auto-join a couple of MUCs and also connect from multiple places to your account. However if you're already logged in and joined all those rooms, a lot MUC software will complain that you are already in the room, when a your second resource tries to auto-join those.

In Psi this situation can be described by a single picture/screenshot. :)
Screenshot of Psi showing tons of dialogs.
A possible solution to this problem is to handle auto-join of chatrooms on login completely differently compared to manual joins. A long requested feature for Psi is to have chatrooms show up in your contact list. This has already been implemented by the Psi+ project and it's working quite nicely.
For auto-join one couldn't show any dialogs at all and just let the items in the contact list indicate whether a join was successful or an error occurred.

The next MUC section that could need some improvement is manually joining a room. You have to go to different places depending if you already know where to join or not.

  1. You already know the conference you want to join.
    Screenshot of Psi's MUC join dialog.
  2. You have the room already bookmarked.
    Screenshot of Psi's bookmark popup menu.
  3. You have no clue what conference to go in, still you want to chat with other people.
    Screenshot of Psi's service discovery.

A possibile solution to this problem is providing a single dialog for the general task of 'Joining a MUC'. More on that is soon to follow.

Posted with tags , , , ,

GSoC '10: Mid-term approaching

Here comes another short update on my Google Summer of Code project.

Stanza acknowledgement is finally done, including representation in the GUI. You can see a short demonstration of the feature in the video below where I'm chatting with Matthew Wild, one of Prosody's main developers. He developed a module for Prosody that implements parts of Stream Management. This made my client side implementation much more easier to test.

The idea is simple: the status icon in the top left corner is replaced with a throbber animation, known to users from recent OSes and browsers, as long as there are messages that haven't been acked by the server.
Psi will at least request an ack after half a minute. However only if there's something to acknowledge for the server.

This week is mid-term evaluation of the Google Summer of Code projects. SCRAM support and stanza acknowledgement, which is the most important part of the Stream Management XEP, are both finished including GUI.

Posted with tags , , ,

GSOC '10: Another Short Update on Stanza Acknowledgement

Here another short update.

Stanza acknowledgement from client to server is working now and chat dialogs get notified when a message sent by them has been acked by the server. The only thing left here is the visual representation in chat dialogs of unacked and acked messages.

After that is done I'll try to add support for session resumption and hopefully get it done till mid-term evaluation next week. Most of this stuff doesn't sound hard to implement however there is currently only one server implementation of XEP-0198 and that one only supports stanza acknowledgement, so it's a bit hard to test my client side implementation of it. XEP-0198 resulted out of long discussions in the XSF on how to improve reliability and the overall experience of chatting over XMPP. It's important that this extension is implemented so we can test in real life whether these changes help or not.

50% of my end semester exams are taken and the rest is following in the next week. After those I'll have plenty of time to start the second big part of this project, UI improvements in Psi.

Posted with tags , , ,

GSoC '10: Stanza Acknowledgement update

Here the short but little late update on my GSoC project status.

Acking requests coming from the server is basically implemented in Iris. Psi as a client can tell Iris that the last received stanza has been handled and Iris will take care of the rest.

Next on the list: Sending own acking requests to the server so we know the stanzas Psi's send have successfully been handled by the server.

Posted with tags , , ,

Sleep Tight at Night Knowing That Your Passwords Are Safe

SCRAM is a new mechanism designed to let you sign in into your servers much more securely than with existing mechanisms. It involves applying various algorithms to your password to make it extremely difficult to hack.

The SCRAM-SHA-1 SASL mechanism was added to the XMPP spec, and replaces the legacy DIGEST-MD5 mechanism. Its various security features include hashed passwords being send over the internet, and hash passwords being stored on disk. Attackers which can read the network traffic or the server's password database are unable to do any harm.

I just finished a SCRAM-SHA-1 implementation for Psi as part of my GSoC project. There are test cases to write and wider testing to be done, but we now have a working implementation.

What does this mean for Psi users? Well, usually, when you don't want to enter your password each time you login, you tell Psi to save it. Most clients, if not all, store it in clear text form if you tell them to store it at all. This is fairly insecure, and anyone with access to the computer could get access to your password. It's a common security advice that you shouldn't use the same password for all your accounts on various services on the internet. However, human laziness is a powerful force. So, if you use the same password for various services, and someone malicious gains access to it, she/he can gain access into all the other accounts where you used that password.

Now SCRAM comes into play. If your server supports this, and your client supports this, you can have them store your password in a form not very usable by attackers. Thankfully Prosody got some new authentication backend APIs thanks to Jeff Mitchell. You can read more about that in Jeff's blog post. Effectively this means that if a malicious person gains read access to your network or the server, they can't do anything, and if they gain access to your computer, they can do nothing more than login into your jabber account.

What are the consequences of using a hash to login? It basically comes down to support. It requires support by both the client and the server. Only when both have support do you get the full security SCRAM offers. On client side, since you're clearing your plaintext password you have to login using SCRAM in the future. On the server side, you're storing the iteration count, salt and the and two keys instead of the plaintext password. This reduces your possible login methods to only PLAIN (this should only be done over TLS) and SCRAM-SHA-1.

This is currently only available in a development branch. It'll soon get merged into the main branch. To enable this in Psi, first make sure your server supports SCRAM. Then go to the Misc tab in your Account Properties dialog and check the checkbox labelled “Store password as salted hash...”.

Account Properties dialog scram checkbox.

Apply the changes. Reconnect.

Now when you visit your Account Properties dialog, you'll notice that the input box for your password is empty. YAY.

Account Properties dialog showing empty password box.

So what's next on the agenda for my Google Summer of Code project? Currently I'm a little behind but I'll get started on implementing stanza acknowledgement and session resumption of XEP-0198 the next couple of days to get back on the track.

Posted with tags , , , , ,

Making the World a More Reliable Place to Chat

So coding time of Google Summer of Code 2010 just began and I wanted to put out some introductory note about what I'm planning to archive this summer.

This summer I'll be working on the Psi project, an advanced cross-platform XMPP client. My official public abstract of my GSoC '10 project is available here.

The project consists of mainly four parts:

  1. implementing the new SCRAM-SHA-1 SASL authentication mechanism

    This should be a rather easy part. Adding SCRAM-SHA-1 to Iris, Psi's core XMPP C++ library based on the Qt framework. Additionally I'll try to introduce the ability to store the password in hashed form, if the user wants to store it at all. However this demands you're using a XMPP server supporting the SCRAM SASL mechanism and it's using a constant salt and iteration count.

  2. implementing XEP-0198 for a more reliable client-to-server connection

    XEP-0198 has been developed in the XMPP community because reliability is still a problem when communication over the internet, even when using TCP. This protocol extension provides features like stanza acknowledgment, session resumption and throttling. Using this XEP clients can tell the user that she/he can be sure that the message reached the server or if there has been a problem. No more strange wondering about messages getting lost.

  3. implementing XEP-0184; this way you can let your chat partner know you've read a message

    The guys from the Psi+ project already wrote a patch for this as I found out a couple of days ago. I'll evaluate the patch, merge it and make it works in a decent way. You don't want to spam people with message receipt requests.

  4. redesigning the multi-user conference join dialog

    Currently you have your bookmarked rooms in a drop-down menu, the recent joined MUC rooms in the join dialog and public rooms available at your server are accessible via the service discovery dialog. I'll try to unify those three use cases in a single MUC dialog which should easy to process of joining MUCs.

So if you have ideas on any of those items let me know. I'm hanging around in the Psi and Psi+ chat rooms.

Posted with tags , , ,


That's right DIGEST-MD5, your job is done here. Now it's time for the new kid to take up your place.

Those of you who ever tried to actually implement the DIGEST-MD5 SASL mechanism for authentication know that it suffers from quite a few problems, including but not limited to a variety of different implementations with a variable level of compliance to the DIGEST-MD5 RFC 2831. These interoperability problems make it quite hard to get a new implementation working with most of the already existing implementations.

SASL mechanisms are designed to be protocol-independent. That's basically the whole idea about SASL; to abstract the authentication part out of the application protocols, and standardize it. However DIGEST-MD5 already started off on the wrong foot.

Intending to interoperate with digest authentication used for HTTP, the requirement for ISO 8859-1 encoding support was added. However XMPP for example, designed internationally and using latest developments in the DNS area, supports UTF-8 usernames, passwords and IDNs. This makes it hard to store a hashed version of the password since you need to be prepared for either encoding, ISO 8859-1 and UTF-8.

Here comes SCRAM to the rescue. So far I have implemented the basic parts of it in my SASL library which is written in Lua and is a part of Prosody. In the SCRAM draft you can read about the design considerations which are clearly more modern and focused on more international usage.

  • Usernames and password are Unicode and are to be string prepped before usage; a feature DIGEST-MD5 doesn't have.
  • You can now easily store the user relevant authentication information in a database which isn't by itself sufficient to impersonate the user; this might be possible somehow with DIGEST-MD5 but it clearly hasn't been made easy for the developers (leading to XMPP servers storing plaintext passwords).
  • You can simply swap out the basis, the hashing algorithm used, in SCRAM. This makes the mechanism more future proof and secure, unless design issues are found in the protocol itself.
  • SCRAM has been designed to be easier to implement than DIGEST-MD5. I totally agree with this. Message parsing has been made easier by using no quotes on values; DIGEST-MD5 used them but only on some values.

One last note on interoperability: I've continuously tested my implementation against Simon Josefsson's implementation for GSASL, which was quite easy to set up thanks to GSASL's command line interface. Shortly after, I also did interoperability testing against the SCRAM implementation by Paul Aurich for Pidgin, in which we both fixed some bugs. A Prosody instance is still running for interoperability testing at scram-interop.ayena.de. Last but not least, we're working nicely against Swift; however no end-users can be happy about that currently;).

So client and server developers, get ready for SCRAM and implement support for it for a more interoperable future; not to forget its being the MTI mechanism for the next XMPP RFC iterations.

Posted with tags , , , , , ,

libidn vs. ICU benchmark in string prepping

Due to Waqas's asking for months now I've finally got around and made some benchmarks on two string prepping libraries. The one being libidn, as far as I know most XMPP servers use this and the other being ICU by IBM.

libidn is LGPL licensed and ICU is MIT licensed.

Current XMPP standard says that every JID must be string prepped so if you don't want to cache because of memory reasons your string prepping routines are called quite often. It turns out ICU is much faster than libidn. For this benchmark I tested a couple of strings and let them go through nameprep, resourceprep and nodeprep profiles via string prepping.

The result is that ICU is about 60 times faster in string prepping. Version of libidn being 1.15 and of ICU being

The source code package of the benchmark is attached to this node. It falls under GPLv3 license.

Posted with tags , , ,

On the new XMPP server and high release frequency

Since quite a month there is a new XMPP server on the market, called Prosody. Today version 0.2.0 has been released with some new features which you can all read up in the release notes.

Version 0.2.0 has been released just one month after version 0.1.0 and four months after development began. With currently nearly one month between a release and the next we have a good release frequency which is fairly important for a project, especially for one which focuses on ease of prototyping new features in and a protocol around it like XMPP with is evolving pretty fast.

With consent of all the other developers, Waqas and my humble self, Matt dictated the use of Lua for the server which has quite some advantages.
Since it's an interpreted language you don't have to wait for building to test some changes you made or don't need stuff like autoconf to build your project cross-platform. However we have some binary modules for encryption and more low-level stuff.

The resulting server is pretty fast, already implements quite some XEPs while not being a memory beast like eJabberd. Not to forget the nice side affect of using Lua as implementation environment which is that you can easily add new protocols to it and test them. This should help us to push the interesting and important XEPs server side so that the client developers get in charge to implement them on their side.

So feel free and grab Prosody from its download page, test it and if you find some please report bugs on the mailing list or in the chat room.


Posted with tags , , , ,

Inter-Project Collaboration during Google Summer of Code 2008

Since XMPP is (becomming) the biggest player from all the instant messaging protocols out there, there are a lot Google Summer of Code™ projects in the XMPP field this year. BOSH, the highly discussed dream team for connecting to XMPP from mobile or other limited network environments, is covered by a lot projects this year. My GSoC project is, not only, about adding BOSH support to libpurple, the C instant messaging library which powers desktop clients like Pidgin and Adium and web clients like Meebo. libpurple doesn’t only cover nearly any proprietary instant messaging protocol but also some open protocols like IRC, SILC and of course XMPP. For XMPP, as the (future) major instant messaging protocol, it’s most important that XEPs get implemented, coded and used in real life. There is a huge number of XEPs which aren’t implemented and may never be, who knows. I will implement BOSH from the beginning since there is no codebase in libpurple in the BOSH field and just contacted Safa Sofuoğlu, a GSoC student for the XSF mentoring organization, who updates Openfire’s BOSH implementation. We plan to test our implementations of the two XEPs, XEP-0124 and XEP-0206, against each other since he’ll write a server side implementation and I’m writing a client side implementation.

Our aim is to have not just working and good performing implementation but moreover implementations according to the two XMPP Enhancement Proposals. I’m looking forward to the inter-project collaboration.

Cheers and good luck to all Google Summer of Code™ students,


Posted with tags , , , ,

What Were Our XEPs Doing The Last Years?

Here I present some charts on XEPs, XMPP Enhancement Proposals, and their development. You can see, most XEPs are either in Experimental or Draft status. The one and only final XEPs are currently:

  • XEP-0077: In-Band Registration
  • XEP-0030: Service Discovery
  • XEP-0009: Jabber-RPC
  • XEP-0004: Data Forms

There have been two XEP source files unprocessable in XMPP's repository because of invalid XML and since the reposity only exists since end 2006 no longer history was accessable.

These charts have been generated by OpenOffice.org and some custom python script which worked on XSF's svn repository.

Posted with tags ,

Finally a Library Which Takes Extensibility in XMPP for Real

I'm working on a new monitor for XMPP networks, in which I most likely take Jabbermonitor down. The upcoming monitor, called xmppmon, is distributed and flexible.

The whole thing is written in C++ using gloox, the 1.0-alpha. This new version introduces gloox::StanzaExtension which makes extending XMPP with your own custom protocols a job of some minutes.

You are just creating a class which you can use on both sides, for receiving and sending the new protocol. Only three tiny steps need to be done if you have the protocol already in mind.

This is our intended protocol for sending and receiving account infos:

You just need to create some function which converts such data into your internal data structure and vice versa and the XPath string that detects such stanza, in this case: /iq/list[@xmlns='http://ayena.de/protocol/xmppmon#serverlistupdate'].

Since gloox now knows about the XPath string it can automatically detect new incoming stanzas which match the new protocol.You can read about the exact mechanics in the gloox documentation.

While most APIs for C++ make you use some complex code the way gloox does this really elegant.

I'm using this a lot in the new xmppmon which I hopefully get finished during the next days.


Posted with tags ,

Application accepted!

My application for Google Summer of Code 2007 has been accepted today so I’ll spend some time of the summer coding a XMPP Data Form Designer Suite. Some short infos for now about that suite. It’ll cover a RAD like GUI designer for the data forms and bots who will later provide the form using Ad-Hoc Commands and Data Forms to the clients which request it. You can use this suite to create specific member bots, run polls and this kind of stuff.

Posted with tags ,