I've finished my implementation of SOCKS5 bytestreams, which is the basis for peer-to-peer file transfer via TCP with Jingle.
The protocol for Jingle describes that both sides of a file-transfer start a SOCKS5 server and send each other a list of IP/port combinations which the other side can then try to connect to. This alone leads to direct peer-to-peer connection as long as only one side is behind a firewall/NAT, or in other words, at least one side is publicly accessible from the internet. Sadly most users are behind some kind of NAT at home (due to WLAN and DSL routers).
Here assisted candidates, IP/port combinations which were acquired with help of UPnP or NAT-PMP, come into play. A lot of home routers and similar devices support one of those protocols, and if enabled they allow a program like Swift to forward certain ports to itself and request the real public IP from the router.
The plan is to use the libraries from the MiniUPnP project. This turned out to be harder than expected, because the library doesn't provide an asynchronous API. I'll still use those libs but will have to change the design and move those API calls to a dedicated thread.
Kind of as a last resort, there are also SOCKS5 proxies. They are used if it's impossible to establish a direct peer-to-peer connection via TCP. This could be due to firewalls or NATs on both sides with no UPnP/NAT-PMP technologies being available at either end.
While this year's Summer of Code nears its end, my next tasks are basically implementing SOCKS5 proxy support, verification of received data using SHA-1 hash, some GUI work and general testing/cleanup.
Here a short update on the UI. The following 4 pictures will roughly show the workflow of sending a file with Swift.
First you right-click on the contact you want to transfer a file to and select "Send File". That item will only be available if the client your contact uses supports Jingle File-Transfer.
A chat view to that contact will appear showing you a chat bubble for the intended transfer. Here you can actually initiate the transfer or cancel it.
After you clicked the send button a input dialog for the description will appear at as soon as you've entered your description the transfer will be negotiated and started.
During the transfer the progress will be shown as a progress bar, like in many other chat applications. The actual presentation could be a bit smoother but this works for starters.
Semester exams are over at my university and now I have 6 weeks to work on my Google Summer of Code project full-time.
I've just finished a set of updates to Swift which allow file transfer via the contact context menu.
The chat log UI in Swift is a WebKit control, with the content being HTML. The new file transfer functionality now adds a UI in the chat log showing the file transfer. This includes an inline progress bar displaying the transfer's progress.
The file transfer is based on the Jingle File Transfer spec, and interoperates with other clients implementing the same spec. I've successfully tested with Pidgin's and Gajim's development branches implementing the same. We should soon begin to see reliable cross-client file transfer for Jabber soon.
What’s next on the list:
- Finishing up the send-a-file use case (cancelation of transfer and error handling)
- Writing the receive-a-file code in the UI
- Finally implementing SOCKS5 transport-method to have real peer-to-peer transfers
Another short update on the progress on the Jingle file transfer for Swift project.
Outgoing file-transfers using Jingle over IBB work now.
The fallback scenario with Pidgin is working now, too. I forgot to send a candidate-error back to Pidgin, which it was waiting for. Now, after Swift send the candidate-error Pidgin correctly answers with a transport-replace. File-transfer using IBB works now from and to Pidgin's jingle-ft branch.
This fallback protocol flow is described in XEP-0260.
Next on the list of things to do would be extending the existing SOCKS5 code connection to connect to other clients and allow connection from other clients. To increase the likelihood of a real peer-to-peer connection, UPnP/NAT-PMP support is planned using libminiupnp and libnatpmp.
Not to forget the Swift integration, meaning GUI work, so end-users can actually send and receive files to other users.
Here a short update on the status.
I've successfully transfered a file from Pidgin to a ReceiveFile test application which is based on the Swiften code I've wrote. This isn't stock Pidgin yet, but a special branch which supports Jingle File Transfers which is maintained by Marcus Lundblad.
The scenario that's working is sending a file smaller than 64 KB from Pidgin to ReceiveFile. Smaller than 64 KB because only then Pidgin uses IBB for transport directly and doesn't try S5B before. My code currently only supports IBB and has implemented fallback so far but Pidgin's falling back from S5B to IBB isn't working in this edge case yet.
To get this working a lot fresh code had to be written including but not limited to classes representing Jingle file transfer description and Jingle transport methods, their serializers to turn them into XML and of course corresponding parser classes that take XML and return the OO representation. Not to forget hundreds of lines of testing code which I've loosely based on the example snippets found in the XEPs.
What's next? Getting the sending case working plus validating the fallback scenario against Pidgin.
Yup, that's right. I'm participating in Google Summer of Code again.
This time it's for the Swift project, a XMPP client project based on a new XMPP library called Swiften. This is a completely fresh codebase, all written in C++ and based on boost, libxml, Expat, OpenSSL and Qt for the GUI.
This year's project is about adding Jingle File Transfer support to Swift(-en) including the transport dependencies and user interface components for Swfit. The plan is to have working file-transfer in Swift(-en) that works in as many network environments as possible, may it be behind some kind of firewall at university or just behind your home router which is doing NAT.
The first three weeks I've implemented a couple of parsers and serializers related to the XEPs around file transfer, including unit tests for them. I've also finished up some already existing code related to Jingle which has been written by Remko Tronçon, my mentor during Google Summer of Code 2011.
What's next? - Next on the list is getting the file transfer logic done and writing ReceiveFile and SendFile command line applications that provide basic testing for file transfers over XMPP.
So expect further and more frequent updates about this project.
I started off implementing a new SASL mechanism, SCRAM-SHA-1, in Psi which will be used
if no external SASL library is available.
Using this mechanism users can login securely even over unencrypted connections and if they want Psi to remember their password, this can be done more securely if SCRAM-SHA-1 is available at the server.
More on this part here.
The second part of the project was implementing Stream Management, XEP-0198, in Psi. Luckily, Matthew Wild, one of Prosody's main developers, started to implement it around the same time so we could easily test each independent implementation against each other.
I've implemented the most important and interesting parts of XEP-0198: stanza acknowledgment and stream resumption. Together they make chatting, but basically everything in XMPP, more reliable.
Especially stream resumption is nice in case your connection is dropped. In this case you don't have to go through the whole roster retrieval and presence distribution steps again. The stream resumption part wasn't that easy to implement, because currently Psi destroys its complete XMPP stack state on disconnection with the server.
During the last couple of weeks I've added a new groupchat join dialog to Psi. This included reusable data models for browsing server room listings, bookmarks and history of joined rooms. Additionally the new dialog lets you choose more than one room to join. I've also modified the still existing old join dialog (It still exists because it also handles join logic.) to support bulk join. This means if you join multiple rooms on login, due to bookmarks, or via the new join dialog, the dialogs indicating join process are hidden as long as no error occurs.
It was quite interesting getting to know Psi's and Iris's codebase which are from quite varying design quality considering their parts
but most of the time it was quite understandable and Justin, Psi's and Iris's original and main developer, was always able to answer my
questions on the code and design.
Coding in the GSoC umbrella organisation XSF was quite fun and well organized. The weekly meetings helped to keep you on track and frequent reports from fellow student kept you up to date on their projects' progress.
All the developed code is available at my github account.
I finally got the new join dialog for Psi finished.
When the dialog opens it shows you the rooms available at your server, if there are any, your recently joined rooms and your bookmarked ones. At the top you simply enter the nickname which you want to use to join the rooms.
There's little special about this dialog and it works how you'd expect it to work, hopefully. If you want to give it a try yourself the code is available here.
Here another short update on my Google Summer of Code project.
I've designed a new dialog for Psi which should ease the process of joining a room. The new dialog's features include:
- all sources of existing rooms in a single dialog
- allow entering a JID of a room manually
- select multiple rooms from server room list, history and bookmarks and join them at once