(I’ve written about the non-technical aspect of my work this summer previously.)

It took me 3 or 4 weeks to get a basic, functioning XEP-0217 implementation. You can probably do this much faster; you have an existing implementation to look at*, and you don’t have to write a general-purpose test suite at the same time.

* (the test suite is under the MIT license; if that’s too restrictive for you, I’d be happy to release it under something else)

Follow XEP-0218

When I wrote my SoC application I spent a lot of time thinking about the proper order to implement things in. XEP-0218 answers that question for you, and gives a good overview of the various specs involved.

Choose a good crypto library

There’s probably at least one for whatever language you’re working in. The minimum needed to implement XEP-0217:

• a cryptographically strong pseudo-random number generator
• HMAC
• SHA256
• AES-128 (CTR mode)

Good documentation is always a plus.

XEP-0116’s only mandatory difference from XEP-0217 is RSA public key signatures and verification. XEP-0116’s Mandatory to Implement Technologies gives an overview of what other options are available.

I ended up using the Python Cryptography Toolkit. It doesn’t support some of XEP-0116’s options (the Twofish and Serpent ciphers and the Whirlpool hash algorithm), but I’ve been happy with it otherwise.

Sessions :(

Easily the trickiest part was getting XEP-0201 sessions working with Gajim’s existing architecture. Sessions are an integral part of XMPP—<thread/> is one of the basic elements provided in the XMPP IM RFC—but there’s little (if any) support for them.

My implementation isn’t great; Gajim can only have one session between two Full JIDs at a time, and will silently drop an existing session if the remote client starts a new one. To do it any other way would have required a massive refactoring of Gajim’s message handling.

This is a bug, it only works because no other clients (that I’m aware of) have any support for sessions, and it will probably poison the well and make it more difficult for other clients to implement sessions properly. I’m not proud of it, and I will fix it if I can manage the refactoring required to do it properly.

I have no advice here; if your client wasn’t designed with client-to-client sessions from the beginning, they will probably cause you pain. Hack through it.

Use my test suite

I’ve found it very useful, as an autoresponding dummy to test the basics against, and as an actual test suite to verify compliance. Documentation is on the way.

Don’t let the crypto scare you

You don’t need to be a cryptography expert. The specs are very complete, and will guide you around most security pitfalls. As long as you have a general idea of how the whole thing works and what you need to avoid exposing, you’ll be fine. It’s fun stuff!

Summer of Code is over. It’s been a great experience, and beats the pants off of most jobs I can imagine. I’ve met lots of interesting people and learned far more about crypto and XMPP than I would have in my spare time.

(Update: I’ve published a more technical overview.)

My Encrypted Sessions implementation is now part of Gajim’s svn trunk. What I’ve completed to date is still just a proof-of-concept. There are three areas in which my implementation is not production-ready:

• Pseudo-random number generation. I’m using Python’s os.urandom; the documentation says it “should be unpredictable enough for cryptographic applications”, but I’m not 100% certain about that.
• Identity verification interface. The dialog I’m using right now assumes that the user has done the verification, and does not prompt again during future negotiations. If you’re using public keys, they’re not verified at all.
• Shared secret storage. Retained secrets are not encrypted. This requires asking the user for a password before they can negotiate an encrypted session.

I left these things to last because they require time-consuming research. In particular, I still haven’t dealt with Gajim’s GUI much. I intend to finish this work and stick around the XMPP community—there are all kinds of interesting things left to be done with this protocol.

I did not complete everything I set out in my application. In particular, “test all MUST and SHOULD level requirements” was unrealistic (to say the least). This is the first programming project I’ve been on with a real deadline, so I was unaccustomed to making these kind of long-term estimates. I erred on the side of making my application look impressive, which was a mistake—I don’t want to be the kind of person who makes claims he can’t back up.

That said, I think the test suite is very useful in its current state, and hope that it will help pave the way for further implementations of Encrypted Sessions. I’ll publish more details about it (and technical details about implementing XEP-0116 and friends) once I’ve finished moving.

On a tangential note, blogging on a regular basis is much harder than it looks. Still, SoC has given this blog a nice PageRank boost, and it actually has some subscribers now.

Nothing exciting this week. I’ve been working on the test suite and laying the groundwork for the three-message negotiation specified in XEP-0116.

I neglected to write about my progress last week, so I’ll stick it in here. I implemented SRS (Shared Retained Secrets). The idea is that when two clients finish negotiating an encrypted session, they store a value which they can later use next time to verify that they’re talking to the same client. This is best used in conjunction with SAS, so that the remote client’s identity only needs to be verified once.

This week I fixed up my test suite enough to make it public. It doesn’t do very much right now, but I wouldn’t have been able to bootstrap my XEP-0217 implementation without it.

You can get it from my darcs repository, or you can use the copy I’m running on this server. Existing tests are (in roughly the order that I wrote and used them):

• xep201@testsuite.necronomicorp.com
• xep155@testsuite.necronomicorp.com
• xep200@testsuite.necronomicorp.com
• xep217@testsuite.necronomicorp.com

Sending one of those a chat message reading ‘help’ should give you a rough idea of how to interact with it.

These reports keep slipping further and further into the week—I need to fix that.

Major bugfixes this week, mostly improving error handling and patching up presence leaks. The one visible change is that I’ve completed SAS support, as can be seen in the screenshot. (yes, the dialog is… verbose. if you have any suggestions for cleaning it up, please share them.)

SAS (short authentication strings) serve the same purpose as a PGP or SSL fingerprint. They all allow you to confirm that you’re communicating directly with the person or computer that you think you are.

The first time you connect to an SSH server you get something very similar. You’re supposed to check with the server’s administrator and make sure the fingerprint you see matches the one ey’s got—few people do this. It’s hard to blame them, it’s a gigantic hexadecimal number.

The difference with a SAS is that it’s, well, short. The version specified in XEP-0116 is only 5 characters long. Hopefully this will encourage people to verify it, rather than just ignoring it.

I’m back on track this week. I’ve implemented some of the more negotiate-y parts of XEPs 0155 and 0217, specifically message logging.

Some people want their encrypted sessions logged, others (I would wager most) don’t. XEP-155 provides a way of expressing your preference and letting both parties negotiate an acceptable situation. It’s important that we both agree, because your logs can potentially incriminate me (and vice versa).

I’ve implemented it like this: an option in Gajim’s Advanced Configuration Editor (which is very similar to Firefox’s about:config) lets you select whether or not you want encrypted sessions logged. When negotiating a session, Gajim will automatically select your preference if the other party offers it as an option; otherwise it will prompt you with a message like the one in the screenshot. This means that when negotiating a session between two Gajims, either the responder (as opposed to the initiator) gets his way or the session is cancelled. The alternative is to prompt the responder if the initiator expresses a preference other than the responder’s; I think my way makes for a minimum of user hassle, but I’m not entirely sure. We’ll see how it works in practice.

With this done I’ve implemented most of XEP-0217. This week: there are still a few bugs to fix, and then I’d best make an interface for SAS, figure out how to securely store SRSs, beef up the quality of my PRNG and start getting the test suite ready for release.

My branch of Gajim has got a functioning XEP-0217 implementation. That is, one client can initiate an encrypted session with another.

I don’t recommend plotting to overthrow any governments just yet; I’m using a weak PRNG, it will still log encrypted messages, I haven’t implemented SAS so there’s no guarantee you’re talking to who you think you are, etc.

And of course I still need to clean up and release the test suite and then get to work on XEP-0116. Things are coming along, though.

I spent more time than I should have hacking away at Gajim to make it allow multiple chat windows per user, only to revert it all back when I realised just how big a task it was. I’ve gone the route of one session per contact; if someone starts a new session, the old one is quietly forgotten about. This is far from ideal, but should be sufficient for now. My philosophy on this project is to get something working as soon as possible, and fill out the details later.

Spent quite a while looking for Python libraries that might implement AES:

• PyAES: dead, replaced by CryptKit
• CryptKit: appears to be dead.
• alo-aes: appears to be dead. no documentation.
• pyopenssl: Gajim already uses this if it’s available, so it would be ideal; unfortunately it doesn’t seem to expose any of OpenSSL’s cipher functions.
• PyCrypto: I went with this because it was the first thing I found that works and has documentation.

I ran across M2Crypto after I’d gotten started; it’s another wrapper for OpenSSL. It might make sense to rewrite the current OpenSSL stuff and use this for everything.

As for interesting things, my branch of Gajim can send and receive encrypted messages - take a look at the screenshot I’ve included. It’s not actually very useful because keys and counters have to be hard-coded, but once I get XEP-0217 up and running (hopefully sometime in week three), that should no longer be an issue.

My lack of cryptographic experience is beginning to show. My mentor (Ian Paterson) has been very helpful in pointing me in the right direction. I’ve also discovered that the Handbook of Applied Cryptography is available online; it’s filling me in on the basics and looks like a good reference.

Despite my laptop’s treachery (and this CRT’s attempts to boil my eyes), things are going well.

I’ve written bots for testing XEPs 201 (Best Practices for Message Threads) and 155 (Stanza Session Negotiation). I’ve also coaxed Gajim into negotiating a session from start to finish (assuming it’s been initiated by some other party).

A lot of my time has been spent figuring out the best way to implement sessions (and fit them into Gajim’s existing structure). All of the XMPP client libraries I’ve seen focus on individual stanzas and the connection to the server. Once you need to keep track of the state of an interaction with another client, things become messy.

The solution is to add another layer between the server session and the stanza handlers. Rather than having a single client that handles everything, you have a generic client that dispatches messages to sessions that handle everything. Even if you’re not implementing anything that depends on stanza sessions, I think it’s a good idea; you can let the session worry about who to send stanzas to, and I think it more closely models the typical useage.

I’ve already rewritten my bots to use this scheme, and cut out a good 25% of their code in doing so. I’m in the process of trying to do something similar with Gajim. The complications of retrofitting existing clients with sessions are going to be a problem for ESessions (and myself, for that matter.)

(By the way, the bots are not up and running right now; they need better error handling before I unleash the public on them.)

I’m very excited to have been accepted for Google’s Summer of Code by the XMPP Standards Foundation. I’ll be writing an end-to-end encryption (ESession) test suite and using it to implement e2e in Gajim.

Continuing my habit of writing x-to-Atom gateways, here’s a little Jabber bot that will post messages you send it to an Atom Publishing Protocol Collection.

It requires atom-tools and the xmpp4r library.

require "atom/collection"
require "xmpp4r"

$coll_user = "bct"
$coll_pass = "atom-password"
$coll_url = "http://necronomicorp.com/testatom?app"

$bot_jid = "test@atompub.necronomicorp.com"
$bot_pass = "xmpp-password"

http = Atom::HTTP.new
http.user = $coll_user
http.pass = $coll_pass

coll = Atom::Collection.new $coll_url, http

cl = Jabber::Client.new($bot_jid)
cl.connect
cl.auth($bot_pass)

cl.add_message_callback do |msg|
  e = Atom::Entry.new  
  e.content = msg.body

  res = coll.post! e

  if res.code == "201"
    cl.send(Jabber::Message.new(msg.from, "success!"))
  else
    cl.send(Jabber::Message.new(msg.from, "#{res.code} #{res.message}"))
  end
end

Thread.stop

Try it out; send a message to test@atompub.necronomicorp.com. It should appear in my test collection.

(The real version is a server component which handles several collections on different JIDs.)

Peter Saint-André (of XMPP fame) posted his thoughts on REST today.

It seems downright silly to say that it’s a good thing to limit the use of other protocols to HTTP-like semantics, given that push semantics have launched not one but two killer apps – email and IM.

That would be downright silly. I don’t think anybody has said that REST is the ultimate architectural style that everything should emulate. REST handles a specific set of problems, one which doesn’t have much overlap with the set of problems encountered generic messaging.

The fuss that’s been kicked up about REST vs. RPC recently is about using HTTP as it was designed, not about devouring all alternatives. XMPP is good at many things that HTTP is not, and that’s a good thing. If it were interchangeable with HTTP, there wouldn’t be much point.

So I don’t know that the Jabberites will ever be good RESTafarians.

Good. HTTP is fine for REST, for now. XMPP’s got a nice niche for itself, and I don’t think that grooming it as a replacement for HTTP would do anyone any good.

I’ll thank you not to characterize it as a religion though; that’s how flame wars get started, and those are never productive.