ralphm's blog

Sunday, 28 November 2004


Look at me...

Just a little note on avatars and Jabber. In x-virge's Jabber Vision, there's a piece on the need for Idavoll to be finished and deployed for finally implementing avatars in the Jabber universe, the right way. Though publish/subscribe is an essential element of the Avatar JEP, there's another dependency: discovery.

In particular, a client has to discover a contacts pubsub node by sending a Disco request to the contact's bare JID. The idea is that the contact's Jabber server responds to this request, and for it to do that, the contact first has to publish this information to its Jabber server. This process is also known as disco-publish. However, as far as I know, jabberd 1.4.x doesn't support disco-publish, and jabberd 2's support is flakey, last time I checked. That covers a big part of the installed base of Jabber servers. How is the support in other servers?

Idavoll 2 now usable

Just what the doctor ordered...

Idavoll 2 is going strong. I can say it is now in a state that should be usuable for developing applications with. A while back, I asked pgmillard what features of JEP-0060 should be implemented, for Idavoll to be deployed on the jabber.org public Jabber server. I believe all of his requirements have now been met. Here the list of supported features:

  • Creating nodes (including so-called instant nodes)

  • Subscribing and unsubscribing

  • Publishing items + subscribers getting notifications

  • Requesting a previously published item

  • Retracting a previously published item

  • Purging all items for a node

  • Deleting a node

  • Configuring a node

  • Requesting current affiliations

  • Disco Info and Disco Items support

Some notes on the above:

  • Subscription requests are always automatically accepted; owner approval using Data Forms is not yet implemented.

  • The only two node configuration options (for now) are whether the node is persistent, and whether notifications should be sent with or without payload.

  • Disco Items is only implemented on the service itself, for requesting all nodes present. There is no hierarchy in the nodes in the current implementation. Because this will generate a lot of data when having more than a few nodes, this functionality can be disabled by the administrator of the service.

Despite the notes, the service should be more than sufficient to finally start implementing all sorts of cool publish/subscribe based features in Jabber clients. More on that in a bit.

What might hold people back in deploying Idavoll right now, is that there is no simple drop-in package, yet. Part of the problem here is that the Jabber support in Twisted needs some patches here and there. One of the patches is stringprep support for JIDs, a must-have, as was pointed out to me by [GNU], earlier today. So I've been busy implementing stringprep support today. And it works!

Twisted development is in a state of flux, as their upcoming 2.0 release will probably be split up in several packages, and the Jabber modules have been moved around. Also, offline offlinedizzyd, who wrote the Jabber support for Twisted is busy with non-Jabber stuff, and I have to somehow get my patches in Twisted's codebase. I'll try and coordinate this with dizzyd in the coming week or so. In the meanwhile, I'll probably have to distribute my patches to Twisted along in the Idavoll release, but I am not sure how to go about that, yet. For now, just checkout the code from the CVS repository, and give it a whirl. I will be more than happy to do some hand-holding.

Today, x-virge wrote an interesting piece on his blog about his Jabber Vision. Intented as some guidance in fixing the biggest pain points currently perstering Jabber, he goes and explains what should be done to get Jabber back up to speed.

One of the things he mentions is finally having a usuable pubsub server component, combined with not one, but two encouragements to help out with finishing Idavoll. Thanks x-virge! I'm not sure if he agrees with my statement above that people could start using Idavoll now for developing their cool stuff, but I can definitely use some help in testing the code, and making it even more useful. If you are interested in helping me out, drop me line!

Monday, 1 November 2004

pubsub.ik.nu uses Idavoll 2

Eating one's own dogfood...

As of today, the pubsub service at ik.nu is powered by Idavoll 2. This means that I think the Twisted rewrite of Idavoll is now good enough to support Mimír and the pubsub features of this site. A nice milestone.

The current implementation has two different backends, one that stores all data in memory, and one that has persistent storage using PostgreSQL. I use the latter for pubsub.ik.nu. This backend is very limited at the moment. It only supports publishing items, storing the items (if the node is so configured) and notifying subscribers. Creating nodes, setting up subscriptions and configuration of nodes is done by updating the database. But that should change soon enough.

I am now at a point that the basic architecture of Idavoll is becoming stable enough to quickly implement new features. I have struggled a bit with finding out how to best use interfaces and adapters in Twisted, and spiv, on the #twisted IRC channel helped me go in the right direction.

The easy part was this: I splitted up the (optional) features of JEP-0060 in nice chunks that are represented in several backend service interfaces. That way, people can code backends with different storage mechanisms and/or business rules, while only having to implement the parts that are useful for the application at hand. So, there is one interface for publishing items (IPublishService), one for subscribing to, and unsubscribing from, a node (ISubscriptionService), and so on.

Next, I wanted to split up the Jabber protocol implementation along similar lines. These would be implemented as adapters from such a backend service interface to the twisted.protocol.jabber.component.IService interface for implementing subservices of a Jabber server component. The problem with this is that you would want to have one backend class implement the choosen interfaces and then instantiate a component.IService with an instance of that backend class. And you can't do that, because it is ambigious!

Taking the two interfaces described above. Suppose I implement a backend (BackendService) that implements IPublishService and ISubscriptionService, and I have adapters to component.IService for both of them. What does the following do?

import my_backend
import pubsub
from twisted.protocols.jabber import component

backend = my_backend.BackendService()
service = component.IService(backend)

As I said, this is ambigious. This could yield either adapter for the implemented backend interfaces. So, what I did was still implement the code for the backend in one class that is MultiService, and have skeleton child services that state the actual interface that is implemented and pass on the method calls to their parent. That way, you can simply call for the adapter from these child service to component.IService, resulting in a component.Service for each backend service.

This probably isn't going to win any beauty contests, but it was the result of my discussion with spiv. Other approaches were even less desirable. I could define all adapters to be incremental and have only one component.Service per specific backend. Another approach would be to have some code that figures out which adapters to fire up from the __implements__ attribute of the backend. I didn't want to go there.

Have a look at the code in the Idavoll CVS repository or the generated API documentation. If you have any questions, comments or suggestions, let me know!