« Yahoo search & the Semantic Web | Main | Technologies for open social networking »

March 16, 2008

Hack Day: and now for something completely different

P1050734 Friday was Hack Day at Eduserv, an internal event that allowed some of our techies to take time away from their normal day-to-day activities in order to think about and work on something new.  The day was one part of a programme of stuff to try and put innovation back at the heart of what Eduserv does.  I think it worked pretty well for a first attempt and I certainly hope we repeat it.  We had people working on things as diverse as integrating OpenID with an open source blogging system, Shibboleth with a commercial social networking tool, MyAthens with Windows Live Messenger, Google Maps to plot usage of SP resources, and a local positioning system for the Eduserv offices based on triangulating the wireless signal strengths from multiple wireless access points.

For my part, I spent some time investigating the possibility exposing an RSS feed of the list of registered services providers within the UK Access Management Federation (UKAMF).  The point of choosing RSS was not to support news and alerting - rather that RSS is a good machine-readable format for anything that looks like a list of URLs.

Plan A was to take the UKAMF Metadata and transform it into RSS using a Yahoo Pipe, Perl script or XSL transformation.  Unfortunately, I quickly realised both that the metadata doesn't contain any information about the human-oriented services associated with the SAML end-points (to be fair, that is not its function) and that the XML file is so large that processing it in anything becomes rather difficult - it is certainly too big to process using Yahoo Pipes.  I must admit, it hadn't occurred to me before now what an odd architectural decision it is to store all the UKAMF metadata in a single XML file at a single point on the network - I suspect this will lead to significant scalability problems as the Federation grows.  The Federation must have been designed by the same person who came up with the Windows registry :-(.

Plan B was less than ideal and involved tagging all the registered UKAMF services in del.icio.us, using the tag 'ukamfsp' as a unique key and a variety of more normal tags to indicate the subject matter of the services.  RSS is one of the main access mechanisms for del.icio.us content, so an RSS feed for the 'ukamfsp' resources is readily available.

Of course, this approach is nothing more than a proof of concept since I am not in a position to maintain the set of resources tagged in del.icio.us.  However, I'd encourage the UKAMF to maintain this RSS feed in some appropriate way, and using an external tool like del.icio.us brings with it some significant advantages.  Having got an RSS feed, writing a Perl, PHP or Ruby script to re-purpose it into XHTML is very easy to do.

This activity raises a couple of other questions...

Firstly, who is likely to be interested in such a list?  Certainly not end-users, who are interested in the set of resources that they need to get a job done but who have no interest in how they are accessed.

Secondly, where and how are services best described?  At the moment the JISC is funding Intute, the IE Service Registry and the UKAMF, all of which contain some aspects of descriptive metadata about available services.  The metadata in each is different, so the split across the three catalogs/registries may be appropriate (though I must admit that I'm  not totally convinced that it is).  However, what seems to be missing at the moment is a unique key to link the three bits of descriptive metadata together and an appropriate API in any of the services to allow client software to say, "tell me what you know about service X".


TrackBack URL for this entry:

Listed below are links to weblogs that reference Hack Day: and now for something completely different:


Looking at the delicious feed you set up, my first thought was: if all those sites had an autodiscoverable opensearch rss link, it would be easy to generate the an unranked, on the fly metasearch engine over all those sites?

In the short term, a delisearch pipe wil search over the domains listed in the feed:


(err, http://tinyurl.com/2h6w2k )

Andy - really pleased to see your comment on where and how are services best described...as I have been boring in people in JISC about this one for a while now as well :-) I can add the list of resources / services on the JISC Collections website and the UK federation metadata itself to your list...and I'm sure there are more. There has to be a better way of managing and distributing all of these service descriptions?

@Tony lol I meant to add that I was surprised how poor the HTML is on some of those service's home pages. A lack of any decent tag on some of them for a start. Amazing for services that are supposed to come from information professionals :-( On that basis, expecting them to support an opensearch autodiscovery link is perhaps being a bit hopeful!?

@Nicole yes, when I said that UKAMF describes the services I actually meant the UKAMF metadata - but you are right, there is also the human-readable Web page which as far as I can tell is currently maintained by hand :-(

FWIW, I parse the UKAMF metadata using Java and XMLBeans. Takes a couple of lines of code to load it into object land. All you need do is run XMLBeans over the SAML2 xsd.
Hack days sound brill - I'm off to hassle my boss :)

The comments to this entry are closed.



eFoundations is powered by TypePad