About a year ago, in Bye, bye Athens... hello UK Federation, I questioned the rather grand claims being made about the then new UK Access Management Federation for Education and Research, notably that it would deliver "improved services to users", and wondered what the reality would be like.
I think we're still waiting to find out to be honest but there doesn't yet seem to be much evidence that anything has really improved over what we had before - certainly not in terms of usability for the end-user!
Last week I attended a meeting set up by the JISC-funded Service Provider Interface Study project, looking specifically at usability issues within the federation as things currently stand.
Firstly... hats off to both the project team and JISC for being so open about the issues. The meeting was a real eye-opener (for me at least), not only in that it demonstrated just how poor usability is across all the players that make up the federation, but also in the realisation that, actually, most service provider access control is done via IP address checking rather than by SAML-based authentication, in part because the usability issues are so great. For most users, SAML only comes into play when they are off-site (at least according to the service providers present at the meeting). Note: I appreciate that this was also the case under the old Athens system... I mention it here only because it seems to me that the continued use of IP address checking hasn't been widely acknowledged in the way the federation is generally presented, so it came as something of a surprise (to me at least).
Usability problems hit almost every aspect of the Federation as it is currently deployed, from the point that the end-user is initially asked to sign-on right thru to the ways in which service provider services are personalised (or not). Overall usability is made worse because the end-to-end experience is distributed across several players - the service provider, the identity provider, and (optionally) a 'where are you from?' service - each of which can, and do, make different decisions about naming and design. The result is a confusing experience for the end-user, combining poor usability of the individual components in the system coupled with a lack of consistency between the different parts, leading to a situation where it must be near impossible, for example, to write user-support documentation (i.e. help pages) covering everything in a comprehensive form. Even trivial issues, such as what 'sign-on' is called and where it is positioned on the page, are handled differently by different players.
It seems to me that privacy and security issues seem to have driven much of the thinking behind the Federation in its current form. At one point I asked the meeting whether anyone had actually asked real end-users whether they would like to have the option of sharing more information between the identity provider and the service provider in order to enjoy a more seamless and usable experience overall (even if it theoretically compromised their privacy in some way)? I'm not sure I got a clear answer... but it is hard not to draw the conclusion that the Federation has been designed by people with a primary interest in the technology rather than the user experience. A bit like the 'good old days' when we let the techies have full control over firewall policies, disregarding the fact that people actually had jobs that they needed to get done.
I'm sorry if all this seems very blunt but the current deployments are so un-friendly that something has got to be done - otherwise we might as well just bite the bullet and go back to having separate login accounts for every service we access (something that most people are perfectly accustomed to these days anyway!).
So... I want to focus on two, related, aspects of usability for the remainder of this post: naming the authentication process and discovering the identity provider.
Firstly... what do you call the process by which you gain access to a service provider in a SAML-based world? How are things 'branded' if you like? This is a non-trivial question to answer but a great example of how largely technical considerations (like the need for federations) have been allowed to get in the way of user-oriented usability issues. It's also something that the OpenID crowd have got cracked. That's not to say that there aren't other problems with OpenID - there are - but at least they have a single global brand (and associated logo) which makes it easy for any user, anywhere in the world to realise when they are being asked to sign-in using their OpenID.
What's the equivalent brand in the SAML world? There isn't one. Nobody pushes the use of a "SAML sign-on" (quite rightly in my opinion) since it would be meaningless to people. Shibboleth, as I've argued before, names a particular bit of software rather than an approach, and so is inappropriate. Some service providers in the UK still use 'Athens' (because it's what end-users are used to!) - again, clearly wrong in a SAML world. That leaves branding at the level of the federation... but who on earth wants to refer to their "UK Access Management Federation login" - what a horrible mouthful that is. And remember... most service providers offer their services globally, so if we brand things at the federation level then service providers have to start asking their users which federation they are part of - something that I suspect most of us neither know nor care!
That brings us on to my second usability issue. In a SAML-based world, service providers have to direct the end-user back to their institution in order that they can sign-in using their institutional username and password before being redirected back to the target service. This is typically done using a 'where are you from' service, either stand-alone on the network or embedded into the service provider website. Typically, this involves the end-user selecting from a pull-down list of identity providers (there are over 500 in the UK Federation currently), optionally preceded by a pull-down list of possible federations. This is horrible.
I'd like to propose a new rule of thumb for the design of user-interfaces in a SAML world... if we ever have to explicitly ask the end-user to choose from a list which federation they are part of, then we have a totally borked approach and we need to do something different. This seems obvious to me - yet it is exactly the direction we are heading in right now :-( .
I'd actually go much further and say that if we ever have to explicitly ask the end-user to tell us which institution they are a member of just so they can sign-in to something, then we have similarly broken the system - but I appreciate that is a more difficult part of the process to remove given the current technology. We can, however, make the selection of the institution rather easier than scrolling thru a list of 500 (or 1000, or 5000) names. How about looking at the way TheTrainLine let you select a station name? How about using the JQuery Auto-Complete function to narrow down the list of available names as the end-user begins to type? Here's a demo of just that. Much more intuitive than a pull-down list. (Thanks to my colleague, Mike Edwards, for the sample code to build this, based on the JISC "what do we do?" page.)
It'll be interesting to see what recommendations the Service Provider Interface Study project comes up with. Here are mine. Let's stop thinking in terms of asking the end-user what federation they belong to and think instead of questions they are likely to know the answers to. What is the name of the institution you belong to? What's the URL of your institutional website? What country are you in? Let's make the machines do the hard work of sorting out which federation is relevent. In short, let's start building user-interfaces, no... scrub that... let's start designing the system as a whole such that usability and the needs of the end-user are put first rather than being tacked on as an after-thought!
Finally... I'm surprised that publishers, and other service providers, aren't driving this much more forcibly than they appear to have done to date. There was a strong feeling in the meeting that things have got much worse (in usability terms) since the demise of Athens - yet the publishers present seemed rather defeatest about what they could do about it. Every time the usability of the system breaks, a service provider somewhere stands to lose a customer and while they are not typically paying for content individually it ultimately all adds up. Publishers should be pressing the UK Federation (and all other federations) for a system that works end-to-end, not just because of their own self-interests, but because of the interests of their primary user-base. I also think that they should be working much more closely together to bring greater consistency to the way that SAML-based sign-on is presented to the end-user.