glennjones.net - Latest Posts

Ident Engine

Sun, 04 Oct 2009 12:23:36 GMT

Without much conscious thought, most of us have built identities across the web. Weve filled in profiles, uploaded photos, videos, reviews and bookmarks. The Ident Engine uses semantic web API’s to bring together these web footprints.

Ident Engine site
A List Apart - Ident Engine

The Ident Engine is part of a year long personal project into combining social graph data with other open data sources. With over 1.3 billion hCard profiles on the web and RDF growing in strength, the semantic web has reached a tipping point. I built the Ident Engine to show that constructing user experiences by blending semantic web APIs and other open data sources is not only practical, but offers exciting opportunities.

Related projects

Identify - Firefox add-on
Identify is a Firefox add-on that combines identities across various social network/media sites. Its uses the Ident Engine library .

Social Graph Explorer
This is the original server-side application on which the Ident Engine library is based.

Microformat API - OAuth
This experimental demo showes how OAuth can be used to add privacy to open data sources like microformats and feeds.

Presentations

I have given a number of talks about blending social graph data and other open data sources over the last year. The slide deck from Twiist.be in Leuven (below) contains the most update version of my talks.

Experiments in Data Portability 2 - Twiist.be – Leuven, Belgium - 15 May 2009
How to connect social media with open standards - Social Web Foocamp - Sebastopol, CA - 18 April 2009
How I built the demo's - Barcamp London 6 – London, UK - 28 March 2009
Microformats: A Quiet Revolution - SXSW – Austin, TX - 14 March 2009
Experiments in Data Portability - Skillswap – Brighton, UK - 25 November 2008

Experiments in Data Portability 2

View more presentations from Glenn Jones.

Articles

Identify: Google People With Two Keystrokes - Read Write Web article on the Identify Firefox addon by Marshall Kirkpatrick

Resources

If you are interested in this area I have listed a number of different resources you may find useful.

Experiments in Data Portability

Sun, 30 Nov 2008 12:58:12 GMT

I have post a screencast with video's of the demo's and synced audio from the podcast

On Tuesday, I gave a skillswap talk about some of the experimental work I have been engaged in. For the last few months I have had a growing interest in the current data portability design patterns used in social media sites. As we start to see the maturing of the earlier technologies such as OpenID and growth of some exciting new ones like OAuth and XRDS-Simple, focus is moving to the user experience. Through this presentation I try to look at current patterns of data portability that could be implemented now and look at some of the lessons that could be leant from the early adopters.

I showed some of what is currently possible using the Google Social Graph API coupled with a Microformats parser in a little application called the Social Network Explorer. The other major demo was my work on a Microformats API which makes use of OAuth to provide user controlled release of private data.

Finally I asked the question do we really own our data in the same way as we own other property. If the value of data decays over time should we be looking more at systems which make use of attention metrics.

Experiments in Data Portability

View SlideShare presentation or Upload your own. (tags: stack open)

Skillswap has always been a vibrant part of the Brighton web scene and it was great to finally get around to giving a talk having listened to many great skillwap talks in the past. Bruce also gave a great talk on OAuth and issues around the password anti-pattern.

James Box will be releasing a podcast of the audio soon.

URL's for topic in the presentation

Madgex Labs
http://lab.madgex.com/
Backnetwork
http://lab.backnetwork.com/
Microformats Parsers
http://microformats.org/wiki/parsers
Microformats Parser - Optimus - XSLT
http://microformatique.com/optimus/
Microformats Parser - Cognition - Perl
http://buzzword.org.uk/cognition/
Google Social Graph API
http://socialgraph.apis.google.com/
Social Graph Node Mapper
http://code.google.com/p/google-sgnodemapper/
URI-Template
http://tools.ietf.org/html/draft-gregorio-uritemplate-03
XRDS-simple
http://xrds-simple.net/core/1.
Portable Contacts API
http://portablecontacts.net/
APML
http://www.apml.org/
TasteBroker experimental API for APML
http://tastebroker.org/
APML JSON Tag Cloud
http://bmn.name/examples/apml/

Microformats to Portable Contacts API converters

Wed, 15 Oct 2008 08:14:58 GMT

I have been doing some research work into the new Portable Contacts API. It’s designed to enable users to securely port their friend lists or address books from one site to another.

Currently most social networking and address book sharing sites have their own proprietary contacts API’s. These API’s often provide some sort of distributed authorisation model for a developer to code against. If you integrate a few of these different interfaces into your site the levels of complexity become nightmarish.

From this tangle was born the password anti-pattern, this is where you ask the user for their username and password and then log in to another site as them and scrape the data you need. This is a bad idea on many counts, but looking at the historical alternatives you can understand why most developers took this route.

OAuth was designed to help remove the complexity of multiple distributed authorisation models. Whereas the Portable Contacts API tackles the second element of friend lists sharing, by providing common API and discovery.

Portable Contacts API is built on open specifications which can be used across sites. It uses a number of pre-existing technologies with its data structures based around OpenSocial and vCard, which should create a common access pattern and contact schema for everyone to use.

I have built a number of interfaces to help evaluate any data loss or added ambiguity that may occur when converting microformats into the Portable Contacts API data schema.

They do not yet provide the querying, sorting and pagination nor the endpoint URL elements of the specification. It’s also worth mentioning that Portable Contacts API is still in development and these interfaces are based on the Draft C specification.

getAttribute href bug

Sun, 19 Feb 2006 16:33:13 GMT

Whilst working on the Ajax Link Tracker and MapSurface I have come across an inconsistency in how the href attribute is retrieved using DOM Scripting.

The href attribute is different to other element attributes in that the value set can be relative to the context of the page URL. If you set a link with a relative href attribute

test page

The browser will look at the pages current URL and derive an absolute URL for the link.

http://www.glenn.jones.net/development/test1.html

This is the root of the problem, some browsers return the text of the attribute and others return the derived absolute URL. The results also differ by the method you use to retrieve the href attribute. There are three common ways to access an attribute:

    linkobj.href;
    linkobj[‘href’];
    linkobj.getAttribute(‘href’);

The linkobj.href and linkobj[‘href’]; methods of accessing the attribute consistently return the derived absolute URL.

Microsoft has tried to address this by ~~problem~~ adding a second parameter to the getAttribute method. The second parameter can be set to 0,1 or 2. If the parameter is set to 2 the method returns the attribute text. Any other setting will return the derived absolute URL.

    linkobj.getAttribute(‘href’);
    linkobj.getAttribute(‘href’,2);

		Derived Absolute URL	Attribute Text
IE	linkobj.href;	x
IE	linkobj.getAttribute(‘href’);	x
IE	linkobj.getAttribute(‘href’,2);		x
Gecko	linkobj.href;	x
Gecko	linkobj.getAttribute(‘href’);		x
Gecko	linkobj.getAttribute(‘href’,2);		x
Opera	linkobj.href;	x
Opera	linkobj.getAttribute(‘href’);	x
Opera	linkobj.getAttribute(‘href’,2);	x

Get attribute test page Test on IE6, Firefox 1.5 and Opera 8.51.

So what should be returned by the getAttribute method? The W3C DOM Level 2 Core specification which sets out the structure of the getAttribute method does not cover this issue. It is not that either approach is wrong or right. On this point the specification is open to interpretation.

~~As a coder I would like to be able to access both values. The DOM Core specification should be updated to address the problem.~~

After a really good exchange with Jim in the comments below, I stand corrected. The specification does say the getAttribute should return the attribute value, not the absolute URL. The Microsoft approach is wrong.

For the time being I am using the old school object property method linkobj.href to return derived absolute URLs. It provides the most consistent results across all browsers.

URLs of interest
W3C REC DOM Level 2 Core specification for getAttribute
Gecko documentation for getAttribute
Microsoft documentation for getAttribute

As usual just as I was finishing this post I found this bug report on the QuickMode site which discusses the same subject.
getAttribute HREF is always absolute.html

OAuth.net

Mon, 08 Sep 2008 10:14:11 GMT

OAuth is one of those technologies that when first understood makes your mind race with the possibilities. Authentication is a dry subject at the best of times, but OAuth shines not because the technology is cool, but because it has the capability to fundamentally change the way people use the web.

Today Madgex open-sourced OAuth.net, which as it sounds is an OAuth library for .net. This is a spin off from an internal research project we are working on.

OAuth.net provides full OAuth consumer and provider support to the core specification. The library facilitates secure API authentication in a simple and standard method for desktop and web applications. We are putting the full source code with a MIT licence on Google code.

http://lab.madgex.com/oauth-net/
http://code.google.com/p/oauth-dot-net/

The thing that really stuck in my mind when I started to investigate OAuth was the joy of the user experience. With sometimes as little as 2 clicks I can share data between sites, which would of taken me ages to re-enter. Somehow this experience has that nice feeling like using the iPhone interface for the first time. Maybe it’s just because I am so sick of re-entering things over and over again.

I have been fascinated with allowing users to share information for over 3 years now, whatever the name used Portable Social Networks, Social Graph or Data Portability.

There is a vast amount of publically available data about each of us embedded in the pages of sites we use. There are hundreds of millions of social network profile pages. The Microformats community has done a great job defining practical ways to extract semantic structures of data, but the issues of privacy, authentication and authorisation are beyond its scope.

OAuth allows users to share data between sites which they do not wish to make public. It does this without resorting to the heinous practice of asking a user to hand over their account details for other sites.

If you have a Google mail account take a look at this demo. It is a very simple demo, but it shows you some of the power of the concept.

http://lab.madgex.com/oauth-net/googlecontacts/

There are also demos for Fire Eagle and extracting protected Microformat resources.

OAuth.net is the first full library for .net. Our hope is that by sharing some of our work it will help move forward the adoption of OAuth. I will be talking about OAuth and OAuth.net at Barcamp London 5 on 27/28 September. Bruce Boughton one of the libraries developers will be talking about the project at Barcamp Brighton 3 on 6/7 September.

Enjoy

Microformats test-suite concept

Sat, 03 May 2008 11:14:29 GMT

I have been working on a concept for a new microformats test-suite. I need a comprehensive test-suite before I can move the development of UfXtract forward. Rather than just build something in isolation I thought it would be nice to find a way to share this work with the community.

I have written two POSH patterns “testsuite” and “testfixture”. They follow the principles of microformats design:

They are self-describing and created with HTML ,with no hidden metadata
Building a test should be easy even for those who are HTML authors
They are not linked to any one programming language and should be easy to share
They allow for the creation of an in-browser Testrunner

For example go to http://ufxtract.com/testsuite/hcard/

The earliest tests for hcard used vcards to describe the expected output. As the community has moved forward it has designed microformats which are independent of external specification. So this test-suite is designed around the concept of a standardised data structure. In this case, expressed in JSON, but they could be converted into XPaths to test XML or other languages.

I have started to build a small console app, which will spider the HTML and create NUnit/C# class files for my build tests. Although this is specific to my own parsers development, it should be easy to do the same for other programming languages and projects.

Parsing “testsuite” and “testfixture”.
I have already setup UfXtract to parse these patterns into JSON/XML. It would not take much for other microformats parser developers to construct profiles for these POSH patterns.

Here is an example of the output.
ttp://lab.backnetwork.com/ufXtract/?url=http%3A%2F%2Fufxtract.com%2Ftestsuite%2Fhcard%2Fhcard1.htm&format=test-fixture&output=json

The Testrunner

If you go to http://ufxtract.com/testsuite/hcard/hcard1.htm and press Alt X you can see a working demonstration of the testrunner.

I have observed that most parser developers are using comparative testing as their main tool to quickly understand how the complex rules and optimizations are applied. So I have built a JavaScript Testrunner which allows for simple comparative testing between parsers.

It uses a number of techniques to standardise both access to the parsers API’s and the JSON output. Please note that at this stage the JSON standardisation process can cause a test to be marked as failed when it could be judged to have passed. Most of the current differences in parser output are down to whether a value is stored as single property or an array of properties.

At the moment the Testrunner is only working with the testfixure , it would not take much to extend the Testrunner to run a whole test-suite.

I would love to add Operator and other parsers to the Testrunner.

Proof of concept

This is very early proof of concept stuff. What I would like to ask is a number of questions before moving it forward.

Are people interested in the idea of shared test-suites?
What do you think to the approach?
Can you see any big issues with the concepts?
If you already have tests/test-suites, would you be willing to add them to the project?
Would you be interested in contributing to a project like this?

Semantic Camp London

Tue, 19 Feb 2008 15:02:41 GMT

I spent this weekend at Semantic Camp learning all about RDF and speaking about parsing microformats. Although the focus of the event was very small, it attracted a nice group of people who were all passionate about open data portability in one way or another.
I think I have a much clearer view of how RDF fits into the concepts of the semantic web. Over the next few weeks I am going to take some time to explore SPARQL. I had a chance to talk to a number of interesting people

I used my talk to try and answer the question Drew McLellan posed a year ago in his presentation Can Your Website be Your API? I used some of my experience building ufXtract and parsing social networks information to see if it is possible in the real world.

Presentation: Can your website be your API and real life (pdf 1.25Mb)

I believe with a few small changes you can create successful ‘read only APIs’ using only microformats embedded in your html. The Q&A moved onto whether you could create ‘read/write APIs’ and some sort of HTTP verb discovery mechanism.

After my talk I spent some time chatting with Dan Brickley author of FOAF. We exchanged a few ideas about where microformats and RDF are going and talked about parsing issues. Dan has made me think hard about the lack of a formal framework for developing microformats parsers. This was reinforced by Gareth Rushgrove discussion on different microformats parsers. There are three things that seem to be missing:

A codified specification, something along the lines of XSD or even a propriety profile that all the current parsers could use.

A strong set of tests which we could use to check whether a parser works to a pre-defined standard. We should have both positive and negative tests to make things like required attribute validation work.

A standard output format so that developers can abstract libraries easily swapping from one to another. It would also allow us to do programmatic comparative testing.

This whole area is something that needs to be raised on the microformats-dev list.

Andrew Walkingshaw did a great talk on automatic indexing using natural-language processing. The whole area of NLP keeps re-surfacing in my work at Madgex and I am becoming more convinced that there is a place for this technology in the aggregation of structured and unstructured information sources. The guys from the BBC should have a special mention for all the great work they demonstrated.

Jon Linklater-Johnson created Semantopoly for the event, yet another great effort from the man who gave us the very cool CSS specificity card game. Just take a look at the flickr pictures, Semantopoly was a great game made better by the twitters that where created around it.

I would like to thank to Tom and Daniel for organising a fantastic event and to the sponsors who helped support it: BBC Backstage, Hedgehog Lab, OpenLink Software, Osmosoft and Talis.

Cake Friday vs HM Revenue & Customs

Sat, 09 Feb 2008 17:44:48 GMT

A couple of weeks ago Madgex had a visit from HM Revenue & Customs to review our PAYE tax payments. While in the office they informed us that providing cake on a Friday maybe a taxable benefit. We received this judgement by letter.

“Friday cake The provision of cakes on a Friday is a different matter altogether. As I understand the position, the company provides cakes for all staff generally on a Friday. What is crucial here is that the meal (cakes) is provided to staff generally, its cost is reasonable in scale and in this instance free for staff. As such the provision of the Friday cakes under the circumstance described do not create a chargeable benefit in accordance with Section 317 Income Tax (Earnings and Pensions) Act 2003(ITEPA 2003). I have used the definition of a meal from the Pocket Oxford Dictionary 1972 Reprint as being ‘taking of food’. If I am incorrect in that regard relief will be available under the departments guidance for dealing with trivial benefits. There is no monetary limit to determine what is a trivial benefit. In essence a chargeable benefit will arise on the provision of cakes however, it is deemed trivial because it is perishable and/or consumable.

I am of the opinion that the cakes provided on Fridays to all staff qualifies for relief under Section 317 ITEPA 2003 and also satisfy the trivial benefit criteria. As such the department will not seek to bring that benefit in to charge. Therefore the company can continue to provide cakes to all staff in the same way as the past without the need to declare the benefit or pay any Class 1A National Insurance.”

We are now considering renaming “Cake Friday” to “Section 317 ITEPA 2003 Friday” in thanks of the enlightened decision of Revenue & Customs. Although we were wondering how they could calculate the national insurance tax due for a slice of cake.

Making your social networking data portable - Marking up profiles and friends with mircoformats

Mon, 03 Dec 2007 22:02:29 GMT

As the ideas of social media extends into the design of many modern websites, the addition of user profiles and friend lists are becoming more common. So what is the best practice for making your social network data portable with mircoformats.

Marking-up profiles and friend lists with mircoformats to any site is a relatively simple HTML coding process and involves the use of hCard and XFN. If you are not yet familiar with mircoformats I would recommend reading microformats.org/wiki/ for a quick overview.

Profiles

Most social network sites follow a standard architecture which has a profile page for each user, often with an easy to remember URL. The information about a user on this page should be marked-up with hCard.

Where you have additional pages containing other user information such as friends list you should link to them using the XFN mircoformats rel=”me” attribute. The use rel=”me” will tell any parser that the linked page is also about this user

rel="me" href="../glennjones/friends/">Your friends

Often friends list can be so long that you need to use pagination and break them into several pages. You should also mark-up the pagination links with rel=”me next” and rel=”me prev”.

More friends: rel="me next" href="../glennjones/friends/2">Page 2

Using rel=”me” to link to profiles on sites allows for a parser to create a consolidated identity from many different sources.

Defining the representative hCard

Often you will find that a profile page will end up with more than one hCard. There could be a compact friends list that uses the hCard-XFN pattern or other content which carries hCard mark-up such hAtom. The representative hCard concept, defines how to mark-up an hCard so that a parser can identify it as representing the profile owner. There are two patterns:

The first is an hCard where the uid, url attribute and the page source url all have the same value.


url uid" href="http://www.glennjones.net/">
Glenn 
Jones

The second is where rel="me" is on the same link as class="url"


url" rel="me" href="http://www.glennjones.net/">
Glenn 
Jones

Personally, I would use the rel=”me” pattern. Because rel=”me” is also the method for telling the parser that this element is a link to another profile.

Friends lists

Traditionally, XFN has been used to define relationships with friends. XFN is a simple mircoformat that allows you to represent these human relationships using rel attribute on hyperlinks.

rel="friend met" href="../adactio" >Jeremy Keith

Although XFN can describe relationships well, it lacks the richness of hCard which is more suited to describing information such as names. The mircoformats community have started to promote the use of the joint hCard-XFN design pattern to take advantage of this. Below are the three most commonly used patterns.

1. A text list of friends full names


fn n url" rel="friend met" href="../adactio">
Jeremy Keith

2. A group of icons with no visible text
When using icons without text, the alt attribute can be used to define the fn (formatted name) of the friend.



fn" src="../icons/adactio.jpg" alt="Jeremy Keith"/>

3. A group of icons with a username marked-up as a nickname
If you wish to use usernames rather than a full name you can change the image attributes to define a nickname.



fn nickname" src="../icons/adactio.jpg" alt="adactio"/>

I would always try and display the full name as visible text. One of the fundamental concepts of mircoformats is not to hide your data. That said, all three patterns will work.

Mapping relationship values

You will often have to create a mapping of values between your sites and XFN. If your system does not differentiate the strength of a relationship, use the default rel=”acquaintance”.

Trying out your mark-up

I have built a portable social network parser which can be used to test your mark-up. There is a public API on the backnetwork labs site which can produce either XML or JSON. There is also a demonstration of how a registration page could consume a site marked-up with mircoformats.

Note

This is an area of ongoing development, so some of this information may change as the portable social network concepts mature. Its always best to check the mircoformats wiki.

ufXtract’s portable social network parser

Mon, 03 Dec 2007 22:01:15 GMT

After 14 months of talking about portable social networks at various events from SXSWi to a small geek dinner, I have finally found the time to build a working example.

ufXtract’s portable social network parser is a combination of the ufXtract microformats parser and a spider which follows rel=”me” links. It has been designed to extract profiles and friends lists from social networks and other sites which have microformats support. The parser returns two main collections of data, all the rel=”me” links and any hCard-XFN patterns.

The parser API
http://lab.backnetwork.com/ufXtract-psn/

A demo using JavaScript and JSON
http://lab.backnetwork.com/ufXtract-psn/demo01.htm

The Parser

You can restrict the parser to a single domain or spider across the whole web. Currently, there are limits to the number of pages which will be parsed. Each collection item is given an additional source-url attribute to identify its origin

There is support for both XML and JSON output, for both client and server-side development.

The parser also uses a version of the representative hCard concept, which tries to identify the hCard representing the profile owner. The implementation is a little more complex than described on the microformats wiki as it extends over multiple pages and domains. This means you may find multiple representative hCards from one call to the API, but there should only ever be one per a URL.

The Demo

I believe there are a number of different ways that this functionality could be designed into web sites. So I have provided a simple interface design to demonstrate one possibility. It’s a bit of a homage to the getsatisfaction.com registration page with a few extra twists. I would like to thank my co-worker James Wragg who created the JavaScript for the demo.

Of the sites listed on the demo last.fm and ma.gnolia.com return the best results. The other sites have differing levels of portable social network support. It also works well against blogs such as adactio.com or tantek.com that are marked-up with rel=”me” . It’s worth trying out the two depth search levels.

Pages not parsing

You may find on some sites like twitter.com only certain pages are parsed. These sites often have good microformats support, but parts of their functionally are locked behind logon's. The parser does not support authenticated sessions as this would mean asking the user to pass me their log-in details which is a really bad idea. If I can lay my hands on a good Open-ID and/or OAuth C# libraries, I will try and implement some different types of authentication.

Research

This is all research work still under development, I placed it on the web for others to experiment with and to help foster discussion. I hope you enjoy playing with this.