glennjones.net - Latest Posts

Microformats test-suite concept

Sat, 03 May 2008 11:14:29 GMT

I have been working on a concept for a new microformats test-suite. I need a comprehensive test-suite before I can move the development of UfXtract forward. Rather than just build something in isolation I thought it would be nice to find a way to share this work with the community.

I have written two POSH patterns “testsuite” and “testfixture”. They follow the principles of microformats design:

They are self-describing and created with HTML ,with no hidden metadata
Building a test should be easy even for those who are HTML authors
They are not linked to any one programming language and should be easy to share
They allow for the creation of an in-browser Testrunner

For example go to http://ufxtract.com/testsuite/hcard/

The earliest tests for hcard used vcards to describe the expected output. As the community has moved forward it has designed microformats which are independent of external specification. So this test-suite is designed around the concept of a standardised data structure. In this case, expressed in JSON, but they could be converted into XPaths to test XML or other languages.

I have started to build a small console app, which will spider the HTML and create NUnit/C# class files for my build tests. Although this is specific to my own parsers development, it should be easy to do the same for other programming languages and projects.

Parsing “testsuite” and “testfixture”.
I have already setup UfXtract to parse these patterns into JSON/XML. It would not take much for other microformats parser developers to construct profiles for these POSH patterns.

Here is an example of the output.
ttp://lab.backnetwork.com/ufXtract/?url=http%3A%2F%2Fufxtract.com%2Ftestsuite%2Fhcard%2Fhcard1.htm&format=test-fixture&output=json

The Testrunner

If you go to http://ufxtract.com/testsuite/hcard/hcard1.htm and press Alt X you can see a working demonstration of the testrunner.

I have observed that most parser developers are using comparative testing as their main tool to quickly understand how the complex rules and optimizations are applied. So I have built a JavaScript Testrunner which allows for simple comparative testing between parsers.

It uses a number of techniques to standardise both access to the parsers API’s and the JSON output. Please note that at this stage the JSON standardisation process can cause a test to be marked as failed when it could be judged to have passed. Most of the current differences in parser output are down to whether a value is stored as single property or an array of properties.

At the moment the Testrunner is only working with the testfixure , it would not take much to extend the Testrunner to run a whole test-suite.

I would love to add Operator and other parsers to the Testrunner.

Proof of concept

This is very early proof of concept stuff. What I would like to ask is a number of questions before moving it forward.

Are people interested in the idea of shared test-suites?
What do you think to the approach?
Can you see any big issues with the concepts?
If you already have tests/test-suites, would you be willing to add them to the project?
Would you be interested in contributing to a project like this?

Semantic Camp London

Tue, 19 Feb 2008 15:02:41 GMT

I spent this weekend at Semantic Camp learning all about RDF and speaking about parsing microformats. Although the focus of the event was very small, it attracted a nice group of people who were all passionate about open data portability in one way or another.
I think I have a much clearer view of how RDF fits into the concepts of the semantic web. Over the next few weeks I am going to take some time to explore SPARQL. I had a chance to talk to a number of interesting people

I used my talk to try and answer the question Drew McLellan posed a year ago in his presentation Can Your Website be Your API? I used some of my experience building ufXtract and parsing social networks information to see if it is possible in the real world.

Presentation: Can your website be your API and real life (pdf 1.25Mb)

I believe with a few small changes you can create successful ‘read only APIs’ using only microformats embedded in your html. The Q&A moved onto whether you could create ‘read/write APIs’ and some sort of HTTP verb discovery mechanism.

After my talk I spent some time chatting with Dan Brickley author of FOAF. We exchanged a few ideas about where microformats and RDF are going and talked about parsing issues. Dan has made me think hard about the lack of a formal framework for developing microformats parsers. This was reinforced by Gareth Rushgrove discussion on different microformats parsers. There are three things that seem to be missing:

A codified specification, something along the lines of XSD or even a propriety profile that all the current parsers could use.

A strong set of tests which we could use to check whether a parser works to a pre-defined standard. We should have both positive and negative tests to make things like required attribute validation work.

A standard output format so that developers can abstract libraries easily swapping from one to another. It would also allow us to do programmatic comparative testing.

This whole area is something that needs to be raised on the microformats-dev list.

Andrew Walkingshaw did a great talk on automatic indexing using natural-language processing. The whole area of NLP keeps re-surfacing in my work at Madgex and I am becoming more convinced that there is a place for this technology in the aggregation of structured and unstructured information sources. The guys from the BBC should have a special mention for all the great work they demonstrated.

Jon Linklater-Johnson created Semantopoly for the event, yet another great effort from the man who gave us the very cool CSS specificity card game. Just take a look at the flickr pictures, Semantopoly was a great game made better by the twitters that where created around it.

I would like to thank to Tom and Daniel for organising a fantastic event and to the sponsors who helped support it: BBC Backstage, Hedgehog Lab, OpenLink Software, Osmosoft and Talis.

Cake Friday vs HM Revenue & Customs

Sat, 09 Feb 2008 17:44:48 GMT

A couple of weeks ago Madgex had a visit from HM Revenue & Customs to review our PAYE tax payments. While in the office they informed us that providing cake on a Friday maybe a taxable benefit. We received this judgement by letter.

“Friday cake The provision of cakes on a Friday is a different matter altogether. As I understand the position, the company provides cakes for all staff generally on a Friday. What is crucial here is that the meal (cakes) is provided to staff generally, its cost is reasonable in scale and in this instance free for staff. As such the provision of the Friday cakes under the circumstance described do not create a chargeable benefit in accordance with Section 317 Income Tax (Earnings and Pensions) Act 2003(ITEPA 2003). I have used the definition of a meal from the Pocket Oxford Dictionary 1972 Reprint as being ‘taking of food’. If I am incorrect in that regard relief will be available under the departments guidance for dealing with trivial benefits. There is no monetary limit to determine what is a trivial benefit. In essence a chargeable benefit will arise on the provision of cakes however, it is deemed trivial because it is perishable and/or consumable.

I am of the opinion that the cakes provided on Fridays to all staff qualifies for relief under Section 317 ITEPA 2003 and also satisfy the trivial benefit criteria. As such the department will not seek to bring that benefit in to charge. Therefore the company can continue to provide cakes to all staff in the same way as the past without the need to declare the benefit or pay any Class 1A National Insurance.”

We are now considering renaming “Cake Friday” to “Section 317 ITEPA 2003 Friday” in thanks of the enlightened decision of Revenue & Customs. Although we were wondering how they could calculate the national insurance tax due for a slice of cake.

Making your social networking data portable - Marking up profiles and friends with mircoformats

Mon, 03 Dec 2007 22:02:29 GMT

As the ideas of social media extends into the design of many modern websites, the addition of user profiles and friend lists are becoming more common. So what is the best practice for making your social network data portable with mircoformats.

Marking-up profiles and friend lists with mircoformats to any site is a relatively simple HTML coding process and involves the use of hCard and XFN. If you are not yet familiar with mircoformats I would recommend reading microformats.org/wiki/ for a quick overview.

Profiles

Most social network sites follow a standard architecture which has a profile page for each user, often with an easy to remember URL. The information about a user on this page should be marked-up with hCard.

Where you have additional pages containing other user information such as friends list you should link to them using the XFN mircoformats rel=”me” attribute. The use rel=”me” will tell any parser that the linked page is also about this user

rel="me" href="../glennjones/friends/">Your friends

Often friends list can be so long that you need to use pagination and break them into several pages. You should also mark-up the pagination links with rel=”me next” and rel=”me prev”.

More friends: rel="me next" href="../glennjones/friends/2">Page 2

Using rel=”me” to link to profiles on sites allows for a parser to create a consolidated identity from many different sources.

Defining the representative hCard

Often you will find that a profile page will end up with more than one hCard. There could be a compact friends list that uses the hCard-XFN pattern or other content which carries hCard mark-up such hAtom. The representative hCard concept, defines how to mark-up an hCard so that a parser can identify it as representing the profile owner. There are two patterns:

The first is an hCard where the uid, url attribute and the page source url all have the same value.


url uid" href="http://www.glennjones.net/">
Glenn 
Jones

The second is where rel="me" is on the same link as class="url"


url" rel="me" href="http://www.glennjones.net/">
Glenn 
Jones

Personally, I would use the rel=”me” pattern. Because rel=”me” is also the method for telling the parser that this element is a link to another profile.

Friends lists

Traditionally, XFN has been used to define relationships with friends. XFN is a simple mircoformat that allows you to represent these human relationships using rel attribute on hyperlinks.

rel="friend met" href="../adactio" >Jeremy Keith

Although XFN can describe relationships well, it lacks the richness of hCard which is more suited to describing information such as names. The mircoformats community have started to promote the use of the joint hCard-XFN design pattern to take advantage of this. Below are the three most commonly used patterns.

1. A text list of friends full names


fn n url" rel="friend met" href="../adactio">
Jeremy Keith

2. A group of icons with no visible text
When using icons without text, the alt attribute can be used to define the fn (formatted name) of the friend.



fn" src="../icons/adactio.jpg" alt="Jeremy Keith"/>

3. A group of icons with a username marked-up as a nickname
If you wish to use usernames rather than a full name you can change the image attributes to define a nickname.



fn nickname" src="../icons/adactio.jpg" alt="adactio"/>

I would always try and display the full name as visible text. One of the fundamental concepts of mircoformats is not to hide your data. That said, all three patterns will work.

Mapping relationship values

You will often have to create a mapping of values between your sites and XFN. If your system does not differentiate the strength of a relationship, use the default rel=”acquaintance”.

Trying out your mark-up

I have built a portable social network parser which can be used to test your mark-up. There is a public API on the backnetwork labs site which can produce either XML or JSON. There is also a demonstration of how a registration page could consume a site marked-up with mircoformats.

Note

This is an area of ongoing development, so some of this information may change as the portable social network concepts mature. Its always best to check the mircoformats wiki.

ufXtract’s portable social network parser

Mon, 03 Dec 2007 22:01:15 GMT

After 14 months of talking about portable social networks at various events from SXSWi to a small geek dinner, I have finally found the time to build a working example.

ufXtract’s portable social network parser is a combination of the ufXtract microformats parser and a spider which follows rel=”me” links. It has been designed to extract profiles and friends lists from social networks and other sites which have microformats support. The parser returns two main collections of data, all the rel=”me” links and any hCard-XFN patterns.

The parser API
http://lab.backnetwork.com/ufXtract-psn/

A demo using JavaScript and JSON
http://lab.backnetwork.com/ufXtract-psn/demo01.htm

The Parser

You can restrict the parser to a single domain or spider across the whole web. Currently, there are limits to the number of pages which will be parsed. Each collection item is given an additional source-url attribute to identify its origin

There is support for both XML and JSON output, for both client and server-side development.

The parser also uses a version of the representative hCard concept, which tries to identify the hCard representing the profile owner. The implementation is a little more complex than described on the microformats wiki as it extends over multiple pages and domains. This means you may find multiple representative hCards from one call to the API, but there should only ever be one per a URL.

The Demo

I believe there are a number of different ways that this functionality could be designed into web sites. So I have provided a simple interface design to demonstrate one possibility. It’s a bit of a homage to the getsatisfaction.com registration page with a few extra twists. I would like to thank my co-worker James Wragg who created the JavaScript for the demo.

Of the sites listed on the demo last.fm and ma.gnolia.com return the best results. The other sites have differing levels of portable social network support. It also works well against blogs such as adactio.com or tantek.com that are marked-up with rel=”me” . It’s worth trying out the two depth search levels.

Pages not parsing

You may find on some sites like twitter.com only certain pages are parsed. These sites often have good microformats support, but parts of their functionally are locked behind logon's. The parser does not support authenticated sessions as this would mean asking the user to pass me their log-in details which is a really bad idea. If I can lay my hands on a good Open-ID and/or OAuth C# libraries, I will try and implement some different types of authentication.

Research

This is all research work still under development, I placed it on the web for others to experiment with and to help foster discussion. I hope you enjoy playing with this.

ufXtract microformats parser

Sun, 25 Nov 2007 17:50:38 GMT

ufXtract is a new microformats parser I have built to help explore the real world issues of creating portable social networks. Although I have previously designed a number spiders that can find the most common hCard and XFN structures, this is my first full blown parser. It has been built from the ground up to take configuration objects which allow the parsing of different microformats or POSH patterns. It was important that I could parse more general patterns such as the joint hCard-XFN being promoted by the microformats community for use with friend’s lists.

http://lab.backnetwork.com/ufXtract/

After some further testing I am going to start to produce a number of portable social network demo’s and posts. This should also provide others with experimental API’s. By sharing this early work I hope in some way to add to the important technical and architectural discussions that are taking place.

I have already added hCard-XFN, rel=”me”, rel=”next” and hAtom to the parser. These are the four cornerstone microformats/patterns required to gather profile and content from other social networks. Although for technical/speed reasons ufXtract is currently only parsing the hEntry sub-element of hAtom.

The component also contains extendable output options, so far, I have built a simple text format for debugging, JSON and XML for building services. For the more technically minded ufXtract is a .net component written in c#. It uses a combination of DOM structures and xPaths. It can typically parse a page in 50-200ms.

At the moment, I am building a test suite to fine tune the components’ compliancy. It still has some small issues with most of the compound microformats, which I am trying to address. If you have any comments or want to point out issues please add them below, I would like as much feedback as possible.

Backnetwork Lab – XFN pagination

Fri, 05 Oct 2007 09:10:00 GMT

Few weeks ago I watched Tantek Çelik give a demonstration of the Plaxo open social graph which is basically a rel=”me” spider. It reminded me how powerful practical demonstrations are at communicating key concepts.

So I have built lab.backnetwork with the idea of constructing a number of service based interfaces which are practical demonstrations of key concepts that are currently under discussion.

I am really interested in focusing on the portable social network area, so I have start with a very simple example of XFN pagination using rel="next". I am hoping to fellow this with a second interface which can parse the XFN/hcard pattern.

XFN pagination interface
Given any Url it will parse XFN relationship links within a page and then fellow any additional links that have the rel=”next” attribute. I have limited system to only spider 20 pages at a time. I have provided two set of test pages with different pagination patterns.

http://lab.backnetwork.com/xfnpagination/

Simple pagination design pattern where the user is viewing pages 2 of a set.

Pages: 1 2 3

The interface will work with any site, but at the moment there are very few if any site which use rel=”next”.

That was Barcamp Brighton

Wed, 12 Sep 2007 20:33:56 GMT

Madgex hosted the first Barcamp Brighton over the weekend, I think it was a really special event. Although I spent most of my time trying to make things run as smoothly as possible, I still sat in on a number of talks and some how managed to join Tantek and Jeremy to discuss portable social networks.

Unlike commercial conferences, everyone who attends Barcamp has to give a talk. This gives the event a uniquely friendly atmosphere. The sessions are passionate and often some of the most informative I have been to. I honestly think I have learned more at the Barcamps than most of the larger conferences I attend.

As Jeremy put it "food was superb and the presentations were fantastic."

The Madgex offices made a great venue, the old building with lots of odd spaces seemed to work well. We had to clear out four offices and rewire our network for the event, but the effort was so worth while.

So many people helped to make the event a success, Andy, Matt and Paul who organised the whole thing and a large number of Madgex staff gave their free time to help. Then Dave Phelan from Pier to Pier who just turned up on the day, created a wi-fi network for us. Of course it’s the participation that makes a great Barcamp.

The food was fantastic, the sponsors did us proud, from a beautiful Mediterranean buffet to sushi, we all ate like kings.

I am currently trying to capture all the sessions onto the electronic schedule grid on the Barcamp Backnetwork sites. If you gave a talk it would be great to write a little outline and/or upload your slides.

So thanks to everyone who came and hopefully we can do it all again soon.

Food was provide by:
Saturday Breakfast: Nixon McInnes
Saturday Lunch: Future of Web Apps
Saturday Dinner: BT Osmosoft
Sunday Breakfast: Microsoft
Sunday Lunch: Realmac Software
Snacks, drinks & water: Inuda Network

Barcamp Brighton

Fri, 31 Aug 2007 13:12:34 GMT

I have been busy over the last few months helping to organise Brighton’s first ever Barcamp on 8-9 Sept. I am really looking forward to what I am sure is going to be a great event.

For those who have not been to a Barcamp before it’s, an un-conference, usually held at the weekend. It’s referred to as an un-conference, because everyone who goes along participates equally, that means being a speaker as well as an attendee. It’s a community driven event to encourage people to share and learn from each other.

It’s great that Madgex has been able to provide the space for the event. In conjunction with our sponsorship of the d.construct conference it shows how strongly we are committed to supporting the local web community.

Interest in the event has been amazing all the places went in 20 minutes. I only wish we could have provided more places. As it is we have reorganised the whole office layout to create the right type of spaces. The moving men are coming on Thursday to help us out.

Over the last few days I have been asked a lot of questions so I thought I would try and answer a few. We will also be sending more detailed emails to everyone who has a place in the next couple of days.

Saturday
We are starting the day a little later to allow for those people who have over done things at the D.construct party the night before. So Registration/Breakfast is between 10:00 and 11:00 and the last session will finish at 20:15.

Sunday
The sessions will start at 10:00 and the event will close at 17:30.

Staying over
You will be able to stay over at our offices on Saturday night; this is a big part of the events culture. Just bring a sleeping bag and camp mat as you'll be sleeping on the floor.

Rooms
We will have five rooms for presentations they will vary in size taking between 20-40 people. Each room will have a video projector, please remember to bring any special laptop to VGA converter/cables for the projectors you may need.

Networking
Having been to lots of events recently where the wi-fi just did not work, we are providing switches in every room where people can connect to the internet using a cable.

SXSW 2007

Thu, 12 Apr 2007 09:43:25 GMT

It always takes me so long to absorb SXSW. It seems so large, 8 tracks at once over a very full 4 days. As with last year I did gain a lot from the event, although I do think that some of the panels did not work so well. Personally I much prefer a more presentational style of delivery. Panels need a certain type of chemistry and it is often hard to find even among people who are excellent individual speakers. That said I do think there are many other benefits to the panel format and its part of what makes SXSW feel like an inclusive event.

This year I helped out on the microformats panel "The Growth and Evolution of Microformats" with Tantek, Frances and Mike. I think it went well. Tantek did the history of microformat's through t-shirts, it had to be seen to be understood. Mike did a very impressive talk on Operator which is developing into a great tool. Jeremy Keith joined us about half way through, as it is now a tradition for him to be a guest on the SXSW microformats panel. Frances covered the all important community component of microformat development.

So here are some of my favourite sessions. Marked up with hReview and hEvent of course.

0.3 event

Web Typography Sucks

SXSWi, Tuesday, 13th March 10:00 am - 11:00 am

Richard and Mark did a great presentation on the current state of typography on the web and what we can do as web designers to improve the layout of web pages. They tried to demonstrate how traditional typographic principles can still be implemented within the current restrictions of HTML. It was one of those eye opening talks that really makes you reassess the design process. I practically like Richard's thoughts on creating rhythem with the correct use of spacing and building grids in relation to your base text size. All the resources from the talk can be found at Web Typography Sucks. Can't wait for the book.

Richard Rutter Production Director, Clearleft Ltd

Mark Boulton Owner, Mark Boulton Design

Review by Glenn Jones on 7 April 2007

0.3 event

Grids Are Good and How to Design with Them

SXSWi, Saturday, 10th March 3:30 pm - 3:55 pm

Over the last few years whilst moving towards web standards I have been fighting against the use of fixed grid layouts in my designs. This is more to do with the fact that the proportions of these designs are often defined by web advertising sizes. Some of the more complex informational structures I now work with could benefit from a more formal approach to grids. Mark and Khoj have really encouraged me to take another look at this methodology.

Mark Boulton Owner, Mark Boulton Design

Khoi Vinh Design Director, The New York Times

Review by Glenn Jones on 7 April 2007

0.3 event

Helvetica

SXSWi, Tuesday 13th March 1:30 pm - 2:50 pm

The feature-length documentary about Helvetica was the big surprise of this year. What can I say, what a great film. I walked away thinking what strange and wonderful complex relationships we have with typography. Helvetica represents this by being unconsciously invisible to us and yet at the same time being so invasive in our urban environment that people do have strong emotional responses to it. It is truly amazing how different typefaces can convey so many values and emotions. The use of interviews with designers to explore the documentary theme were edited together well. If you like design or are interested in typography see this film. http://www.helveticafilm.com/

Gary Hustwit Director,

Review by Glenn Jones on 7 April 2007

0.3 event

Design Patterns: Defining and Sharing Web Interface Design Languages

SXSWi, Monday, 12th March 4:05 pm - 4:30 pm

I have been interested in design patterns for a long time, a great deal of Madgex's design is based on interface patterns that are reused across multiple sites. This concept has been around for some time, but is gaining greater amount of attention as the likes of Yahoo start to publish their pattern libraries. Luke talked about how to define patterns and communicating how they can be applied. What was interesting was that Luke also touched on the usability studies which back up some of the pattern design choices. The extension of documenting patterns with usability research could be very powerful. Made me think about how contextual usability tests have to be and whether you can truly test a pattern in isolation.

Luke Wroblewski Principal Designer, Yahoo!

Review by Glenn Jones on 7 April 2007

0.3 event

Web 2.0 and Semantic Web: The Impact on Scientific Publishing

SXSWi, Saturday, 10th March 5:00 pm - 6:00 pm

This was one of those small sessions on a specialist subject that sometimes just work because of the passion of the people involved. Scientific publishing is a world unto itself, with complex traditions of publishing which are part of the validation and developmental processes of science itself. Building online peer reviews and citing is complex. The session dealt with many of the issues facing Web 2.0 information system design. Do you create top down categorisation or allow users to create a folksomony through tagging? The group seemed to suggest that hybrid categorisation could be the solution. The online scientific publishing seems to be moving into an interesting use of layered document structures to reveal complex metadata.

John Wilbanks Exec Dir of Science Commons, Creative Commons

Matthew Cockerill Publisher, BioMed Central Ltd

Melissa Hagemann Program Mgr, Open Society Institute

Timo Hannay Dir of Web Pub, Nature Publishing Group

Amit Kapoor Managing Dir, Topaz

Review by Glenn Jones on 7 April 2007

0.3 event

How to Bluff Your Way in Web 2.0

SXSWi, Saturday, 10th March 11:30 am - 12:30 pm

Well what can you say about a talk displaying a slide with "AJAX" as "Accessibility Just Ain’t Xciting". There is a lot of worth in the concept like social networking and user generated content, the problem that a lot of people are coming against is hype around “umbrella buzzwords” like Web 2.0. Sometimes a little humor can be the best tool to cut through prevailing ideas to show true value. The video is well worth a watch http://blog.viddler.com/cdevroe/sxsw-bluff-20/

Andy Budd Creative Director, Clearleft Ltd

Jeremy Keith Web Developer, Clearleft Ltd

Review by Glenn Jones on 7 April 2007

0.3 event

Learning Interaction Design From Las Vegas

SXSWi, Sunday, 11th March 3:30 pm - 3:55 pm

There was something about Dan Saffer's talk on learning interaction design from Las Vegas which I loved. I think it made be the sideways pop at us designers for just designing for an "idealized person--a person that not surprisingly turns out to be someone almost exactly like ourselves". Dan did a great job of pointing out that the brutal direct nature of Las Vegas has valuable lessons, if we can see pass modern design values.

Dan Saffer Interaction Designer, Adaptive Path

Review by Glenn Jones on 7 April 2007