glennjones.net

Microformats test-suite concept

I have been working on a concept for a new microformats test-suite. I need a comprehensive test-suite before I can move the development of UfXtract forward. Rather than just build something in isolation I thought it would be nice to find a way to share this work with the community.

I have written two POSH patterns “testsuite” and “testfixture”. They follow the principles of microformats design:

• They are self-describing and created with HTML ,with no hidden metadata
• Building a test should be easy even for those who are HTML authors
• They are not linked to any one programming language and should be easy to share
• They allow for the creation of an in-browser Testrunner

For example go to http://ufxtract.com/testsuite/hcard/

The earliest tests for hcard used vcards to describe the expected output. As the community has moved forward it has designed microformats which are independent of external specification. So this test-suite is designed around the concept of a standardised data structure. In this case, expressed in JSON, but they could be converted into XPaths to test XML or other languages.

I have started to build a small console app, which will spider the HTML and create NUnit/C# class files for my build tests. Although this is specific to my own parsers development, it should be easy to do the same for other programming languages and projects.

Parsing “testsuite” and “testfixture”.
I have already setup UfXtract to parse these patterns into JSON/XML. It would not take much for other microformats parser developers to construct profiles for these POSH patterns.

Here is an example of the output.
ttp://lab.backnetwork.com/ufXtract/?url=http%3A%2F%2Fufxtract.com%2Ftestsuite%2Fhcard%2Fhcard1.htm&format=test-fixture&output=json

 

The Testrunner

If you go to http://ufxtract.com/testsuite/hcard/hcard1.htm and press Alt X you can see a working demonstration of the testrunner.

I have observed that most parser developers are using comparative testing as their main tool to quickly understand how the complex rules and optimizations are applied. So I have built a JavaScript Testrunner which allows for simple comparative testing between parsers.

It uses a number of techniques to standardise both access to the parsers API’s and the JSON output. Please note that at this stage the JSON standardisation process can cause a test to be marked as failed when it could be judged to have passed. Most of the current differences in parser output are down to whether a value is stored as single property or an array of properties.

At the moment the Testrunner is only working with the testfixure , it would not take much to extend the Testrunner to run a whole test-suite.

I would love to add Operator and other parsers to the Testrunner.

 

Proof of concept

This is very early proof of concept stuff. What I would like to ask is a number of questions before moving it forward.

• Are people interested in the idea of shared test-suites?
• What do you think to the approach?
• Can you see any big issues with the concepts?
• If you already have tests/test-suites, would you be willing to add them to the project?
• Would you be interested in contributing to a project like this?

Navigation

• Home
• Archive
• Links
• About

Copyright ©

Glenn Jones

Creative commons license: Some rights reserved