After saying goodbye to DERI in my previous post, now it’s time to say a few words about my new job: two weeks ago on the 3rd of October I joined the consulting team at Talis. As a company, Talis has pretty amazing track record of being a market leader in Semantic Web and linked data technology (e.g, through their linked data platform, their Aspire e-learning system and recently through Kasabi). Beyond these products, the consulting team helps clients (see the case studies for a list of previous projects) to learn about the possibilities of the technology, and eventually design and develop individual linked data solutions.

I’m pretty excited about this move – after years in academia, I will finally be able to apply my know-how to real-world problems and use cases of actual, paying customers! And as I said in my previous post: now it is the perfect time to do this, as semantic technologies are more and more moving into the mainstream.

For the coming six months I’ll be based in Birmingham (coincidentally the home of metal!), but will eventually move to Berlin and work from there. Germany is still a little behind in the whole open data movement, but things are happening there as well. See you soon!

It feels strange, but last Friday, after a good 7 1/2 years (that’s 2829 days!), I finally had my last day at DERI, the Digital Enterprise Resarch Institute at the National University of Ireland in Galway.

Coming from a background as a linguist and knowing very little about the Semantic Web, I started as a fresh PhD student in January 2004, when DERI was still only a handful of researchers. Very few people had ever heard about this “Semantic Web” (let alone “linked data” – that label was only coined a few years later), and those who did mostly considered it to be a rather far-fetched, purely academic exercise. I experienced the somewhat crazy early years at DERI (read about it in the paper…), saw the institute grow, change management, change location and eventually turn into the largest (currently 137 members) and probably most successful SW research institute world-wide. I’m pretty sure that for many if not most SW or linked data-related projects and activities you will come across today, there will be someone involved who either did or does work at DERI. Or someone who will work at DERI in the future – during the almost 8 years I have spent there, many outstanding personalities I met in the community eventually joined our little institute.

I experienced DERI as a fantastic place to work: I learned an immense amount of things (skills and experiences that definitely helped me find my new job), made good friends from all over the world (some of the for life, I’m sure), had the opportunity to work and engage with some of the most interesting and influential people in the community (both at DERI and in collaboration with outside partners) and even managed to finish a PhD along the way. Of course, part of the DERI experience is the (mostly) beautiful city of Galway, where the institute is located – but that’s a whole different story. I feel privileged having been very close to the centre of a development which saw the idea of a meaningful, machine-interpretable, “smarter” Web evolve from something that was either ignored or laughed at, into something that is now (in one form or the other) on the agenda of virtually all the big players who define what the Web is today – to pick a few arbitrary examples, just look at schema.org (Google, Yahoo!, Microsoft), Opengraph (facebook) or the adoption of linked data by the BBC.

So, now that my time at DERI is over, I’d like to say “thank you” once more to everyone I have met there, worked with, laughed with, argued with, drank Guinness, whiskey and wine with (or coffee and tea), or walked through the rain with – go raibh míle maith agat! We’ll meet again!

This little project was an opportunity to try out the very cool ScraperWiki. One thing led to another, after a short while I arrived at this scraper, and eventually the final Jargon RDF was done. Now on to the actual task at hand…

The new version of the LOD cloud will be published later today or tomorrow, but you get a sneak peak here first! ;)

Semantic agents are agents which are aware of RDF data and actively request it.

To know whether or not an agent requests RDF, we could look at the header of an individual HTTP request and check if the agent had specified Accept: application/rdf+xml. However, the Apache server log files unfortunately don’t tell us anything about the request header. Luckily though, there is an indirect way of finding out about this. If our linked data site uses best practice content negotiation and 303 redirects, we can look at pairs of requests in the log files. E.g., the Semantic Web Dog Food site uses a particular URI pattern for resources and their HTML and RDF representations. E.g.:

http://data.semanticweb.org/organization/deri-nui-galway

http://data.semanticweb.org/organization/deri-nui-galway/html

http://data.semanticweb.org/organization/deri-nui-galway/rdf

If the plain URI is requested, the server will either redirect to the HTML or the RDF representation, based on what was specified by the agent. Therefore, if we find a request for a plain URI and a request for the corresponding RDF URI, from the same IP address and the same agent, within a short time frame (e.g. 5 seconds), then we can infer that the agent had requested application/rdf+xml and can therefore be classified as a semantic agent.

90.21.243.141 - - [06/Oct/2008:16:07:58 +0100] "GET /organization/vrije-universiteit-amsterdam-the-netherlands HTTP/1.1" 303 7592 "-" "rdflib-2.4.0 (http://rdflib.net/; eikeon@eikeon.com)"
90.21.243.141 - - [06/Oct/2008:16:08:02 +0100] "GET /organization/vrije-universiteit-amsterdam-the-netherlands/rdf HTTP/1.1" 200 45358 "-" "rdflib-2.4.0 (http://rdflib.net/; eikeon@eikeon.com)"

The example above shows this: the “rdflib.net” agent requested the plain URI .../organization/vrije-universiteit-amsterdam-the-netherlands and was 303 redirected to .../organization/vrije-universiteit-amsterdam-the-netherlands/rdf a few seconds later. From this we can automatically infer that “rdflib.net” is a semantic agent.

A list of 423 semantic agents found in this way for the dog food site from 10/2008-10/2009 is here. Looking at the list, we can find a lot of agents that are clearly “semantic”, such as the “SindiceFetcher” or a SIOC browser. However, most of them are actually not what I would normally consider “semantic”, such as hordes of “Mozilla”-branded agents or dodgy looking bots. More research is awaiting…

• “International Semantic Web Conference” to “Incredible Semantic Web Conference”
• “Asian Semantic Web Conference” to “Amazing Semantic Web Conference”
• “European Semantic Web Conference” to “Extraordinary Semantic Web Conference”

to lod (verb): lod, lodded, lodding – the act of publishing linked open data on the World Wide Web, adhering to the rules of linked data.