Ivan’s private site

July 6, 2007

SPARQL Endpoint interface to Python

Filed under: Code,Python,Semantic Web,Work Related — Ivan Herman @ 12:43
Tags: ,

I played with SPARQL on my local machine, and I also got inspired by Lee’s SPARQL library for Javascript. But, well, I prefer Python… So I made a set of utility classes first for myself, but then I decided to package it more properly. Maybe others can find it useful, too.

The goal is to give some help in turning a SPARQL query into the corresponding HTTP GET Protocol, send it to a SPARQL endpoint somewhere on the Web, and do something with the results. The simplest usage is something like:

from SPARQL import SPARQLWrapper
queryString = "SELECT * WHERE { ?s ?p ?o. }"
sparql = SPARQLWrapper("http://localhost:2020/sparql")
# add a default graph, though that can also be done in the query string
sparql.addDefaultGraph("http://www.example.com/data.rdf")
sparql.setQuery(queryString)
try :
    ret = sparql.query() # ret is a stream with the results in XML, it is a file like object
except:
    deal_with_the_exception() # eg, syntax error

To make it even easier to use, conversions to more Python-friendly formats can also done on the results: eg, turn it into a proper DOM tree if the result is XML, use Bob Ippolito’s simplejson module to convert a return format in JSON into Python dictionary, or parse it with RDFLib and return an RDFLib Graph in case the return is in RDF/XML. Ie, one could have done:

try :
    sparql.setReturnFormat(SPARQL.JSON)
    ret = sparql.query()
    dict = ret.convert()
except:
    deal_with_the_exception()

where “dict” is a Python dictionary. There are some more tricks in the library, but that essentially it…

The code is available from my site; the API documentation is included in the distribution (and is also available online).

It is an early release. There are some problems, and I expect some more. I have primarily tested it with two different SPARQL endpoints running on my local machine (joseki3 and virtuoso) and also with some public SPARQL endpoints. There are some differences on the return media type for, eg, JSON or N3, the non-standard arguments (eg, setting the return format) still diverge a bit, etc. But I would expect these to converge over time. However, I am sure that my code will have problems with some of the endpoints at least on those grounds (or others)…

About these ads

9 Comments

  1. OMG!

    On friday I was working in the same idea [1], but my implementation needs more work. I will test your lib ;-)

    BTK, do you know pysparql [2][3]? The author also got inspired by sparql.js xD

    [1] http://svn.berlios.de/svnroot/repos/swaml/trunk/apps/prototypes/sparql-endpoint/client/sparlclient.py
    [2] http://labs.cavorite.com/python/sparql/
    [3] http://code.google.com/p/pysparql/

    Comment by Sergio Fernandez — July 8, 2007 @ 10:56

  2. Oops… I must admit I did not know about pysparql, my bad. I should have made homework, I guess… Having said that, the web site is currently down :-(

    Thanks for the pointer!

    Comment by Ivan Herman — July 8, 2007 @ 11:36

  3. Don’t worry, pysparql is under development, your library seems a better implementation.

    Comment by Sergio Fernandez — July 8, 2007 @ 12:23

  4. :-) Nevertheless, I should have looked at it…

    Comment by Ivan Herman — July 8, 2007 @ 16:24

  5. Hi Ivan,

    I want to write a Python SPARQL client, I think. In particular I want to query http://doapstore.org/

    Do you have any example code for pysparql that performs a query on a live site I can use? I’m completely new to SPARQL and I’m suffering from extreme acronym-overload and don’t know where to begin.

    Thanks!
    Rob

    Comment by Rob Cakebread — July 20, 2007 @ 0:33

  6. Rob,

    I do not know about pysparql, that is another package. For the code I talked about, there are some example codes in the package’s documentation. I hope that helps.

    Ivan

    Comment by Ivan Herman — July 20, 2007 @ 9:25

  7. Have you seen what you can do with Yahoo Pipes? http://pipes.yahoo.com/pipes/
    Maybe your SPARQL endpoint ambitions are too broad. Just make a simple module that someone can plug into, and let them worry about the rest of it.

    Comment by Joy — October 18, 2007 @ 14:42

  8. Joy,

    I am not sure I understand what you mean. I do not have problems with yahoo pipes, it is a great stuff, but it does not really have anything to do with the SPARQL interface…

    Comment by Ivan Herman — October 18, 2007 @ 15:03

  9. Good to see this evolving. I should try to rev http://svn.foaf-project.org/foaftown/2008/ruby-sparql-client/ or find someone who’ll really get it working well…

    Comment by Dan Brickley — February 14, 2008 @ 16:18


RSS feed for comments on this post.

The Rubric Theme. Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 3,616 other followers

%d bloggers like this: