1:- module(sparqlprog,[
    2      sparql_endpoint/2
    3   ,  sparql_endpoint/3
    4   ,  current_sparql_endpoint/5
    5   ,  sparql_endpoint_url/2
    6   ,  srule/2
    7   ,  srule/3
    8   ,  srule/4
    9%   ,  query_goal/3     % Endpoint, Context, Opts
   10%   ,  query_phrase/3   % Endpoint, QueryPhrase, Result
   11%   ,  query_sparql/3 % Endpoint,QueryText,Result
   12   ,  create_sparql_select/2
   13   ,  create_sparql_select/3
   14   ,  create_sparql_select/4
   15   ,  create_sparql_construct/3
   16   ,  create_sparql_construct/4
   17   ,  inject_label_query/5
   18   ,  service_query_all/4
   19   ,  service/2
   20   ,  service/3
   21   ,  (??)/1
   22   ,  (??)/2
   23   ,  (??)/3
   24   ,  (??)/4
   25   ,  op(1150,xfy,join)
   26   ,  op(1150,fx,??)
   27   ,  op(1150,xfy,??)
   28   ]).   29
   30:- op(1150,xfy,join).

sparqlprog - logic programming with SPARQL

This library provides a prolog interface to SPARQL queries. It allows logic program queries to be compiled to SPARQL, and then executed on a remote SPARQL server.

Quickstart

The following can be entered interactively on the prolog console:

[library(sparqlprog)].
rdf_register_prefix(dbont,'http://dbpedia.org/ontology/').
sparql_endpoint( dbp, 'http://dbpedia.org/sparql/').
dbp ?? rdf(B,rdf:type,dbont:'Band'), rdf(B,dbont:bandMember,M).

This performs the following steps:

  1. For convenience, the prefix dbont is registered using rdf_register_prefix/2
  2. Next we register the short name dbp for the DBPedia endoint using sparql_endpoint/2.
  3. Then we query for all bands and their members using by queried by calling ??/2 - the query is specified using the standard rdf/3 predicate.

On the console the results should look like:

|    dbp ?? rdf(B,rdf:type,dbont:'Band'), rdf(B,dbont:bandMember,M).
B = 'http://dbpedia.org/resource/Alice_in_Chains',
M = 'http://dbpedia.org/resource/Sean_Kinney' ;
B = 'http://dbpedia.org/resource/Alice_in_Chains',
M = 'http://dbpedia.org/resource/William_DuVall' ;
B = 'http://dbpedia.org/resource/Alice_in_Chains',
M = 'http://dbpedia.org/resource/Jerry_Cantrell' ;
B = 'http://dbpedia.org/resource/Alice_in_Chains',
M = 'http://dbpedia.org/resource/Mike_Inez' ;
B = 'http://dbpedia.org/resource/Anthrax_(American_band)',
M = 'http://dbpedia.org/resource/Scott_Ian' .

Using user-defined predicates

You can define your own predicates for use in queries. So long as these stay within the sparqlprog subset, they can be rewritten into a query formed from rdf/3 terms.

For example, you can create a file dbpedia.pl with the following content:

band(X) :- rdf(X,rdf:type,dbont:'Band').
band_member(S,O) :- rdf(S,dbont:bandMember,O).

The original query can then be rewritten as:

dbp ?? band(B), band_member(B,M).

library(sparqlprog/ontologies/dbpedia) provides basic wrapper predicates for dbpedia.

This becomes more advantageous where we want to re-use predicates that encapsulate some query logic, for example, the following 3-ary predicate connects two bands by a shared member:

has_shared_band_member(B1,B2,A) :-
      rdf(A,dbo:associatedBand,B1),
      rdf(A,dbo:associatedBand,B2),
      B1\=B2.

library(sparqlprog/ontologies/dbpedia/dbpedia_matcher) shows how to construct a more advanced example for being able to perform semantic similarity of bands based on shared genres.

sparqlprog is distributed with a number of modules for existing triplestore schemas (with a bias towards life sciences triplestores).

In future some of these will have their own distribution. Some examples:

Note that library(sparqlprog/ontologies/wikidata) is deprecated, instead use library(sparqlprog_wikidata), a separate distribution

using OWL

library(sparqlprog/owl_util) provides predicates for working with OWL ontologies.

For example, owl_edge/4 provides an easy way to extract 'edges' from an ontology (e.g subClassOf between named classes, or involving existential restrictions).

library(sparqlprog/owl_search_viz) provides predicates for searching and visualizing OWL ontologies

sparqlprog language

Any program composed of sparqlprog primitive predicates and the following connectors is considered to be a sparqlprog program, and can be translated to SPARQL.

The connectors allowed are:

Note that the cut operator ! is not allowed.

The following are sparqlprog primitives:

Additionally, all SPARQL functions are treated as built-in predicates, e.g. regex/3, str_starts/2, lcase/2

Running sparqlprog programs over in-memory database

SWI-Prolog has its own in-memory database that can be interrogated via rdf/3.

sparqlprog programs can be executed over this in-memory database, as well as remote databases. See sparqlprog/emulate_builtins.pl for examples

Mixing remote and local execution

One of the challenges of using SPARQL with a traditional programming language is the impedance mismatch when combining query logic and programmatic logic. With sparqlprog, both programs and queries are specified in the same language.

Authors

  185:- use_module(library(sandbox)).  186:- use_module(library(settings)).  187:- use_module(library(semweb/sparql_client)).  188:- use_module(library(semweb/rdf11)).  189:- use_module(library(dcg_core)).  190:- use_module(library(dcg_codes)).  191:- use_module(sparql_dcg).  192:- use_module(concurrency).  193
  194:- dynamic srule/4.  195:- multifile srule/4.  196
  197:- dynamic sparql_endpoint/5.  198:- dynamic sparql_endpoint_url/2.  199:- multifile sparql_endpoint/5.  200:- set_prolog_flag(double_quotes, codes).  201
  202:- setting(limit,integer,1000,'Default SPARQL SELECT limit').  203:- setting(select_options,list,[distinct(true)],'Default select options').  204:- setting(user_agent,atom,'sparqlprog','Value for http user-agent').  205
  206:- meta_predicate query_phrase(+,//,-).  207
  208sandbox:safe_meta(sparql_dcg:phrase_to_sparql(Phr,_),[Phr]).
  209sandbox:safe_primitive(sparql_dcg:select(_,_,_,_,_)).
  210sandbox:safe_primitive(sparql_dcg:describe(_,_,_,_)).
  211sandbox:safe_primitive(sparql_dcg:describe(_,_,_)).
  212sandbox:safe_primitive(sparql_dcg:ask(_,_,_)).
  213
  214service_query_all(EP,Template,Spec,Results) :-
  215        setof(Template,??(EP,Spec),Results),
  216        !.
  217service_query_all(_,_,_,[]).
  218
  219service(Service,Query) :- ??(Service,Query).
  220service(Service,Query,S) :- ??(Service,Query,S).
 ??(?EP, +Goal:sparql_goal, +SelectTerm) is nondet
 ??(?EP, +Goal:sparql_goal, +SelectTerm, +Opts:list) is nondet
Query endpoint EP using Goal, selecting variables in SelectTerm

EP should be a name declared using sparql_endpoint/2. IF EP is unbound on entry, it is bound to the endpoint from which the current bindings were obtained.

Goal is any prolog query that conforms to the sparqlprog subset. i.e. it consists of sparqlprog predicates such as rdf/3, or defined predicates that can be compiled down to basic predicates.

SelectTerm is any prolog term, the variables used in this term will be used to determine the SELECT in the SPARQL query

Example:

??(dbp, band_member(B,M), row([B,M]))

Note in many cases the SELECT variables can be determined from the query in which case ??/2 is more convenient

  247??(EP,Spec,SelectTerm) :-
  248        ??(EP,Spec,SelectTerm,[]).
  249??(EP,Spec,SelectTerm,OptsOrig) :-
  250        copy_term(OptsOrig,Opts),
  251        debug(sparqlprog,'Finding subqueries: ~q opts=~q',[Spec,Opts]),
  252        %expand_opts(OptsIn,Opts),
  253        expand_subqueries(Spec,Spec2,EP),
  254        debug(sparqlprog,'Rewriting goal: ~q',[Spec2]),
  255        rewrite_goal(Spec2,SpecRewrite,Opts),
  256        debug(sparqlprog,'Rewritten goal: ~q',[SpecRewrite]),
  257        Goal=SpecRewrite,
  258        %spec_goal_opts(SpecRewrite,Goal,Opts),
  259        %debug(sparqlprog,'selecting opts...',[]),
  260        setting(select_options,Opts0),
  261        debug(sparqlprog,'Opts: ~q',[Opts0]),
  262        merge_options(Opts,Opts0,Opts1),
  263        query_goal(EP,Goal,SelectTerm,Opts1).
 ??(?EP, +Goal:sparql_goal) is nondet
Equivalent to ??/3 where the SELECT variables are extracted from variables in Goal.

Note: if Goal contains an aggregate query then ??/3 should be used.

  272??(EP,Spec) :-
  273        ??(EP,Spec,Spec).
 ??(+Goal:sparql_goal) is nondet
equivalent to ??/2, calling all known endpoints in parallel.
  279??(Spec) :- ??(_,Spec).
  280
  281
  282
  283
  284/*
  285expand_opts(OptsIn,OptsOut) :-
  286        select(program(P),OptsIn,T),
  287        !,
  288        tmp_file(prog,F),
  289        open(F,write,WS,[]),
  290        write(F),
  291        write(WS,P),
  292        close(WS),
  293        open(F,read,S,[]),
  294        findall(rule(T),read_term(S,T,[]),Rules),
  295        close(S),
  296        append(Rules,T,OptsOut).
  297expand_opts(Opts,Opts).
  298*/
  299
  300
  301
  302%spec_goal_opts(Opts ?? Goal, Goal, Opts) :- !.
  303%spec_goal_opts(Goal,Goal,[]).
  304
  305subq_term(subq(Q),EP,Q,EP).
  306subq_term(subq(Q,EP),_,Q,EP).
  307subq_term(x:Q,EP,Q,EP).
  308
  309expand_subqueries(V,V,_) :-
  310        var(V),
  311        !.
  312expand_subqueries(In,Right,EP) :-
  313        nonvar(In),
  314        In = (Left join Right),
  315        !,
  316        ??(EP, Left).
  317expand_subqueries(In,true,EP) :-
  318        nonvar(In),
  319        subq_term(In,EP,Q,EP2),
  320        !,
  321        ??(EP2, Q).
  322expand_subqueries([],[],_) :- !.
  323expand_subqueries([H|L],[H2|L2],EP) :-
  324        !,
  325        expand_subqueries(H,H2,EP),
  326        expand_subqueries(L,L2,EP).
  327expand_subqueries(In,Out,EP) :-
  328        compound(In),
  329        !,
  330        In =.. [P|Args],
  331        expand_subqueries(Args,Args2,EP),
  332        Out =.. [P|Args2].
  333expand_subqueries(X,X,_) :- !.
  334
  335
  336% convert a prolog list [A,B,..] to a disjunctive term A;B;...
  337plist_to_disj([_-X],X) :- !.
  338plist_to_disj([_-X|T],(X;T2)) :- plist_to_disj(T,T2).
  339
  340
  341
  342:- multifile rewrite_goal_hook/2.
 rewrite_goal(+InGoal, ?OutGoalDisjunction) is semidet
non-deterministic top-level goal rewrite if multiple goals possible, return a disjunction of goals (G1;G2;...;Gn) TODO: may be nondet if select variables in In...
  350rewrite_goal(In,OutDisj,Opts) :-
  351        debug(sparqlprog,'XXX: Rewriting goal: ~q',[In]),
  352        setof(Out,rewrite_goal(In,Out,1,Opts),Outs),
  353        list_to_disj(Outs,OutDisj).
  354
  355/*
  356% deterministic version
  357xxxrewrite_goal(In,OutDisj) :-
  358        debug(sparqlprog,'XXX: Rewriting goal: ~q',[In]),
  359        setof(In-Out,rewrite_goal(In,Out,1),InOuts),
  360        debug(sparqlprog,'YYY: Rewritten goal: ~q -> ~q',[In,InOuts]),
  361        unify_keys(In,InOuts),
  362        plist_to_disj(InOuts,OutDisj),
  363        !.
  364
  365unify_keys(_,[]).
  366unify_keys(Term,[TermX-_ | T]) :-
  367        term_variables(Term, Vars),
  368        term_variables(TermX, Vars),
  369        unify_keys(Term,T).
  370  
  371*/
 rewrite_goal(+InGoal, ?OutGoal, +Depth) is semidet
typically deterministic, but non-deterministic if multiple possible paths
  377rewrite_goal(In,Out,N,Opts) :-
  378        debug(sparqlprog,'rewriting: ~q',[In]),
  379        rewrite_goal_hook(In,X), !,
  380        debug(sparqlprog,'Using hook to transform ~q -> ~q',[In,X]),
  381        rewrite_goal(X,Out,N,Opts).
  382
  383
  384
  385
  386
  387
  388
  389% ------
  390% terminals
  391% ------
  392rewrite_goal(T, T2,_, _Opts) :- T=rdf(_,_,_), !, replace_string_unification(T,T2).
  393rewrite_goal(T, T2,_, _Opts) :- T=rdf(_,_,_,_), !, replace_string_unification(T,T2).
  394rewrite_goal(filter(A), filter(A),_, _Opts) :- !.
  395rewrite_goal(rdf_has(S,P,O), T2,_, _Opts) :- !, replace_string_unification(rdf(S,P,O),T2).
  396rewrite_goal(service(S,G), service(S,G2), D, Opts) :- !, rewrite_goal(G,G2,D, Opts).
  397
  398% TODO: consider adding semantics
  399%rewrite_goal(rdf(S,P,O), rdf_has(S,P,O),_, _Opts) :- !.
  400
  401%rewrite_goal('??'(Opts,Q), '??'(Opts,Q2), _, Opts) :- !, rewrite_goal(Q,Q2).
  402
  403rewrite_goal(aggregate(A,G,V), aggregate(A,G2,V), D, Opts) :- !, rewrite_goal(G,G2,D, Opts).
  404rewrite_goal(aggregate_group(A,GVs,G,V), aggregate_group(A,GVs,G2,V), D, Opts) :- !, rewrite_goal(G,G2,D, Opts).
  405rewrite_goal(aggregate_group(A,GVs,G,H,V), aggregate_group(A,GVs,G2,H2,V), D, Opts) :- !, rewrite_goal(G,G2,D, Opts), rewrite_goal(H,H2,D, Opts).
  406
  407% RDFStar
  408rewrite_goal(with(G,EdgeProps), rdfstar(S,P,O,EdgeProps), D, Opts) :- !, rewrite_goal(G,rdf(S,P,O), D, Opts).
  409
  410% direct evaluation
  411rewrite_goal( (pre(G),G2), G3, D, Opts) :- G, !, rewrite_goal(G2,G3, D, Opts).
  412
  413
  414% rdfs terminals
  415rewrite_goal(rdf_where(Q), rdf_where(Q2), D, Opts) :-
  416        !,
  417        rewrite_goal(Q, Q2, D, Opts).
  418rewrite_goal({Q}, {Q2}, D, Opts) :-
  419        !,
  420        rewrite_goal(Q, Q2, D, Opts).
  421rewrite_goal(optional(Q), optional(Q2), _, Opts) :-
  422        !,
  423        rewrite_goal(Q,Q2, Opts).
  424rewrite_goal(exists(Q), exists(Q2), _, Opts) :-
  425        !,
  426        rewrite_goal(Q,Q2, Opts).
  427rewrite_goal(rdfs_subclass_of(C,P), rdf(C,zeroOrMore(rdfs:subClassOf),P),_, _Opts) :- !.
  428rewrite_goal(rdfs_subproperty_of(C,P), rdf(C,zeroOrMore(rdfs:subPropertyOf),P),_, _Opts) :- !.
  429rewrite_goal(rdfs_individual_of(I,C), (rdf(I,rdf:type,X),rdf(X,zeroOrMore(rdfs:subClassOf),C)),_, _Opts) :- !.
  430rewrite_goal(a(I,C), rdf(I,rdf:type,C),_, _Opts) :- !.
  431%rewrite_goal(rdf_member(X,L), rdf(L,oneOrMore(rdf:rest)/rdf:last,X),_, _Opts) :- !.
  432rewrite_goal(rdf_member(X,L), rdf(L,(zeroOrMore(rdf:rest)/(rdf:first)),X),_, _Opts) :- !.
  433
  434% rdf11 rules
  435rewrite_goal(substring(S,P), contains(S,P), _, _Opts) :- !.
  436rewrite_goal(prefix(S,P), str_starts(S,P), _, _Opts) :- !.
  437rewrite_goal(the(_,_), true, _, _Opts) :- !.
  438
  439% Match any literal that matches Pattern case insensitively, where the `*' character in Pattern matches zero or more characters
  440rewrite_goal(like(S,P1), regex(S,P2,i), _, _Opts) :- !, like_to_regex(P1,P2).
  441
  442% TODO
  443%rewrite_goal(word(S,P1), regex(S,P2,i), _, Opts) :- !, like_to_regex(P1,P2).
  444
  445
  446% ------
  447% non-terminals
  448% ------
  449rewrite_goal((A,B),(A2,B2),D, Opts) :-
  450        !,
  451        rewrite_goal(A,A2,D, Opts),
  452        rewrite_goal(B,B2,D, Opts).
  453
  454rewrite_goal((A;B),(A2;B2),D, Opts) :-
  455        !,
  456        rewrite_goal(A,A2,D, Opts),
  457        rewrite_goal(B,B2,D, Opts).
  458rewrite_goal(\+A, \+A2, D, Opts) :-
  459        !,
  460        rewrite_goal(A,A2,D, Opts).
  461
  462% EXPERIMENTING WITH NONDET
  463rewrite_goal(A,A2,D, Opts) :-
  464        % TODO: see refl/2 test in test_aux
  465        setof(A-Clause,safe_clause(A,Clause),Clauses),
  466        !,
  467        debug(sparqlprog,' ~q CLAUSES==> ~q ~q',[A,Clauses,A2]),
  468        increase_depth(D,D2),
  469        %list_to_disj(Clauses,X),
  470        member(A-Clause,Clauses),
  471        rewrite_goal(Clause,A2,D2, Opts).
  472rewrite_goal(A,A2,D, Opts) :-
  473        % user defined expansion rule
  474        copy_term(Opts,OptsCopy),
  475        setof(A-Body,member( rule( (A:-Body) ) ,OptsCopy),Clauses),
  476        !,
  477        debug(sparqlprog,' ~q RULE==> ~q ~q',[A,Clauses,A2]),
  478        increase_depth(D,D2),
  479        %list_to_disj(Clauses,X),
  480        member(A-Clause,Clauses),
  481        rewrite_goal(Clause,A2,D2, Opts).
  482        %rewrite_goal(X,A2,D2, Opts).
  483rewrite_goal(A,A2,D, Opts) :-
  484        % as above, but parse the rule strinmg
  485        member(rule( Atom ),Opts),
  486        atom(Atom),
  487        read_term_from_atom(Atom, (Head :- Body), []),
  488        Head=A,
  489        !,
  490        debug(sparqlprog,' ~q RULE==> ~q',[A,A2]),
  491        increase_depth(D,D2),
  492        rewrite_goal(Body,A2,D2, Opts).
  493rewrite_goal(A,A,_,_).
  494
  495
  496% for certain predicates, do not use replace_string_unification(T,T3)
  497% TODO: make this a hook
  498nofilter(P) :-
  499        ground(P),
  500        rdf_global_id(P,Px),
  501        atomic(Px),
  502        atom_concat('http://www.bigdata.com/rdf/search#',_,Px).
  503
  504% rdf11 'like' construct
  505like_to_regex(Like,Re) :-
  506        concat_atom(Parts,'*',Like),
  507        (   Parts=[''|_]
  508        ->  Init=''
  509        ;   Init='^'),
  510        (   reverse(Parts,[''|_])
  511        ->  Last=''
  512        ;   Last='$'),
  513        concat_atom(Parts,'.*',Re1),
  514        concat_atom([Init,Re1,Last],Re).
  515
  516        
  517
  518% We avoid translation rdf(X,rdf:label,"foo") to a direct
  519% triple in the SPARQL query, since this may fail to
  520% match (e.g. if xsd:string is the actual type)
  521% instead we translate to  (rdf(X,rdf:label,VAR),VAR=="foo")
  522% because this yields a FILTER in the SPARQL giving desired results
  523replace_string_unification(T,T) :-
  524        (   T=rdf(_,P,_)
  525        ;   T=rdf(_,P,_,_)),
  526        nofilter(P),
  527        !.
  528replace_string_unification(T,T3) :-
  529        debug(sparqlprog,'replacing string unification: ~q',[T]),
  530        T =.. [P|Args],
  531        replace_string_unification_args(Args,Args2,T2,T3),
  532        T2 =.. [P|Args2],
  533        debug(sparqlprog,'replaced string unification: ~q',[T3]).
  534
  535replace_string_unification_args([],[],T,T).
  536replace_string_unification_args([anystr(A)|Args],[A2|Args2],T,(T2,FreshVar==A)) :-
  537        string(A),
  538        !,
  539        A2 = FreshVar,
  540        replace_string_unification_args(Args,Args2,T,T2).
  541replace_string_unification_args([A|Args],[A|Args2],T,T2) :-
  542        replace_string_unification_args(Args,Args2,T,T2).
  543
  544
  545no_rewrite(rdf_graph(_)).
  546no_rewrite(rdf_predicate(_)).
  547no_rewrite(rdf_is_iri(_)).
  548no_rewrite(member(_,_)).
  549no_rewrite(sparqlprog:member(_,_)).
  550
  551no_rewrite_mod(emulate_builtins).
  552no_rewrite_mod(rdf11).
  553
  554
  555:- meta_predicate safe_clause(:,?).  556
  557safe_clause(Goal,Body) :-
  558        % TODO: come up with a more extensible way to prevent SPARQL builtins being expanded
  559        \+ \+ \+ no_rewrite(Goal),
  560        catch(clause(Goal,Body,Ref),_,fail),
  561        \+ \+ \+ ((clause_property(Ref,module(Mod)),
  562                   no_rewrite_mod(Mod))).
  563
  564
  565
  566% TODO: attempt at expanding clauses not exported
  567% not clear this is a good idea...
  568todo__safe_clause(Goal,LocalBody) :-
  569        catch(clause(Goal,Body,Ref),_,fail),
  570        clause_property(Ref,module(M)),
  571        (   M\=system,
  572            \+ ((Body=..[rdf|_])),
  573            catch(clause(M:Body,_),_,fail)
  574        ->  LocalBody = M:Body
  575        ;   LocalBody=Body).
  576
  577
  578        
  579
  580
  581list_to_disj([X],X) :- !.
  582list_to_disj([X|T],(X;T2)) :- list_to_disj(T,T2).
  583
  584
  585increase_depth(D,_) :-
  586        D > 10,
  587        !,
  588        throw(error(max_depth_exceeded(D))).
  589increase_depth(D,D2) :-
  590        D2 is D+1.
 srule(+Pred, +Args) is det
 srule(+Pred, +Args, +Desc) is det
declare a new sparql rule
  598srule(P,A) :- srule(P,A,'').
  599srule(P,A,D) :-
  600        current_module(M),
  601        assert(srule(P,A,D,M)).
 sparql_endpoint(+EP:ground, +URL:atom, +Options) is det
 sparql_endpoint(+EP:ground, +URL:atom) is det
Declares EP as a short name for a SPARQL endpoint with the given URL.

No options are defined at the moment.

Example:

sparql_endpoint( dbp, 'http://dbpedia.org/sparql/').
  614sparql_endpoint(EP,Url) :- sparql_endpoint(EP,Url,[]).
  615sparql_endpoint(EP,Url,Options) :-
  616   url_endpoint(Url,Host,Port,Path),
  617   !,
  618   retract_declared_endpoint(EP,Url),     
  619   debug(sparqlprog,'Asserting SPARQL end point ~q: ~q ~q ~q ~q.',[EP,Host,Port,Path,Options]),
  620   assert(sparql_endpoint(EP,Host,Port,Path,Options)),
  621   retractall(sparql_endpoint_url(EP,_)),
  622   assert(sparql_endpoint_url(EP,Url)).
  623
  624retract_declared_endpoint(EP,Url) :-
  625   sparql_endpoint(EP,Host,Port,Path,_),
  626   debug(info,'% WARNING: Updating already registered SPARQL end point ~q.\n',[Url]),
  627   retractall(sparql_endpoint(EP,Host,Port,Path,_)),
  628   !.
  629retract_declared_endpoint(_,_).
  630
  631%user:term_expansion(:-(sparql_endpoint(EP,Url)), Expanded) :- 
  632%   endpoint_declaration(EP,Url,[],Expanded).
  633%user:term_expansion(:-(sparql_endpoint(EP,Url,Options)), Expanded) :- 
  634%   endpoint_declaration(EP,Url,Options,Expanded).
  635
  636
  637
  638endpoint_declaration(EP,Url,Options, sparqlprog:sparql_endpoint(EP,Host,Port,Path,Options)) :-
  639	debug(sparqlprog,'Declaring SPARQL end point ~q: ~q ~q ~q ~q.',[EP,Host,Port,Path,Options]),
  640   url_endpoint(Url,Host,Port,Path).
  641
  642url_endpoint(Url,Host,Port,Path) :-
  643	parse_url(Url,Parsed),
  644	member(host(Host),Parsed),
  645	member(path(Path),Parsed),
  646	(member(port(Port),Parsed);Port=80).
 current_sparql_endpoint(-EP:ground, -Host:atom, -Port:natural, -Path:atom, -Options:list) is nondet
Succeeds once for each known endpoint.
  652current_sparql_endpoint(EP,Host,Port,Path,Options) :-
  653   sparql_endpoint(EP,Host,Port,Path,Options).
  654
  655
  656% ----------------------------------------------------
  657% Goal-based queries 
  658% These get translated into phrase-based queries.
 query_goal(+EP, +Goal:sparql_goal, +Opts) is nondet
query_goal(-EP, +Goal:sparql_goal, +Opts) is nondet
Runs a SPARQL query against one or more SPARQL endpoints. Goal is converted into a textual SPARQL query using the DCG defined in sparql_dcg.pl.

If EP is ground on entry, the query is run against the specified endpoint. If EP is unbound on entry, the query is run agains all endpoints in parallel, possibly returning multiple results from each.

(The following applies only to queries that return bindings, not to simple boolean questions, which return only true or false.) Options are as follows:

limit(L:natural)
At-most this many bindings will be returned per SPARQL call.
offset(O:natural)
Begin returning bindings from the Oth result on.
autopage(Auto:bool)
If false, a single SPARQL call is made using any limit and offset options if supplied. If true, then the offset option is ignored and multiple SPARQL queries are made as necessary to supply results, using the limit option to determine the number of results retrieved from the endpoint at a time. Other options are passed to phrase_to_sparql/2.
  686query_goal(EP,Goal,Opts) :- 
  687        query_goal(EP,Goal,Goal,Opts).
  688
  689query_goal(EP,Goal,SelectTerm,Opts) :- 
  690   findall(EP,sparql_endpoint(EP,_,_,_,_),EPs1),
  691   sort(EPs1,EPs),
  692   term_variables(SelectTerm,Vars),
  693   (  Vars = [] % if no variables, do an ASK query, otherwise, SELECT
  694   -> phrase_to_sparql(ask(Goal),SPARQL),
  695      parallel_query(simple_query(SPARQL),EPs,EP-true)
  696   ;  Result =.. [row|Vars],
  697      setting(limit,DefaultLimit),
  698      call_dcg((  option_default_select(limit(Limit),DefaultLimit),
  699                  option_default_select(autopage(Auto),true),
  700                  (  {Auto=true}
  701                  -> {Query = autopage_query(Limit,SPARQL)},
  702                     option_default_select(offset(_),_)
  703                  ;  {Query = simple_query(SPARQL)},
  704                     cons(limit(Limit))
  705                  ) 
  706               ), Opts, Opts1),
  707      debug(sparqlprog,'DCG: ~q ~q ~q',[Vars,Goal,Opts1]),
  708      phrase_to_sparql(select(Vars,Goal,Opts1),SPARQL),
  709      debug(sparqlprog,'Executing parallel query: ~w // ~w // ~w',[SPARQL,Query,EPs]),
  710      parallel_query(Query,EPs,EP-Result),
  711      debug(sparqlprog,'RR=~q',[Result])
  712   ).
 create_sparql_select(+SelectTerm, +Goal, -SPARQL, +Opts) is det
 create_sparql_select(+Goal, -SPARQL, +Opts) is det
 create_sparql_select(+Goal, -SPARQL) is det
Generates a sparql SELECT or ASK statement for a prolog goal without executing it.

Goal can be any sparqlprog program, see ??/3

  722create_sparql_select(Goal,SPARQL) :-
  723        create_sparql_select(Goal,SPARQL,[]).
  724
  725create_sparql_select(Goal,SPARQL,Opts) :-
  726        create_sparql_select(Goal,Goal,SPARQL,Opts).
  727
  728create_sparql_select(Select,Goal,SPARQL,Opts) :-
  729        select(inject_labels(true),Opts,OptsRest),
  730        !,
  731        inject_label_query(Select,Goal,Select2,Goal2,Opts),
  732        create_sparql_select(Select2,Goal2,SPARQL,OptsRest).
  733
  734                
  735create_sparql_select(Select,Goal,SPARQL,Opts) :-
  736        filter_opts(Opts,OptsFiltered),
  737        rewrite_goal(Goal,Goal2,Opts),
  738        debug(sparqlprog,'Rewritten goal2: ~q',[Goal2]),
  739        term_variables(Select,Vars),
  740        debug(sparqlprog,'Vars: ~q',[Vars]),
  741        (   Vars = [] % if no variables, do an ASK query, otherwise, SELECT
  742        ->  phrase_to_sparql(ask(Goal2),SPARQL)
  743        ;   setting(limit,DefaultLimit),
  744            call_dcg((  option_default_select(limit(Limit),DefaultLimit),
  745                        option_default_select(autopage(Auto),true),
  746                        (   {Auto=true}
  747                        ->  {Query = autopage_query(Limit,SPARQL)},
  748                            option_default_select(offset(_),_)
  749                        ;   {Query = simple_query(SPARQL)},
  750                            cons(limit(Limit))
  751                        ) 
  752                     ), OptsFiltered, Opts1),
  753            phrase_to_sparql(select(Vars,Goal2,Opts1),SPARQL,Opts)).
  754
  755filter_opts([],[]).
  756filter_opts([H|T],[H|T2]) :-
  757        H=..[P|_],
  758        P\=inject_labels,
  759        P\=bindings,
  760        P\=label_predicate,
  761        !,
  762        filter_opts(T,T2).
  763filter_opts([_|T],T2) :-
  764        filter_opts(T,T2).
 inject_label_query(+Select, +Query, ?Select2, ?Query2, +Opts) is det
Add an optional(rdf(X,rdfs:label,XL)) for every variable X in Select
  770inject_label_query(Select, Goal, Select2, (Goal,ConjGoal), Opts) :-
  771        term_variables(Select,Vars),
  772        option(label_predicate(P),Opts,rdfpred(rdfs:label)),
  773        inject_label_for_vars(Vars,ConjGoal,LabelVars,P),
  774        conjoin(Select,LabelVars,Select2).
  775
  776inject_label_for_vars([Var],Goal,[LabelVar],P) :-
  777        !,
  778        inject_label_for_var(Var,Goal,LabelVar,P).
  779inject_label_for_vars([Var|Vars],(Goal1,Goal2),[LabelVar|LabelVars],P) :-
  780        !,
  781        inject_label_for_var(Var,Goal1,LabelVar,P),
  782        inject_label_for_vars(Vars,Goal2,LabelVars,P).
  783inject_label_for_var(Var,optional(rdf(Var,P,VarLabel)),VarLabel,rdfpred(P)) :- !.  % e.g. rdfs:label
  784inject_label_for_var(Var,optional(Goal),VarLabel,P) :-  Goal =.. [P,Var,VarLabel].  % prolog predicate, e.g. enlabel/2
  785
  786
  787conjoin(Term,L,T2) :-
  788        is_list(Term),
  789        !,
  790        append(Term,L,T2).
  791conjoin(Term,L,[Term|L]) :-
  792        \+compound(Term),
  793        !.
  794conjoin(Term,L,T2) :-
  795        Term =.. [P|Args],
  796        append(Args,L,Args2),
  797        T2 =.. [P|Args2].
 create_sparql_construct(+Head, +Goal, -SPARQL, +Opts) is det
 create_sparql_construct(+Head, +Goal, -SPARQL) is det
Generates a sparql CONSTRUCT statement for a prolog goal without executing it.

Goal or Head can be any prolog goal consisting of based rdf/3 or rdf/4 statements, filters, or terms that can be rewritten in this way

the Head forms the head part of the CONSTRUCT

  817create_sparql_construct(Head,Goal,SPARQL) :-
  818   create_sparql_construct(Head,Goal,SPARQL,[]).
  819create_sparql_construct(Head,Goal,SPARQL,Opts) :-
  820   rewrite_goal(Goal,Goal2, Opts),
  821   rewrite_goal(Head,Head2, Opts),
  822   debug(sparqlprog,'Rewritten: ~q <- ~q',[Head2,Goal2]),        
  823   phrase_to_sparql(construct(Head2,Goal2,Opts),SPARQL).
  824
  825
  826cons(X,T,[X|T]).
  827option_default_select(Opt,Def,O1,O2) :- select_option(Opt,O1,O2,Def).
  828simple_query(SPARQL,EP,EP-Result) :- query_sparql(EP,SPARQL,Result).
  829autopage_query(Limit,SPARQL,EP,EP-Result) :- autopage(EP,SPARQL,Limit,0,Result).
  830
  831autopage(EP,SPARQL,Limit,Offset,Result) :-
  832   format(string(Q),'~s LIMIT ~d OFFSET ~d',[SPARQL,Limit,Offset]),
  833   findall(R,query_sparql(EP,Q,R),Results),
  834   (  member(Result,Results)
  835   ;  length(Results,Limit),     % no next page if length(Results) < Limit
  836      Offset1 is Offset + Limit, % next batch of results
  837      autopage(EP,SPARQL,Limit,Offset1,Result)
  838   ).
 parallel_query(+Query, +EPs:list, ?EPResultPairs:list) is nondet
  841parallel_query(_,[],_) :- !, format(user_error, 'No endpoints',[]), fail.
  842parallel_query(P,[X],Y) :- !,
  843        debug(sparqlprog,'Bypassing parallelism, endpoints= ~q',[X]),
  844        call(P,X,Y).            % no parallel
  845parallel_query(P,Xs,Y) :-
  846   maplist(par_goal(P,Y),Xs,Goals),
  847   concurrent_or(Y,Goals,[on_error(continue)]).
  848
  849par_goal(P,Y,X,call(P,X,Y)).
 query_phrase(+EP, +Q:sparqle_phrase(R), R) is nondet
query_phrase(-EP, +Q:sparqle_phrase(R), R) is nondet
Phrase-based queries using the DCG defined in sparql_dcg.pl. The return type depends on the query:
select(V:list(var), sparql_goal, options) :: sparql_phrase(row(N)) :- length(V,N).
describe(resource,sparql_goal)            :: sparql_phrase(rdf).
describe(resource)                        :: sparql_phrase(rdf).
ask(sparql_goal)                          :: sparql_phrase(bool).

rdf  ---> rdf(resource,resource,object).
bool ---> true; false.

row(N) is the type of terms of functor row/N.

  869query_phrase(EP,Phrase,Result) :- 
  870        phrase_to_sparql(Phrase,SPARQL),
  871        query_sparql(EP,SPARQL,Result).
  872
  873
  874phrase_to_sparql(Phrase,SPARQL) :-
  875        phrase_to_sparql(Phrase,SPARQL,[]).
  876
  877phrase_to_sparql(Phrase,SPARQL,Opts) :-
  878        option(bindings(Bindings),Opts),
  879        !,
  880        term_variables(Phrase,Vars),
  881        copy_term(t(Vars,Phrase,Bindings),t(Vars1,Phrase1,Bindings1)),
  882        assign_vars_using_bindings(Vars1,Bindings1),
  883        phrase_vars_to_sparql(Phrase1,Vars1,SPARQL).
  884
  885phrase_to_sparql(Phrase,SPARQL,_Opts) :-
  886        term_variables(Phrase,Vars),
  887        copy_term(t(Vars,Phrase),t(Vars1,Phrase1)),
  888        phrase_vars_to_sparql(Phrase1,Vars1,SPARQL).
  889
  890phrase_vars_to_sparql(Phrase1,Vars1,SPARQL) :-
  891        numbervars(Vars1,0,_),
  892        (   phrase(Phrase1,Codes)
  893        ->  true
  894        ;   throw(unrecognised_query(Phrase1))
  895        ),
  896        string_codes(SPARQL,Codes),
  897        debug(sparqlprog,'SPARQL query: ~s',[SPARQL]).
 assign_vars_using_bindings(+Vars:list, +Bindings:list) is det
Bindings = [A='A',B='B', ...] assign all variables in Bindings to a variable v(Name)

rationale: by default, we assign vars to numbers using numbervars/3 these are then translated by the DCG to ?v1, ?v2, ... etc if the SPARQL is intended to be seen, then it's preferable to use the user's own meaningful assigned variable names

  908assign_vars_using_bindings(_,[]).
  909assign_vars_using_bindings(Vars1,[VN=v(VN2)|Bindings]) :-
  910        downcase_first_char(VN,VN2),
  911        assign_vars_using_bindings(Vars1,Bindings).
  912
  913downcase_first_char(A,A2) :-
  914        atom_chars(A,[C|Chars]),
  915        downcase_atom(C,C2),
  916        atom_chars(A2,[C2|Chars]).
  917
  918        
  919        
  920% ----------------------------------------------------
  921% In the end, everything comes through this.
 query_sparql(?EP, SPARQL, -Result) is nondet
Runs textual SPARQL query against an endpoint, exactly as with sparql_query/3. If EP is unbound on entry, all known endpoints will be tried sequentially.
  928query_sparql(EP,SPARQL,Result) :-
  929   sparql_endpoint(EP,Host,Port,Path,EPOpts),
  930   debug(sparqlprog,'Querying endpoint http://~q:~q~q - ~w',[Host,Port,Path,SPARQL]),
  931   setting(user_agent,UserAgent),
  932   sparql_query(SPARQL,Result,[user_agent(UserAgent),host(Host),port(Port),path(Path)|EPOpts])