Hugh,
Thanks for your reply, I've seen the bulk loader, but I need my application
itself to upload and download multiple whole graphs. I don't really want to
have to automate the copying of files using scp. Anyway, I have worked
around this by parsing the files using Jena, writing out as N-Triples and
then uploading in chunks.
However, I'm now facing the problem of how to do the download.
Approaches I have tried so far:
- Use the Graph Crud Endpoint. This works for smallish graphs (once
the ResultSetMaxRows
parameter is increased). But for larger graphs (my example is around 5M
triples), I get this error message: "Virtuoso 22023 Error SR...: The result
vector is too large". This looks to be related to this open issue on
GitHub: https://github.com/openlink/virtuoso-opensource/issues/119 so I
guess this approach is not going to bear fruit.
- I have also tried using Jena Streaming as detailed here:
https://jena.apache.org/documentation/io/rdf-output.html So I'm using
RDFDataMgr.write() to attempt to serialize a whole graph to an output
stream. This works fine for small graphs, but even for a graph of 500K
triples I get the error "SR177: Misc Transaction Error" which doesn't give
me much to go on.
At this point I think I may have to resort to issuing a query for a whole
graph using SPARQL to get a stream of triples and serialize them at the
client. Any other suggestions?
Thanks
Chris
On 28 February 2015 at 05:31, Hugh Williams <hwilli...@openlinksw.com>
wrote:
> Hi Chris,
>
>
> For large datasets files as you indicate having, it is generally recommend
> to use the Virtuoso RDF Bulk loader to upload the files as detailed at:
>
>
> http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtRDFInsert#Loading%20RDF%20datasets%20into%20one%20or%20more%20Virtuoso%20Graph%20IRIs
>
> Best Regards
> Hugh Williams
> Professional Services
> OpenLink Software, Inc. // http://www.openlinksw.com/
> Weblog -- http://www.openlinksw.com/blogs/
> LinkedIn -- http://www.linkedin.com/company/openlink-software/
> Twitter -- http://twitter.com/OpenLink
> Google+ -- http://plus.google.com/100570109519069333827/
> Facebook -- http://www.facebook.com/OpenLinkSoftware
> Universal Data Access, Integration, and Management Technology Providers
>
> On 26 Feb 2015, at 17:19, Chris Rimmer <chris.rim...@67bricks.com> wrote:
>
> Hi,
>
> I am building an application which needs to programmatically upload RDF
> files to be stored in Virtuoso. I've been using the Graph Crud Endpoint,
> which is fine for files that are not too large, but I've found it getting
> progressively slower once the file gets beyond 200MB or so.
>
> I'm now looking at handling these larger files via WebDAV instead. What I
> want to know is, if a file is placed into the rdf_sink directory, is there
> any way to get some indication of the progress in ingesting it, or of any
> errors that occurred while the process was running?
>
> Thanks
>
> Chris
>
> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming The Go Parallel Website,
> sponsored
> by Intel and developed in partnership with Slashdot Media, is your hub for
> all
> things parallel software development, from weekly thought leadership blogs
> to
> news, videos, case studies, tutorials and more. Take a look and join the
> conversation now.
> http://goparallel.sourceforge.net/_______________________________________________
> Virtuoso-users mailing list
> Virtuoso-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/virtuoso-users
>
>
>
------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users
