Hi Hugh, No not really, just trying another knob to see if things could go faster… >From the perspective of the data warehousing set up we would use, directly >loading triples from a ssh & gunzip pipe into virtuoso would be nice (as our data production and data hosting are in different data centres). But not something you should prioritise IMHO (it’s nice to have)
Regards, Jerven On 20 Jan 2014, at 19:17, Hugh Williams <hwilli...@openlinksw.com> wrote: > Hi Jerven, > > In speaking to development the Virtuoso "file_io..." function were written > for use with regular files and thus do not support named pipes. Do you have a > specific need for this support for data loads ? > > Best Regards > Hugh Williams > Professional Services > OpenLink Software, Inc. // http://www.openlinksw.com/ > Weblog -- http://www.openlinksw.com/blogs/ > LinkedIn -- http://www.linkedin.com/company/openlink-software/ > Twitter -- http://twitter.com/OpenLink > Google+ -- http://plus.google.com/100570109519069333827/ > Facebook -- http://www.facebook.com/OpenLinkSoftware > Universal Data Access, Integration, and Management Technology Providers > > On 17 Jan 2014, at 14:08, Jerven Bolleman <jerven.bolle...@isb-sib.ch> wrote: > >> Hi Hugh, >> >> The virtuoso server log does not show any error messages. >> The version is Version 7.0.1-dev.3207-pthreads as of Jan 16 2014 >> >> If I use file_open instead of file_to_string_output I get a error >> message (FA25 Seek error in file , which makes sense for a pipe). >> >> i.e. DEBUG: Using script:DB.DBA.TTLP(file_open >> ('/tmp/virtuosoJavaCommunicationFifo7003142399157620167.ttl'), >> 'http://beta.sparql.uniprot.org/locations/','http://beta.sparql.uniprot.org/locations/') >> 2014-01-17 13:29:35 +0000 >> org.expasy.sesame.virtuoso.VirtuosoBulkStatementTransaction >> ERROR: >> virtuoso.jdbc4.VirtuosoException: FA025: Seek error in file >> '/tmp/virtuosoJavaCommunicationFifo7003142399157620167.ttl', error : >> >> Then in my code writing to the named pipe fails on the 87th triple just >> as in the case with file_to_string_output. (this seems to be very close >> to 4096 bytes written before failing). >> >> However, with file_to_string_output I see the same broken pipe from the >> java side. But nothing in the virtuoso.log. Just the normal start up >> logging. The process also fails on the 87th triple. >> >> Secondly as a second trial, I speed up the writing into the turtle file >> then the process is successful. e.g. cat in to a named pipe or less >> checks on the java side. This makes me think that the >> file_to_string_output method does not correctly check if the named pipe >> is finished or if it just has no bytes available at this time. >> If that is the case then it would be a bug, but someone would need to >> check on a code level if my idea is correct. >> >> Regards, >> Jerven >> >> >> On 17/01/14 02:42, Hugh Williams wrote: >>> Hi Jerven, >>> >>> What is the version of Virtuoso being used, please provide the output of >>> running: >>> >>> virtuoso-t -? >>> >>> What are the actual errors being reported on the server side, please >>> provide a copy or snippet of the virtuoso.log show these ? >>> >>> Are you able to provide a simple test case for recreation in-house ? >>> >>> Best Regards >>> Hugh Williams >>> Professional Services >>> OpenLink Software, Inc. // http://www.openlinksw.com/ >>> Weblog -- http://www.openlinksw.com/blogs/ >>> LinkedIn -- http://www.linkedin.com/company/openlink-software/ >>> Twitter -- http://twitter.com/OpenLink >>> Google+ -- http://plus.google.com/100570109519069333827/ >>> Facebook -- http://www.facebook.com/OpenLinkSoftware >>> Universal Data Access, Integration, and Management Technology Providers >>> >>> On 16 Jan 2014, at 16:44, Jerven Bolleman <jerven.bolle...@isb-sib.ch >>> <mailto:jerven.bolle...@isb-sib.ch>> wrote: >>> >>>> Hi Virtuoso Devs, >>>> >>>> I am trying to use file_to_string_output to read from a named pipe. >>>> However, this seems to break without log message from virtuoso >>>> >>>> e.g. >>>> >>>> DB.DBA.TTLP(file_to_string_output >>>> ('/tmp/virtuosoJavaCommunicationFifo7287643597025348653.ttl'), >>>> 'http://beta.sparql.uniprot.org/locations/','http://beta.sparql.uniprot.org/locations/') >>>> >>>> Which on the other side gives me Broken Pipe IOExceptions. >>>> >>>> Do you know if anyone else has tried this before or if this >>>> fundamentally will never work? >>>> >>>> The /tmp is in the DirsAllowed >>>> >>>> Regards, >>>> Jerven >>>> >>>> -- >>>> ------------------------------------------------------------------- >>>> Jerven Bolleman jerven.bolle...@isb-sib.ch >>>> <mailto:jerven.bolle...@isb-sib.ch> >>>> SIB Swiss Institute of Bioinformatics Tel: +41 (0)22 379 58 85 >>>> CMU, rue Michel Servet 1 Fax: +41 (0)22 379 58 58 >>>> 1211 Geneve 4, >>>> Switzerland www.isb-sib.ch <http://www.isb-sib.ch> - www.uniprot.org >>>> <http://www.uniprot.org> >>>> Follow us at https://twitter.com/#!/uniprot >>>> ------------------------------------------------------------------- >>>> >>>> ------------------------------------------------------------------------------ >>>> CenturyLink Cloud: The Leader in Enterprise Cloud Services. >>>> Learn Why More Businesses Are Choosing CenturyLink Cloud For >>>> Critical Workloads, Development Environments & Everything In Between. >>>> Get a Quote or Start a Free Trial Today. >>>> http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk >>>> _______________________________________________ >>>> Virtuoso-users mailing list >>>> Virtuoso-users@lists.sourceforge.net >>>> https://lists.sourceforge.net/lists/listinfo/virtuoso-users >>> >> >> -- >> ------------------------------------------------------------------- >> Jerven Bolleman jerven.bolle...@isb-sib.ch >> SIB Swiss Institute of Bioinformatics Tel: +41 (0)22 379 58 85 >> CMU, rue Michel Servet 1 Fax: +41 (0)22 379 58 58 >> 1211 Geneve 4, >> Switzerland www.isb-sib.ch - www.uniprot.org >> Follow us at https://twitter.com/#!/uniprot >> ------------------------------------------------------------------- >> >> ------------------------------------------------------------------------------ >> CenturyLink Cloud: The Leader in Enterprise Cloud Services. >> Learn Why More Businesses Are Choosing CenturyLink Cloud For >> Critical Workloads, Development Environments & Everything In Between. >> Get a Quote or Start a Free Trial Today. >> http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk >> _______________________________________________ >> Virtuoso-users mailing list >> Virtuoso-users@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/virtuoso-users > ------------------------------------------------------------------- Jerven Bolleman jerven.bolle...@isb-sib.ch SIB Swiss Institute of Bioinformatics Tel: +41 (0)22 379 58 85 CMU, rue Michel Servet 1 Fax: +41 (0)22 379 58 58 1211 Geneve 4, Switzerland www.isb-sib.ch - www.uniprot.org Follow us at https://twitter.com/#!/uniprot ------------------------------------------------------------------- ------------------------------------------------------------------------------ CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments & Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk _______________________________________________ Virtuoso-users mailing list Virtuoso-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/virtuoso-users
