Re: [Virtuoso-users] Fastest way to run lots of XSLT transformations

2009-11-13 Thread Cristian Vasquez
Hello Ivan, Thank you for the clarification. It seems that there is an error at one page of the documentation [1], where the function 'file_to_string_session' is mentioned, but It doesn't seem to exist (Undefined procedure). I'll use file_to_string_output. Cristian. [1] http://docs.openlinksw

Re: [Virtuoso-users] Fastest way to run lots of XSLT transformations

2009-11-13 Thread Ivan Mikhailov
Christian, You may wish to use string session instead of string. file_to_string_output() will produce one without even placing whole text to memory (if it's longer than the threshold then most of it will stay on disk). At the same time string session is a valid input for both xper_doc() and xtree_

Re: [Virtuoso-users] Fastest way to run lots of XSLT transformations

2009-11-13 Thread Cristian Vasquez
Sorry, It was an error when trying load an entity object using xtree_doc from a string longer than 10 megabytes. this issue was solved using xper_doc to operate over an 'XPER entity'. Thank you for your answer. Cristian. On Fri, Nov 13, 2009 at 2:28 PM, Ivan Mikhailov wrote: > Aldo, > > What l

Re: [Virtuoso-users] Fastest way to run lots of XSLT transformations

2009-11-13 Thread Ivan Mikhailov
Aldo, What limit do you mean? We've handled tens of megabytes of XML as XML trees and gigabytes of XMLs as "persistent XMLs" without any issues. Best Regards, Ivan Mikhailov OpenLink Software http://virtuoso.openlinksw.com On Fri, 2009-11-13 at 06:24 -0300, Aldo Bucchi wrote: > Ivan, > > Than

Re: [Virtuoso-users] Web Robot/Copy

2009-11-13 Thread Rumi Tsekova
Hi Aldo, You can take a look at the following links: 1. http://docs.openlinksw.com/virtuoso/rdfinsertmethods.html#rdfinsertmethodvirtuosocrawler 2. http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtRDFInsert#ExampleUsingVirtuosoCrawler 3. http://docs.openlinksw.com/virtuoso/virtuoso

[Virtuoso-users] Web Robot/Copy

2009-11-13 Thread Aldo Bucchi
Hi I just discovered the web robot (web copy) and will see if I can use that to integrate some XML load/transform pipeline into Virtuoso (right now we use shell scripts et al to get the XML files) I read the docs and they leave me with tons of questions. Before shooting, is this working?

Re: [Virtuoso-users] Fastest way to run lots of XSLT transformations

2009-11-13 Thread Aldo Bucchi
Ivan, Thanks for the info. One more related question. If the XML files are in the DB, will the mem limit for the transformations still exist? Today we find it necessary to split them into ~1MB files. On Nov 13, 2009, at 4:12, Ivan Mikhailov wrote: Hello Aldo, If you transform each fil

Re: [Virtuoso-users] Fastest way to run lots of XSLT transformations

2009-11-13 Thread Ivan Mikhailov
Hello Aldo, If you transform each file only once then it does not matter (or filesystem may be even faster). More important is that XSLT is a single-thread thing so you may wish to occupy all your CPUs by running an async queue of works or split the set of files in few parts and process each part