Re: [Virtuoso-users] Long query - "Lost connection to server"

[Pantelis Natsiavas]

Thank you Hugh for your answer.

I have actually already used the respective pragma before the SPARUL query.

SPARQL DEFINE sql:log-enable 3
INSERT 

The documentation of the virtuoso functions is not clear for me. However, I
think I have already done what you suggest. Right?

The graph the SPARUL is run against is rather big. It is the
"ClinicalTrials.gov" graph of the Bio2RDF, namely 152.725.529 triplets. My
VM has 32GBs RAM and the memory consumption of the virtuoso process starts
from 40% and constantly increases until the break down (I have seen 70%
memory consumption but I cannot be certain for the peak). When the
operation breaks down, the memory is not fully released. It drops down to
around 40%. Even though in the log file it is shown that virtuoso restarts, I
have to restart the VM in order to get my full memory back.

I have repeated the process today and I am attaching the respective log
file part (I though it might be useful). Perhaps you can understand more
from the log, or you could give me instructions for more detailed logging.

Please note that I have already followed in the instructions regarding the
buffering (General Memory Usage Settings) and the swapping (Linux-only --
"swappiness") on the http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/
Main/VirtRDFPerformanceTuning.

I really appreciate your help.

Kind regards,
Pantelis Natsiavas

2016-08-10 4:09 GMT+03:00 Hugh Williams :

> Hi Pantelis,
>
> What is the memory consumption of Virtuoso whilst running the insert query
> and what is the size of the graph the insert is being performed against ?
> As note for SPARUL operation against graph with large amounts of data it is
> recommended queries are performed in row-wise auto commit mode to reduce
> memory consumption which case otherwise be depleted causing hangs or the
> server crash if memory cannot be allocated, with the Virtuoso log_enable
> function as detailed at:
>
> http://docs.openlinksw.com/virtuoso/fn_log_enable.html
> http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/V
> irtTipsAndTricksGuideDeleteLargeGraphs
>
> Best Regards
> Hugh Williams
> Professional Services
> OpenLink Software, Inc.  //  http://www.openlinksw.com/
> Weblog   -- http://www.openlinksw.com/blogs/
> LinkedIn -- http://www.linkedin.com/company/openlink-software/
> Twitter  -- http://twitter.com/OpenLink
> Google+  -- http://plus.google.com/100570109519069333827/
> Facebook -- http://www.facebook.com/OpenLinkSoftware
> Universal Data Access, Integration, and Management Technology Providers
>
> On 9 Aug 2016, at 12:11, Pantelis Natsiavas  wrote:
>
> I have just retried the above long-time query and checked the
> virtuoso.log. I see that after 45 minutes I get the following log entries
> (I executed the query around 10.00):
>
> 10:24:23 * Monitor: High disk read (2)
> 10:41:23 * Monitor: High disk read (2)
> 10:43:24 * Monitor: High disk read (2)
> 10:43:41 mmap failed with 12
> 10:43:45 mmap failed with 12
> 10:43:45 mmap failed with 12
> 10:43:45 mmap failed with 12
> 10:43:46 mmap failed with 12
> 10:43:46 mmap failed with 12
> 10:43:46 GPF: Dkpool.c:1634 could not allocate memory with mmap
>
> The same messages (more or less) had in the previous "lost connection"
> incident.
>
> Could somebody provide some hints?
>
> Kind regards,
> Pantelis Natsiavas
>
>
> 2016-08-09 9:58 GMT+03:00 Pantelis Natsiavas :
>
>> Hi.
>>
>> I am trying to execute a SPARQL INSERT-WHERE taking a lot time to
>> complete.
>>
>> INSERT {
>> GRAPH > >
>> {
>> ?s  ?newLabel
>> }
>> }
>> WHERE {
>> GRAPH > >
>> {
>> ?s ?p ?o .
>> FILTER (REGEX(STR(?o),"abc","i")) .
>> BIND(REPLACE(?o, "abc", "", "i") AS ?newLabel)
>> }
>> };
>>
>> Trying to execute it through the web environment, I get a timeout. I
>> thought that the most appropriate way to do it would be through the isql-v
>> command line environment. When I try to do it, after some hours I get
>>
>>  Error 08S01: [Virtuoso Driver]CL065: Lost connection to server*
>>
>> This is rather peculiar as the isql-v runs on the same VM as the
>> virtuoso. It should also be noted that the virtuoso instance responds and
>> has not collapsed.
>>
>> I have three questions:
>> 1. Is this "lost connection" behavior in the isql-v environment normal?
>> 2. Does this "lost connection" message mean that the transaction has
>> rolled-back or is the command still running?
>> 3. Is there a better way to run long-time queries against virtuoso?
>>
>> Kind regards,
>> Pantelis Natsiavas
>>
>
> 
> --
> What NetFlow Analyzer can do for you? Monitors network bandwidth and
> traffic
> patterns at an interface-level. Reveals which users, apps, and protocols
> are
> consuming the most bandwidth. Provides multi-vendor support for NetFlow,
> J-Flow, 

[Virtuoso-users] URL Rewrite

[Jürgen Jakobitsch]

hi,

i'm having trouble creating a suitable url-rewrite, can someone help me
with this?

i have a virtual directory (/search) which i want to redirect to a sparql
fulltextquery, but in the url rewrite's "destination path format" i don't
know how to access a url-parameter.

what i want to do is to redirect from

/search?query=myquerystring

to

/sparql=SELECT * WHERE { ?s ?p ?o
FILTER(REGEX(STR(?o),'myquerystring','i'))}

any pointer greatly appreciated

krj


*Jürgen Jakobitsch*
Innovation Director
Semantic Web Company GmbH
EU: +43-1-4021235-0
Mobile: +43-676-6212710
http://www.semantic-web.at
http://www.poolparty.biz


PERSONAL INFORMATION
| web   : http://www.turnguard.com
| foaf  : http://www.turnguard.com/turnguard
| g+: https://plus.google.com/111233759991616358206/posts
| skype : jakobitsch-punkt
| xmlns:tg  = "http://www.turnguard.com/turnguard#";
--
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are 
consuming the most bandwidth. Provides multi-vendor support for NetFlow, 
J-Flow, sFlow and other flows. Make informed decisions using capacity 
planning reports. http://sdm.link/zohodev2dev___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

[Virtuoso-users] DESCRIBE does not return single owl:sameAs

[Adrian Gschwend]

Hi list,

I provide some URIs in Virtuoso which consist only of one triple,
pointing to the main resource in the same store via owl:sameAs:

http://lindas-data.ch/sparql?default-graph-uri=&query=CONSTRUCT+WHERE+%7B%0D%0A%0D%0A%3Chttp%3A%2F%2Flod.opentransportdata.swiss%2Fdidok%2FBSGB%3E+%3Fp+%3Fo%0D%0A%7D&should-sponge=&format=text%2Fhtml&timeout=0&debug=on

SELECT & CONSTRUCT work fine and I get the triple back. If I do the same
with DESCRIBE I get an empty resource back.

Sample: DESCRIBE 

Endpoint: http://lindas-data.ch/sparql


Is that as designed and if so, how can I force to give the triple back
anyways? Can't use it for dereferencing that way.

regards

Adrian

--
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are 
consuming the most bandwidth. Provides multi-vendor support for NetFlow, 
J-Flow, sFlow and other flows. Make informed decisions using capacity 
planning reports. http://sdm.link/zohodev2dev
___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users

[Virtuoso-users] Debugging JDBC Virtuoso connection

[Davis, Daniel (NIH/NLM) [C]]

I've done my UI using Java Spring, but so far, I've been doing it as an old 
dinosaur, running my build and my changes on the Linux host where Virtuoso is 
running.
Now, I'm trying to use Virtuoso remotely from another host.

>From Python, I can connect to port , but when I try to connect to Virtuoso 
>from Java, it does not seem to be working.

I would prefer to enable some logging from Virtuoso JDBC to see what the 
problem is - it is throwing connection refused, but the connection should be 
open.
What is it actually trying to connect with?

Thanks,

Dan Davis, Systems/Applications Architect (Contractor),
Office of Computer and Communications Systems,
National Library of Medicine, NIH

--
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are 
consuming the most bandwidth. Provides multi-vendor support for NetFlow, 
J-Flow, sFlow and other flows. Make informed decisions using capacity 
planning reports. http://sdm.link/zohodev2dev___
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users