Hi Pantelis, I don’t see the indicated attachment to your email with the virtuoso log ?
Is that status(); command output complete as it normally should the CPU %
being consume by the server when running and apart from the status command, I
don’t see any other processes indicated as running ?
Note you can use the Virtuoso PL Debugger to see what procedures/statements are
running on the server continuously as follows:
$ isql 1111 dba <pwd> -D
OpenLink Interactive PL Debugger (Virtuoso).
Type EXIT to exit, HELP for help
Debug session estabilished to 1111
DEBUG> info threads
@1111:4 in DB.DBA.test_proc1 () at 11
DEBUG>
See the following documentation of additional details on usage:
http://docs.openlinksw.com/virtuoso/pldebugger.html#pldebugger
<http://docs.openlinksw.com/virtuoso/pldebugger.html#pldebugger>
Also, do you have the Linux swappiness parameter set to minimise swapping to
disk which will degrade performance as detailed at:
http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtRDFPerformanceTuning#Linux-only%20--%20”swappiness
<http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtRDFPerformanceTuning#Linux-only%20--%20%E2%80%9Dswappiness>”
Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc. // http://www.openlinksw.com/
Weblog -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter -- http://twitter.com/OpenLink
Google+ -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers
> On 7 Jul 2016, at 15:46, Pantelis Natsiavas <natsia...@gmail.com> wrote:
>
> Thank you Hugh for your answer.
>
> Virtuoso runs on a VM with 32GBytes of RAM. This VM is dedicated to the
> specific virtuoso instance and hosted in cloud infrastructure.
>
> I have left my transaction to run until Tuesday (about 4 days !!!) and still
> had no response. The VM reported 95% usage of memory but still no response.
> The response in the status() command was the following:
> OpenLink Virtuoso Server
> Version 07.20.3214-pthreads for Linux as of Oct 14 2015
> Started on: 2016-07-02 16:13 GMT+3
>
> Database Status:
> File size 0, 10407936 pages, 3793820 free.
> 2720000 buffers, 130665 used, 4 dirty 0 wired down, repl age 0 0 w. io 0
> w/crsr.
> Disk Usage: 131941 reads avg 0 msec, 0% r 0% w last 0 s, 31533 writes
> flush 4.255 MB,
> 562 read ahead, batch = 224. Autocompact 0 in 0 out, 0% saved.
> Gate: 388 2nd in reads, 0 gate write waits, 0 in while read 0 busy scrap.
> Log = /usr/local/var/lib/virtuoso/db/virtuoso.trx, 4146 bytes
> 6613693 pages have been changed since last backup (in checkpoint state)
> Current backup timestamp: 0x0000-0x00-0x00
> Last backup date: unknown
> Clients: 1 connects, max 1 concurrent
> RPC: 8 calls, 1 pending, 1 max until now, 0 queued, 0 burst reads (0%), 0
> second 0M large, 22M max
> Checkpoint Remap 0 pages, 0 mapped back. 35 s atomic time.
> DB master 10407936 total 3793820 free 0 remap 0 mapped back
> temp 256 total 251 free
>
> Lock Status: 0 deadlocks of which 0 2r1w, 39 waits,
> Currently 1 threads running 0 threads waiting 0 threads in vdb.
> Pending:
>
> Client 1111:1:-3: Account: dba, 215 bytes in, 289 bytes out, 1 stmts.
> PID: 30445, OS: unix, Application: unknown, IP#: 127.0.0.1
> Transaction status: PENDING, 1 threads.
> Locks:
>
>
> Running Statements:
> Time (msec) Text
> 567 status()
>
>
> Hash indexes
>
>
> On Wednesday, I restarted the VM in order to make sure that the transaction
> was cleared, and executed the query through the isql-v while enabling log.
> The usage of memory is currently about 60% and I got plenty of log messages
> that I cannot really interpret. I attach the log file in the current email.
> Perhaps, you could provide me with some guidance on what I should do, based
> on the log file.
>
> Please note the following:
>
> The values in the virtuoso.ini configuration file are:
> NumberOfBuffers = 2720000
> MaxDirtyBuffers = 2000000
>
> The response in the status(); command in the SQL is the following:
>
> OpenLink Virtuoso Server
> Version 07.20.3214-pthreads for Linux as of Oct 14 2015
> Started on: 2016-07-06 20:17 GMT+3
>
> Database Status:
> File size 0, 11027968 pages, 3947693 free.
> 2720000 buffers, 483125 used, 31 dirty 0 wired down, repl age 0 0 w. io 0
> w/crsr.
> Disk Usage: 472040 reads avg 0 msec, 0% r 0% w last 0 s, 936646 writes
> flush 10.35 MB,
> 2533 read ahead, batch = 168. Autocompact 530282 in 446316 out, 15%
> saved col ac: 45742 in 12% saved.
> Gate: 446 2nd in reads, 0 gate write waits, 0 in while read 0 busy scrap.
> Log = /usr/local/var/lib/virtuoso/db/virtuoso.trx, 1403 bytes
> 7079218 pages have been changed since last backup (in checkpoint state)
> Current backup timestamp: 0x0000-0x00-0x00
> Last backup date: unknown
> Clients: 1 connects, max 1 concurrent
> RPC: 6 calls, 1 pending, 1 max until now, 0 queued, 0 burst reads (0%), 0
> second 0M large, 152M max
> Checkpoint Remap 589 pages, 0 mapped back. 464 s atomic time.
> DB master 11027968 total 3947693 free 589 remap 2 mapped back
> temp 256 total 251 free
>
> Lock Status: 0 deadlocks of which 0 2r1w, 20 waits,
> Currently 1 threads running 0 threads waiting 0 threads in vdb.
> Pending:
>
> Client 1111:1:-2: Account: dba, 215 bytes in, 289 bytes out, 1 stmts.
> PID: 10273, OS: unix, Application: unknown, IP#: 127.0.0.1
> Transaction status: PENDING, 1 threads.
> Locks:
>
>
> Running Statements:
> Time (msec) Text
> 673 status()
>
>
> Hash indexes
>
> Thank you in advance for your help.
>
> Kind regards,
> Pantelis Natsiavas
>
>
>
>
> 2016-07-02 23:22 GMT+03:00 Hugh Williams <hwilli...@openlinksw.com
> <mailto:hwilli...@openlinksw.com>>:
> Hi Pantelis,
>
> SPARUL operations are performed as part of a transaction, which is stored in
> memory until the operation is completed and committed to the database, and
> for such a large graph would consume a significant amount system memory,
> possibly all available memory and not being able to complete. Thus how much
> memory is available on the system and how much is allocated for Virtuoso data
> buffers ie NumberOfBuffers INI file param and how much memory is left ? You
> can run the “status();” command from isql to see the Buffers setting and
> other Virtuoso server status metrics.
>
> You can also try prepending the query with the "DEFINE sql:log-enable 3”
> pragma which sets transaction log mode to autocommit during the insert
> significantly reducing the memory requirement which may then allow the
> operation to complete, as is required when deleting large graphs as
> documented at:
>
>
> http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtTipsAndTricksGuideDeleteLargeGraphs
>
> <http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtTipsAndTricksGuideDeleteLargeGraphs>
>
> I also note you are running a 3214 build which is from June 2015 and would
> recommend updating to the latest 3217 build from April 2016 …
>
> Best Regards
> Hugh Williams
> Professional Services
> OpenLink Software, Inc. // http://www.openlinksw.com/
> <http://www.openlinksw.com/>
> Weblog -- http://www.openlinksw.com/blogs/
> <http://www.openlinksw.com/blogs/>
> LinkedIn -- http://www.linkedin.com/company/openlink-software/
> <http://www.linkedin.com/company/openlink-software/>
> Twitter -- http://twitter.com/OpenLink <http://twitter.com/OpenLink>
> Google+ -- http://plus.google.com/100570109519069333827/
> <http://plus.google.com/100570109519069333827/>
> Facebook -- http://www.facebook.com/OpenLinkSoftware
> <http://www.facebook.com/OpenLinkSoftware>
> Universal Data Access, Integration, and Management Technology Providers
>
> > On 2 Jul 2016, at 15:06, Pantelis Natsiavas <natsia...@gmail.com
> > <mailto:natsia...@gmail.com>> wrote:
> >
> > Hi everybody.
> >
> > I am trying to execute the following query
> >
> > INSERT INTO GRAPH <targetGraph> {
> > ?a <someProperty> ?b
> > }
> > WHERE {
> > GRAPH <sourceGraph> {
> > ?a <someOtherProperty> ?c .
> > BIND(REPLACE(STR(?a),"something:", "", "i") AS ?b) }
> > }
> >
> > The sourceGraph contains about 94 million triplets. When I execute the
> > above SPARQL, I get no error, however, I get no response either. I check
> > after some minutes and the <targetGraph> has not yet been created. The
> > system acts as if I had never executed the statement.
> >
> > I have tried executing the above through /sparql and through the /conductor
> > web interfaces.
> >
> > Is there a way to see what is really going on? Am I doing something wrong?
> >
> > I have also already tried to debug through the isql-v localhost:8890 dba
> > <password> -D but I get a totally uncomprehensible error:
> >
> > isql-v() [0x44986a]
> > isql-v() [0x4498c8]
> > isql-v() [0x44b00e]
> > isql-v() [0x42fc65]
> > isql-v() [0x4108b0]
> > isql-v() [0x4043ee]
> > /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7fc06a645ec5]
> > isql-v() [0x40441e]
> > GPF: Dkernel.c:3080 can't free if in served sessions
> > Segmentation fault (core dumped)
> >
> > Please note that I am using Version: 07.20.3214, Build: Oct 14 2015 on
> > ubuntu server.
> >
> > Kind regards,
> > Pantelis Natsiavas
> > ------------------------------------------------------------------------------
> > Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
> > Francisco, CA to explore cutting-edge tech and listen to tech luminaries
> > present their vision of the future. This family event has something for
> > everyone, including kids. Get more information and register today.
> > http://sdm.link/attshape_______________________________________________
> > <http://sdm.link/attshape_______________________________________________>
> > Virtuoso-users mailing list
> > Virtuoso-users@lists.sourceforge.net
> > <mailto:Virtuoso-users@lists.sourceforge.net>
> > https://lists.sourceforge.net/lists/listinfo/virtuoso-users
> > <https://lists.sourceforge.net/lists/listinfo/virtuoso-users>
>
>
> ------------------------------------------------------------------------------
> Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
> Francisco, CA to explore cutting-edge tech and listen to tech luminaries
> present their vision of the future. This family event has something for
> everyone, including kids. Get more information and register today.
> http://sdm.link/attshape_______________________________________________
> Virtuoso-users mailing list
> Virtuoso-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/virtuoso-users
smime.p7s
Description: S/MIME cryptographic signature
------------------------------------------------------------------------------ What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic patterns at an interface-level. Reveals which users, apps, and protocols are consuming the most bandwidth. Provides multi-vendor support for NetFlow, J-Flow, sFlow and other flows. Make informed decisions using capacity planning reports.http://sdm.link/zohodev2dev
_______________________________________________ Virtuoso-users mailing list Virtuoso-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/virtuoso-users
