Hi Pantelis, The error in the log snippet you provided:
19:49:11 It is impossible to have a database file
/media/VirtuosoDBDrive/virtuoso.db with a length not multiple of 2MB.
19:49:11 The process must have last terminated while growing the file.
19:49:11 Please contact OpenLink Customer Support
implies that Virtuoso probably crashed or shutdown unexpectedly when Virtuoso
was in the middle of increasing the database size by 2MB segment. Although is
seems to the start successfully as indicated in the log, but should run a
database integrity check to ensure there is no corruption of the database by
running the following from isql:
backup ‘/dev/null’;
See, http://docs.openlinksw.com/virtuoso/databaseadmsrv.html#diagnosingrepairing
Also, are any reports in the system log ie /var/log/messages to indicate a
problem with Virtuoso ?
Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc. // http://www.openlinksw.com/
Weblog -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter -- http://twitter.com/OpenLink
Google+ -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers
> On 14 Jul 2016, at 07:54, Pantelis Natsiavas <natsia...@gmail.com> wrote:
>
> Hi Hugh.
>
> The part of interest in the log file is currently attached. Sorry for not
> attaching it in my previous email. I have reduced only to the log messages
> that could be of interest, in the specific time span, around "Please contact
> OpenLink Customer Support" message.
>
> Yes I have changed the swappiness parameter to vm.swappiness = 10. I have not
> yet managed to use the debugger as the above error occurs. I will not insist
> on that.
>
> The status(); response has been copied and pasted as it came out in the
> isql-v command line.
>
> Actually, what happened in my last two tries executing the above, is that the
> SPARQL query executed (partially) as isql-v stopped the transaction because
> the "connection with the server had been lost" (or something like that).
>
> Thank you once more for your time.
>
> Kind regards,
> Pantelis Natsiavas
>
> 2016-07-12 14:30 GMT+03:00 Hugh Williams <hwilli...@openlinksw.com
> <mailto:hwilli...@openlinksw.com>>:
> Hi Pantelis,
>
> I don’t see the indicated attachment to your email with the virtuoso log ?
>
> Is that status(); command output complete as it normally should the CPU %
> being consume by the server when running and apart from the status command,
> I don’t see any other processes indicated as running ?
>
> Note you can use the Virtuoso PL Debugger to see what procedures/statements
> are running on the server continuously as follows:
>
> $ isql 1111 dba <pwd> -D
> OpenLink Interactive PL Debugger (Virtuoso).
> Type EXIT to exit, HELP for help
> Debug session estabilished to 1111
> DEBUG> info threads
> @1111:4 in DB.DBA.test_proc1 () at 11
> DEBUG>
>
> See the following documentation of additional details on usage:
>
> http://docs.openlinksw.com/virtuoso/pldebugger.html#pldebugger
> <http://docs.openlinksw.com/virtuoso/pldebugger.html#pldebugger>
>
>
> Also, do you have the Linux swappiness parameter set to minimise swapping to
> disk which will degrade performance as detailed at:
>
>
> http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtRDFPerformanceTuning#Linux-only%20--%20”swappiness
>
> <http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtRDFPerformanceTuning#Linux-only%20--%20%E2%80%9Dswappiness>”
>
>
> Best Regards
> Hugh Williams
> Professional Services
> OpenLink Software, Inc. // http://www.openlinksw.com/
> <http://www.openlinksw.com/>
> Weblog -- http://www.openlinksw.com/blogs/
> <http://www.openlinksw.com/blogs/>
> LinkedIn -- http://www.linkedin.com/company/openlink-software/
> <http://www.linkedin.com/company/openlink-software/>
> Twitter -- http://twitter.com/OpenLink <http://twitter.com/OpenLink>
> Google+ -- http://plus.google.com/100570109519069333827/
> <http://plus.google.com/100570109519069333827/>
> Facebook -- http://www.facebook.com/OpenLinkSoftware
> <http://www.facebook.com/OpenLinkSoftware>
> Universal Data Access, Integration, and Management Technology Providers
>
>> On 7 Jul 2016, at 15:46, Pantelis Natsiavas <natsia...@gmail.com
>> <mailto:natsia...@gmail.com>> wrote:
>>
>> Thank you Hugh for your answer.
>>
>> Virtuoso runs on a VM with 32GBytes of RAM. This VM is dedicated to the
>> specific virtuoso instance and hosted in cloud infrastructure.
>>
>> I have left my transaction to run until Tuesday (about 4 days !!!) and still
>> had no response. The VM reported 95% usage of memory but still no response.
>> The response in the status() command was the following:
>> OpenLink Virtuoso Server
>> Version 07.20.3214-pthreads for Linux as of Oct 14 2015
>> Started on: 2016-07-02 16:13 GMT+3
>>
>> Database Status:
>> File size 0, 10407936 pages, 3793820 free.
>> 2720000 buffers, 130665 used, 4 dirty 0 wired down, repl age 0 0 w. io 0
>> w/crsr.
>> Disk Usage: 131941 reads avg 0 msec, 0% r 0% w last 0 s, 31533 writes
>> flush 4.255 MB,
>> 562 read ahead, batch = 224. Autocompact 0 in 0 out, 0% saved.
>> Gate: 388 2nd in reads, 0 gate write waits, 0 in while read 0 busy scrap.
>> Log = /usr/local/var/lib/virtuoso/db/virtuoso.trx, 4146 bytes
>> 6613693 pages have been changed since last backup (in checkpoint state)
>> Current backup timestamp: 0x0000-0x00-0x00
>> Last backup date: unknown
>> Clients: 1 connects, max 1 concurrent
>> RPC: 8 calls, 1 pending, 1 max until now, 0 queued, 0 burst reads (0%), 0
>> second 0M large, 22M max
>> Checkpoint Remap 0 pages, 0 mapped back. 35 s atomic time.
>> DB master 10407936 total 3793820 free 0 remap 0 mapped back
>> temp 256 total 251 free
>>
>> Lock Status: 0 deadlocks of which 0 2r1w, 39 waits,
>> Currently 1 threads running 0 threads waiting 0 threads in vdb.
>> Pending:
>>
>> Client 1111:1:-3: Account: dba, 215 bytes in, 289 bytes out, 1 stmts.
>> PID: 30445, OS: unix, Application: unknown, IP#: 127.0.0.1
>> Transaction status: PENDING, 1 threads.
>> Locks:
>>
>>
>> Running Statements:
>> Time (msec) Text
>> 567 status()
>>
>>
>> Hash indexes
>>
>>
>> On Wednesday, I restarted the VM in order to make sure that the transaction
>> was cleared, and executed the query through the isql-v while enabling log.
>> The usage of memory is currently about 60% and I got plenty of log messages
>> that I cannot really interpret. I attach the log file in the current email.
>> Perhaps, you could provide me with some guidance on what I should do, based
>> on the log file.
>>
>> Please note the following:
>>
>> The values in the virtuoso.ini configuration file are:
>> NumberOfBuffers = 2720000
>> MaxDirtyBuffers = 2000000
>>
>> The response in the status(); command in the SQL is the following:
>>
>> OpenLink Virtuoso Server
>> Version 07.20.3214-pthreads for Linux as of Oct 14 2015
>> Started on: 2016-07-06 20:17 GMT+3
>>
>> Database Status:
>> File size 0, 11027968 pages, 3947693 free.
>> 2720000 buffers, 483125 used, 31 dirty 0 wired down, repl age 0 0 w. io 0
>> w/crsr.
>> Disk Usage: 472040 reads avg 0 msec, 0% r 0% w last 0 s, 936646 writes
>> flush 10.35 MB,
>> 2533 read ahead, batch = 168. Autocompact 530282 in 446316 out, 15%
>> saved col ac: 45742 in 12% saved.
>> Gate: 446 2nd in reads, 0 gate write waits, 0 in while read 0 busy scrap.
>> Log = /usr/local/var/lib/virtuoso/db/virtuoso.trx, 1403 bytes
>> 7079218 pages have been changed since last backup (in checkpoint state)
>> Current backup timestamp: 0x0000-0x00-0x00
>> Last backup date: unknown
>> Clients: 1 connects, max 1 concurrent
>> RPC: 6 calls, 1 pending, 1 max until now, 0 queued, 0 burst reads (0%), 0
>> second 0M large, 152M max
>> Checkpoint Remap 589 pages, 0 mapped back. 464 s atomic time.
>> DB master 11027968 total 3947693 free 589 remap 2 mapped back
>> temp 256 total 251 free
>>
>> Lock Status: 0 deadlocks of which 0 2r1w, 20 waits,
>> Currently 1 threads running 0 threads waiting 0 threads in vdb.
>> Pending:
>>
>> Client 1111:1:-2: Account: dba, 215 bytes in, 289 bytes out, 1 stmts.
>> PID: 10273, OS: unix, Application: unknown, IP#: 127.0.0.1
>> Transaction status: PENDING, 1 threads.
>> Locks:
>>
>>
>> Running Statements:
>> Time (msec) Text
>> 673 status()
>>
>>
>> Hash indexes
>>
>> Thank you in advance for your help.
>>
>> Kind regards,
>> Pantelis Natsiavas
>>
>>
>>
>>
>> 2016-07-02 23:22 GMT+03:00 Hugh Williams <hwilli...@openlinksw.com
>> <mailto:hwilli...@openlinksw.com>>:
>> Hi Pantelis,
>>
>> SPARUL operations are performed as part of a transaction, which is stored in
>> memory until the operation is completed and committed to the database, and
>> for such a large graph would consume a significant amount system memory,
>> possibly all available memory and not being able to complete. Thus how much
>> memory is available on the system and how much is allocated for Virtuoso
>> data buffers ie NumberOfBuffers INI file param and how much memory is left ?
>> You can run the “status();” command from isql to see the Buffers setting and
>> other Virtuoso server status metrics.
>>
>> You can also try prepending the query with the "DEFINE sql:log-enable 3”
>> pragma which sets transaction log mode to autocommit during the insert
>> significantly reducing the memory requirement which may then allow the
>> operation to complete, as is required when deleting large graphs as
>> documented at:
>>
>>
>> http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtTipsAndTricksGuideDeleteLargeGraphs
>>
>> <http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtTipsAndTricksGuideDeleteLargeGraphs>
>>
>> I also note you are running a 3214 build which is from June 2015 and would
>> recommend updating to the latest 3217 build from April 2016 …
>>
>> Best Regards
>> Hugh Williams
>> Professional Services
>> OpenLink Software, Inc. // http://www.openlinksw.com/
>> <http://www.openlinksw.com/>
>> Weblog -- http://www.openlinksw.com/blogs/
>> <http://www.openlinksw.com/blogs/>
>> LinkedIn -- http://www.linkedin.com/company/openlink-software/
>> <http://www.linkedin.com/company/openlink-software/>
>> Twitter -- http://twitter.com/OpenLink <http://twitter.com/OpenLink>
>> Google+ -- http://plus.google.com/100570109519069333827/
>> <http://plus.google.com/100570109519069333827/>
>> Facebook -- http://www.facebook.com/OpenLinkSoftware
>> <http://www.facebook.com/OpenLinkSoftware>
>> Universal Data Access, Integration, and Management Technology Providers
>>
>> > On 2 Jul 2016, at 15:06, Pantelis Natsiavas <natsia...@gmail.com
>> > <mailto:natsia...@gmail.com>> wrote:
>> >
>> > Hi everybody.
>> >
>> > I am trying to execute the following query
>> >
>> > INSERT INTO GRAPH <targetGraph> {
>> > ?a <someProperty> ?b
>> > }
>> > WHERE {
>> > GRAPH <sourceGraph> {
>> > ?a <someOtherProperty> ?c .
>> > BIND(REPLACE(STR(?a),"something:", "", "i") AS ?b) }
>> > }
>> >
>> > The sourceGraph contains about 94 million triplets. When I execute the
>> > above SPARQL, I get no error, however, I get no response either. I check
>> > after some minutes and the <targetGraph> has not yet been created. The
>> > system acts as if I had never executed the statement.
>> >
>> > I have tried executing the above through /sparql and through the
>> > /conductor web interfaces.
>> >
>> > Is there a way to see what is really going on? Am I doing something wrong?
>> >
>> > I have also already tried to debug through the isql-v localhost:8890 dba
>> > <password> -D but I get a totally uncomprehensible error:
>> >
>> > isql-v() [0x44986a]
>> > isql-v() [0x4498c8]
>> > isql-v() [0x44b00e]
>> > isql-v() [0x42fc65]
>> > isql-v() [0x4108b0]
>> > isql-v() [0x4043ee]
>> > /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7fc06a645ec5]
>> > isql-v() [0x40441e]
>> > GPF: Dkernel.c:3080 can't free if in served sessions
>> > Segmentation fault (core dumped)
>> >
>> > Please note that I am using Version: 07.20.3214, Build: Oct 14 2015 on
>> > ubuntu server.
>> >
>> > Kind regards,
>> > Pantelis Natsiavas
>> > ------------------------------------------------------------------------------
>> > Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
>> > Francisco, CA to explore cutting-edge tech and listen to tech luminaries
>> > present their vision of the future. This family event has something for
>> > everyone, including kids. Get more information and register today.
>> > http://sdm.link/attshape_______________________________________________
>> > <http://sdm.link/attshape_______________________________________________>
>> > Virtuoso-users mailing list
>> > Virtuoso-users@lists.sourceforge.net
>> > <mailto:Virtuoso-users@lists.sourceforge.net>
>> > https://lists.sourceforge.net/lists/listinfo/virtuoso-users
>> > <https://lists.sourceforge.net/lists/listinfo/virtuoso-users>
>>
>>
>> ------------------------------------------------------------------------------
>> Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
>> Francisco, CA to explore cutting-edge tech and listen to tech luminaries
>> present their vision of the future. This family event has something for
>> everyone, including kids. Get more information and register today.
>> http://sdm.link/attshape_______________________________________________
>> <http://sdm.link/attshape_______________________________________________>
>> Virtuoso-users mailing list
>> Virtuoso-users@lists.sourceforge.net
>> <mailto:Virtuoso-users@lists.sourceforge.net>
>> https://lists.sourceforge.net/lists/listinfo/virtuoso-users
>> <https://lists.sourceforge.net/lists/listinfo/virtuoso-users>
>
>
> <virtuoso.log.part>
smime.p7s
Description: S/MIME cryptographic signature
------------------------------------------------------------------------------ What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic patterns at an interface-level. Reveals which users, apps, and protocols are consuming the most bandwidth. Provides multi-vendor support for NetFlow, J-Flow, sFlow and other flows. Make informed decisions using capacity planning reports.http://sdm.link/zohodev2dev
_______________________________________________ Virtuoso-users mailing list Virtuoso-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/virtuoso-users
