Hi Jörn,
I don’t see any indication of you having provided a copy of your “virtuoso.ini”
thus can you please provide one ?
So you have about 6billion triples with 128GB RAM available , which should be
sufficient.
Can you provide some details of database space consumption by querying the
DB.DBA.SYS_INDEX_SPACE_STATS view as detailed at:
http://docs.openlinksw.com/virtuoso/databaseadmsrv.html#statusfunc
and also query the DB.DBA.sys_col_info to obtain information on column store
space utlliization as detailed at:
http://docs.openlinksw.com/virtuoso/coredbengine.html#colstorespaceutil
The second status output after about 5mins shows "10900000 buffers, 3662255
used," what activities would have been performed on the database up to this
time ie would you have executed the typical query work load to run on the
system such that the database fully warmed up to that point only, as only about
a third of the allocated buffers are in use ?
The following error in the log you asked about can be ignored as they are not
dangerous and more for information as the server has corrected the condition:
04:32:47 cpt/lt unusual cond rltrx.c:1437
15:01:58 The __sys_ext_map free pages incorrect 270 != 140 actually free
For the following errors:
12:54:51 Checkpoint removed 2004 MB of remapped pages, leaving 0 MB.
Duration 18.78 s. To save this time, increase MaxCheckpointRemap and/or
set Unremap quota to 0 in ini file.
It is recommended the MaxCheckpointRemap is set to 1/4 of the database size,
thus what is yours set to ? See:
http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtRDFPerformanceTuning
http://docs.openlinksw.com/virtuoso/checkpoint.html#checkpointparams
Given the size of the database do you have striping enabled especially if you
have multiple file systems the database can be striped across as this can
improve performance of IO operations to disk, see:
http://docs.openlinksw.com/virtuoso/databaseadmsrv.html#ini_Striping
http://docs.openlinksw.com/virtuoso/databaseadmsrv.html#IOQS
What type of disks are being used are they SSD or normal or other ?
Best Regards
Hugh Williams
Professional Services
OpenLink Software, Inc. // http://www.openlinksw.com/
Weblog -- http://www.openlinksw.com/blogs/
LinkedIn -- http://www.linkedin.com/company/openlink-software/
Twitter -- http://twitter.com/OpenLink
Google+ -- http://plus.google.com/100570109519069333827/
Facebook -- http://www.facebook.com/OpenLinkSoftware
Universal Data Access, Integration, and Management Technology Providers
> On 14 Nov 2015, at 17:52, Jörn Hees <j_h...@cs.uni-kl.de> wrote:
>
> Hi Hugh,
>
>
>> If all datasets are indicate to have loaded with ll_state = 2 and not errors
>> reported in ll_error then the datasets have been loaded successfully.
>
> ok, cool :)
>
>
>> Did you run the checkpoint command when the bulk load completed as that
>> should always be done ? Also after restart is the high CPU consumption still
>> occurring ?
>
> yes, i ran "checkpoint; commit work; checkpoint;" a couple of times already,
> but after a short while the DB is busy again... the same after a full restart
> of the machine.
> I attached the log since restart yesterday at the end of this mail (including
> some comments)... at start time the instance contained the DBpedia EN + DE +
> pagelinks dumps.
> I then started to load the Freebase, LinkedGeoData and Wikidata dumps with a
> single loader.
> Afterwards the endpoint held ~ 5.96 G triples.
>
>
>> What does the “status();” command run from isql as that will show the
>> status of the machine and what might be consuming resources.
>
> Actually funny, cause when i ran that the CPU load instantly seems to have
> dropped to 0 and no IO activity anymore, but the command never returned...
> i'm waiting for > 10 minutes now:
> OpenLink Interactive SQL (Virtuoso), version 0.9849b.
> Type HELP; for help and EXIT; to exit.
> SQL> status();
> Connected to OpenLink Virtuoso
> Driver: 07.20.3214 OpenLink Virtuoso ODBC Driver
>
> I opened some other isql-vt sessions but since then the DB doesn't respond at
> all.
> It also doesn't seem to respond to "/etc/init.d/virtuoso-opensource-7 stop"
> or "sudo kill ..." within 5 Minutes...
> After that i tried several other kill signals like SIGHUP, SIGINT and i think
> SIGUSR1 finally did the trick.
> Starting took awefully long and seems to have replayed a lot of transactions
> which i don't fully understand as there were several checkpoints since the
> import...
> Should i rather use the backup from before?
>
>
------------------------------------------------------------------------------
_______________________________________________
Virtuoso-users mailing list
Virtuoso-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/virtuoso-users
