Hi Alex,
I would definitely say your Windows Vista 32bit machine is under
resourced for hosting a fully loaded DBpedia instance in
Virtuoso,
being
basically a desktop spec machine. I would recommend a higher
spec.
machine and a Server based OS ie Windows Server 2003 or 2008
or Linux
which is what most would typically setup DBpedia on. You could
also
consider what DBpedia datasets are required for you need to
load a
minimum set requiring less resources.
Assuming you are using the settings in the default dbpeda.ini
file
the
NumberOfBuffers parameter is set to 100000 meaning some 800MB of
memory
will be used by the Virtuoso Server, and it that much memory
is not
available when the server attempts to allocate it this would
account for
the out of memory errors you have been receiving.
Personally I would recommend if possible you setup a 64bit
Linux or
Windows Server machine with at least 8GB of RAM as
receommended. You
also need a new set of binaries with the fixes for the "Can
not set
NULL
to not nullable column 'DB.DBA.RDF_QUAD.O" errors. If you
intend to
use
a Linux build we have a patched archive snap shot that can be
made
available requiring that you compile and build the binaries. On
Windows
you would need to wait for us to schedule a new set of open
source
binaries for this OS.
Best Regards
Hugh Williams
Professional Services
OpenLink Software
Web: http://www.openlinksw.com
Support: http://support.openlinksw.com
Forums: http://boards.openlinksw.com/support
On 20 Sep 2009, at 01:39, Alex wrote:
Hi Hugh,
Yeah, on second thought I was rather ignoring those errors
mainly
because I didn't want to have to deal with them!
Here are the details of the machine on which I am installing:
1. Windows Vista Ultimate 32-bit
2. 2GB physical RAM
3. Intel(R) Core(TM)2 CPU @ 2.13 GHz
Now that you say the minimum for Amazon EC2 hosting is a
machine with
8GB memory, I'm starting to see that the specs may be a
problem. It
didn't actually cross my mind initially, since this machine was
relatively high-end for 3 years ago when I bought it. Optimism
again,
perhaps...
Finally, here is the version info I got from running `virtuoso-
t --
help`.
Virtuoso Open Source Edition (multi threaded)
Version 5.0.11.3039-threads as of Apr 21 2009
Compiled for 32 Bit Windows Operating Environments
Copyright (C) 1999-2009 OpenLink Software
Anyway, thanks for continued speedy replies and sticking with
me on
all
the issues. It has certainly not been a straightforward matter
to
troubleshoot, and I'm just hoping there will be some fruits to
these
labours. If it helps at all, I should probably state that my
overall aim
is simply to set up a DBpedia SPARQL endpoint that can serve
over
1000
results per query (the current restriction on the DBpedia.org
endpoint).
Regards,
Alex
Hugh Williams wrote:
Hi Alex,
I would say the "out of memory" errors in the installer log
are
a bit
more than minor, as the system would appear not to have the
resources
for hosting the datasets being loaded.
What are the details of the machine you are attempting to
host
your
DBpedia instance on:
1. What is OS name and bit'ness ? You mentioned using
cygwin,
thus I
presume you are on Windows ? What windows variant and is it
32
or 64
bit ?
2. What is available memory consumption ? I believe you
said 2GB
in a
previous post ?
3. What is type and speed of Processor ?
For hosting the DBpedia datasets on Amazon EC2 the 3.2
datasets
we have
currently recommend a minimum Linux x86_64 machine with 8GB
memory.
I also not a few "Can not set NULL to not nullable column
'DB.DBA.RDF_QUAD.O" errors in your load log which is an issue
that has
been fixed in our latest builds thus you would benefit from
an
updated
build also, as those datasets have probably failed to load.
Please confirm the version number of build date of the server
you are
using.
Best Regards
Hugh Williams
Professional Services
OpenLink Software
Web: http://www.openlinksw.com
Support: http://support.openlinksw.com
Forums: http://boards.openlinksw.com/support
On 19 Sep 2009, at 18:31, Alex wrote:
Hi Hugh,
I suspected you'd ask for them... no idea how pertinent
they are
to the
issue though. I've attached the console log and the load log
that was
generated.
Cheers,
Alex
Hugh Williams wrote:
Hi Alex,
What are the minor errors you indicate having occurred ?
The SPARQL query timeout is set to 60 by default, thus you
might
want to
try increasing the MaxQueryExecutionTime paramter in the
[SPARQL]
section of the dbpedia.ini file which controls the
timeouts and
restart
the server.
Should the problems persist please try enabling tracing on
the server
as detailed at:
http://docs.openlinksw.com/virtuoso/fn_trace_on.html
This will write trace info to the dbpedia.log file, please
provide
this
and the dbpedia.ini file for analysis.
You should also try running the same query via the
Virtuoso
isql
problem, preceded with the "sparql" keyword to direct it
to the
SPARQL
processor ...
Best Regards
Hugh Williams
Professional Services
OpenLink Software
Web: http://www.openlinksw.com
Support: http://support.openlinksw.com
Forums: http://boards.openlinksw.com/support
On 19 Sep 2009, at 16:19, Alex wrote:
Hi Hugh,
The installation seems to have completed fine now (a few
minor
errors),
and I've just tested the SPARQL endpoint in the browser,
but
the
server
seems to be complaining.
S1T00 Error SR171: Transaction timed out
SPARQL query:
...
I received this message for the recommended sample query
in
README.txt,
as well as several others. Any suggestions please?
Regards,
Alex
Started.
Checking files
load_nt.sh : present
dbpedia_dav.vad : present
dbpedia_post.sql : present
dbpedia-ontology.owl : present
umbel_class_hierarchy_v071.n3 : present
umbel_abstract_concepts.n3 : present
umbel_external_ontologies_linkage.n3 : present
yago-class-hierarchy_en.nt : present
umbel_subject_concepts.n3 : present
opencyc-2008-06-10.owl : present
opencyc-2008-06-10-readable.owl : present
Installing dbpedia_dav.vad
Starting...
Logging into: load_nt.sh.log
=======================================
Loading started.
=======================================
Loading data/externallinks_en.nt (3653630 lines) 13:25:50
Retryng from line 1235724
Loaded.
Loading data/geo_en.nt (1110708 lines) 13:41:43
Loaded.
Loading data/homepage_en.nt (65178 lines) 13:47:38
Loaded.
Loading data/image_en.nt (2775360 lines) 13:49:16
Loaded.
Loading data/infobox-mappingbased-loose.nt (8238555 lines)
14:11:35
Loaded.
Loading data/infoboxproperties_en.nt (98244 lines) 15:07:53
Loaded.
Loading data/infobox_en.nt (39855823 lines) 15:12:25
Loaded.
Loading data/instancetype_en.nt (3790668 lines) 16:04:50
Loaded.
Loading data/longabstract_en.nt (2788812 lines) 16:31:05
Loaded.
Loading data/pagelinks_en.nt (77092350 lines) 19:11:57
Retryng from line 16209818
Loaded.
Loading data/persondata_en.nt (118002 lines) 14:44:55
Loaded.
Loading data/redirect_en.nt (3494567 lines) 15:14:27
Loaded.
Loading data/shortabstract_en.nt (2787499 lines) 16:36:46
Loaded.
Loading data/skoscategories_en.nt (1844055 lines) 23:28:05
Loaded.
Loading data/wikipage_en.nt (3201391 lines) 00:35:10
Loaded.
Loading data/yagolink_en.nt (1889985 lines) 02:44:03
Loaded.
Loading data/yago_en.nt (4645108 lines) 08:32:36
Loaded.
=======================================
Final checkpoint.
=======================================
Check bad.nt file for skipped triples.
=======================================
Running post-install scripts
PASSED: RDF_OBJ_FT_RULE_ADD : STATE=OK MESSAGE=OK
PASSED: VT_INC_INDEX_DB_DBA_RDF_OBJ : STATE=OK MESSAGE=OK
PASSED: RDF_QUAD_POGS : STATE=OK MESSAGE=OK
PASSED: RDF_QUAD_PSOG : STATE=OK MESSAGE=OK
PASSED: RDF_QUAD_SOPG : STATE=OK MESSAGE=OK
PASSED: Load 2008-10-31_dbpedia.owl : STATE=OK MESSAGE=OK
PASSED: Load 2008-10-31_dbpedia.owl : STATE=OK MESSAGE=OK
PASSED: DBpedia inference : STATE=OK MESSAGE=OK
***FAILED: Load umbel_class_hierarchy_v071 : STATE=23000
MESSAGE=[OpenLink][Virt
uoso ODBC Driver][Virtuoso Server]SR133: Can not set NULL to not
nullable column
'DB.DBA.RDF_QUAD.O'
PASSED: Load umbel_subject_concepts : STATE=OK MESSAGE=OK
PASSED: Load umbel_abstract_concepts : STATE=OK MESSAGE=OK
PASSED: Load umbel_external_ontologies_linkage : STATE=OK
MESSAGE=OK
PASSED: UMBEL inference : STATE=OK MESSAGE=OK
PASSED: Load yago-class-hierarchy_en : STATE=OK MESSAGE=OK
PASSED: Yago inference : STATE=OK MESSAGE=OK
PASSED: Load opencyc_class_hierarchy : STATE=OK MESSAGE=OK
***FAILED: Opencyc inference : STATE=40006 MESSAGE=[OpenLink]
[Virtuoso
ODBC Driv
er][Virtuoso Server]SR337: Transaction aborted because the
server is
out of memo
ry
***FAILED: opencyc-2008-06-10-readable : STATE=40006
MESSAGE=[OpenLink][Virtuoso
ODBC Driver][Virtuoso Server]SR337: Transaction aborted
because the
server is o
ut of memory
***FAILED: DBpedia property inference : STATE=40006
MESSAGE=[OpenLink][Virtuoso
ODBC Driver][Virtuoso Server]SR337: Transaction aborted
because the
server is ou
t of memory
===============================
Install finished.
Sat Sep 19 06:39:55 GMTDT 2009
===============================
The MAIL_SERVER environment variable is not set, e.g. export
MAIL_SERVER=mail.do
main:25
===============================
Install started
Tue Sep 15 13:25:15 GMTDT 2009
===============================
Checking for VOS setup
Starting Virtuoso server, please wait ...
Cannot start Virtuoso server, please consult dbpedia.log file
Started.
Checking files
load_nt.sh : present
dbpedia_dav.vad : present
dbpedia_post.sql : present
dbpedia-ontology.owl : present
umbel_class_hierarchy_v071.n3 : present
umbel_abstract_concepts.n3 : present
umbel_external_ontologies_linkage.n3 : present
yago-class-hierarchy_en.nt : present
umbel_subject_concepts.n3 : present
opencyc-2008-06-10.owl : present
opencyc-2008-06-10-readable.owl : present
Installing dbpedia_dav.vad
00000
No errors detected
00000
Installation of "DBpedia" is complete.
00000
Now making a final checkpoint.
00000
Final checkpoint is made.
00000
SUCCESS
=======================================
Loading started.
=======================================
Loading data/externallinks_en.nt (3653630 lines) 13:25:50
Retryng from line 1235724
Loading data/geo_en.nt (1110708 lines) 13:41:43
Loading data/homepage_en.nt (65178 lines) 13:47:38
Loading data/image_en.nt (2775360 lines) 13:49:16
Loading data/infobox-mappingbased-loose.nt (8238555 lines)
14:11:35
Loading data/infoboxproperties_en.nt (98244 lines) 15:07:53
Loading data/infobox_en.nt (39855823 lines) 15:12:25
Loading data/instancetype_en.nt (3790668 lines) 16:04:50
Loading data/longabstract_en.nt (2788812 lines) 16:31:05
Loading data/pagelinks_en.nt (77092350 lines) 19:11:57
Retryng from line 16209818
Loading data/persondata_en.nt (118002 lines) 14:44:55
Loading data/redirect_en.nt (3494567 lines) 15:14:27
Loading data/shortabstract_en.nt (2787499 lines) 16:36:46
Loading data/skoscategories_en.nt (1844055 lines) 23:28:05
Loading data/wikipage_en.nt (3201391 lines) 00:35:10
Loading data/yagolink_en.nt (1889985 lines) 02:44:03
Loading data/yago_en.nt (4645108 lines) 08:32:36
Final checkpoint.
=======================================
Check bad.nt file for skipped triples.
=======================================
Running post-install scripts
PASSED: RDF_OBJ_FT_RULE_ADD : STATE=OK MESSAGE=OK
PASSED: VT_INC_INDEX_DB_DBA_RDF_OBJ : STATE=OK MESSAGE=OK
PASSED: RDF_QUAD_POGS : STATE=OK MESSAGE=OK
PASSED: RDF_QUAD_PSOG : STATE=OK MESSAGE=OK
PASSED: RDF_QUAD_SOPG : STATE=OK MESSAGE=OK
PASSED: Load 2008-10-31_dbpedia.owl : STATE=OK MESSAGE=OK
PASSED: Load 2008-10-31_dbpedia.owl : STATE=OK MESSAGE=OK
PASSED: DBpedia inference : STATE=OK MESSAGE=OK
*** Error 23000: [OpenLink][Virtuoso ODBC Driver][Virtuoso
Server]SR133: Can not set NULL to not nullable column
'DB.DBA.RDF_QUAD.O'
at line 36 of Top-Level:
ttlp_mt (file_to_string_output
('umbel_class_hierarchy_v071.n3'), '',
'http://dbpedia.org/resource/classes/umbel#')
***FAILED: Load umbel_class_hierarchy_v071 : STATE=23000
MESSAGE=[OpenLink][Virtuoso ODBC Driver][Virtuoso Server]
SR133: Can
not set NULL to not nullable column 'DB.DBA.RDF_QUAD.O'
PASSED: Load umbel_subject_concepts : STATE=OK MESSAGE=OK
PASSED: Load umbel_abstract_concepts : STATE=OK MESSAGE=OK
PASSED: Load umbel_external_ontologies_linkage : STATE=OK
MESSAGE=OK
PASSED: UMBEL inference : STATE=OK MESSAGE=OK
PASSED: Load yago-class-hierarchy_en : STATE=OK MESSAGE=OK
PASSED: Yago inference : STATE=OK MESSAGE=OK
PASSED: Load opencyc_class_hierarchy : STATE=OK MESSAGE=OK
*** Error 40006: [OpenLink][Virtuoso ODBC Driver][Virtuoso
Server]SR337: Transaction aborted because the server is out of
memory
at line 80 of Top-Level:
rdfs_rule_set ('http://dbpedia.org/resource/inference/rules/
opencyc#',
'http://dbpedia.org/resource/classes/opencyc#')
*** Error 40006: [OpenLink][Virtuoso ODBC Driver][Virtuoso
Server]SR337: Transaction aborted because the server is out of
memory
at line 80 of Top-Level:
rdfs_rule_set ('http://dbpedia.org/resource/inference/rules/
opencyc#',
'http://dbpedia.org/resource/classes/opencyc#')
***FAILED: Opencyc inference : STATE=40006 MESSAGE=[OpenLink]
[Virtuoso
ODBC Driver][Virtuoso Server]SR337: Transaction aborted
because the
server is out of memory
*** Error .....: [OpenLink][Virtuoso ODBC Driver][Virtuoso
Server]SQ200: Out of memory
at line 84 of Top-Level:
checkpoint
*** Error 40006: [OpenLink][Virtuoso ODBC Driver][Virtuoso
Server]SR337: Transaction aborted because the server is out of
memory
at line 84 of Top-Level:
checkpoint
*** Error .....: [OpenLink][Virtuoso ODBC Driver][Virtuoso
Server]SQ200: Out of memory
at line 85 of Top-Level:
DB..RDF_LOAD_RDFXML_MT (file_to_string_output
('opencyc-2008-06-10-readable.owl'),
'http://sw.opencyc.org/2008/06/10/concept/',
'opencyc-2008-06-10-readable')
*** Error 40006: [OpenLink][Virtuoso ODBC Driver][Virtuoso
Server]SR337: Transaction aborted because the server is out of
memory
at line 85 of Top-Level:
DB..RDF_LOAD_RDFXML_MT (file_to_string_output
('opencyc-2008-06-10-readable.owl'),
'http://sw.opencyc.org/2008/06/10/concept/',
'opencyc-2008-06-10-readable')
***FAILED: opencyc-2008-06-10-readable : STATE=40006
MESSAGE=[OpenLink][Virtuoso ODBC Driver][Virtuoso Server]SR337:
Transaction aborted because the server is out of memory
*** Error .....: [OpenLink][Virtuoso ODBC Driver][Virtuoso
Server]SQ200: Out of memory
in lines 89-98 of Top-Level:
#line 89 "(console)"
ttlp (
'@prefix rdfs: .
rdfs:subPropertyOf
.
rdfs:subPropertyOf
.
rdfs:subPropertyOf
.
rdfs:subPropertyOf
.
rdfs:subPropertyOf
.
rdfs:subPropertyOf
.
rdfs:subPropertyOf
.',
'','http://dbpedia.org/schema/property_rules#')
*** Error 40006: [OpenLink][Virtuoso ODBC Driver][Virtuoso
Server]SR337: Transaction aborted because the server is out of
memory
in lines 89-98 of Top-Level:
#line 89 "(console)"
ttlp (
'@prefix rdfs: .
rdfs:subPropertyOf
.
rdfs:subPropertyOf
.
rdfs:subPropertyOf
.
rdfs:subPropertyOf
.
rdfs:subPropertyOf
.
rdfs:subPropertyOf
.
rdfs:subPropertyOf
.',
'','http://dbpedia.org/schema/property_rules#')
*** Error .....: [OpenLink][Virtuoso ODBC Driver][Virtuoso
Server]SQ200: Out of memory
at line 100 of Top-Level:
rdfs_rule_set ('http://dbpedia.org/schema/property_rules#',
'http://dbpedia.org/schema/property_rules#')
*** Error 40006: [OpenLink][Virtuoso ODBC Driver][Virtuoso
Server]SR337: Transaction aborted because the server is out of
memory
at line 100 of Top-Level:
rdfs_rule_set ('http://dbpedia.org/schema/property_rules#',
'http://dbpedia.org/schema/property_rules#')
***FAILED: DBpedia property inference : STATE=40006
MESSAGE=[OpenLink][Virtuoso ODBC Driver][Virtuoso Server]SR337:
Transaction aborted because the server is out of memory
*** Error .....: [OpenLink][Virtuoso ODBC Driver][Virtuoso
Server]SQ200: Out of memory
at line 104 of Top-Level:
checkpoint
*** Error 40006: [OpenLink][Virtuoso ODBC Driver][Virtuoso
Server]SR337: Transaction aborted because the server is out of
memory
at line 104 of Top-Level:
checkpoint
===============================
Install finished.
Sat Sep 19 06:39:55 GMTDT 2009
===============================
The MAIL_SERVER environment variable is not set, e.g. export
MAIL_SERVER=mail.domain:25
