Hi Bastian, Some additional information. Below are the relevant INI file params we used on our hosted Dbpedia server for importing and running this service:
[Parameters] . . . ServerThreads = 1000 CheckpointInterval = 60; AutoCheckpointLogSize = 10000000 NumberOfBuffers = 300000 MaxDirtyBuffers = 4000 MaxCheckpointRemap = 40000 . . . Note the server we are using has less capacity in terms of Processors speeds and Memory than your, so this should suffice for your needs and could also be increased if need be ... Regards, Hugh On 4/7/07 19:10, "Hugh Williams" <hwilli...@openlinksw.com> wrote: > Hi Bastian > > You most probably need to update or add the NumberOfBuffers ini file parameter > to the [Parameters] section of the Virtuoso configuration file as detailed at: > > http://docs.openlinksw.com/virtuoso/dbadm.html#ini_Parameters > > Note that Dbpedia is actually hosted on a Virtuoso server see > http://dbpedia.org for more details and can be interacted > with directly via http://dbpedia.openlinksw.com:8890/sparql or > http://dbpedia.openlinksw.com:8890/isparql . > > Below are the details of how we imported the Dbpedia dataset into Virtuoso: > > * Download the DBpedia datasets from http://dbpedia.org/docs/#down > > * Unzip and place all the the datasets file (*.nt) into a directory called > "dbpedia_data" > > * Ensure the "dbpedia_data" directory is added to your "DirsAllowed?" > paramter in the Virtuoso Server configuration file and restart the server. > > * The NumberOfBuffers? paramter of the Virtuoso Server configuration file > is set to a suitable value to perform the import,noting that each buffer is 8K > bytes of memory. (In our case NumberOfBuffers=300000) > > * Run the script below from the parent directory of the "dbpedia_data" > directory to loop through and import each dataset into your Virtuoso Server > > #!/bin/sh > rm -f load_dbpedia.log > for f in `find dbpedia_data -name '*.nt'` > do > echo "Loading $f `date \"+%H:%M:%S\"`" | tee -a load_dbpedia.log > isql 1112 dba dba verbose=on banner=off prompt=off echo=ON errors=stdout > exec="ttlp_mt (file_to_string_output ('$f'), '', 'http://dbpedia.org'); > checkpoint;" >> load_dbpedia.log > done > > Best Regards, > Hugh Williams > OpenLink Software > > On 4/7/07 18:41, "Bastian Quilitz" <quil...@informatik.hu-berlin.de> wrote: > >> Hello, >> >> I am experimenting with virtuoso open source as rdf store. I tried to import >> a dbpedia dataset (1.6M triples) into the demo database (demo.ini) on a >> 8xOpteron(dualCore) with a lot of ram. This took around 3h and I am >> wondering whether this is a reasonable time? Although there were some other >> users using the machine (total cpu avg. was around 50%, RAM available >5GB) >> this seems to be quiet slow (~130triples/sec). I used the ttpl_mp() function >> to import the triples from a local file. >> >> Am I doing something wrong or are there tuning options I should use? >> >> >> Kind regards, >> Bastian >> >> >> ------------------------------------------------------------------------- >> This SF.net email is sponsored by DB2 Express >> Download DB2 Express C - the FREE version of DB2 express and take >> control of your XML. No limits. Just data. Click to get it now. >> http://sourceforge.net/powerbar/db2/ >> _______________________________________________ >> Virtuoso-users mailing list >> Virtuoso-users@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/virtuoso-users > > Best Regards, Hugh Williams VP Product Development OpenLink Software Tel: +44 (0) 8681 7701 Web: http://www.openlinksw.com Universal Data Access & Data Integration Technology Providers
