Hi Egon,
Can you please provide the following for review:
1. The output of running the "status();" run from command from the Virtuoso
"isql" commandline tool
2. The triple count of the database with "select count(*) where {?s ?p ?o};"If you load the data in the Quad Store from an empty database then it will be running in column store mode by default. Looking at your INI file I see you have the "NumberOfBuffers" param, which is the key to performance in terms of being able to load the database working set into memory, set twice: NumberOfBuffers = 680000 MaxDirtyBuffers = 500000 and NumberOfBuffers = 10000 MaxDirtyBuffers = 6000 The latter should be commented out or removed and the server restarted, the "status()" command will show the buffers allocated and used , which is useful to see ... Best Regards Hugh Williams Professional Services OpenLink Software, Inc. // http://www.openlinksw.com/ Weblog -- http://www.openlinksw.com/blogs/ LinkedIn -- http://www.linkedin.com/company/openlink-software/ Twitter -- http://twitter.com/OpenLink Google+ -- http://plus.google.com/100570109519069333827/ Facebook -- http://www.facebook.com/OpenLinkSoftware Universal Data Access, Integration, and Management Technology Providers On 4 Oct 2013, at 10:18, Jerven Bolleman <jerven.bolle...@isb-sib.ch> wrote: > Hi Egon, > > Not a virtuoso expert. But about the mondica, I would not expect > a different result between VOS6 and VOS7. The query mondica uses for > measuring uptime is > > ASK { ?s ?p ?o .} > > This query is very cheap to answer no matter which store is used. > > So in the mondica is measuring the time needed for http transfer etc... > This highly dependant on the mondica server and infrastructure > implementation. > > It is around 1 second for the uniprot one and that is quite slow for a > http connection setup like this. > > When I try these endpoints it is in the 100 ms range which is more normal. > > for server in "http://beta.sparql.uniprot.org/"; > "http://rdf.farmbio.uu.se/chembl";do time wget "$server/sparql?query=ASK > {?s ?p ?o}";done > > So no VOS7 is not going to affect mondica response times. It is going to > affect complex query times. > > Otherwise it looks to me like you have both the settings for 8gb and the > default settings uncommented, I suspect that the defaults win in that > case as these are the last ones. > > Regards, > Jerven > > On 04/10/13 10:59, Egon Willighagen wrote: >> Hi VOS community, >> >> we have recently upgraded our VOS7 SPARQL end point for the ChEMBL-RDF >> data [0]: http://rdf.farmbio.uu.se/chembl/sparql >> >> It's crashed a few times (not sure why; I'm now monitoring it day by >> day), but I was originally looking forward to the faster performance. >> I started from a fresh empty database, and reloaded all data, but when >> it was back online, I was expecting it to be faster, but when looking >> at the Mondeca SPARQL uptime service, it reported the same response >> times... does that make sense? >> >> I'm no expert at all in the VOS config files, and am wondering if it >> is sensible at all (given below), and if there is anything I can do to >> improve the performance of the SPARQL end point... I have no clue >> where to start, and do not have time to deeply explore this either (as >> a chemist you do not get funding for maintenance of this kind at >> all...) >> >> What are things I can try? Did I need to do some specific indexing to >> take advantage of the column store speed up? >> >> Am I doing something stupid in my config (below)? The machine this >> runs on is an 8 core, but with VOS that should not help. It has 8GB of >> memory. >> >> Egon >> >> 0.http://www.jcheminf.com/content/5/1/23 >> >> the config file: >> >> ========================================================== >> ; >> ; virtuoso.ini >> ; >> ; Configuration file for the OpenLink Virtuoso VDBMS Server >> ; >> ; To learn more about this product, or any other product in our >> ; portfolio, please check out our web site at: >> ; >> ; http://virtuoso.openlinksw.com/ >> ; >> ; or contact us at: >> ; >> ; general.informat...@openlinksw.com >> ; >> ; If you have any technical questions, please contact our support >> ; staff at: >> ; >> ; technical.supp...@openlinksw.com >> ; >> >> ; >> ; Database setup >> ; >> [Database] >> DatabaseFile = virtuoso.db >> ErrorLogFile = virtuoso.log >> LockFile = virtuoso.lck >> TransactionFile = virtuoso.trx >> xa_persistent_file = virtuoso.pxa >> ErrorLogLevel = 7 >> FileExtend = 200 >> MaxCheckpointRemap = 2000 >> Striping = 0 >> TempStorage = TempDatabase >> >> >> [TempDatabase] >> DatabaseFile = virtuoso-temp.db >> TransactionFile = virtuoso-temp.trx >> MaxCheckpointRemap = 2000 >> Striping = 0 >> >> >> ; >> ; Server parameters >> ; >> [Parameters] >> ServerPort = 1113 >> LiteMode = 0 >> DisableUnixSocket = 1 >> DisableTcpSocket = 0 >> ;SSLServerPort = 2111 >> ;SSLCertificate = cert.pem >> ;SSLPrivateKey = pk.pem >> ;X509ClientVerify = 0 >> ;X509ClientVerifyDepth = 0 >> ;X509ClientVerifyCAFile = ca.pem >> ServerThreads = 20 >> CheckpointInterval = 60 >> O_DIRECT = 0 >> CaseMode = 2 >> MaxStaticCursorRows = 5000 >> CheckpointAuditTrail = 0 >> AllowOSCalls = 0 >> SchedulerInterval = 10 >> DirsAllowed = ., /var/data/egonw/chembl/vad >> ThreadCleanupInterval = 0 >> ThreadThreshold = 10 >> ResourcesCleanupInterval = 0 >> FreeTextBatchSize = 100000 >> SingleCPU = 0 >> VADInstallDir = >> /usr/local/virtuoso-opensource/share/virtuoso/vad/ >> PrefixResultNames = 0 >> RdfFreeTextRulesSize = 100 >> IndexTreeMaps = 256 >> MaxMemPoolSize = 200000000 >> PrefixResultNames = 0 >> MacSpotlight = 0 >> IndexTreeMaps = 64 >> ;; >> ;; When running with large data sets, one should configure the Virtuoso >> ;; process to use between 2/3 to 3/5 of free system memory and to stripe >> ;; storage on all available disks. >> ;; >> ;; Uncomment next two lines if there is 2 GB system memory free >> ;NumberOfBuffers = 170000 >> ;MaxDirtyBuffers = 130000 >> ;; Uncomment next two lines if there is 4 GB system memory free >> ;NumberOfBuffers = 340000 >> ; MaxDirtyBuffers = 250000 >> ;; Uncomment next two lines if there is 8 GB system memory free >> NumberOfBuffers = 680000 >> MaxDirtyBuffers = 500000 >> ;; Uncomment next two lines if there is 16 GB system memory free >> ;NumberOfBuffers = 1360000 >> ;MaxDirtyBuffers = 1000000 >> ;; Uncomment next two lines if there is 32 GB system memory free >> ;NumberOfBuffers = 2720000 >> ;MaxDirtyBuffers = 2000000 >> ;; Uncomment next two lines if there is 48 GB system memory free >> ;NumberOfBuffers = 4000000 >> ;MaxDirtyBuffers = 3000000 >> ;; Uncomment next two lines if there is 64 GB system memory free >> ;NumberOfBuffers = 5450000 >> ;MaxDirtyBuffers = 4000000 >> ;; >> ;; Note the default settings will take very little memory >> ;; but will not result in very good performance >> ;; >> NumberOfBuffers = 10000 >> MaxDirtyBuffers = 6000 >> >> >> [HTTPServer] >> ServerPort = 8892 >> ServerRoot = >> /usr/local/virtuoso-opensource/var/lib/virtuoso/vsp >> ServerThreads = 20 >> DavRoot = DAV >> EnabledDavVSP = 0 >> HTTPProxyEnabled = 0 >> TempASPXDir = 0 >> DefaultMailServer = localhost:25 >> ServerThreads = 10 >> MaxKeepAlives = 10 >> KeepAliveTimeout = 10 >> MaxCachedProxyConnections = 10 >> ProxyConnectionCacheTimeout = 15 >> HTTPThreadSize = 280000 >> HttpPrintWarningsInOutput = 0 >> Charset = UTF-8 >> ;HTTPLogFile = logs/http.log >> >> [AutoRepair] >> BadParentLinks = 0 >> >> [Client] >> SQL_PREFETCH_ROWS = 100 >> SQL_PREFETCH_BYTES = 16000 >> SQL_QUERY_TIMEOUT = 0 >> SQL_TXN_TIMEOUT = 0 >> ;SQL_NO_CHAR_C_ESCAPE = 1 >> ;SQL_UTF8_EXECS = 0 >> ;SQL_NO_SYSTEM_TABLES = 0 >> ;SQL_BINARY_TIMESTAMP = 1 >> ;SQL_ENCRYPTION_ON_PASSWORD = -1 >> >> [VDB] >> ArrayOptimization = 0 >> NumArrayParameters = 10 >> VDBDisconnectTimeout = 1000 >> KeepConnectionOnFixedThread = 0 >> >> [Replication] >> ServerName = db-WS1 >> ServerEnable = 1 >> QueueMax = 50000 >> >> >> ; >> ; Striping setup >> ; >> ; These parameters have only effect when Striping is set to 1 in the >> ; [Database] section, in which case the DatabaseFile parameter is ignored. >> ; >> ; With striping, the database is spawned across multiple segments >> ; where each segment can have multiple stripes. >> ; >> ; Format of the lines below: >> ; Segment<number> = <size>, <stripe file name> [, <stripe file name> .. ] >> ; >> ; <number> must be ordered from 1 up. >> ; >> ; The <size> is the total size of the segment which is equally divided >> ; across all stripes forming the segment. Its specification can be in >> ; gigabytes (g), megabytes (m), kilobytes (k) or in database blocks >> ; (b, the default) >> ; >> ; Note that the segment size must be a multiple of the database page size >> ; which is currently 8k. Also, the segment size must be divisible by the >> ; number of stripe files forming the segment. >> ; >> ; The example below creates a 200 meg database striped on two segments >> ; with two stripes of 50 meg and one of 100 meg. >> ; >> ; You can always add more segments to the configuration, but once >> ; added, do not change the setup. >> ; >> [Striping] >> Segment1 = 100M, db-seg1-1.db, db-seg1-2.db >> Segment2 = 100M, db-seg2-1.db >> ;... >> >> ;[TempStriping] >> ;Segment1 = 100M, db-seg1-1.db, db-seg1-2.db >> ;Segment2 = 100M, db-seg2-1.db >> ;... >> >> ;[Ucms] >> ;UcmPath = <path> >> ;Ucm1 = <file> >> ;Ucm2 = <file> >> ;... >> >> >> [Zero Config] >> ServerName = virtuoso (WS1) >> ;ServerDSN = ZDSN >> ;SSLServerName = >> ;SSLServerDSN = >> >> >> [Mono] >> ;MONO_TRACE = Off >> ;MONO_PATH = <path_here> >> ;MONO_ROOT = <path_here> >> ;MONO_CFG_DIR = <path_here> >> ;virtclr.dll = >> >> >> [URIQA] >> DynamicLocal = 0 >> DefaultHost = localhost:8890 >> >> >> [SPARQL] >> DefaultGraph = http://linkedchemistry.info/chembl/ >> ImmutableGraphs = http://linkedchemistry.info/chembl/ >> ResultSetMaxRows = 1000000 >> MaxQueryCostEstimationTime = 5000 ; in seconds >> MaxQueryExecutionTime = 80 ; in seconds >> DefaultQuery = select distinct * where >> {<http://linkedchemistry.info/chembl/molecule/m443> ?p ?o} >> ;ExternalQuerySource = 1 >> ;ExternalXsltSource = 1 >> DeferInferenceRulesInit = 0 ; controls inference rules loading >> ;PingService = http://rpc.pingthesemanticweb.com/ >> >> >> [Plugins] >> LoadPath = >> /usr/local/virtuoso-opensource/lib/virtuoso/hosting >> Load1 = plain, wikiv >> Load2 = plain, mediawiki >> Load3 = plain, creolewiki >> Load4 = plain, im >> ;Load5 = plain, wbxml2 >> ;Load6 = plain, hslookup >> ;Load7 = attach, libphp5.so >> ;Load8 = Hosting, hosting_php.so >> ;Load9 = Hosting,hosting_perl.so >> ;Load10 = Hosting,hosting_python.so >> ;Load11 = Hosting,hosting_ruby.so >> ;Load12 = msdtc,msdtc_sample >> ========================================================== >> >> > > > -- > ------------------------------------------------------------------- > Jerven Bolleman jerven.bolle...@isb-sib.ch > SIB Swiss Institute of Bioinformatics Tel: +41 (0)22 379 58 85 > CMU, rue Michel Servet 1 Fax: +41 (0)22 379 58 58 > 1211 Geneve 4, > Switzerland www.isb-sib.ch - www.uniprot.org > Follow us at https://twitter.com/#!/uniprot > ------------------------------------------------------------------- > > ------------------------------------------------------------------------------ > October Webinars: Code for Performance > Free Intel webinars can help you accelerate application performance. > Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from > the latest Intel processors and coprocessors. See abstracts and register > > http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktrk > _______________________________________________ > Virtuoso-users mailing list > Virtuoso-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/virtuoso-users
smime.p7s
Description: S/MIME cryptographic signature
------------------------------------------------------------------------------ October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register > http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktrk
_______________________________________________ Virtuoso-users mailing list Virtuoso-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/virtuoso-users
