Hi VOS community, we have recently upgraded our VOS7 SPARQL end point for the ChEMBL-RDF data [0]: http://rdf.farmbio.uu.se/chembl/sparql
It's crashed a few times (not sure why; I'm now monitoring it day by day), but I was originally looking forward to the faster performance. I started from a fresh empty database, and reloaded all data, but when it was back online, I was expecting it to be faster, but when looking at the Mondeca SPARQL uptime service, it reported the same response times... does that make sense? I'm no expert at all in the VOS config files, and am wondering if it is sensible at all (given below), and if there is anything I can do to improve the performance of the SPARQL end point... I have no clue where to start, and do not have time to deeply explore this either (as a chemist you do not get funding for maintenance of this kind at all...) What are things I can try? Did I need to do some specific indexing to take advantage of the column store speed up? Am I doing something stupid in my config (below)? The machine this runs on is an 8 core, but with VOS that should not help. It has 8GB of memory. Egon 0.http://www.jcheminf.com/content/5/1/23 the config file: ========================================================== ; ; virtuoso.ini ; ; Configuration file for the OpenLink Virtuoso VDBMS Server ; ; To learn more about this product, or any other product in our ; portfolio, please check out our web site at: ; ; http://virtuoso.openlinksw.com/ ; ; or contact us at: ; ; general.informat...@openlinksw.com ; ; If you have any technical questions, please contact our support ; staff at: ; ; technical.supp...@openlinksw.com ; ; ; Database setup ; [Database] DatabaseFile = virtuoso.db ErrorLogFile = virtuoso.log LockFile = virtuoso.lck TransactionFile = virtuoso.trx xa_persistent_file = virtuoso.pxa ErrorLogLevel = 7 FileExtend = 200 MaxCheckpointRemap = 2000 Striping = 0 TempStorage = TempDatabase [TempDatabase] DatabaseFile = virtuoso-temp.db TransactionFile = virtuoso-temp.trx MaxCheckpointRemap = 2000 Striping = 0 ; ; Server parameters ; [Parameters] ServerPort = 1113 LiteMode = 0 DisableUnixSocket = 1 DisableTcpSocket = 0 ;SSLServerPort = 2111 ;SSLCertificate = cert.pem ;SSLPrivateKey = pk.pem ;X509ClientVerify = 0 ;X509ClientVerifyDepth = 0 ;X509ClientVerifyCAFile = ca.pem ServerThreads = 20 CheckpointInterval = 60 O_DIRECT = 0 CaseMode = 2 MaxStaticCursorRows = 5000 CheckpointAuditTrail = 0 AllowOSCalls = 0 SchedulerInterval = 10 DirsAllowed = ., /var/data/egonw/chembl/vad ThreadCleanupInterval = 0 ThreadThreshold = 10 ResourcesCleanupInterval = 0 FreeTextBatchSize = 100000 SingleCPU = 0 VADInstallDir = /usr/local/virtuoso-opensource/share/virtuoso/vad/ PrefixResultNames = 0 RdfFreeTextRulesSize = 100 IndexTreeMaps = 256 MaxMemPoolSize = 200000000 PrefixResultNames = 0 MacSpotlight = 0 IndexTreeMaps = 64 ;; ;; When running with large data sets, one should configure the Virtuoso ;; process to use between 2/3 to 3/5 of free system memory and to stripe ;; storage on all available disks. ;; ;; Uncomment next two lines if there is 2 GB system memory free ;NumberOfBuffers = 170000 ;MaxDirtyBuffers = 130000 ;; Uncomment next two lines if there is 4 GB system memory free ;NumberOfBuffers = 340000 ; MaxDirtyBuffers = 250000 ;; Uncomment next two lines if there is 8 GB system memory free NumberOfBuffers = 680000 MaxDirtyBuffers = 500000 ;; Uncomment next two lines if there is 16 GB system memory free ;NumberOfBuffers = 1360000 ;MaxDirtyBuffers = 1000000 ;; Uncomment next two lines if there is 32 GB system memory free ;NumberOfBuffers = 2720000 ;MaxDirtyBuffers = 2000000 ;; Uncomment next two lines if there is 48 GB system memory free ;NumberOfBuffers = 4000000 ;MaxDirtyBuffers = 3000000 ;; Uncomment next two lines if there is 64 GB system memory free ;NumberOfBuffers = 5450000 ;MaxDirtyBuffers = 4000000 ;; ;; Note the default settings will take very little memory ;; but will not result in very good performance ;; NumberOfBuffers = 10000 MaxDirtyBuffers = 6000 [HTTPServer] ServerPort = 8892 ServerRoot = /usr/local/virtuoso-opensource/var/lib/virtuoso/vsp ServerThreads = 20 DavRoot = DAV EnabledDavVSP = 0 HTTPProxyEnabled = 0 TempASPXDir = 0 DefaultMailServer = localhost:25 ServerThreads = 10 MaxKeepAlives = 10 KeepAliveTimeout = 10 MaxCachedProxyConnections = 10 ProxyConnectionCacheTimeout = 15 HTTPThreadSize = 280000 HttpPrintWarningsInOutput = 0 Charset = UTF-8 ;HTTPLogFile = logs/http.log [AutoRepair] BadParentLinks = 0 [Client] SQL_PREFETCH_ROWS = 100 SQL_PREFETCH_BYTES = 16000 SQL_QUERY_TIMEOUT = 0 SQL_TXN_TIMEOUT = 0 ;SQL_NO_CHAR_C_ESCAPE = 1 ;SQL_UTF8_EXECS = 0 ;SQL_NO_SYSTEM_TABLES = 0 ;SQL_BINARY_TIMESTAMP = 1 ;SQL_ENCRYPTION_ON_PASSWORD = -1 [VDB] ArrayOptimization = 0 NumArrayParameters = 10 VDBDisconnectTimeout = 1000 KeepConnectionOnFixedThread = 0 [Replication] ServerName = db-WS1 ServerEnable = 1 QueueMax = 50000 ; ; Striping setup ; ; These parameters have only effect when Striping is set to 1 in the ; [Database] section, in which case the DatabaseFile parameter is ignored. ; ; With striping, the database is spawned across multiple segments ; where each segment can have multiple stripes. ; ; Format of the lines below: ; Segment<number> = <size>, <stripe file name> [, <stripe file name> .. ] ; ; <number> must be ordered from 1 up. ; ; The <size> is the total size of the segment which is equally divided ; across all stripes forming the segment. Its specification can be in ; gigabytes (g), megabytes (m), kilobytes (k) or in database blocks ; (b, the default) ; ; Note that the segment size must be a multiple of the database page size ; which is currently 8k. Also, the segment size must be divisible by the ; number of stripe files forming the segment. ; ; The example below creates a 200 meg database striped on two segments ; with two stripes of 50 meg and one of 100 meg. ; ; You can always add more segments to the configuration, but once ; added, do not change the setup. ; [Striping] Segment1 = 100M, db-seg1-1.db, db-seg1-2.db Segment2 = 100M, db-seg2-1.db ;... ;[TempStriping] ;Segment1 = 100M, db-seg1-1.db, db-seg1-2.db ;Segment2 = 100M, db-seg2-1.db ;... ;[Ucms] ;UcmPath = <path> ;Ucm1 = <file> ;Ucm2 = <file> ;... [Zero Config] ServerName = virtuoso (WS1) ;ServerDSN = ZDSN ;SSLServerName = ;SSLServerDSN = [Mono] ;MONO_TRACE = Off ;MONO_PATH = <path_here> ;MONO_ROOT = <path_here> ;MONO_CFG_DIR = <path_here> ;virtclr.dll = [URIQA] DynamicLocal = 0 DefaultHost = localhost:8890 [SPARQL] DefaultGraph = http://linkedchemistry.info/chembl/ ImmutableGraphs = http://linkedchemistry.info/chembl/ ResultSetMaxRows = 1000000 MaxQueryCostEstimationTime = 5000 ; in seconds MaxQueryExecutionTime = 80 ; in seconds DefaultQuery = select distinct * where {<http://linkedchemistry.info/chembl/molecule/m443> ?p ?o} ;ExternalQuerySource = 1 ;ExternalXsltSource = 1 DeferInferenceRulesInit = 0 ; controls inference rules loading ;PingService = http://rpc.pingthesemanticweb.com/ [Plugins] LoadPath = /usr/local/virtuoso-opensource/lib/virtuoso/hosting Load1 = plain, wikiv Load2 = plain, mediawiki Load3 = plain, creolewiki Load4 = plain, im ;Load5 = plain, wbxml2 ;Load6 = plain, hslookup ;Load7 = attach, libphp5.so ;Load8 = Hosting, hosting_php.so ;Load9 = Hosting,hosting_perl.so ;Load10 = Hosting,hosting_python.so ;Load11 = Hosting,hosting_ruby.so ;Load12 = msdtc,msdtc_sample ========================================================== -- Dr E.L. Willighagen Postdoctoral Researcher Department of Bioinformatics - BiGCaT Maastricht University (http://www.bigcat.unimaas.nl/) Homepage: http://egonw.github.com/ LinkedIn: http://se.linkedin.com/in/egonw Blog: http://chem-bla-ics.blogspot.com/ PubList: http://www.citeulike.org/user/egonw/tag/papers ORCID: 0000-0001-7542-0286 ------------------------------------------------------------------------------ October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register > http://pubads.g.doubleclick.net/gampad/clk?id=60134791&iu=/4140/ostg.clktrk _______________________________________________ Virtuoso-users mailing list Virtuoso-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/virtuoso-users
