Thank you that is very helpful. Cheers, Andy
> On 23 Apr 2015, at 03:46, Hugh Williams <hwilli...@openlinksw.com> wrote: > > Hi Andy, > > The amount of memory and hence NumberOfBuffers a dataset requires for being > hosted/cached in memory for best performance varies depending on how well a > given dataset can be compressed, which varies based on its structure (data > regularity) and size of literals etc. and thus makes this hard to predict > with any accuracy before hand. As a general rule for column store we say the > average storage requirement is 10 bytes per quad ie 10GB RAM per billion > triples , but as said this varies some maybe less and some more. For example > we have seen 9 bytes/quad with DBpedia, 7 bytes/quad with BSBM , 14 > bytes/quad with web crawl data (not very regular). > > Once a dataset is loaded its space utilisation can be determined as detailed > at: > > http://docs.openlinksw.com/virtuoso/coredbengine.html#colstorespaceutil > > I hope this helps ... > > Best Regards > Hugh Williams > Professional Services > OpenLink Software, Inc. // http://www.openlinksw.com/ > Weblog -- http://www.openlinksw.com/blogs/ > LinkedIn -- http://www.linkedin.com/company/openlink-software/ > Twitter -- http://twitter.com/OpenLink > Google+ -- http://plus.google.com/100570109519069333827/ > Facebook -- http://www.facebook.com/OpenLinkSoftware > Universal Data Access, Integration, and Management Technology Providers > >> On 22 Apr 2015, at 17:01, Andy Jenkinson <andy.jenkin...@ebi.ac.uk> wrote: >> >> Hi, >> >> I would like to find a more efficient way to work out how much memory to >> allocate to some new Virtuoso VMs, which is directly dependent on how many >> buffers I allocate to Virtuoso. I want to allocate enough memory so that the >> column store is effectively operating in-memory. To work this out, currently >> we have to allocate a large number of buffers and then monitor the service >> to see how many are actually used, but this is: >> a) time consuming >> b) an inefficient use of memory in the meantime >> c) difficult to plan >> >> I realise one would “normally" allocate the buffers according to the >> available memory, but I want to do the reverse. Is there a rough guide to >> calculate an appropriate number of buffers from the number of triples in the >> dataset? >> >> Cheers, >> Andy >> ------------------------------------------------------------------------------ >> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT >> Develop your own process in accordance with the BPMN 2 standard >> Learn Process modeling best practices with Bonita BPM through live exercises >> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_ >> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF >> _______________________________________________ >> Virtuoso-users mailing list >> Virtuoso-users@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/virtuoso-users > ------------------------------------------------------------------------------ BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT Develop your own process in accordance with the BPMN 2 standard Learn Process modeling best practices with Bonita BPM through live exercises http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_ source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF _______________________________________________ Virtuoso-users mailing list Virtuoso-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/virtuoso-users
