Maybe also have a look at these links. http://www.hathitrust.org/blogs/large-scale-search/performance-5-million-volumes http://www.hathitrust.org/blogs/large-scale-search
On Fri, 13 Jan 2012 15:49:06 +0100, Daniel Brügge <dan...@bruegge.eu> wrote: > Hi, > > it's definitely a problem to store 5TB in Solr without using sharding. I > try to split data over solr instances, > so that the index will fit in my memory on the server. > > I ran into trouble with a Solr using 50G index. > > Daniel > > On Jan 13, 2012, at 1:08 PM, mustafozbek wrote: > >> I am an apache solr user about a year. I used solr for simple search >> tools >> but now I want to use solr with 5TB of data. I assume that 5TB data will >> be >> 7TB when solr index it according to filter that I use. And then I will >> add >> nearly 50MB of data per hour to the same index. >> 1- Are there any problem using single solr server with 5TB data. (without >> shards) >> a- Can solr server answers the queries in an acceptable time >> b- what is the expected time for commiting of 50MB data on 7TB index. >> c- Is there an upper limit for index size. >> 2- what are the suggestions that you offer >> a- How many shards should I use >> b- Should I use solr cores >> c- What is the committing frequency you offered. (is 1 hour OK) >> 3- are there any test results for this kind of large data >> >> There is no available 5TB data, I just want to estimate what will be the >> result. >> Note: You can assume that hardware resourses are not a problem. >> >> >> -- >> View this message in context: >> http://lucene.472066.n3.nabble.com/Can-Apache-Solr-Handle-TeraByte-Large-Data-tp3656484p3656484.html >> Sent from the Solr - User mailing list archive at Nabble.com.
