Actually, I have a dream to exceed those two billions. It seems possible, to move to Vint in fileformat and change int docnums to longs in Lucene API. Does anyone know whether it's possible? And this question is not so esoteric if we are talking about SolrCloud, which can hold more that 2bn docs in few smaller shards. Any experience?
On Thu, Feb 7, 2013 at 5:46 PM, Rafał Kuć <r....@solr.pl> wrote: > Hello! > > Right, my bad - ids are still using int32. However, that still > gives us 2,147,483,648 possible identifiers right per single index, > which is not close to the 13,5 millions mentioned in the first mail. > > -- > Regards, > Rafał Kuć > Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - ElasticSearch > > > Rafal, > > > What about docnums, don't they are limited by int32 ? > > 07.02.2013 15:33 пользователь "Rafał Kuć" <r....@solr.pl> написал: > > >> Hello! > >> > >> Practically there is no limit in how many documents can be stored in a > >> single index. In your case, as you are using Solr from 2011, there is > >> a limitation regarding the number of unique terms per Lucene segment > >> ( > >> > http://lucene.apache.org/core/old_versioned_docs/versions/3_0_0/fileformats.html#Limitations > >> ). > >> However I don't think you've hit that. Solr by itself doesn't remove > >> documents unless told to do so. > >> > >> Its hard to guess what can be the reason and as you said, you see > >> updates coming to your handler. Maybe new documents have the same > >> identifiers that the ones that are already indexed ? As I said, this > >> is only a guess and we would need to have more information. Are there > >> any exceptions in the logs ? Do you run delete command ? Are your > >> index files changed ? How do you run commit ? > >> > >> -- > >> Regards, > >> Rafał Kuć > >> Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - > ElasticSearch > >> > >> > I have searched this forum but not yet found a definitive answer, I > >> think the > >> > answer is "There is No Limit depends on server specification". But > never > >> the > >> > less I will say what I have seen and then ask the questions. > >> > >> > From scratch (November 2011) I have set up our SOLR which contains > data > >> from > >> > various sources, since March 2012 , the number of indexed records > (unique > >> > ID's) reached 13.5 million , which was to be expected. However for the > >> last > >> > 8 months the number of records in the index has not gone above 13.5 > >> million, > >> > yet looking at the request handler outputs I can safely say at least > >> > anywhere from 50 thousand to 100 thousand records are being indexed > >> daily. > >> > So I am assuming that earlier records are being removed, and I do not > >> want > >> > that. > >> > >> > Question: If there is a limit to the number of records the index can > >> store > >> > where do I find this and change it? > >> > Question: If there is no limit does anyone have any idea why for the > last > >> > months the number has not gone beyond 13.5 million, I can safely say > >> that at > >> > least 90% are new records. > >> > >> > thanks > >> > >> > macroman > >> > >> > >> > >> > -- > >> > View this message in context: > >> > > >> > http://lucene.472066.n3.nabble.com/Maximum-Number-of-Records-In-Index-tp4038961.html > >> > Sent from the Solr - User mailing list archive at Nabble.com. > >> > >> > > -- Sincerely yours Mikhail Khludnev Principal Engineer, Grid Dynamics <http://www.griddynamics.com> <mkhlud...@griddynamics.com>
