Partial updates is nothing as clever as I may have made it sound, it is just
changing a record value , for example last name from Smith to Jones, that's
my partial update.
No errors at all in indexing, I have not yet checked the logs , but the DIH
output counts show no errors, here is an example
A couple of things to check.
1) Have you retained your solr logs. If so, take a look in them for
indexing errors.
2) What is the difference between maxdocs and numdocs. This will give an
indication if a large number of records are being deleted or updated.
3) Can you explain your partial updates?
Our document ID's are most definately distinct and there are partial updates
to existing records, I have run SQL queries outside of SOLR to validate
records going in and only about 1% are updates to existing records. There
are no deletes underway every day new records are added or updated. Example
We don't run one ourselves at Sematext, but know of people who do have
large ES clusters, one with > 10B docs.
Otis
--
Solr & ElasticSearch Support
http://sematext.com/
On Mon, Feb 11, 2013 at 8:41 AM, Mikhail Khludnev <
mkhlud...@griddynamics.com> wrote:
> Otis,
> Do you run 4bn docs SolrCl
Otis,
Do you run 4bn docs SolrCloud or ElasticSearch or aware of somebody who do?
10.02.2013 4:54 пользователь "Otis Gospodnetic"
написал:
> Exceeding 2B is no problem. But it won't happen in a single Lucene index
> any time soon, so...
>
> Otis
> Solr & ElasticSearch Support
> http://sematext.c
Actually, I have a dream to exceed those two billions. It seems possible,
to move to Vint in fileformat and change int docnums to longs in Lucene
API. Does anyone know whether it's possible?
And this question is not so esoteric if we are talking about SolrCloud,
which can hold more that 2bn docs in
Hello!
Right, my bad - ids are still using int32. However, that still
gives us 2,147,483,648 possible identifiers right per single index,
which is not close to the 13,5 millions mentioned in the first mail.
--
Regards,
Rafał Kuć
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - Elast
Rafal,
What about docnums, don't they are limited by int32 ?
07.02.2013 15:33 пользователь "Rafał Kuć" написал:
> Hello!
>
> Practically there is no limit in how many documents can be stored in a
> single index. In your case, as you are using Solr from 2011, there is
> a limitation regarding the
