A couple of things to check. 1) Have you retained your solr logs. If so, take a look in them for indexing errors. 2) What is the difference between maxdocs and numdocs. This will give an indication if a large number of records are being deleted or updated. 3) Can you explain your partial updates? Are you sending the entire document again for the partial update?
Try to debug your next load. Perform the load, and watch the logs for errors. Write a program that loops through each doc and checks to see if the doc is present in the index. On Tue, Feb 12, 2013 at 6:19 AM, Macroman <peter0...@hotmail.com> wrote: > Our document ID's are most definately distinct and there are partial > updates > to existing records, I have run SQL queries outside of SOLR to validate > records going in and only about 1% are updates to existing records. There > are no deletes underway every day new records are added or updated. Example > for today. Before Data Handler ran, 13,586,537 records in SOLR all distinct > ID's. Records extracted from 7 different sources to go into SOLR index was > , > 45,345, of these 1,912 were updates to existing records. Thus 43,433 were > new records each with a new ID. I made sure ID's we always distinct. Yet > our > index now says 13,589,646. Indicating that only 3,109 new records went into > the index. Thus missing 40,324 records. I use Date Facet Range and can see > that there is an increase for January and February this year. In conclusion > I have to say that it must be removing earlier records somehow despite no > knowing where this may be controlled/set if at all. If there is a possible > configuration to remove or weed records where is this configured? Our SOLR > is virtually out of the box and only SOLCONFIG and SCHEMA amended to suit > the needs of our business for fields and field types indexed. We also have > configured the macro.s "Velocity" to display results. So none the wiser and > thank you to all whom have responded so far. > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Maximum-Number-of-Records-In-Index-tp4038961p4039908.html > Sent from the Solr - User mailing list archive at Nabble.com. > -- Joel Bernstein Professional Services LucidWorks
