This is what we do, A user logs in - enter s a few documents in a particular domain, say A, B or C - logs out. Say B is the most commonly used domain. The increase in index size is drastic only in this particular domain.
So unless a user logs in there s no question of documents being submitted or any indexing activity. >From the logs i see no user logged in during that time frame. Adding to this we have to take a back up of the index every day and that s how we even came to know that such a problem even existed. Is there a possibility that i could schedule an optimize to run at a specific time during the day and hence try control the index s file size? On Sat, Mar 16, 2013 at 6:51 PM, Erick Erickson [via Lucene] < ml-node+s472066n4047962...@n3.nabble.com> wrote: > Well, if nothing is going on at all, it's hard to see why the index would > increase. So I suspect _something_ is going on. Possibilities: > > 1> you have indexing activity going on. Even if it's just replacing docs > that already exist, which is actually an add and a delete the index will > grow for a while; the deleted info isn't removed from the index until the > segment is merged, which happens unpredictably (well, actually predictably > but not on a schedule you enforce). So the index would > grow/shrink/grow/shrink. Do you have automatic process in the background > that push docs to the index? > > 2> You forceMerge (optimize), in which case the index will at least double > in size temporarily. > > 3> You are replicating. While the replication goes on, especially if your > index has changed greatly, then your index could double. > > None of these fit the symptoms you describe very well mind you. It's > suspicious that these increases last long enough for you to see them > predictably in the morning, unless, say, a background process indexes > things regularly and you always look at the same time... unlikely at best. > > But the fact that your index goes from 18G to 8G strongly suggests that > you're doing a forceMerge/optimize when you see it bump up to 18G. > > Best > Erick > > > > > On Sat, Mar 16, 2013 at 4:52 AM, eanand333 <[hidden > email]<http://user/SendEmail.jtp?type=node&node=4047962&i=0>> > wrote: > > > Hi, I am kind of new in here. Got the same question... > > I am using Java version 1.6 and Lucene version 3.3. > > Can the index file size increase automatically over night? > > During the evening i see the size around 11GB, next day morning i see it > to > > be 18GB and again the size reduces around 8GB. > > I have checked the logs and i am sure that there was no user activity > > during > > this period. > > > > Mr. Eric has suggested this to be a user error, if that s true, i would > > like > > to know what are the possible errors which could result in the rapid > > increase in index file size ? > > > > Or what are the other possibilities for the index size to increase > > exponentially? > > > > Thanks > > > > > > > > -- > > View this message in context: > > > http://lucene.472066.n3.nabble.com/Can-index-size-increase-when-no-updates-optimizes-are-happening-tp3334022p4047945.html > > Sent from the Solr - User mailing list archive at Nabble.com. > > > > > ------------------------------ > If you reply to this email, your message will be added to the discussion > below: > > http://lucene.472066.n3.nabble.com/Can-index-size-increase-when-no-updates-optimizes-are-happening-tp3334022p4047962.html > To unsubscribe from Can index size increase when no updates/optimizes are > happening?, click > here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=3334022&code=ZWFuYW5kLjA2OUBnbWFpbC5jb218MzMzNDAyMnwtNTIyMjg4NDE1> > . > NAML<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > -- View this message in context: http://lucene.472066.n3.nabble.com/Can-index-size-increase-when-no-updates-optimizes-are-happening-tp3334022p4048444.html Sent from the Solr - User mailing list archive at Nabble.com.
