I've seen similar errors when large background merges happen while
looping in a result set. See http://lucene.grantingersoll.com/2008/07/16/mysql-solr-and-communications-link-failure/
On Jan 9, 2009, at 12:50 PM, Mark Miller wrote:
Your basically writing segments more often now, and somehow avoiding
a longer merge I think. Also, likely, deduplication is probably
adding enough extra data to your index to hit a sweet spot where a
merge is too long. Or something to that effect - MySql is especially
sensitive to timeouts when doing a select * on a huge db in my
testing. I didnt understand your answer on the autocommit - I take
it you are using it? Or no?
All a guess, but it def points to a merge taking a bit long and
causing a timeout. I think you can relax the MySql timeout settings
if that is it.
I'd like to get to the bottom of this as well, so any other info you
can provide would be great.
- Mark
Marc Sturlese wrote:
Hey Shalin,
In the begining (when the error was appearing) i had
<ramBufferSizeMB>32</ramBufferSizeMB>
and no maxBufferedDocs set
Now I have:
<ramBufferSizeMB>32</ramBufferSizeMB>
<maxBufferedDocs>50</maxBufferedDocs>
I think taht setting maxBufferedDocs to 50 I am forcing more disk
writting
than I would like... but at least it works fine (but a bit
slower,opiously).
I keep saying that the most weird thing is that I don't have that
problem
using solr1.3, just with the nightly...
Even that it's good that it works well now, would be great if
someone can
give me an explanation why this is happening
Shalin Shekhar Mangar wrote:
On Fri, Jan 9, 2009 at 9:23 PM, Marc Sturlese
<marc.sturl...@gmail.com>wrote:
hey there,
I hadn't autoCommit set to true but I have it sorted!!!!! The error
stopped
appearing after setting the property maxBufferedDocs in
solrconfig.xml. I
can't exactly undersand why but it just worked.
Anyway, maxBufferedDocs is deprecaded, would ramBufferSizeMB do
the same?
What I find strange is this line in the exception:
"Last packet sent to the server was 202481 ms ago."
Something took very very long to complete and the connection got
closed by
the time the next row was fetched from the opened resultset.
Just curious, what was the previous value of maxBufferedDocs and
what did
you change it to?
--
View this message in context:
http://www.nabble.com/Deduplication-patch-not-working-in-nightly-build-tp21287327p21374908.html
Sent from the Solr - User mailing list archive at Nabble.com.
--
Regards,
Shalin Shekhar Mangar.
--------------------------
Grant Ingersoll
Lucene Helpful Hints:
http://wiki.apache.org/lucene-java/BasicsOfPerformance
http://wiki.apache.org/lucene-java/LuceneFAQ
