Re: Solr hanging when extracting a some broken .doc files

2013-12-19 Thread Augusto Camarotti
Hey Andrea! thanks for answering, this is the complete stack trace is following below. (the other is just the same): I'm going to try that modification of the logging level but i'm really considering to debug tika and try to correct it myself. 03:38:23ERRORSolrCoreorg.apache.solr.common.Solr

Re: Solr hanging when extracting a some broken .doc files

2013-12-19 Thread Raymond Wiker
On Thu, Dec 19, 2013 at 10:01 AM, Charlie Hull wrote: > On 18/12/2013 09:03, Alexandre Rafalovitch wrote: > >> Charlie, >> >> Does it mean you are talking to it from a client program? Or are you >> running Tika in a listen/server mode and build some adapters for standard >> Solr processes? >> > >

Re: Solr hanging when extracting a some broken .doc files

2013-12-19 Thread Charlie Hull
On 18/12/2013 09:03, Alexandre Rafalovitch wrote: Charlie, Does it mean you are talking to it from a client program? Or are you running Tika in a listen/server mode and build some adapters for standard Solr processes? If we're writing indexers in Python we usually run Tika as a server - which

Re: Solr hanging when extracting a some broken .doc files

2013-12-18 Thread Alexandre Rafalovitch
Charlie, Does it mean you are talking to it from a client program? Or are you running Tika in a listen/server mode and build some adapters for standard Solr processes? Regards, Alex. Personal website: http://www.outerthoughts.com/ LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch - Ti

Re: Solr hanging when extracting a some broken .doc files

2013-12-18 Thread Charlie Hull
On 17/12/2013 15:29, Augusto Camarotti wrote: Hi guys, I'm having a problem with solr when trying to index some broken .doc files. I have set up a test case using Solr to index all the files the users save on the shared directorys of the company that i work for and Solr is hanging when tr

Re: Solr hanging when extracting a some broken .doc files

2013-12-17 Thread Andrea Gazzarini
Hi Augusto, I don't believe the mailing list allows attachments. Could you please post the complete stacktrace? In addition, set the logging level of tika classes to FINEST in solr console, maybe can be helpful Best, Andrea On 17 Dec 2013 16:30, "Augusto Camarotti" wrote: > Hi guys, > >I'm

Re: solr hanging, possible memory issues?

2012-09-21 Thread Otis Gospodnetic
Hi Kevin, Try taking a heap dump snapshot and analyzing it with something like YourKit to see what's eating the memory. SPM for Solr (see signature) will show you JVM heap and GC numbers/graphs/activity that may shed some light on the issue. You could also turn on verbose GC logging and/or use jst

Re: solr hanging regularly, thread dump attached, PDFOperator

2012-08-30 Thread Jack Krupansky
I've heard of a couple of situations where the parsing of multiple PDF files in parallel was problematic with lots of blocked threads. There may or may not be bugs in PDFBox, but one workaround is simply to only send one PDF at a time to Solr, or at least a smaller number in parallel. I don't

Re: Solr hanging

2012-04-25 Thread Trym R. Møller
Hi Chris Hostetter Does that mean, that the last two questions I have posted hasn't reached the mailing list? Best regards Trym Den 25-04-2012 19:58, Chris Hostetter skrev: : Subject: Solr hanging : References:<31fdac6b-c4d9-4383-865d-2faca0f09...@geekychris.com> : : In-Reply-To: : https://

Re: Solr hanging

2012-04-25 Thread Chris Hostetter
: Subject: Solr hanging : References: <31fdac6b-c4d9-4383-865d-2faca0f09...@geekychris.com> : : In-Reply-To: : https://people.apache.org/~hossman/#threadhijack Thread Hijacking on Mailing Lists When starting a new discussion on a mailing list, please do not reply to an existing message,

Re: Solr Hanging

2012-04-23 Thread Mark Miller
And see https://issues.apache.org/jira/browse/SOLR-683 as it also may be related or have helpful info... On Apr 23, 2012, at 8:17 AM, Mark Miller wrote: > Perhaps related is > http://www.lucidimagination.com/search/document/6d0e168c82c86a38#45c945b2de6543f4 > > On Apr 23, 2012, at 5:37 AM, Try

Re: Solr Hanging

2012-04-23 Thread Mark Miller
Perhaps related is http://www.lucidimagination.com/search/document/6d0e168c82c86a38#45c945b2de6543f4 On Apr 23, 2012, at 5:37 AM, Trym R. Møller wrote: > Hi > > I have succeeded in reproducing the scenario with two Solr instances running. > They cover a single collection with two slices and tw

Re: Solr Hanging

2012-04-23 Thread Trym R. Møller
Hi I have succeeded in reproducing the scenario with two Solr instances running. They cover a single collection with two slices and two replica, two cores in each Solr instance. I have changed the number of threads that Jetty is allowed to use as follows: 3 3 0 And when indexing a single do

Re: Solr Hanging

2012-04-19 Thread Trym R. Møller
Thanks for your answer. I am running an (older) revision of solr from around the 29/2-2012 I suspect that the thread I have included is the leader of the shard? The Solr instance, I have the dump from, contains more than one leader, so I don't know which shard (slice) the thread is working on.

Re: Solr Hanging

2012-04-19 Thread Yonik Seeley
On Thu, Apr 19, 2012 at 4:25 AM, "Trym R. Møller" wrote: > Hi > > I am using Solr trunk and have 7 Solr instances running with 28 leaders and > 28 replicas for a single collection. > After indexing a while (a couple of days) the solrs start hanging and doing > a thread dump on the jvm I see blocke

Re: Solr Hanging While Building Suggester Index

2011-09-28 Thread Stephen Duncan Jr
No, this is on a test system that is still smallish, approx 100,000 records of dummy data with Wikipedia articles as content at the time this occurred. I wouldn't expect rebuilding the index to stall the entire JVM, that seems excessive... Stephen Duncan Jr www.stephenduncanjr.com On Wed, Sep

Re: Solr Hanging While Building Suggester Index

2011-09-28 Thread Markus Jelsma
Is this a huge index? Keep in mind that most spellchecker implementations rebuild the index which can stall the entire process if there are millions of full text documents to process. There is a new implementation called DirectSolrSpellchecker that doens't so a complete rebuild but i haven't tr

Re: Solr Hanging all of sudden with update/csv

2011-03-11 Thread danomano
Sweet, those links very very useful :). and should most definitely help :) One overriding concern I have: 1) if I were to simply update the config to use a different mergeFactor, and restart the solr server, (would it then adjust the segments accordingly?) or would I need to start from scratch..(

Re: Solr Hanging all of sudden with update/csv

2011-03-09 Thread Jason Rutherglen
You will need to cap the maximum segment size using LogByteSizeMergePolicy.setMaxMergeMB. As then you will only have segments that are of an optimal size, and Lucene will not try to create gigantic segments. I think though on the query side you will run out of heap space due to the terms index si

Re: Solr Hanging all of sudden with update/csv

2011-03-09 Thread Otis Gospodnetic
m/ :: Solr - Lucene - Nutch Lucene ecosystem search :: http://search-lucene.com/ - Original Message > From: danomano > To: solr-user@lucene.apache.org > Sent: Wed, March 9, 2011 1:17:08 PM > Subject: Re: Solr Hanging all of sudden with update/csv > > After About 4-5 ho

Re: Solr Hanging all of sudden with update/csv

2011-03-09 Thread danomano
After About 4-5 hours the merge completed (ran out of heap)..as you suggested..it was having memory issues.. Read queries during the merge were working just fine (they were taking longer then normal ~30-60seconds). I think I need to do more reading on understanding the merge/optimization processe

Re: Solr Hanging all of sudden with update/csv

2011-03-08 Thread Jason Rutherglen
> The index size itself is about 270Gb, (we are hopping to support upto > 500-1TB), and have supplied the system with ~3TB diskspace. That's simply massive for a single node. When the system tries to merge the segments the queries are probably not working? And the merges will take quite a while.

Re: Solr Hanging all of sudden with update/csv

2011-03-08 Thread danomano
Actually this is definitely not a ram issue. I have visualVM connected and MAX Ram available for the JavaVM is ~7GB, but the system is only using ~5.5GB, with a MAX so far of 6.5GB consumed. I think..well I'm guessing the system hit a merge threshold, but I can't tell for sure..I have seen the in

Re: Solr Hanging all of sudden with update/csv

2011-03-08 Thread Jonathan Rochkind
My guess is that you're running out of RAM. Actual Java profiling is beyond me, but I have seen issues on updating that were solved by more RAM. If you are updating every few minutes, and your new index takes more than a few minutes to warm, you could be running into overlapping warming index