Re: concurrency problem with delta-import (indexing various cores simultaniously)

2009-02-28 Thread Marc Sturlese
are being hit by garbage collection. Memory utilization >>> should >>> go down once an import completes. Which GC are you using? There have >>> been >>> a >>> few recent threads on GC settings. Perhaps you can try out a few of >>> those >&

Re: concurrency problem with delta-import (indexing various cores simultaniously)

2009-02-27 Thread Ryuuichi KUMAI
go down once an import completes. Which GC are you using? There have been >> a >> few recent threads on GC settings. Perhaps you can try out a few of those >> settings. I don't know how big your documents/index are but if possible >> give >> it more memory. >> >> -- >> Regards, >> Shalin Shekhar Mangar. >> >> > > -- > View this message in context: > http://www.nabble.com/concurrency-problem-with-delta-import-%28indexing-various-cores-simultaniously%29-tp22120430p22125716.html > Sent from the Solr - User mailing list archive at Nabble.com. > >

Re: concurrency problem with delta-import (indexing various cores simultaniously)

2009-02-20 Thread Marc Sturlese
try out a few of those > settings. I don't know how big your documents/index are but if possible > give > it more memory. > > -- > Regards, > Shalin Shekhar Mangar. > > -- View this message in context: http://www.nabble.com/concurrency-problem-with-delta-import-%28indexing-various-cores-simultaniously%29-tp22120430p22125716.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: concurrency problem with delta-import (indexing various cores simultaniously)

2009-02-20 Thread Shalin Shekhar Mangar
On Fri, Feb 20, 2009 at 11:23 PM, Marc Sturlese wrote: > > Yes, > Now it's almost tree days non-stop since I am running updates with the 3 > cores with cron jobs. If there are updates of 1 docs everything is > alrite. When I start doing updates of 30 is when that core runs really > slow. I

Re: concurrency problem with delta-import (indexing various cores simultaniously)

2009-02-20 Thread Marc Sturlese
f Ryuuichi's fixes? > > -- > Regards, > Shalin Shekhar Mangar. > > -- View this message in context: http://www.nabble.com/concurrency-problem-with-delta-import-%28indexing-various-cores-simultaniously%29-tp22120430p22125443.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: concurrency problem with delta-import (indexing various cores simultaniously)

2009-02-20 Thread Shalin Shekhar Mangar
On Fri, Feb 20, 2009 at 10:43 PM, Marc Sturlese wrote: > > Hey, > Yeah, I patched the bug reported by Ryuuichi of the SimpleDateFormat > aswell. > Is there any other known concurrency bug that maybe I am missing? > In my use case I could manage to index not concurrently but would like to > discove

Re: concurrency problem with delta-import (indexing various cores simultaniously)

2009-02-20 Thread Marc Sturlese
ixed > recently. > > See https://issues.apache.org/jira/browse/SOLR-1017 (which was fixed on > 11th > Feb) > -- > Regards, > Shalin Shekhar Mangar. > > -- View this message in context: http://www.nabble.com/concurrency-problem-with-delta-import-%28indexing-various-cores-simultaniously%29-tp22120430p22123287.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: concurrency problem with delta-import (indexing various cores simultaniously)

2009-02-20 Thread Shalin Shekhar Mangar
On Fri, Feb 20, 2009 at 8:41 PM, Marc Sturlese wrote: > > Hey there, > I am indexing 3 cores concurrently from 3 diferent mysql tables (I do it > every 5 minutes with a cron job). > The three cores use JdbcDataSource as datasource in data-config.xml > Reached a point, the core that fetches more my

concurrency problem with delta-import (indexing various cores simultaniously)

2009-02-20 Thread Marc Sturlese
(but the other tow keep working fine)...but java doesn't throw and exception... I am using a nightly from early january. I found someone experienced the same problem and uploaded a templateString patch to make it thread-save. http://www.nabble.com/Concurrency-problem-with-delta-import-td216

Re: Concurrency problem with delta-import

2009-01-26 Thread Ryuuichi KUMAI
Hello Shalin, Thank you for your reply. I opened the issue and attached the patch. https://issues.apache.org/jira/browse/SOLR-985 > The lists are OK since they are modified only in the constructor. The map > needs to be changed to a ConcurrentHashMap as you did in the patch. I understood. Many

Re: Concurrency problem with delta-import

2009-01-26 Thread Shalin Shekhar Mangar
The lists are OK since they are modified only in the constructor. The map needs to be changed to a ConcurrentHashMap as you did in the patch. On Mon, Jan 26, 2009 at 7:23 PM, Shalin Shekhar Mangar < shalinman...@gmail.com> wrote: > Wow, well spotted. TemplateString is not thread-safe but it is be

Re: Concurrency problem with delta-import

2009-01-26 Thread Shalin Shekhar Mangar
Wow, well spotted. TemplateString is not thread-safe but it is being used concurrently by many cores due to the static instance. Apart from the cache map, the lists will also need to be taken care of. Can you please open an issue and attach this patch? https://issues.apache.org/jira/browse/SOLR

Concurrency problem with delta-import

2009-01-26 Thread Ryuuichi KUMAI
Hello, I'm using Solr 1.3 and I have a problem with DataImportHandler. My environment: - Solr 1.3 - MySQL 5.1.30, Connector/J 5.1.6 - Linux 2.6.9 x86_64 (RHEL4) - Sun JDK 1.6.0_11 - Apache Tomcat 6.0.18 Our Solr server has multi core, and the schema in each core is the same. When delta-impo