Re: Multiple indices
: norms, term vectors, field caches, etc. It almost sounds like one : would want multiple instances of Solr in the same app container. But : if that's the case, you aren't saving much over just having multiple : app servers. I didn't say anything yesterday because i pretty much agreed with yonik ... but it occured to me this morning that if you have a finite amout of ram on a box (which we all do) 'R' gigs, and you want to run a variable number of solr indexs 'N", the only way to do this currently is to run N appserver instances and explicitly configure the JVM for each to use at most R/N Gigs max heap ... which would bnot only make addign a new index a pain (you have to reconfigure every appserver instance) but it also means that you are managine your memory instead of letting java do it. if it was possible to run several indexes in one appserver, then you wouldn't have to wworry about this (but if you wnated to worry about it, you could run some or all of your indexes in their own appserver instances) perhaps one way to go would be if solr paid attention to what webapp name it was being run in, and used that to determine the directory it looked in for it's configs/index? ... so most people would just intall "solr.war" and it would look for everything in ./solr/ -- but if i want multiple idexes i copy solr.war was solr-people.war, solr-places.war, and solr-things.war, and now i've got 3 webapps running solr, and 3 seperate directories containing 3 seperate indexes and configs... ./solr-people/, ./solr-places/, ./solar-things, etc... -Hoss
Re: Multiple indices
Just a quick comment not sure how things are really set up for Solr, but one could use the /solr/index/XXX URL "scheme" to determine which index the request is for. Configs for different indices could be named after that "index" part in the URL, and that "index" part could be used as the key in various data structures that store index-specific information. Otis - Original Message From: Chris Hostetter <[EMAIL PROTECTED]> To: solr-user@lucene.apache.org Sent: Wednesday, March 22, 2006 3:10:36 PM Subject: Re: Multiple indices : norms, term vectors, field caches, etc. It almost sounds like one : would want multiple instances of Solr in the same app container. But : if that's the case, you aren't saving much over just having multiple : app servers. I didn't say anything yesterday because i pretty much agreed with yonik ... but it occured to me this morning that if you have a finite amout of ram on a box (which we all do) 'R' gigs, and you want to run a variable number of solr indexs 'N", the only way to do this currently is to run N appserver instances and explicitly configure the JVM for each to use at most R/N Gigs max heap ... which would bnot only make addign a new index a pain (you have to reconfigure every appserver instance) but it also means that you are managine your memory instead of letting java do it. if it was possible to run several indexes in one appserver, then you wouldn't have to wworry about this (but if you wnated to worry about it, you could run some or all of your indexes in their own appserver instances) perhaps one way to go would be if solr paid attention to what webapp name it was being run in, and used that to determine the directory it looked in for it's configs/index? ... so most people would just intall "solr.war" and it would look for everything in ./solr/ -- but if i want multiple idexes i copy solr.war was solr-people.war, solr-places.war, and solr-things.war, and now i've got 3 webapps running solr, and 3 seperate directories containing 3 seperate indexes and configs... ./solr-people/, ./solr-places/, ./solar-things, etc... -Hoss
SEVERE: java.lang.OutOfMemoryError: Java heap space
Occasionally when inserting I get the error message SEVERE: java.lang.OutOfMemoryError: Java heap space Any clues how to track down when&where it's happening? Or any good way I can get better clues how to track it down? I'm doing inserts of about 1000 documents at a time between commits. Would doing a smaller number avoid this problem. Or, better - any way I can calculate how many docs I can insert in a batch without triggering this?
Re: SEVERE: java.lang.OutOfMemoryError: Java heap space
On 3/22/06, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > > Occasionally when inserting I get the error message > SEVERE: java.lang.OutOfMemoryError: Java heap space > Any clues how to track down when&where it's happening? > Or any good way I can get better clues how to track it down? What's the heap size of the JVM. The default is normally pretty small. If you are using the example, just put it on the command line... java -Xmx256m -jar start.jar > I'm doing inserts of about 1000 documents at a time between > commits. Would doing a smaller number avoid this problem. No, that shouldn't be a problem. Solr only keeps a single hashtable entry for each unique id of uncommitted documents so it can delete duplicates during the commit. The exception probably hits when lucene needs to merge segments. There isn't much you can do to avoid that memory usage except increase the heap size. BTW, How many indexed fields do you have? -Yonik
