Re: Java heap space
Yonik Seeley wrote: Yes, on a delete operation. I'm not doing any commits until the end of all delete operations. I assume this is a delete-by-id and not a delete-by-query? They work very differently. Yes, all queries are delete-by-id. If you are first deleting so you can re-add a newer version of the document, you don't need too... overwriting older documents based on the uniqueKeyField is something Solr does for you! Yes, I know. But the articles in our (sql-)database get new IDs when they are changed so they need to be deleted an re-inserted into the index. Is it possible to use a profiler to see where all the memory is going? It sounds like you may have uncovered a memory leak somewhere. I'm not that experienced concerning Java, but maybe if you give me some advice I'm glad if I can help. So far I had a quick look at JMP once but that's all. Don't hesitate to write me a PM on that subject. Also what OS, what JVM, what appserver are you using? OS: Linux (Debian GNU/Linux i686) JVM: Java HotSpot(TM) Server VM (build 1.5.0_06-b05, mixed mode) of Sun's JDK 5.0. Currently I'm using the Jetty installation from the solr nightly builds for test purposes. Marcus
Ruby bindings/SolRuby?
Is anyone working on Ruby bindings? I noticed a page called "SolRuby" on the wiki - what is this? I would like to make Solr work like a model in Rails if possible, but don't want to reinvent the wheel. Thanks, Brian
Re: Java heap space
: > If you are first deleting so you can re-add a newer version of the : > document, you don't need too... overwriting older documents based on : > the uniqueKeyField is something Solr does for you! : : Yes, I know. But the articles in our (sql-)database get new IDs when : they are changed so they need to be deleted an re-inserted into the index. this is off the subject of the heap space issue ... but if the id changes, then maybe it shouldn't be the uniqueId of your index? .. your code must have someone of recognizing that article B with id 222 is a changed version of article A with id 111 (otherwise how would you know to delete 111 when you insert 222?) ..whatever that mechanism is, perhaps it should determine your uniqueKey? -Hoss
Re: Java heap space
Chris Hostetter wrote: > this is off the subject of the heap space issue ... but if the id changes, > then maybe it shouldn't be the uniqueId of your index? .. your code must > have someone of recognizing that article B with id 222 is a changed > version of article A with id 111 (otherwise how would you know to delete > 111 when you insert 222?) ..whatever that mechanism is, perhaps it should > determine your uniqueKey? No, there is no "key" or something that reveals a relation between new article B and old article A. After B is inserted and A is deleted, all of A's existence is gone and we do not even know that B is A's "successor". Changes are simply kept in a table which tells the system which IDs to delete and which new (or changed) articles to insert, automatically giving them new IDs. I know this may not be (or at least sound) perfect and it is not the way things are handled normally. But this works fine for our needs. We gather information about changes to our data during the day and apply them on a nightly update (which, I know, does not imply that IDs have to change). So, yes, I'm sure I got the right uniqueKey. ;-) Marcus
Adding xml to SolrQueryResponse
Is there a way to add attributes besides name to an xml node returned from SolrQueryResponse? I've looked at the SolrQueryResponse.add and it looks like a NamedList is my only option. I know that I can get by with nodes that have only the name attribute but it would make life a little easier to throw some more attributes on a node. Thanks, Mike Austin
Re: Adding xml to SolrQueryResponse
On 5/1/06, Mike Austin <[EMAIL PROTECTED]> wrote: Is there a way to add attributes besides name to an xml node returned from SolrQueryResponse? I've looked at the SolrQueryResponse.add and it looks like a NamedList is my only option. I know that I can get by with nodes that have only the name attribute but it would make life a little easier to throw some more attributes on a node. There isn't currently, and making things too extensible will get you back to unconstrained XML again (might as well use DOM or something). Hmmm, but one could put random XML in the response by making an XML type that you would get if you put a DOM object in the NamedList for the response. // normal Solr result
Re: Adding xml to SolrQueryResponse
: Is there a way to add attributes besides name to an xml node returned from : SolrQueryResponse? I've looked at the SolrQueryResponse.add and it looks : like a NamedList is my only option. I know that I can get by with nodes : that have only the name attribute but it would make life a little easier to : throw some more attributes on a node. FYI: The orriginal thinking behind the current XML format was that itshould only contain concepts that were easy to map to the "primitive" types found in most type safe langauges: int/float/date/string/etc... exists to provide a simple mechanism to support both: Ordered lists of heterogenously typed objects; and Un-Ordered name/value pairs ... all depending on what you prefer. Allowing arbitrarry attributes would complicate that simple mapping of tag to primitive object -- Which doesn't mean I'm opposed to the idea, I acctually think it would be nice if there was a simple way to add a DOM sub-trees directly to the response -- but I'd advocate not having any core Solr code use this ... mainly because one of the "Near Term" goals is to get an XML DTD/Schema that validates all legal responses from any (real or theoretical) RequestHandler. Allowing arbitrary XL attributes would illiminate SolrCore's ability to ensure that the XML responses it generates validate with that DTD/Schema -- so if we did add support for it, we'd want to treat it special in some way. -Hoss
