Re: Index complex JSON data in SOLR

2014-11-20 Thread Renaud Delbru
Hi David, you might want to look at SIREn 1.4 [1], a plugin for Lucene/Solr, that includes a update handler [2] which mimics elasticsearch index api. You can push JSON documents to the API and it will dynamically flatten and index the JSON documents into a set of fields (similar to Elasticsea

Re: Index complex JSON data in SOLR

2014-11-15 Thread David Lee
Thanks Alex and William for the suggestions. I'll try out the approach storing the JSON string. On Sat, Nov 15, 2014 at 5:27 PM, William Bell wrote: > You can take 4.* of Solr and just apply my fix. > > Store JSON stringified into a string field (make sure the field name ends > in _json). Then

Re: Index complex JSON data in SOLR

2014-11-15 Thread William Bell
You can take 4.* of Solr and just apply my fix. Store JSON stringified into a string field (make sure the field name ends in _json). Then you can output with: wt=json&json.fsuffix=_json OK? Use SOLR-4685. On Sat, Nov 15, 2014 at 5:07 PM, Alexandre Rafalovitch wrote: > It sounds to me that y

Re: Index complex JSON data in SOLR

2014-11-15 Thread Alexandre Rafalovitch
It sounds to me that you are not actually searching on the state or price. So, does it make sense to store it in Solr? Maybe it should stay in external database and you merge it. Or store (not index) that json as pure text field and parse what you need out of it manually, as you would with Elastics

Re: Index complex JSON data in SOLR

2014-11-15 Thread David Lee
Assume that we are selling a product online to 50 states in the USA. But each state has its own price. ALthough the base product information is the same, the index size will increase 50 times if we index that way. The usage is similar as searching a product; but based on the location of the us

Re: Index complex JSON data in SOLR

2014-11-15 Thread Alexandre Rafalovitch
The first link shows how to create children with specific content, but you need to use "_childDocuments_":... explicitly instead of the "prices: " and perhaps add "type: price" or some such to differentiate record types. But I am not quite following why you say it will increase 50 times. By compar

Re: Index complex JSON data in SOLR

2014-11-15 Thread David Lee
Thanks Alex. I take a look at the approach of transforming JSON document before mapping it to the Solr schema at http://lucidworks.com/blog/indexing-custom-json-data/ . It's a walk-around. But in my case, if every state has its own price, the number of documents needs to be indexed will inc

Re: Index complex JSON data in SOLR

2014-11-15 Thread Alexandre Rafalovitch
It's simple in Elasticsearch, but what you actually get is a single document and all it's children data ({state, price}) entries are joined together behind the scenes into the multivalued fields. Which may or may not be an issue for you. For Solr, nested documents need to be parent/child separate