Hi,

 

I wonder if anyone could shed some insight on a dynamic indexing question...?

 

The basic requirement is this:

 

Indexing:

A process writes to an index, and when it reaches a certain size (say, 1GB), a 
new index (core) is 'automatically' created/deployed (i.e. the process doesn't 
know about it) and further indexing now goes into the new core. When that one 
reaches its threshold size, a new index is deplyoed, and so on.

The process that is writing to the indices doesn't actually know that it is 
writing to different cores.

 

Searching:

When a search is directed at the above index, the actual search is a 
distrbitued shard search across all the shards that have been deployed. Again, 
the searcher process doesn't know this, but gets back the aggregated results, 
as if it had specified all the shards in the request URL, but as these are 
changing dynamically, it of course can't know what they all are at any given 
time.

 

This requirement sounds to me perhaps like a Katta thing. I've had a look at 
Solr-1395, and there's questions in Lucid that sound similar (e.g. 
http://www.lucidimagination.com/search/document/4b3d00055413536d/solr_katta_integration#4b3d00055413536d),
 so I guess (hope) I'm not the only one with this requirement.

 

I couldn't find anything in either Katta or SOLR-1395 that fit both the writing 
and searching requirement, but I could easily have missed it.

 

Is Katta/Solr-1395 the way to go to achieve this? Would such a solution be 
'production-ready'? Has anyone deployed this type of thing in a production 
environment?

 

Any insight/advice would be greatly appreciated.

 

Thanks!

Peter

 

 
                                          
_________________________________________________________________
Do you have a story that started on Hotmail? Tell us now
http://clk.atdmt.com/UKM/go/195013117/direct/01/

Reply via email to