Re: Need guidance solrcloud shardings with date interval

2017-07-25 Thread Erick Erickson
ing specific shard name >> to load data. >> >> but my R&D is to find a way, not to use shard name using loading >> time,solrcloud automatically load data into predefined shard/date specific >> shard. >> >> Is there any way to perfome this? >&

Re: Need guidance solrcloud shardings with date interval

2017-07-25 Thread Walter Underwood
> > > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Need-guidance-solrcloud-shardings-with-date-interval-tp4346936p4347250.html > Sent from the Solr - User mailing list archive at Nabble.com.

Re: Need guidance solrcloud shardings with date interval

2017-07-23 Thread Susheel Kumar
If you decide to go with multiple collection and aliasing, this would be useful https://blog.cloudera.com/blog/2013/10/collection-aliasing-near-real-time-search-for-really-big-data/ On Sat, Jul 22, 2017 at 10:37 AM, Shawn Heisey wrote: > On 7/22/2017 5:02 AM, m rehman kahloon wrote: > > but my

Re: Need guidance solrcloud shardings with date interval

2017-07-22 Thread Shawn Heisey
On 7/22/2017 5:02 AM, m rehman kahloon wrote: > but my R&D is to find a way, not to use shard name using loading > time,solrcloud automatically load data into predefined shard/date specific > shard. The implicit router is the only one you can use when you're doing time interval sharding, because i

Re: Need guidance solrcloud shardings with date interval

2017-07-22 Thread m rehman kahloon
name using loading time,solrcloud automatically load data into predefined shard/date specific shard. Is there any way to perfome this? Once again thanks Sir. -- View this message in context: http://lucene.472066.n3.nabble.com/Need-guidance-solrcloud-shardings-with-date-inte

Re: Need guidance solrcloud shardings with date interval

2017-07-22 Thread m rehman kahloon
shard name, document automatially load with respect to its date. thanks, waiting still guidance -- View this message in context: http://lucene.472066.n3.nabble.com/Need-guidance-solrcloud-shardings-with-date-interval-tp4346936p4347249.html Sent from the Solr - User mailing list archive at

RE: Need guidance solrcloud shardings with date interval

2017-07-20 Thread Davis, Daniel (NIH/NLM) [C]
[mailto:wun...@wunderwood.org] Sent: Thursday, July 20, 2017 1:24 PM To: solr-user@lucene.apache.org Subject: Re: Need guidance solrcloud shardings with date interval Well, you have bad problem. You have a requirement that forces you to build an expensive, unreliable search system. You need to

Re: Need guidance solrcloud shardings with date interval

2017-07-20 Thread Walter Underwood
Well, you have bad problem. You have a requirement that forces you to build an expensive, unreliable search system. You need to do specific shard creation at specific times every day. What happens if that fails? Does search go down until it is fixed because all searches are going to a shard tha

Re: Need guidance solrcloud shardings with date interval

2017-07-20 Thread Erick Erickson
bq: that is our requirmnt to load data into specific shard and later after rentention time we will delete that shard Why is it necessary to delete a shard when deleting the old data by query removes it? This sounds like an XY problem. Someone has "required" that you enforce data retention by dele

Re: Need guidance solrcloud shardings with date interval

2017-07-20 Thread rehman kahloon
blockquote, div.yahoo_quoted { margin-left: 0 !important; border-left:1px #715FFA solid !important; padding-left:1ex !important; background-color:white !important; } Hi Eric,   Thank you very much for your guidance. No sir that is our requirmnt to load data into specific shard and later after r

Re: Need guidance solrcloud shardings with date interval

2017-07-20 Thread Susheel Kumar
Agree. One should first try to measure the performance with standard/common approach. On Thu, Jul 20, 2017 at 11:00 AM, Walter Underwood wrote: > I agree. Use the standard shard distribution and delete by query to remove > older documents. > > Much, much simpler and probably faster at query time

Re: Need guidance solrcloud shardings with date interval

2017-07-20 Thread Walter Underwood
I agree. Use the standard shard distribution and delete by query to remove older documents. Much, much simpler and probably faster at query time. I’m seeing a lot of e-mails about people trying to do fancy things with sharding before they’ve even tried and measured the performance. wunder Walt

Re: Need guidance solrcloud shardings with date interval

2017-07-20 Thread Erick Erickson
Use the "implicit" router (being renamed "manual". that takes the value of a particular field (_route_ by default) and sends docs to that exact shard. But I also question whether sharding on this schema is a good idea. If you have an access pattern where most queries are for, say, the last two day

Need guidance solrcloud shardings with date interval

2017-07-20 Thread rehman kahloon
Hi Sir,             Taken your id from your document on SlideShare. Need your guidance on my plan ,My target is to create sub-collection/shards within a collection. e.g          Currently 1 have 10 days data and want to store data against each date in separate partitions.  like oracle partition