bq: that is our requirmnt to load data into specific shard and later after rentention time we will delete that shard
Why is it necessary to delete a shard when deleting the old data by query removes it? This sounds like an XY problem. Someone has "required" that you enforce data retention by deleting a shard so you're asking about deleting shards. Whereas the problem is to purge old data that _could_ be accomplished by rotating shards, but is _also_ accomplished by just issuing a "delete all data more than 10 days old" query. But if it's a requirement, se the reference guide for "implicit" routing in the document routing sections. Best, Erick On Thu, Jul 20, 2017 at 8:15 AM, Susheel Kumar <susheel2...@gmail.com> wrote: > Agree. One should first try to measure the performance with standard/common > approach. > > On Thu, Jul 20, 2017 at 11:00 AM, Walter Underwood <wun...@wunderwood.org> > wrote: > >> I agree. Use the standard shard distribution and delete by query to remove >> older documents. >> >> Much, much simpler and probably faster at query time. >> >> I’m seeing a lot of e-mails about people trying to do fancy things with >> sharding before they’ve even tried and measured the performance. >> >> wunder >> Walter Underwood >> wun...@wunderwood.org >> http://observer.wunderwood.org/ (my blog) >> >> >> > On Jul 20, 2017, at 7:57 AM, Erick Erickson <erickerick...@gmail.com> >> wrote: >> > >> > Use the "implicit" router (being renamed "manual". that takes the >> > value of a particular field (_route_ by default) and sends docs to >> > that exact shard. >> > >> > But I also question whether sharding on this schema is a good idea. If >> > you have an access pattern where most queries are for, say, the last >> > two days then all the work will be done on only 2 machines and all the >> > rest will be idle. You should at least consider just using normal >> > routing that distributes the data across all shards and then use >> > delete-by-query to delete the data older than 10 days. >> > >> > Best, >> > Erick >> > >> > On Thu, Jul 20, 2017 at 12:51 AM, rehman kahloon >> > <mrehman_kahl...@yahoo.com.invalid> wrote: >> >> >> >> Hi Sir, >> >> Taken your id from your document on SlideShare. >> >> Need your guidance on my plan ,My target is to create >> sub-collection/shards within a collection. >> >> e.g >> >> Currently 1 have 10 days data and want to store data against >> each date in separate partitions. like oracle partition concepts (one >> table can have many partitions) >> >> Plan is to store each date data with in separate node, Total physical >> nodes are 10 and after 10 days, 11th date data load in node1 and existing >> data backup (oldest date data with purge and backed up). >> >> Please guide me how can i perform that using SolrCloud. 1 collection >> with unlimited sub collection. >> >> >> >> Thank you very much in advanced. >> >> >> >> Kind Regards,Muhammad Rehman Kahloon. >> >>
