Re: need suggestions for storing TBs of strutucred data in SolrCloud

2014-03-06 Thread Shawn Heisey
On 3/6/2014 12:17 AM, Chia-Chun Shih wrote: > I am planning a system for searching TB's of structured data in SolrCloud. > I need suggestions for handling such huge amount of data in SolrCloud. > (e.g., number of shards per collection, number of nodes, etc.) > > Here are some specs of the system:

Re: need suggestions for storing TBs of strutucred data in SolrCloud

2014-03-06 Thread Toke Eskildsen
On Thu, 2014-03-06 at 08:17 +0100, Chia-Chun Shih wrote: >1. Raw data is 35,000 CSV files per day. Each file is about 5 MB. >2. One collection serves one day. 200-day history data is required. So once your data are indexed, they will not change? If seems to me that 1 shard/day is a fine ch

need suggestions for storing TBs of strutucred data in SolrCloud

2014-03-05 Thread Chia-Chun Shih
Hi, I am planning a system for searching TB's of structured data in SolrCloud. I need suggestions for handling such huge amount of data in SolrCloud. (e.g., number of shards per collection, number of nodes, etc.) Here are some specs of the system: 1. Raw data is 35,000 CSV files per day. Each