Re: Size of ID field

2013-10-06 Thread Zahoor Mohamed
Thanks On Fri, Oct 4, 2013 at 5:56 PM, Jack Krupansky wrote: > It all depends. I mean, if you have 20 million URLs averaging 40 > characters each, that's 80 MB, not a big deal at all, but if you have 20 > billion URLs that would take up 80 GB, which might be a big deal. But if > you shard those

Re: Size of ID field

2013-10-04 Thread Jack Krupansky
It all depends. I mean, if you have 20 million URLs averaging 40 characters each, that's 80 MB, not a big deal at all, but if you have 20 billion URLs that would take up 80 GB, which might be a big deal. But if you shard those 20 billion documents into 10 shards, 8 GB may or may not be a big dea

Re: Size of ID field

2013-10-04 Thread Dmitry Kan
Using arbitrary strings affects at least on the traffic between the shard(s) and a querying client or shards and a frontend solr instance. We have actually hit such an issue, described here: https://issues.apache.org/jira/browse/SOLR-4903, which has triggered the suggestion for ids compaction: http