Re: Using join vs flattening structure

2015-08-31 Thread Erick Erickson
Mostly just do the most naive data-flattening you can and see how big the index is. You really have to generate the index then run representative queries at it. But naively flattening the data in this case approaches 15B documents, which is a problem, you're sharding over quite a few shards etc.

Re: Using join vs flattening structure

2015-08-31 Thread Brian Narsi
We have about 15 million items. Each item has 10 attributes that we are indexing at this time. We are planning on adding 15 more attributes in future. We have about 1 customers. Each of the items mentioned above can have special pricing, etc for each of the customers. There are 6 attributes of

Re: Using join vs flattening structure

2015-08-31 Thread Erick Erickson
For 1-3, test and see. The problem I often see is that it is _assumed_ that flattening the data will cost a lot in terms of index size and maintenance. Test that assumption before going down the relational road. You haven't talked about how many documents you have, how much data would have to be r