1. Relatively small numbers of dynamic fields are fine. 20-40 would be fine.
1,000 would become problematic. "hundreds" would also likely be problematic,
but it would depend on the application and its data.
2. Sparse dynamic fields are less problematic than larger numbers of fully
populated fields.
3. Keep the field names to Java-like rules, no special characters, just
letters, digits, and underscores.
4. Keep the field name length relative short - don't store large data values
in the field names themselves.
Generally, "know your data". Static is preferable, but if your data gets too
dynamic and you no longer have any clue as to how fields there are, how many
are populated, and what the profile is for the field values, then you won't
be able to keep on top of what's going on in your index.
-- Jack Krupansky
-----Original Message-----
From: Marko Asplund
Sent: Saturday, April 20, 2013 4:16 PM
To: solr-user
Subject: Re: Dynamic data model design questions
Jack Krupansky wrote:
In general, Solr is much more friendly towards static data models. Yes,
you
can use dynamic fields, but use them in moderation. The more heavily you
lean on them, the more likely that you will eventually become unhappy with
Solr.
Can you concrete examples of what kinds of issues should I expect to
face when using a data model with only dynamic fields?
We've requirements that quite explicitly direct us into using dynamic
fields and I'd like to understand what kinds of problems we might end
up having.
How many fields are we talking about here?
The data model is designed to be dynamic, so the number is not fixed,
but I'm expecting there'll be perhaps about 20-40 fields.
The trick with Solr is not to brute-force flatten your data model (as you
appear to be doing), but to REDESIGN your data model so that it is more
amenable to a flat data model, and takes advantage of Solr's features. You
can use multiple collections for different types of data. And you can
simulate joins across tables by doing a sequence of queries (although it
would be nice to have a SolrJ client-side method to do that in one API
call.)
We're storing arbitrarily complex object graphs in a data store and
want to use Solr for implementing search property field search.
It may be difficult to use a flatter data model, but I'll consider
this option as well.
thanks!
marko
