Hi Martin, Perhaps you could make a small change in Solr to add "don't reload EFF if it hasn't been modified since it was last opened". I assume you commit pretty often, but don't modify EFF files that often, so this could save you some needless loading. That said, I'd be surprised EFF doesn't already do this... I didn't check.
Otis -- Search Analytics - http://sematext.com/search-analytics/index.html Performance Monitoring - http://sematext.com/spm/index.html On Mon, Oct 8, 2012 at 4:55 AM, Martin Koch <m...@issuu.com> wrote: > Hi List > > We're using Solr-4.0.0-Beta with a 7M document index running on a single > host with 16 shards. We'd like to use an ExternalFileField to hold a value > that changes often. However, we've discovered that the file is apparently > re-read by every shard/core on *every commit*; the index is unresponsive in > this period (around 20s on the host we're running on). This is unacceptable > for our needs. In the future, we'd like to add other values as > ExternalFileFields, and this will make the problem worse. > > It would be better if the external file were instead read in in the > background, updating previously read relevant values for each shard as they > are read in. > > I guess a change in the ExternalFileField code would be required to achieve > this, but I have no experience here, so suggestions are very welcome. > > Thanks, > /Martin Koch - Issuu - Senior Systems Architect.
