I would like to ask the Solr organization an enhancement request.
The FileFloatSource creates a cache value from an external file when a Core
is reloaded and/or a new searcher is opened. Nevertheless, the external
files can be changed less frequently.
With a larger document and larger external files, it consumes lots of CPU
and memory. Also, it makes a Core reloading process longer.
Until the getFloats method of the FileFloatSource is completed and replace
the cache value in the floatCache --> readerCache, two larger float arrays
are in the memory. When the external file was not changed, these two
larger float array contains same values.
Current format of the external field file is this:
Id=float_value
Id=float_value
Id=float_value
: :
If we support this kind format, we can only create the large array
object(s) when a version of the file is changed.
Version (such as system current time as an example)
Id=float_value
Id=float_value
Id=float_value
: :
When the version was not changed, we can still use the cached array without
creating a new one.
Brief pseudo code, which should be added to the getFloats method, is
something like this.
String currentVersion = r.readLine();
if(latestVerstion != null && latestVerstion.equals(currentVersion)){
Object cacheVal = latestCache.get(ffs.field.getName());
if(null != cacheVal){
return (float[])cacheVal;
}
}
//Create float array after the version check.
vals = new float[reader.maxDoc()];
if (ffs.defVal != 0) {
Arrays.fill(vals, ffs.defVal);
}
latestVerstion = currentVersion;
latestCache.put(ffs.field.getName(), vals);
//This is existing codes
for (String line; (line=r.readLine())!=null;) {
int delimIndex = line.lastIndexOf(delimiter);
if (delimIndex < 0) continue;
int endIndex = line.length();
*How can I file the enhancement request?*
Thanks.
