Shawn, would you mind to provide some numbers?
I'm experimenting with lucene 8.0.0.
I have 100 shard index of 100M docs with 2000 docVals only updateable
fields. Searching for such field turns to be blazingly fast
$ curl 'localhost:39200/books/_search?pretty&size=20' -d '
{"query": {"bool": {"filter": {"range": {"subscription_0x1": {"lte": 666,
"gte": 666}}}}}}'
{
"took" : 148,
"timed_out" : false,
"_shards" : { "total" : 100, "successful" : 100, "skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : { "value" : 1, "relation" : "eq"
},
"max_score" : 0.0,
"hits" : [
{
"_index" : "books",
"_type" : "books",
"_id" : "28113070",
"_score" : 0.0
}
]
}
}
I've just updated this field in this particular doc. Other 245K of 100M
docs has 1 in it
$ curl -H 'Content-Type:application/json'
'localhost:39200/books/_search?pretty&size=20' -d '
{"track_total_hits": true, "query": {"bool": {"filter": {"range":
{"subscription_0x1": {"lte": 1, "gte":1}}}}}}'
{
"took" : 16,
"timed_out" : false,
"_shards" : {
"total" : 100,
"successful" : 100,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 245335,
"relation" : "eq"
},
"max_score" : 0.0,
"hits" : [
{
"_index" : "books",
"_type" : "books",
"_id" : "30155366",
"_score" : 0.0
},
It's dv field without index
$ curl -s
'localhost:39200/books/_mapping/field/subscription_0x1?pretty&include_defaults=true'
{
"books" : {
"mappings" : {
"subscription_0x1" : {
"full_name" : "subscription_0x1",
"mapping" : {
"subscription_0x1" : {
"type" : "integer",
"boost" : 1.0,
"index" : false,
"store" : false,
"doc_values" : true,
"term_vector" : "no",
"norms" : false,
"eager_global_ordinals" : false,
"similarity" : "BM25",
"ignore_malformed" : false,
"coerce" : true,
"null_value" : null
}
}
}
}
}
}
On Tue, Sep 10, 2019 at 4:55 PM Shawn Heisey <[email protected]> wrote:
> On 9/10/2019 7:15 AM, Doss wrote:
> > 4 to 5 million documents.
> >
> > For an NTR index, we need a field to be updated very frequently and
> filter
> > results based on it. Will In-Place updates help us?
> >
> > <field name="status" type="pint" indexed="false" stored="false"
> > docValues="true" />
>
> Although you CAN search on docValues-only fields, the performance is
> terrible. So the answer I have for you is "maybe, but you won't like
> it." For good filtering performance, you need the field to be indexed.
> Which means you can't do in-place updates.
>
> Thanks,
> Shawn
>
--
Sincerely yours
Mikhail Khludnev
