gsmiller commented on a change in pull request #443:
URL: https://github.com/apache/lucene/pull/443#discussion_r752355654
##########
File path:
lucene/facet/src/java/org/apache/lucene/facet/taxonomy/OrdinalMappingLeafReader.java
##########
@@ -107,6 +113,64 @@ public BytesRef binaryValue() {
}
}
+ private class OrdinalMappingSortedNumericDocValues extends
FilterSortedNumericDocValues {
+ private final IntArrayList currentValues;
+ private int currIndex;
+
+ OrdinalMappingSortedNumericDocValues(SortedNumericDocValues in) {
+ super(in);
+ currentValues = new IntArrayList(32);
+ }
+
+ @Override
+ public boolean advanceExact(int target) throws IOException {
+ boolean result = in.advanceExact(target);
+ if (result) {
+ reloadValues();
+ }
+ return result;
+ }
+
+ @Override
+ public int advance(int target) throws IOException {
+ int result = in.advance(target);
+ if (result != DocIdSetIterator.NO_MORE_DOCS) {
+ reloadValues();
+ }
+ return result;
+ }
+
+ @Override
+ public int nextDoc() throws IOException {
+ int result = in.nextDoc();
+ if (result != DocIdSetIterator.NO_MORE_DOCS) {
+ reloadValues();
+ }
+ return result;
+ }
+
+ @Override
+ public int docValueCount() {
+ return currentValues.elementsCount;
+ }
+
+ private void reloadValues() throws IOException {
+ currIndex = 0;
+ currentValues.clear();
+ for (int i = 0; i < in.docValueCount(); i++) {
+ currentValues.add(ordinalMap[(int) in.nextValue()]);
Review comment:
> @gsmiller put another way, I think it would be much more interesting
to think about something like a new 'taxonomy' or 'hierarchy' type that
supports the additional features the current taxo index provides, rather than
providing a lot of hooks for apps to manage ordinals themselves. It could
probably ultimately perform better and be easier to use (e.g. not juggle a
sidecar index).
I like this way of thinking about the problem. As far as benchmarking
performance goes, the big improvement was getting away from the custom binary
format (which was packing ordinals into a single binary payload using a v-int
style approach) and using SORTED_NUMERIC instead to encode the ordinals for
each doc. So there could be room for more improvement (or at least remaining
performance neutral) by having an "ordinal specific" type doc value field. But
I like your way of thinking and wouldn't hold up this PR by any means to
explore something different at this point. Going to SORTED_NUMERIC is a good
step forward for now. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
