vignesh-manel opened a new pull request, #28529: URL: https://github.com/apache/camel/pull/28529
https://issues.apache.org/jira/browse/CAMEL-26227 ## Problem 1. **Inserted fields are stored but never indexed.** `LuceneIndexer.createFieldType` sets `stored` and `tokenized` but no index options, so the fields keep Lucene's default `IndexOptions.NONE`. A `lucene:...:query` never finds what `lucene:...:insert` (or the `srcDir` indexing) wrote. This dates from the Lucene 5 upgrade in 7d86043c7963 (CAMEL-9360), which commented out `answer.setIndexed(true)` (removed in Lucene 5) without adding `setIndexOptions`. This change restores the pre-Lucene-5 behaviour; it does not revert a deliberate choice. 2. **Each field is its own document.** `add(field, value, analyzed)` created a new `Document` per field, so one exchange became one document for `exchangeId`, one per header and one for `contents` (and `path` / `contents` per source file). A hit on the body could not be tied to its headers or file path. ## Change - `createFieldType` sets `IndexOptions.DOCS_AND_FREQS_AND_POSITIONS` on both field types. - One `Document` per exchange (`exchangeId`, headers, `contents`) and one per `srcDir` file (`path`, `contents`). - New `LuceneInsertAndQueryTest`: insert then query finds the body, one exchange is one document (a query on a header returns the body and the full document), and a `srcDir` file is found with its untokenized `path` in the same document. No existing test covered insert followed by query: the query tests build their own index with `TextField`, and the ITs never assert hit counts. - Component doc: describes the fields of a document (and fixes the field name, `contents` not "content"); catalog copy updated. - Upgrade guide 4.24 entry. ## Breaking change (in the upgrade guide) Lucene does not allow a field to change from not indexed to indexed within an index. Inserting with this change into an index directory written by an earlier Camel version fails with `IllegalArgumentException: cannot change field "exchangeId" from index options=NONE to inconsistent index options=DOCS_AND_FREQS_AND_POSITIONS`. The index must be deleted and rebuilt. Nothing searchable is lost, because none of its fields were indexed. A query on such an index still returns 0 hits without an error. I verified both behaviours with throwaway tests that are not committed. ## Verification (local, JDK 21.0.12.1, Maven 3.9.12) - Before the fix, the new test failed 3/3: `expected: <1> but was: <0>` hits for body and source file, and `expected: <1> but was: <3>` documents for one exchange. - After the fix: `mvn verify` in `components/camel-lucene` passes 12 unit tests and 6 ITs (`LuceneIndexAndQueryProducerIT`, `LuceneQueryProcessorIT`). - Mutation checks: removing `setIndexOptions`, splitting the exchange back into several documents, and splitting `path` from `contents` each make the new test fail. - `-Psourcecheck` (formatter validate, impsort check, license) passes for the module. - I have not run the full build; this is a draft until CI is green. _Claude Code on behalf of vignesh-manel_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
