Croway opened a new pull request, #30169: URL: https://github.com/apache/camel/pull/30169
[CAMEL-27011](https://issues.apache.org/jira/browse/CAMEL-27011) ### What was wrong With `outputType=StreamList` and `outputClass`, each row is mapped to the output class when the route reads it. When a row cannot be mapped (e.g. a value that does not convert to the type of its property), the first pass of the split fails as expected, but, as in CAMEL-26996, the failure comes from the splitter's own iteration, so the error handler redelivers to the splitter with the same iterator, which no longer stops at that row: - **camel-jdbc** (and camel-spring-jdbc): the body is camel-support's `StreamListIterator`, whose `next()` takes the row from the `ResultSetIterator` first (the result set is then past it) and then maps it. A redelivered split goes on with the next row: the exchange succeeds **without the row that could not be mapped**. - **camel-sql**: `ResultSetIterator.next()` maps the row with Spring's `BeanPropertyRowMapper`. CAMEL-26996 keeps the iterator failing after an `SQLException`, but a `RuntimeException` of the row mapper (Spring's `TypeMismatchException` for a value that does not convert to an enum property, or a custom `RowMapperFactory`) was not caught. The splitter closes the iterator body when the split fails (`SplitterIterable.close()` → `IOHelper.closeIterator`), `hasNext()` is then `!closed`, and a redelivered split finds no more rows: the exchange succeeds **without that row and all the rows after it**. This was noted (not changed) in #30082. ### The fix Same as CAMEL-26996: once mapping a row has failed, the iterator keeps failing. - `StreamListIterator` (camel-support) remembers the `RuntimeException` of the row mapper; from then on `hasNext()` and `next()` throw a `RuntimeCamelException` with it as cause. - camel-sql `ResultSetIterator.next()` also catches the `RuntimeException` of the row mapper: it remembers it, closes the iterator and rethrows it; `hasNext()` throws as after a read failure, also after `close()`. The first failure is unchanged (the exception of the mapper as it is). ### Design choices for reviewers - **Keep failing, rather than mapping the same row again on the next call.** A mapping failure is deterministic in practice, and in camel-sql the row cannot be read again anyway once the splitter has closed the iterator. Keeping the failure is the CAMEL-26996 behaviour (`hasNext()` throws, so a caller that catches the exception of `next()` and loops does not spin on the same row). - **The fix is in `StreamListIterator`** (where the row is taken from the delegate before it is mapped), not in camel-jdbc only. Its other user, camel-google-bigquery (`outputType=STREAM_LIST`), gets the same behaviour; its `FieldValueListMapper` only copies values into a `Map`, so it is not expected to fail there. - Only failures of the mapper are remembered by `StreamListIterator`: a failure of the delegate itself is left to the delegate, which knows whether it can go on (camel-jdbc's `ResultSetIterator` already keeps failing after a read error since CAMEL-26996). - Behaviour change for camel-jdbc code that reads the iterator itself and skips rows that fail to map: `hasNext()` now throws after such a row (documented in the upgrade guide, with the alternative: read without `outputClass` and map in the route). ### Tests - `StreamListIteratorTest` (camel-support): a mapper failure is thrown as it is, then `hasNext()`/`next()` throw a `RuntimeCamelException` with it as cause, and the row after it is never returned. - `JdbcProducerStreamListMappingFailureTest` (camel-jdbc) and `SqlProducerStreamListMappingFailureTest` (camel-sql): H2 table with rows `NEW`, `LOST`, `DONE` mapped to a bean with an enum property (row 2 cannot be mapped), split with `maximumRedeliveries(2)`; the exchange must fail after 2 redeliveries with only row 1 routed. camel-sql also checks the iterator keeps failing after `close()`. - **Fail before the fix**: camel-jdbc `the exchange succeeded with the rows [1, 3]`, camel-sql `the exchange succeeded with the rows [1]` (and `hasNext()` does not throw), camel-support `keepsFailingOnceMappingARowHasFailed` fails. - After the fix: the new tests pass, and so do the full camel-jdbc (72 tests, including the CAMEL-26996 ones), camel-sql (398), camel-spring-jdbc (2) suites, and the camel-google-bigquery SQL tests (40, including `GoogleBigQuerySQLProducerStreamListTest`). ### Docs and upgrade guide The `StreamList` paragraphs of `jdbc-component.adoc` and `sql-component.adoc` (added by CAMEL-26996) now say the same applies to a row that cannot be mapped to the `outputClass` (catalog copies updated). Upgrade guide 4.24: new section after the CAMEL-26996 one. _Claude Code on behalf of Croway_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
