Croway opened a new pull request, #30169:
URL: https://github.com/apache/camel/pull/30169

   [CAMEL-27011](https://issues.apache.org/jira/browse/CAMEL-27011)
   
   ### What was wrong
   
   With `outputType=StreamList` and `outputClass`, each row is mapped to the 
output class when the route reads it. When a row cannot be mapped (e.g. a value 
that does not convert to the type of its property), the first pass of the split 
fails as expected, but, as in CAMEL-26996, the failure comes from the 
splitter's own iteration, so the error handler redelivers to the splitter with 
the same iterator, which no longer stops at that row:
   
   - **camel-jdbc** (and camel-spring-jdbc): the body is camel-support's 
`StreamListIterator`, whose `next()` takes the row from the `ResultSetIterator` 
first (the result set is then past it) and then maps it. A redelivered split 
goes on with the next row: the exchange succeeds **without the row that could 
not be mapped**.
   - **camel-sql**: `ResultSetIterator.next()` maps the row with Spring's 
`BeanPropertyRowMapper`. CAMEL-26996 keeps the iterator failing after an 
`SQLException`, but a `RuntimeException` of the row mapper (Spring's 
`TypeMismatchException` for a value that does not convert to an enum property, 
or a custom `RowMapperFactory`) was not caught. The splitter closes the 
iterator body when the split fails (`SplitterIterable.close()` → 
`IOHelper.closeIterator`), `hasNext()` is then `!closed`, and a redelivered 
split finds no more rows: the exchange succeeds **without that row and all the 
rows after it**.
   
   This was noted (not changed) in #30082.
   
   ### The fix
   
   Same as CAMEL-26996: once mapping a row has failed, the iterator keeps 
failing.
   
   - `StreamListIterator` (camel-support) remembers the `RuntimeException` of 
the row mapper; from then on `hasNext()` and `next()` throw a 
`RuntimeCamelException` with it as cause.
   - camel-sql `ResultSetIterator.next()` also catches the `RuntimeException` 
of the row mapper: it remembers it, closes the iterator and rethrows it; 
`hasNext()` throws as after a read failure, also after `close()`.
   
   The first failure is unchanged (the exception of the mapper as it is).
   
   ### Design choices for reviewers
   
   - **Keep failing, rather than mapping the same row again on the next call.** 
A mapping failure is deterministic in practice, and in camel-sql the row cannot 
be read again anyway once the splitter has closed the iterator. Keeping the 
failure is the CAMEL-26996 behaviour (`hasNext()` throws, so a caller that 
catches the exception of `next()` and loops does not spin on the same row).
   - **The fix is in `StreamListIterator`** (where the row is taken from the 
delegate before it is mapped), not in camel-jdbc only. Its other user, 
camel-google-bigquery (`outputType=STREAM_LIST`), gets the same behaviour; its 
`FieldValueListMapper` only copies values into a `Map`, so it is not expected 
to fail there.
   - Only failures of the mapper are remembered by `StreamListIterator`: a 
failure of the delegate itself is left to the delegate, which knows whether it 
can go on (camel-jdbc's `ResultSetIterator` already keeps failing after a read 
error since CAMEL-26996).
   - Behaviour change for camel-jdbc code that reads the iterator itself and 
skips rows that fail to map: `hasNext()` now throws after such a row 
(documented in the upgrade guide, with the alternative: read without 
`outputClass` and map in the route).
   
   ### Tests
   
   - `StreamListIteratorTest` (camel-support): a mapper failure is thrown as it 
is, then `hasNext()`/`next()` throw a `RuntimeCamelException` with it as cause, 
and the row after it is never returned.
   - `JdbcProducerStreamListMappingFailureTest` (camel-jdbc) and 
`SqlProducerStreamListMappingFailureTest` (camel-sql): H2 table with rows 
`NEW`, `LOST`, `DONE` mapped to a bean with an enum property (row 2 cannot be 
mapped), split with `maximumRedeliveries(2)`; the exchange must fail after 2 
redeliveries with only row 1 routed. camel-sql also checks the iterator keeps 
failing after `close()`.
   - **Fail before the fix**: camel-jdbc `the exchange succeeded with the rows 
[1, 3]`, camel-sql `the exchange succeeded with the rows [1]` (and `hasNext()` 
does not throw), camel-support `keepsFailingOnceMappingARowHasFailed` fails.
   - After the fix: the new tests pass, and so do the full camel-jdbc (72 
tests, including the CAMEL-26996 ones), camel-sql (398), camel-spring-jdbc (2) 
suites, and the camel-google-bigquery SQL tests (40, including 
`GoogleBigQuerySQLProducerStreamListTest`).
   
   ### Docs and upgrade guide
   
   The `StreamList` paragraphs of `jdbc-component.adoc` and 
`sql-component.adoc` (added by CAMEL-26996) now say the same applies to a row 
that cannot be mapped to the `outputClass` (catalog copies updated). Upgrade 
guide 4.24: new section after the CAMEL-26996 one.
   
   _Claude Code on behalf of Croway_
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to