davsclaus opened a new pull request, #3239:
URL: https://github.com/apache/camel-kamelets/pull/3239

   Part of #3170: `azure-servicebus-source` can process several messages at the 
same time, and move a message whose processing fails to the dead-letter queue 
straight away. `azure-servicebus-sink` can send the body as bytes, and have 
Service Bus hold each message for a delay.
   
   ### The options
   
   | Kamelet | Option | Template mapping | What it does |
   |---|---|---|---|
   | source | `maxConcurrentCalls` | `maxConcurrentCalls: 
"{{?maxConcurrentCalls}}"` | How many messages are processed at the same time. 
When unset, the component default of 1 applies. |
   | source | `enableDeadLettering` | `enableDeadLettering: 
"{{?enableDeadLettering}}"` | `true` dead-letters a message whose processing 
fails, instead of abandoning it. |
   | sink | `binary` | `binary: "{{?binary}}"` | `true` sends the body as bytes 
instead of text. |
   | sink | `scheduledEnqueueDelay` | When set: a `choice` sets the 
`CamelAzureServiceBusProducerOperation` header to `scheduleMessages`, and 
`CamelAzureServiceBusScheduledEnqueueTime` to 
`${bean:type:java.time.OffsetDateTime?method=now()}` plus 
`{{?scheduledEnqueueDelay}}` seconds, before the send; a second `choice` 
removes both headers after it | Service Bus holds each message for that many 
seconds. |
   
   - **Defaults:** all four options are optional and deliberately have no 
`default`, as in #3174. When they are unset, the optional placeholders drop 
them and the `choice` does nothing, so the endpoint URIs are identical to 
`main`'s.
   - **`maxConcurrentCalls`:**
     - Above 1, messages are processed in no particular order.
     - With `sessionEnabled`, it counts for each session. The messages of one 
session are then processed at the same time too, and lose their order. The 
descriptions say so.
   - **`enableDeadLettering`:**
     - The dead letter gets the exception as its reason 
(`java.lang.IllegalStateException: …`) and the stack trace as its error 
description. The component does this.
     - It works only with `PEEK_LOCK`.
     - It does nothing while reading a dead-letter queue with `subQueue`: the 
component abandons the message there, as before.
   - **`binary`:**
     - Without it, the producer reads the body as a String. Bytes that are not 
valid UTF-8 come out damaged.
     - With it, a Map body fails (`NullPointerException: 'content' cannot be 
null`). The descriptions say to turn it into JSON first.
   - **`scheduledEnqueueDelay` is a delay, not the component's 
`scheduledEnqueueTime`.** The component has a `scheduledEnqueueTime` endpoint 
option, but it cannot be used from a Kamelet:
     - It is a fixed `OffsetDateTime`, and Camel has no type converter from a 
String. `scheduledEnqueueTime=2026-11-18T23:00:00+01:00` fails with `No type 
converter available to convert from type: java.lang.String to the required 
type: java.time.OffsetDateTime`.
     - It is read only by the `scheduleMessages` operation. The Kamelet uses 
`sendMessages`, which ignores it.
     - A fixed instant would also be of little use for a stream of messages.
   - **How the delay is set:** the template computes now plus the delay in 
seconds, and sets it in the `CamelAzureServiceBusScheduledEnqueueTime` header. 
It also sets the operation header for that send, so the endpoint stays 
`producerOperation=sendMessages`, and no second sender client is created.
     - Both headers are set only when the option is set, so the delay is fixed 
for the Kamelet.
     - Both are removed after the send, so the message leaves the Kamelet 
unchanged ("Output: the message is unchanged."). Without that removal, a test 
showed the caller getting 
`CamelAzureServiceBusProducerOperation=scheduleMessages` back, which would 
schedule its next send to Service Bus as well.
     - The computation uses the bean language (`${bean:type:…}` and an OGNL 
call), so the sink now declares `camel:bean`. The catalog SBOM was regenerated 
by the build.
   - **Descriptions and partials:**
     - Source: "Example Processing Messages in Parallel" and "Example 
Dead-Lettering Failed Messages".
     - Sink: "Example Sending Binary Data" and "Example with a Delay".
     - The Good-to-know paragraphs cover ordering, sessions, the dead-letter 
reason, and the binary and delay behaviour.
   
   ### Verification
   
   Tested with Camel JBang 4.24.0-SNAPSHOT against the Azure Service Bus 
emulator (`mcr.microsoft.com/azure-messaging/servicebus-emulator`, with 
`mcr.microsoft.com/mssql/server:2022-latest`). It was configured through 
`Config.json` with:
   - plain queues with `MaxDeliveryCount: 10`;
   - two queues with `RequiresSession: true`;
   - a topic with a subscription.
   
   Processing was failed on purpose with `throwException`. Received bytes were 
compared through base64.
   
   | Case | Result |
   |---|---|
   | Options unset, both Kamelets, `main` and this branch | `camel get endpoint 
--json`: identical URIs 
(`azure-servicebus://plainq?connectionString=…&credentialType=CONNECTION_STRING&producerOperation=sendMessages&serviceBusType=queue`,
 and `…&serviceBusReceiveMode=PEEK_LOCK&serviceBusType=queue`); message sent 
and received |
   | All four options set | Sink endpoint has `binary=true` (still 
`producerOperation=sendMessages`). Source endpoint has 
`enableDeadLettering=true&maxConcurrentCalls=4`. Message received 1 s later 
(`scheduledEnqueueDelay: 1`) |
   | 10 messages, synchronous 1 s step, `maxConcurrentCalls` unset | P-1 … P-10 
in order, one thread, one per second: 10 s |
   | The same, `maxConcurrentCalls: 5` | P-1 … P-5 at 21:48:47.015 on 5 
threads, then P-6 … P-10 at 21:48:48.049: 2 s, out of order (P-7 before P-6) |
   | Session queue, 6 messages of session A, `sessionEnabled: true`, 
`maxConcurrentCalls: 3`, 1 s step | A-1, A-2, A-3 at the same time, then A-4, 
A-5, A-6: three at once within one session |
   | Plain queue, E-1 fails, `enableDeadLettering` unset | E-1 received 10 
times (delivery count 0 … 9), then dead-lettered: `MaxDeliveryCountExceeded` |
   | The same with `enableDeadLettering: true` | D-1 received once, then D-2 
processed. In the dead-letter queue (`subQueue: DEAD_LETTER_QUEUE`): D-1 with 
reason `java.lang.IllegalStateException: processing failed on purpose` and the 
stack trace as description |
   | `enableDeadLettering: true` on a session queue, S-2 fails | S-1, S-2 
(once), S-3; S-2 is in the dead-letter queue with the exception as reason |
   | `enableDeadLettering: true` on a topic subscription, T-1 fails | T-1 once, 
T-2 processed; T-1 is in the subscription's dead-letter queue with the 
exception as reason |
   | `enableDeadLettering: true` with `RECEIVE_AND_DELETE`, R-1 fails | 
Received once; neither in the queue nor in the dead-letter queue afterwards 
(lost, as before) |
   | `enableDeadLettering: true` with `subQueue: DEAD_LETTER_QUEUE`, processing 
fails | The dead letter is received again 501 times in 12 s: not dead-lettered 
again |
   | Sink, body bytes 0x00 … 0xFF (256 bytes), `binary` unset | Received 512 
bytes: each byte ≥ 0x80 became U+FFFD (`efbfbd`) |
   | The same with `binary: true` | Received 256 bytes, identical |
   | `file:` route sending a 1 KiB binary file, `binary: true` (the partial's 
example) | Received file identical (`cmp`) |
   | `binary: true`, text `blåbærgrød` | Received as 13 bytes, `blåbærgrød` 
(UTF-8) |
   | `binary: true`, Map body | Fails: `NullPointerException: 'content' cannot 
be null` |
   | `scheduledEnqueueDelay: 20`, 2 messages, reader running | Sent at 
21:56:32.365. Received at 21:56:52.367 with enqueued time 20:56:52.340Z (UTC): 
20 s later. After the sink, `CamelAzureServiceBusProducerOperation` and 
`CamelAzureServiceBusScheduledEnqueueTime` are empty |
   | `scheduledEnqueueDelay: 10` with `sessionId: B` to a session queue | 
Received 10 s later, with session B |
   | `scheduledEnqueueDelay` set, template without the removal step | The 
caller sees `op=scheduleMessages` and the scheduled time after the sink, which 
is why the step is there |
   | Sink with no option set, message carries only a 
`CamelAzureServiceBusScheduledEnqueueTime` header (OffsetDateTime, now + 15 s) 
| Received at once: the header is not used with `sendMessages` |
   
   **Not verified:**
   - `maxConcurrentCalls` above 1 combined with `maxConcurrentSessions` above 
1, for the total number of messages in parallel.
   - Asynchronous steps with `maxConcurrentCalls` unset on a queue without 
sessions. The description says that the next message may then start before the 
previous one has finished. This follows from the consumer code, which is the 
same as for sessions, where #3207 observed it.
   
   `./mvnw clean verify -DskipTests` and then `./mvnw clean verify` passed from 
the root:
   - Before building, the library copies of the two Kamelets were refreshed 
with `./mvnw -pl library/camel-kamelets 
resources:copy-resources@copy-resource-one`.
   - `KameletsCatalogTest` needed no change.
   - The build regenerated `kamelets-index.json` and the catalog SBOM 
(`camel-bean` for the sink).
   
   _Claude Code on behalf of Claus Ibsen_
   
   🤖 Generated with [Claude Code](https://claude.com/claude-code)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to