davsclaus opened a new pull request, #3207:
URL: https://github.com/apache/camel-kamelets/pull/3207
Part of #3170: `azure-servicebus-source` can read a queue or subscription
with sessions enabled, and its dead-letter queue. `azure-servicebus-sink` can
send to a queue with sessions enabled. Both can log in with an Azure identity.
### The problem
- `azure-servicebus-source` had no `sessionEnabled`. On a queue with
sessions, it started but received nothing, and every 5 seconds it logged `It is
not possible for an entity that requires sessions to create a non-sessionful
message receiver`.
- It had no `subQueue` either, so it could not read the dead-letter queue.
- `azure-servicebus-sink` had no `sessionId`. Every send to a queue with
sessions failed with `The SessionId was not set on a message, and it cannot be
sent to the entity`.
- `credentialType` offered `AZURE_IDENTITY` and `TOKEN_CREDENTIAL`, but both
need `fullyQualifiedNamespace`, which the Kamelets did not set. They also
required `connectionString`. So an Azure identity login worked only through a
component-level setting plus a dummy connection string.
### The options
| Kamelet | Option | Template mapping | What it does |
|---|---|---|---|
| source | `sessionEnabled` | `sessionEnabled: "{{?sessionEnabled}}"` |
`true` reads a queue or subscription with sessions, through the session
processor. |
| source | `maxConcurrentSessions` | `maxConcurrentSessions:
"{{?maxConcurrentSessions}}"` | How many sessions are read at the same time.
When unset, the component default of 1 applies. |
| source | `subQueue` | `subQueue: "{{?subQueue}}"` | `DEAD_LETTER_QUEUE` or
`TRANSFER_DEAD_LETTER_QUEUE`. |
| sink | `sessionId` | `sessionId: "RAW({{?sessionId}})"` | Every message is
sent with this session ID. |
| both | `fullyQualifiedNamespace` | `fullyQualifiedNamespace:
"{{?fullyQualifiedNamespace}}"` | The namespace host name, needed by
`AZURE_IDENTITY` and `TOKEN_CREDENTIAL`. |
| both | `connectionString` (changed) | `RAW({{connectionString}})` becomes
`RAW({{?connectionString}})`, no longer `required` | Can be left out with an
Azure identity. |
- **Defaults:** all the new options are optional and deliberately have no
`default`, as in #3174. When they are unset, the optional placeholders drop
them, and the endpoint URIs are identical to `main`'s.
- With the default `CONNECTION_STRING` and no `connectionString`, the
integration does not start: `Azure ServiceBus ConnectionString or FQNS must be
specified`.
- **`sessionId` is a fixed option, not a header.** camel-azure-servicebus
4.24 reads the session ID only from the endpoint configuration. Its
`CamelAzureServiceBusSessionId` header is a consumer header: the producer does
not read it.
- The Kamelet does not build the URI from a header with `toD`. That would
put message data into the endpoint URI.
- The descriptions say so, and say to use one sink per session ID. A
per-message session ID needs the component to read the header, which is an
`apache/camel` change.
- `RAW()` keeps the value as written: `a+b %20c&d=1` arrived as `a+b
%20c&d=1`.
- **`pattern:`:**
- `fullyQualifiedNamespace`: `^[A-Za-z0-9][A-Za-z0-9.-]*$`, a host name,
so `sb://…` or `https://…` is rejected.
- `sessionId`: `^.{1,128}$`, the Service Bus limit.
- **Headers:** the source's `dataTypes` now list
`CamelAzureServiceBusSessionId`, `CamelAzureServiceBusDeadLetterReason` and
`CamelAzureServiceBusDeadLetterErrorDescription`. The consumer already set
them; this lists them as metadata only, with no transformer.
- **Descriptions:**
- When sessions are needed, and what the error looks like without them.
- Ordering: the messages of one session are received in the order they
were sent. The next one comes only after the previous one has been processed,
unless a step works asynchronously. A failing message is received again before
the next one of its session, and different sessions have no order between them.
- `maxConcurrentSessions` 1 holds a session until it has been idle for a
minute.
- The dead-letter queue has no sessions.
- A failing dead letter is received again at once, over and over.
- For a topic, a message without a session ID is accepted, but a
subscription with sessions dead-letters it.
- How to log in with `AZURE_IDENTITY`.
- **Partial pages:** the authentication section no longer says that the
Kamelet cannot set the namespace. New sections: "Example with Sessions" and
"Example Reading the Dead-Letter Queue" for the source, and "Example with a
Session ID" for the sink.
### Verification
Tested with Camel JBang 4.24.0-SNAPSHOT against the Azure Service Bus
emulator (`mcr.microsoft.com/azure-messaging/servicebus-emulator`, with
`mcr.microsoft.com/mssql/server:2022-latest`). The emulator was configured
through `Config.json` with:
- `sessq`: a queue with `RequiresSession: true`;
- `dlq`: a queue with `MaxDeliveryCount: 2`;
- `plainq`: a plain queue;
- `sesstopic`: a topic, with the subscription `sesssub` that has
`RequiresSession: true`.
Messages were dead-lettered by failing the processing on purpose
(`throwException`).
| Case | Result |
|---|---|
| Options unset, both Kamelets, `main` and this branch, `plainq` | `camel
get endpoint --json`: identical URIs
(`azure-servicebus://plainq?connectionString=…&credentialType=CONNECTION_STRING&serviceBusReceiveMode=PEEK_LOCK&serviceBusType=queue`,
and `…&producerOperation=sendMessages&serviceBusType=queue`); message sent and
received |
| Sink to `sessq`, no `sessionId` | Fails: `The SessionId was not set on a
message, and it cannot be sent to the entity` |
| Sink to `sessq`, `sessionId: A` | Sent; the reader sees
`CamelAzureServiceBusSessionId=A` |
| Sink, `sessionId: "a+b %20c&d=1"` | Endpoint has `sessionId=RAW(…)`;
received with the session ID `a+b %20c&d=1`, as written |
| Sink to `plainq` (no sessions), `sessionId: X` | Sent and received; the
session ID is kept on the message |
| Sink to `sesstopic`, no `sessionId` | Sent. In `sesssub` it lands in the
dead-letter queue: reason `Session id is null.`, `Session enabled entity
doesn't allow a message whose session identifier is null.` |
| Source on `sessq`, `sessionEnabled` unset | Starts, receives nothing; WARN
every ~5 s: `It is not possible for an entity that requires sessions to create
a non-sessionful message receiver` |
| Source on `sessq`, `sessionEnabled: true` | Receives everything, with the
session ID header |
| Same, sessions A and B (5 each), `maxConcurrentSessions` unset | All of A
at 15:24:36, then all of B at 15:25:37: one session at a time, the next after
~60 s idle |
| Same, `maxConcurrentSessions: 2`, synchronous 200 ms step | A and B
interleaved; within each, 1→5 in order, ~230 ms apart (one at a time) |
| Same with an asynchronous delay of 200 ms | Within a session still 1→5 in
order, but 3 ms apart: the next message starts before the previous one is
finished |
| `sessionEnabled: true`, A-2 fails on every try | A-1, then A-2 ten times
(delivery count 0…9), then A-3, A-4: order kept. A-2 dead-lettered |
| `subQueue: DEAD_LETTER_QUEUE` on `sessq`, `sessionEnabled: true` | WARN:
`Cannot create a MessageSession for a sub-queue` |
| `subQueue: DEAD_LETTER_QUEUE` on `sessq`, `sessionEnabled` unset |
Receives A-2: reason `MaxDeliveryCountExceeded`, `Message could not be consumed
after 10 delivery attempts.` |
| `sessionEnabled: true` on `plainq` | WARN every ~5 s: `A sessionful
message receiver cannot be created on an entity that does not require sessions`
|
| `dlq`, 2 messages, processing always fails | Each delivered twice, then
dead-lettered |
| `subQueue: DEAD_LETTER_QUEUE` on `dlq` | Both received, with
`CamelAzureServiceBusDeadLetterReason=MaxDeliveryCountExceeded`; a second run
receives nothing (completed) |
| The same, processing fails | The first dead letter is received 871 times
in 15 s; the second never arrives |
| `subQueue: TRANSFER_DEAD_LETTER_QUEUE` | Endpoint has
`subQueue=TRANSFER_DEAD_LETTER_QUEUE`; the consumer starts without error |
| Topic `sesstopic` / `sesssub`, `sessionEnabled` unset, then `true` |
Unset: the same WARN as the queue. `true`: T-1, T-2, T-3 received in order |
| `credentialType: AZURE_IDENTITY`, `fullyQualifiedNamespace: localhost`, no
`connectionString` | Endpoints have
`credentialType=AZURE_IDENTITY&fullyQualifiedNamespace=localhost`, with no
`connectionString`; the client connects to `localhost:5671` (`NAMESPACE:
localhost`) |
| `AZURE_IDENTITY`, no `fullyQualifiedNamespace` | Does not start: `Azure
ServiceBus ConnectionString or FQNS must be specified` |
| `CONNECTION_STRING`, no `connectionString` | Does not start: the same
message |
| `AZURE_IDENTITY` without `connectionString`, `main`'s Kamelets | Does not
start: `Failure creating route from template` (missing required
`connectionString`) |
**Not verified:**
- The Azure identity login itself. The emulator accepts only its SAS
connection string and speaks plain AMQP on 5672, so the identity case stops at
the connection to `localhost:5671`. It does show that the namespace reaches the
client.
- Dead-lettering of expired messages. The sink cannot set a time to live.
- On the emulator, a dead letter's delivery count stayed at 2 while it was
received again and again.
`./mvnw clean verify -DskipTests` and then `./mvnw clean verify` passed from
the root:
- Before building, the library copies of the two Kamelets were refreshed
with `./mvnw -pl library/camel-kamelets
resources:copy-resources@copy-resource-one`.
- `KameletsCatalogTest` now expects 10 headers for
`azure-servicebus-source`, up from 7.
- The build regenerated `kamelets-index.json`.
_Claude Code on behalf of Claus Ibsen_
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]