davsclaus opened a new pull request, #3304:
URL: https://github.com/apache/camel-kamelets/pull/3304
Part of #3117. Since CAMEL-27258, Camel applies the default output data type
of a source Kamelet when the message is handed to the consuming route, not
after that route has run. The sources that already had an `out` default
therefore now give the consuming route the declared body and `Content-Type`.
This PR updates their Citrus expectations and the `Output:` paragraphs of their
descriptions to match. The Kamelets' data types themselves do not change.
> **Draft:** the updated tests need CAMEL-27258, which was merged in Camel
main today, after the published Camel 4.24.0-SNAPSHOT that CI uses was built.
With that snapshot the old expectations still hold, so these tests fail on CI
until a newer snapshot is published. Once it is, these tests fail on `main`
without this PR.
## What the consuming route gets
"Before" is what CI gives today, from the tests; "after" is Camel main with
CAMEL-27258.
| Kamelet | default | body / Content-Type before → after |
|---|---|---|
| `aws-sqs-source` | `text` | `String`, no `Content-Type` → `String`,
`text/plain` |
| `azure-storage-queue-source` | `text` | `ByteArrayInputStreamCache`, none
→ `String`, `text/plain` |
| `aws-ddb-streams-source` | `json` | JSON text `String` → JSON `byte[]`,
`application/json` |
| `azure-cosmosdb-source` | `json` | `InputStreamCache`, `application/json`
→ `byte[]`, `application/json` |
| `google-calendar-source` | `json` | the stream of `marshal json`,
`application/json` → `byte[]`, `application/json` (no test) |
| `google-sheets-source` | `json` | same as `google-calendar-source` (no
test) |
| `slack-source` | `json` | the JSON text of `marshal json`,
`application/json` → `byte[]`, `application/json` |
| `azure-eventhubs-source` | `binary` | `byte[]`, none → `byte[]`,
`application/octet-stream` |
| `aws-s3-source` | `binary` | `byte[]`, none → `byte[]`,
`application/octet-stream` |
| `azure-storage-blob-source` | `binary` | `InputStreamCache`, none →
`InputStreamCache` (kept), `application/octet-stream` |
| `azure-storage-datalake-source` | `binary` | same as
`azure-storage-blob-source` |
| `google-storage-source` | `binary` | same as `azure-storage-blob-source` |
| `ibm-cos-source` | `binary` | same as `azure-storage-blob-source` |
| `azure-storage-files-source` | `binary` | the file → `byte[]`,
`application/octet-stream` (no test, no emulator; from the transformer code) |
Left out, as agreed: `aws-cloudtrail-source` (#3297 changes it to `json` and
updates its test) and `google-mail-source` (#3296 changes its template; it has
no test).
## Changes
- **Descriptions:** the `Output:` paragraph of each Kamelet above now names
the `Content-Type` header, and the body where it changed.
`azure-storage-queue-source` gives text, not "(as a stream)", and
`azure-storage-files-source` gives bytes, not "text, bytes or a stream".
- **Tests that record the body:** the Azure, Google Cloud Storage and IBM
COS source tests expect the values in the table.
- **Tests that only logged:** the `aws-sqs-source`, `aws-s3-source`,
`aws-ddb-streams-source` and `slack-source` routes now log the body class and
`Content-Type` (`Kamelet result: ...`). Their tests verify that line, as #3297
did for `aws-kinesis-source`.
- **The binary source routes** (Event Hubs, Blob, Data Lake, Cloud Storage,
IBM COS) remove `Content-Type` and send the body as text to the Citrus server,
because Citrus reads an `application/octet-stream` body as a byte array.
## Tests
Run locally with Camel main + CAMEL-27258 in `~/.m2`:
- `AwsIT#awsS3+awsSqs+awsDdb+awsEventBridge`: 17 pass. This includes the
other S3/SQS/DDB tests, the event listener of `aws-eventbridge-sink`, and
`aws-s3-to-http`, which runs `aws-s3-source` through the CloudEvents data types.
- `AzureIT` 15, `GoogleCloudIT` 6, `SlackIT` 2 and `ServicesIT#ibmCos` 2 all
pass.
- None of the binary sources gave an empty or short body: the Blob, Data
Lake, Cloud Storage and IBM COS tests check the content.
- `aws-ddb-streams-source` and `slack-source` do not depend on
`camel:jackson3`. Their `json` transformer was still found in these runs, so
the proposal's concern about a missing transformer did not show up.
- `./mvnw clean install` from the root passes, run twice.
## Overlap with open PRs
- **#3301** (Azure and Google defaults) changes the expected `ct=` of the
`azure-servicebus-source` and `google-pubsub-source` tests. This PR does not
touch those files.
- **#3297** (AWS defaults) changes the `aws-cloudtrail-source` test, and
other AWS tests that this PR does not touch.
- **#3296** changes `google-mail-source`, which this PR leaves out.
- **#3294** changes the same `Output:` sentence of `azure-cosmosdb-source`,
to "as JSON text (a stream)" with `application/json`. That matches the CI
snapshot, but with CAMEL-27258 the body is `byte[]`. Whichever PR merges second
resolves the conflict, and the merged text should say "in bytes", as this PR
does.
- **#3285** changes `azure-storage-datalake-source` (an option) and adds a
delete test. The `Output:` sentence and the existing test here are different
lines.
- **#3287** adds the `verified` label to most of these Kamelets: different
lines.
- All of them regenerate `kamelets-index.json` and the copies under
`library/camel-kamelets`, so the PR that merges second regenerates them.
## Upgrade note for #3190
The CAMEL-27258 bullet in #3190 already covers this. For more detail, it
could add:
```markdown
- `azure-storage-queue-source` now gives a `String` (it was a stream), and
`azure-cosmosdb-source`, `google-calendar-source`, `google-sheets-source`,
`slack-source` and `aws-ddb-streams-source` give `byte[]` JSON. The binary
sources set `Content-Type: application/octet-stream`, and the text ones
`text/plain`.
```
_Claude Code on behalf of Claus Ibsen_
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_016Y4dDmBDsSWV117kccuNf9
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]