This is an automated email from the ASF dual-hosted git repository.
davsclaus pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
The following commit(s) were added to refs/heads/main by this push:
new 45eb08fa4 chore: the Elasticsearch, OpenSearch and Solr Kamelets take
timeouts, and Solr a request handler (#3170) (#3226)
45eb08fa4 is described below
commit 45eb08fa499e2ca1940de558c9f0dee2cfec01e3
Author: Claus Ibsen <[email protected]>
AuthorDate: Wed Nov 18 22:35:10 2026 +0100
chore: the Elasticsearch, OpenSearch and Solr Kamelets take timeouts, and
Solr a request handler (#3170) (#3226)
elasticsearch-index-sink and opensearch-index-sink take connectionTimeout
and socketTimeout, in milliseconds. When they are not set, the
components wait 30 seconds, as before.
Their certificate option, the CA to trust for a cluster with a private
or self-signed certificate, is now mapped inside RAW(), as #3173 did for
Solr. A DER certificate encoded in base64 contains '+', which became a
space, and the integration failed to start with "Illegal base64
character 20". Both PEM and DER now work.
solr-sink and solr-source take requestHandler, the path of another
handler of the collection, and connectionTimeout and requestTimeout. When
they are not set, the documents go to /update, the query runs on
/select, and Solr is waited for 60 seconds to connect and 10 minutes to
answer, as before.
All new options are optional without a default, so with them unset the
endpoint URIs are the same as before. maxRetryTimeout is left out: it is
deprecated in both components and has no effect.
Co-authored-by: Claude Opus 5.5 <[email protected]>
---
.../elasticsearch-index-sink-description.adoc | 38 ++++++++++++++++++++--
.../opensearch-index-sink-description.adoc | 38 +++++++++++++++++++++-
.../ROOT/partials/solr-sink-description.adoc | 29 ++++++++++++++++-
.../ROOT/partials/solr-source-description.adoc | 27 ++++++++++++++-
kamelets/elasticsearch-index-sink.kamelet.yaml | 16 +++++++--
kamelets/opensearch-index-sink.kamelet.yaml | 16 +++++++--
kamelets/solr-sink.kamelet.yaml | 20 ++++++++++--
kamelets/solr-source.kamelet.yaml | 20 ++++++++++--
.../main/resources/META-INF/kamelets-index.json | 8 ++---
.../kamelets/elasticsearch-index-sink.kamelet.yaml | 16 +++++++--
.../kamelets/opensearch-index-sink.kamelet.yaml | 16 +++++++--
.../main/resources/kamelets/solr-sink.kamelet.yaml | 20 ++++++++++--
.../resources/kamelets/solr-source.kamelet.yaml | 20 ++++++++++--
13 files changed, 254 insertions(+), 30 deletions(-)
diff --git
a/docs/modules/ROOT/partials/elasticsearch-index-sink-description.adoc
b/docs/modules/ROOT/partials/elasticsearch-index-sink-description.adoc
index 3be25f039..63b2fd36c 100644
--- a/docs/modules/ROOT/partials/elasticsearch-index-sink-description.adoc
+++ b/docs/modules/ROOT/partials/elasticsearch-index-sink-description.adoc
@@ -8,9 +8,41 @@ This Kamelet expects JSON-formatted data as input and outputs
plain text respons
This Kamelet supports username and password authentication for connecting to
ElasticSearch clusters.
-=== SSL Configuration
-
-SSL connection is enabled by default. You can provide a base64-encoded
certificate for secure connections to your ElasticSearch cluster.
+=== Trusting a Private Certificate Authority
+
+The Kamelet connects with HTTPS by default (`enableSSL`). The certificate of
the cluster is checked against the certificate authorities (CA) that Java
trusts, so a cluster whose certificate is self-signed, or signed by a private
CA, fails with `PKIX path building failed`.
+
+Give the certificate of that CA in `certificate`, as a PEM or DER file encoded
in base64 on one line. Elasticsearch generates such a CA when it starts with
security on, in `config/certs/http_ca.crt`. Encode it with `base64 -w0
http_ca.crt` (`base64 -i http_ca.crt` on macOS). Only this CA is trusted then.
+
+=== Timeouts
+
+When Elasticsearch does not answer, the message fails after 30 seconds.
`connectionTimeout` is how long to wait for the connection, and `socketTimeout`
how long to wait for data from Elasticsearch, both in milliseconds.
+
+=== Example
+
+[source,yaml]
+----
+- route:
+ from:
+ uri: "kamelet:timer-source"
+ parameters:
+ message: '{"order": 42, "status": "shipped"}'
+ contentType: application/json
+ steps:
+ - to:
+ uri: "kamelet:elasticsearch-index-sink"
+ parameters:
+ hostAddresses: "es.example.com:9200"
+ clusterName: "quickstart"
+ indexName: "orders"
+ user: "elastic"
+ password: "{{search.password}}"
+ certificate: "{{search.ca}}"
+ connectionTimeout: 5000
+ socketTimeout: 10000
+----
+
+Here `search.ca` is an application property that holds the certificate of the
CA, encoded in base64.
=== Index Configuration
diff --git a/docs/modules/ROOT/partials/opensearch-index-sink-description.adoc
b/docs/modules/ROOT/partials/opensearch-index-sink-description.adoc
index 257c1590d..7b8246bd3 100644
--- a/docs/modules/ROOT/partials/opensearch-index-sink-description.adoc
+++ b/docs/modules/ROOT/partials/opensearch-index-sink-description.adoc
@@ -23,4 +23,40 @@ OpenSearch is designed as a distributed search engine that
can:
=== Open Source Alternative
-OpenSearch serves as an open-source alternative to Elasticsearch, maintaining
compatibility while providing community-driven development and governance.
\ No newline at end of file
+OpenSearch serves as an open-source alternative to Elasticsearch, maintaining
compatibility while providing community-driven development and governance.
+
+=== Trusting a Private Certificate Authority
+
+The Kamelet connects with HTTPS by default (`enableSSL`). The certificate of
the cluster is checked against the certificate authorities (CA) that Java
trusts, so a cluster whose certificate is self-signed, or signed by a private
CA, fails with `PKIX path building failed`.
+
+Give the certificate of that CA in `certificate`, as a PEM or DER file encoded
in base64 on one line. The demo configuration of OpenSearch uses such a CA, in
`config/root-ca.pem`. Encode it with `base64 -w0 root-ca.pem` (`base64 -i
root-ca.pem` on macOS). Only this CA is trusted then.
+
+=== Timeouts
+
+When OpenSearch does not answer, the message fails after 30 seconds.
`connectionTimeout` is how long to wait for the connection, and `socketTimeout`
how long to wait for data from OpenSearch, both in milliseconds.
+
+=== Example
+
+[source,yaml]
+----
+- route:
+ from:
+ uri: "kamelet:timer-source"
+ parameters:
+ message: '{"order": 42, "status": "shipped"}'
+ contentType: application/json
+ steps:
+ - to:
+ uri: "kamelet:opensearch-index-sink"
+ parameters:
+ hostAddresses: "opensearch.example.com:9200"
+ clusterName: "opensearch-cluster"
+ indexName: "orders"
+ user: "admin"
+ password: "{{search.password}}"
+ certificate: "{{search.ca}}"
+ connectionTimeout: 5000
+ socketTimeout: 10000
+----
+
+Here `search.ca` is an application property that holds the certificate of the
CA, encoded in base64.
diff --git a/docs/modules/ROOT/partials/solr-sink-description.adoc
b/docs/modules/ROOT/partials/solr-sink-description.adoc
index d105ceaee..65706aa5f 100644
--- a/docs/modules/ROOT/partials/solr-sink-description.adoc
+++ b/docs/modules/ROOT/partials/solr-sink-description.adoc
@@ -33,4 +33,31 @@ Solr provides advanced search features including:
- Highlighting and suggestions
- Spatial and geographical search
- Complex query syntax and boosting
-- Analytics and aggregations
\ No newline at end of file
+- Analytics and aggregations
+
+=== Request Handler and Timeouts
+
+The documents are sent to the `/update` handler of the collection. To send
them to another update handler, defined in the configuration of the collection,
for example with its own update chain, give its path in `requestHandler`.
+
+When Solr does not answer, a message fails only after 10 minutes.
`requestTimeout` is how long to wait for the answer of Solr to each message,
and `connectionTimeout` how long to wait for the connection, both in
milliseconds. Over plain HTTP, the Solr client first checks the server with a
`HEAD` request, so the first message can wait twice as long.
+
+=== Example
+
+[source,yaml]
+----
+- route:
+ from:
+ uri: "kamelet:timer-source"
+ parameters:
+ message: '{"id": "order-42", "status_s": "shipped"}'
+ contentType: application/json
+ steps:
+ - to:
+ uri: "kamelet:solr-sink"
+ parameters:
+ servers: "solr.example.com:8983"
+ collection: "orders"
+ requestHandler: "/update/orders"
+ connectionTimeout: 5000
+ requestTimeout: 30000
+----
diff --git a/docs/modules/ROOT/partials/solr-source-description.adoc
b/docs/modules/ROOT/partials/solr-source-description.adoc
index 189054f1b..2ee2e3611 100644
--- a/docs/modules/ROOT/partials/solr-source-description.adoc
+++ b/docs/modules/ROOT/partials/solr-source-description.adoc
@@ -19,7 +19,13 @@ The Kamelet requires the following properties:
- `query`: the query, in the Solr query syntax, such as `status_s:new`
- `period`: the time between two queries, in milliseconds
-Optional properties include `size` (the most documents each query returns),
`enableSSL` and `certificate` (HTTPS), and `username` and `password` (Basic
authentication).
+Optional properties include `size` (the most documents each query returns),
`requestHandler` (the search handler that runs the query), `connectionTimeout`
and `requestTimeout`, `enableSSL` and `certificate` (HTTPS), and `username` and
`password` (Basic authentication).
+
+=== Request handler and timeouts
+
+The query runs on the `/select` handler of the collection. To run it on
another search handler, defined in the configuration of the collection, for
example with its own defaults or filters, give its path in `requestHandler`,
such as `/query`.
+
+When Solr does not answer, the query fails only after 10 minutes.
`requestTimeout` is how long to wait for the answer of Solr to each query, and
`connectionTimeout` how long to wait for the connection, both in milliseconds.
A query that fails is logged, and the next one runs at the next period.
=== Usage example
@@ -37,3 +43,22 @@ Optional properties include `size` (the most documents each
query returns), `ena
- to:
uri: "kamelet:log-sink"
----
+
+=== Example with more than 10 documents and a timeout
+
+[source,yaml]
+----
+- route:
+ from:
+ uri: "kamelet:solr-source"
+ parameters:
+ servers: "solr.example.com:8983"
+ collection: "orders"
+ query: "status_s:new"
+ period: 60000
+ size: 500
+ requestTimeout: 30000
+ steps:
+ - to:
+ uri: "kamelet:log-sink"
+----
diff --git a/kamelets/elasticsearch-index-sink.kamelet.yaml
b/kamelets/elasticsearch-index-sink.kamelet.yaml
index a11a0e40b..8795a2659 100644
--- a/kamelets/elasticsearch-index-sink.kamelet.yaml
+++ b/kamelets/elasticsearch-index-sink.kamelet.yaml
@@ -39,7 +39,7 @@ spec:
Output: when it is used inside a route, the body afterwards is the id of
the document, as text.
- Good to know: a body that is not JSON fails the message, and so does a
message with no index, from the option or the header. It connects with TLS by
default (`enableSSL`); for a server with a self-signed certificate, give its
certificate in `certificate`.
+ Good to know: a body that is not JSON fails the message, and so does a
message with no index, from the option or the header. It connects with TLS by
default (`enableSSL`). For a cluster whose certificate is self-signed, or
signed by a private certificate authority (CA), give the certificate of that CA
in `certificate`; without it, the message fails with `PKIX path building
failed`. When Elasticsearch does not answer, the message fails after 30
seconds; `connectionTimeout` and `sock [...]
required:
- clusterName
- hostAddresses
@@ -80,8 +80,18 @@ spec:
example: orders
certificate:
title: Certificate
- description: The certificate of the certificate authority (CA) that
signed the certificate of the cluster, such as the `http_ca.crt` file of
Elasticsearch, as PEM text encoded in base64. Needed when the cluster uses a
self-signed certificate.
+ description: The certificate of the certificate authority (CA) that
signed the certificate of the cluster, such as the `http_ca.crt` file that
Elasticsearch generates, as a PEM or DER file encoded in base64 on one line,
such as the output of `base64 -w0 http_ca.crt` (`base64 -i http_ca.crt` on
macOS). Needed when the cluster uses a self-signed certificate, or one signed
by a private CA. When it is set, only this CA is trusted; when it is not set,
the certificates that Java trusts [...]
type: string
+ connectionTimeout:
+ title: Connection Timeout
+ description: How long to wait for the connection to Elasticsearch, in
milliseconds. When it is not set, it is 30000, which is 30 seconds.
+ type: integer
+ example: 5000
+ socketTimeout:
+ title: Socket Timeout
+ description: How long to wait for data from Elasticsearch, in
milliseconds. When Elasticsearch sends nothing for this long, during the TLS
handshake or while it stores the document, the message fails with a
`SocketTimeoutException`. When it is not set, it is 30000, which is 30 seconds.
+ type: integer
+ example: 10000
dataTypes:
in:
headers:
@@ -165,4 +175,4 @@ spec:
- unmarshal:
json: {}
- to:
- uri:
"{{local-es}}:{{clusterName}}?operation=INDEX&hostAddresses={{hostAddresses}}&enableSSL={{enableSSL}}&indexName={{?indexName}}&certificatePath=base64:{{?certificate}}"
+ uri:
"{{local-es}}:{{clusterName}}?operation=INDEX&hostAddresses={{hostAddresses}}&enableSSL={{enableSSL}}&indexName={{?indexName}}&certificatePath=RAW(base64:{{?certificate}})&connectionTimeout={{?connectionTimeout}}&socketTimeout={{?socketTimeout}}"
diff --git a/kamelets/opensearch-index-sink.kamelet.yaml
b/kamelets/opensearch-index-sink.kamelet.yaml
index 11ee72a8b..aa9865adf 100644
--- a/kamelets/opensearch-index-sink.kamelet.yaml
+++ b/kamelets/opensearch-index-sink.kamelet.yaml
@@ -39,7 +39,7 @@ spec:
Output: when it is used inside a route, the body afterwards is the id of
the document, as text.
- Good to know: a body that is not JSON fails the message, and so does a
message with no index, from the option or the header. It connects with TLS by
default (`enableSSL`); for a server with a self-signed certificate, give its
certificate in `certificate`.
+ Good to know: a body that is not JSON fails the message, and so does a
message with no index, from the option or the header. It connects with TLS by
default (`enableSSL`). For a cluster whose certificate is self-signed, or
signed by a private certificate authority (CA), give the certificate of that CA
in `certificate`; without it, the message fails with `PKIX path building
failed`. When OpenSearch does not answer, the message fails after 30 seconds;
`connectionTimeout` and `socketT [...]
required:
- clusterName
- hostAddresses
@@ -85,8 +85,18 @@ spec:
example: "#bean:mySslContext"
certificate:
title: Certificate
- description: The certificate of the certificate authority (CA) that
signed the certificate of the cluster, as PEM text encoded in base64. Needed
when the cluster uses a self-signed certificate.
+ description: The certificate of the certificate authority (CA) that
signed the certificate of the cluster, such as the `root-ca.pem` file of the
OpenSearch demo configuration, as a PEM or DER file encoded in base64 on one
line, such as the output of `base64 -w0 root-ca.pem` (`base64 -i root-ca.pem`
on macOS). Needed when the cluster uses a self-signed certificate, or one
signed by a private CA. When it is set, only this CA is trusted; when it is not
set, the certificates that Jav [...]
type: string
+ connectionTimeout:
+ title: Connection Timeout
+ description: How long to wait for the connection to OpenSearch, in
milliseconds. When it is not set, it is 30000, which is 30 seconds.
+ type: integer
+ example: 5000
+ socketTimeout:
+ title: Socket Timeout
+ description: How long to wait for data from OpenSearch, in
milliseconds. When OpenSearch sends nothing for this long, for example while it
stores a document, the message fails. When it is not set, it is 30000, which is
30 seconds.
+ type: integer
+ example: 10000
dataTypes:
in:
headers:
@@ -170,4 +180,4 @@ spec:
- unmarshal:
json: {}
- to:
- uri:
"{{local-opensearch}}:{{clusterName}}?operation=INDEX&hostAddresses={{hostAddresses}}&enableSSL={{enableSSL}}&indexName={{?indexName}}&certificatePath=base64:{{?certificate}}&sslContextParameters={{?sslContextParameters}}"
+ uri:
"{{local-opensearch}}:{{clusterName}}?operation=INDEX&hostAddresses={{hostAddresses}}&enableSSL={{enableSSL}}&indexName={{?indexName}}&certificatePath=RAW(base64:{{?certificate}})&sslContextParameters={{?sslContextParameters}}&connectionTimeout={{?connectionTimeout}}&socketTimeout={{?socketTimeout}}"
diff --git a/kamelets/solr-sink.kamelet.yaml b/kamelets/solr-sink.kamelet.yaml
index c5078dc4f..f2fec6a0a 100644
--- a/kamelets/solr-sink.kamelet.yaml
+++ b/kamelets/solr-sink.kamelet.yaml
@@ -39,7 +39,7 @@ spec:
Output: when it is used inside a route, the body afterwards is the
answer of Solr, as a Map-like object with its `responseHeader`.
- Good to know: a body that is not JSON fails the message. With
`autocommit` false, the default, the documents become searchable only when Solr
commits them, as set in the configuration of the collection. It connects with
plain HTTP by default; set `enableSSL` to true for a Solr that is reached with
HTTPS, and give the certificate in `certificate` when Solr uses a self-signed
certificate.
+ Good to know: a body that is not JSON fails the message. With
`autocommit` false, the default, the documents become searchable only when Solr
commits them, as set in the configuration of the collection. It connects with
plain HTTP by default; set `enableSSL` to true for a Solr that is reached with
HTTPS, and give the certificate in `certificate` when Solr uses a self-signed
certificate. The documents go to the `/update` handler of the collection, or to
the one in `requestHandler`. [...]
required:
- servers
- collection
@@ -82,6 +82,22 @@ spec:
title: Certificate
description: The certificate of Solr, or of the certificate authority
(CA) that signed it, as a PEM or DER file encoded in base64 on one line, such
as the output of `base64 -w0 solr.crt` (`base64 -i solr.crt` on macOS). Needed
when Solr uses a self-signed certificate, or one that Java does not trust. When
it is not set, the certificates that Java trusts are used. Only used when
`enableSSL` is true.
type: string
+ requestHandler:
+ title: Request Handler
+ description: The path of the request handler of the collection to send
the documents to, such as `/update/catalog`, a handler defined in the
configuration of the collection, for example with its own defaults or update
chain. When it is not set, `/update` is used.
+ type: string
+ pattern: "^/[A-Za-z0-9_./-]+$"
+ example: "/update/catalog"
+ connectionTimeout:
+ title: Connection Timeout
+ description: How long to wait for the connection to Solr, in
milliseconds. When it is not set, it is 60000, which is one minute. Over plain
HTTP, the first message can wait twice as long, as the Solr client first checks
the server with a `HEAD` request.
+ type: integer
+ example: 5000
+ requestTimeout:
+ title: Request Timeout
+ description: How long to wait for the answer of Solr to each message,
in milliseconds. When Solr has not answered in time, the message fails. When it
is not set, it is 600000, which is 10 minutes. Over plain HTTP, the first
message can wait twice as long, as the Solr client first checks the server with
a `HEAD` request.
+ type: integer
+ example: 30000
types:
in:
mediaType: application/json
@@ -108,4 +124,4 @@ spec:
- removeHeader:
name: content-type
- to:
- uri:
"solr:{{servers}}/solr/?autocommit={{autocommit}}&password=RAW({{?password}})&username=RAW({{?username}})&enableSSL={{enableSSL}}&certificatePath=RAW(base64:{{?certificate}})"
+ uri:
"solr:{{servers}}/solr/?autocommit={{autocommit}}&password=RAW({{?password}})&username=RAW({{?username}})&enableSSL={{enableSSL}}&certificatePath=RAW(base64:{{?certificate}})&requestHandler={{?requestHandler}}&connectionTimeout={{?connectionTimeout}}&requestTimeout={{?requestTimeout}}"
diff --git a/kamelets/solr-source.kamelet.yaml
b/kamelets/solr-source.kamelet.yaml
index 784ed1f0d..b3fad5885 100644
--- a/kamelets/solr-source.kamelet.yaml
+++ b/kamelets/solr-source.kamelet.yaml
@@ -37,7 +37,7 @@ spec:
Output: the body is JSON text, the answer of Solr: `responseHeader`, and
under `response` the list of the documents found, each with its fields. The
`Content-Type` header is `application/json`.
- Good to know: every query returns all the matching documents again, not
only the new ones; filter on a date in the query to get only new documents.
Each query returns only the first documents found: as many as `size`, or 10
with the usual Solr settings when `size` is not set. It connects with plain
HTTP by default; set `enableSSL` to true for a Solr that is reached with HTTPS,
and give the certificate in `certificate` when Solr uses a self-signed
certificate.
+ Good to know: every query returns all the matching documents again, not
only the new ones; filter on a date in the query to get only new documents.
Each query returns only the first documents found: as many as `size`, or 10
with the usual Solr settings when `size` is not set. It connects with plain
HTTP by default; set `enableSSL` to true for a Solr that is reached with HTTPS,
and give the certificate in `certificate` when Solr uses a self-signed
certificate. The query runs on the [...]
required:
- period
- servers
@@ -93,6 +93,22 @@ spec:
title: Certificate
description: The certificate of Solr, or of the certificate authority
(CA) that signed it, as a PEM or DER file encoded in base64 on one line, such
as the output of `base64 -w0 solr.crt` (`base64 -i solr.crt` on macOS). Needed
when Solr uses a self-signed certificate, or one that Java does not trust. When
it is not set, the certificates that Java trusts are used. Only used when
`enableSSL` is true.
type: string
+ requestHandler:
+ title: Request Handler
+ description: The path of the search handler of the collection that
runs the query, such as `/query` or `/newOrders`, a handler defined in the
configuration of the collection, for example with its own defaults or filters.
When it is not set, `/select` is used.
+ type: string
+ pattern: "^/[A-Za-z0-9_./-]+$"
+ example: "/query"
+ connectionTimeout:
+ title: Connection Timeout
+ description: How long to wait for the connection to Solr, in
milliseconds. When it is not set, it is 60000, which is one minute.
+ type: integer
+ example: 5000
+ requestTimeout:
+ title: Request Timeout
+ description: How long to wait for the answer of Solr to each query, in
milliseconds. When Solr has not answered in time, the query fails, the error is
logged, and the next query runs at the next period. When it is not set, it is
600000, which is 10 minutes.
+ type: integer
+ example: 30000
types:
out:
mediaType: application/json
@@ -126,7 +142,7 @@ spec:
- removeHeader:
name: content-type
- to:
- uri:
"solr:{{servers}}/solr/?password=RAW({{?password}})&username=RAW({{?username}})&enableSSL={{enableSSL}}&certificatePath=RAW(base64:{{?certificate}})&size={{?size}}"
+ uri:
"solr:{{servers}}/solr/?password=RAW({{?password}})&username=RAW({{?username}})&enableSSL={{enableSSL}}&certificatePath=RAW(base64:{{?certificate}})&size={{?size}}&requestHandler={{?requestHandler}}&connectionTimeout={{?connectionTimeout}}&requestTimeout={{?requestTimeout}}"
- marshal:
json:
library: Jackson
diff --git
a/library/camel-kamelets/src/main/resources/META-INF/kamelets-index.json
b/library/camel-kamelets/src/main/resources/META-INF/kamelets-index.json
index 409ab4f22..ed249d7ec 100644
--- a/library/camel-kamelets/src/main/resources/META-INF/kamelets-index.json
+++ b/library/camel-kamelets/src/main/resources/META-INF/kamelets-index.json
@@ -715,7 +715,7 @@
"supportLevel" : "Stable",
"title" : "ElasticSearch Index Sink",
"summary" : "Stores each message, a JSON object, as a document in an index
of Elasticsearch, so that it can be searched.",
- "description" : "Stores each message, a JSON object, as a document in an
index of Elasticsearch, so that it can be searched.\n\nUse it to make records
searchable, such as orders, log entries or products, or to feed dashboards in
Kibana. For OpenSearch, use `opensearch-index-sink`.\n\nInput: the body is the
document, as JSON text, such as `{\"order\": 42, \"status\": \"shipped\"}`. The
optional `indexId` header (or `ce-indexid`) is the id of the document: a
document with the same id i [...]
+ "description" : "Stores each message, a JSON object, as a document in an
index of Elasticsearch, so that it can be searched.\n\nUse it to make records
searchable, such as orders, log entries or products, or to feed dashboards in
Kibana. For OpenSearch, use `opensearch-index-sink`.\n\nInput: the body is the
document, as JSON text, such as `{\"order\": 42, \"status\": \"shipped\"}`. The
optional `indexId` header (or `ce-indexid`) is the id of the document: a
document with the same id i [...]
"group" : "ElasticSearch",
"namespace" : "Search",
"provider" : "Apache Software Foundation"
@@ -1705,7 +1705,7 @@
"supportLevel" : "Stable",
"title" : "OpenSearch Index Sink",
"summary" : "Stores each message, a JSON object, as a document in an index
of OpenSearch, so that it can be searched.",
- "description" : "Stores each message, a JSON object, as a document in an
index of OpenSearch, so that it can be searched.\n\nUse it to make records
searchable, such as orders, log entries or products, or to feed dashboards in
OpenSearch Dashboards. For Elasticsearch, use
`elasticsearch-index-sink`.\n\nInput: the body is the document, as JSON text,
such as `{\"order\": 42, \"status\": \"shipped\"}`. The optional `indexId`
header (or `ce-indexid`) is the id of the document: a document [...]
+ "description" : "Stores each message, a JSON object, as a document in an
index of OpenSearch, so that it can be searched.\n\nUse it to make records
searchable, such as orders, log entries or products, or to feed dashboards in
OpenSearch Dashboards. For Elasticsearch, use
`elasticsearch-index-sink`.\n\nInput: the body is the document, as JSON text,
such as `{\"order\": 42, \"status\": \"shipped\"}`. The optional `indexId`
header (or `ce-indexid`) is the id of the document: a document [...]
"group" : "OpenSearch",
"namespace" : "Search",
"provider" : "Apache Software Foundation"
@@ -2085,7 +2085,7 @@
"supportLevel" : "Stable",
"title" : "Solr Sink",
"summary" : "Adds each message, a JSON object or a JSON array of objects,
as one or more documents to a collection or core of Apache Solr, so that they
can be searched.",
- "description" : "Adds each message, a JSON object or a JSON array of
objects, as one or more documents to a collection or core of Apache Solr, so
that they can be searched.\n\nUse it to make records searchable in Solr, such
as products, articles or orders. To run a query on a schedule, use
`solr-source`.\n\nInput: the body is JSON text: an object for one document,
such as `{\"id\": \"order-42\", \"status_s\": \"shipped\"}`, or an array of
objects for several. A document with the same [...]
+ "description" : "Adds each message, a JSON object or a JSON array of
objects, as one or more documents to a collection or core of Apache Solr, so
that they can be searched.\n\nUse it to make records searchable in Solr, such
as products, articles or orders. To run a query on a schedule, use
`solr-source`.\n\nInput: the body is JSON text: an object for one document,
such as `{\"id\": \"order-42\", \"status_s\": \"shipped\"}`, or an array of
objects for several. A document with the same [...]
"group" : "Solr",
"namespace" : "Search",
"provider" : "Apache Software Foundation"
@@ -2095,7 +2095,7 @@
"supportLevel" : "Stable",
"title" : "Solr Source",
"summary" : "Runs the same query on a collection or core of Apache Solr at
a regular interval, and sends the result of each query as one message.",
- "description" : "Runs the same query on a collection or core of Apache
Solr at a regular interval, and sends the result of each query as one
message.\n\nUse it to watch a Solr collection on a schedule, for example to
send the documents that match a query to another system. To add documents to
Solr, use `solr-sink`.\n\nOutput: the body is JSON text, the answer of Solr:
`responseHeader`, and under `response` the list of the documents found, each
with its fields. The `Content-Type` head [...]
+ "description" : "Runs the same query on a collection or core of Apache
Solr at a regular interval, and sends the result of each query as one
message.\n\nUse it to watch a Solr collection on a schedule, for example to
send the documents that match a query to another system. To add documents to
Solr, use `solr-sink`.\n\nOutput: the body is JSON text, the answer of Solr:
`responseHeader`, and under `response` the list of the documents found, each
with its fields. The `Content-Type` head [...]
"group" : "Solr",
"namespace" : "Search",
"provider" : "Apache Software Foundation"
diff --git
a/library/camel-kamelets/src/main/resources/kamelets/elasticsearch-index-sink.kamelet.yaml
b/library/camel-kamelets/src/main/resources/kamelets/elasticsearch-index-sink.kamelet.yaml
index a11a0e40b..8795a2659 100644
---
a/library/camel-kamelets/src/main/resources/kamelets/elasticsearch-index-sink.kamelet.yaml
+++
b/library/camel-kamelets/src/main/resources/kamelets/elasticsearch-index-sink.kamelet.yaml
@@ -39,7 +39,7 @@ spec:
Output: when it is used inside a route, the body afterwards is the id of
the document, as text.
- Good to know: a body that is not JSON fails the message, and so does a
message with no index, from the option or the header. It connects with TLS by
default (`enableSSL`); for a server with a self-signed certificate, give its
certificate in `certificate`.
+ Good to know: a body that is not JSON fails the message, and so does a
message with no index, from the option or the header. It connects with TLS by
default (`enableSSL`). For a cluster whose certificate is self-signed, or
signed by a private certificate authority (CA), give the certificate of that CA
in `certificate`; without it, the message fails with `PKIX path building
failed`. When Elasticsearch does not answer, the message fails after 30
seconds; `connectionTimeout` and `sock [...]
required:
- clusterName
- hostAddresses
@@ -80,8 +80,18 @@ spec:
example: orders
certificate:
title: Certificate
- description: The certificate of the certificate authority (CA) that
signed the certificate of the cluster, such as the `http_ca.crt` file of
Elasticsearch, as PEM text encoded in base64. Needed when the cluster uses a
self-signed certificate.
+ description: The certificate of the certificate authority (CA) that
signed the certificate of the cluster, such as the `http_ca.crt` file that
Elasticsearch generates, as a PEM or DER file encoded in base64 on one line,
such as the output of `base64 -w0 http_ca.crt` (`base64 -i http_ca.crt` on
macOS). Needed when the cluster uses a self-signed certificate, or one signed
by a private CA. When it is set, only this CA is trusted; when it is not set,
the certificates that Java trusts [...]
type: string
+ connectionTimeout:
+ title: Connection Timeout
+ description: How long to wait for the connection to Elasticsearch, in
milliseconds. When it is not set, it is 30000, which is 30 seconds.
+ type: integer
+ example: 5000
+ socketTimeout:
+ title: Socket Timeout
+ description: How long to wait for data from Elasticsearch, in
milliseconds. When Elasticsearch sends nothing for this long, during the TLS
handshake or while it stores the document, the message fails with a
`SocketTimeoutException`. When it is not set, it is 30000, which is 30 seconds.
+ type: integer
+ example: 10000
dataTypes:
in:
headers:
@@ -165,4 +175,4 @@ spec:
- unmarshal:
json: {}
- to:
- uri:
"{{local-es}}:{{clusterName}}?operation=INDEX&hostAddresses={{hostAddresses}}&enableSSL={{enableSSL}}&indexName={{?indexName}}&certificatePath=base64:{{?certificate}}"
+ uri:
"{{local-es}}:{{clusterName}}?operation=INDEX&hostAddresses={{hostAddresses}}&enableSSL={{enableSSL}}&indexName={{?indexName}}&certificatePath=RAW(base64:{{?certificate}})&connectionTimeout={{?connectionTimeout}}&socketTimeout={{?socketTimeout}}"
diff --git
a/library/camel-kamelets/src/main/resources/kamelets/opensearch-index-sink.kamelet.yaml
b/library/camel-kamelets/src/main/resources/kamelets/opensearch-index-sink.kamelet.yaml
index 11ee72a8b..aa9865adf 100644
---
a/library/camel-kamelets/src/main/resources/kamelets/opensearch-index-sink.kamelet.yaml
+++
b/library/camel-kamelets/src/main/resources/kamelets/opensearch-index-sink.kamelet.yaml
@@ -39,7 +39,7 @@ spec:
Output: when it is used inside a route, the body afterwards is the id of
the document, as text.
- Good to know: a body that is not JSON fails the message, and so does a
message with no index, from the option or the header. It connects with TLS by
default (`enableSSL`); for a server with a self-signed certificate, give its
certificate in `certificate`.
+ Good to know: a body that is not JSON fails the message, and so does a
message with no index, from the option or the header. It connects with TLS by
default (`enableSSL`). For a cluster whose certificate is self-signed, or
signed by a private certificate authority (CA), give the certificate of that CA
in `certificate`; without it, the message fails with `PKIX path building
failed`. When OpenSearch does not answer, the message fails after 30 seconds;
`connectionTimeout` and `socketT [...]
required:
- clusterName
- hostAddresses
@@ -85,8 +85,18 @@ spec:
example: "#bean:mySslContext"
certificate:
title: Certificate
- description: The certificate of the certificate authority (CA) that
signed the certificate of the cluster, as PEM text encoded in base64. Needed
when the cluster uses a self-signed certificate.
+ description: The certificate of the certificate authority (CA) that
signed the certificate of the cluster, such as the `root-ca.pem` file of the
OpenSearch demo configuration, as a PEM or DER file encoded in base64 on one
line, such as the output of `base64 -w0 root-ca.pem` (`base64 -i root-ca.pem`
on macOS). Needed when the cluster uses a self-signed certificate, or one
signed by a private CA. When it is set, only this CA is trusted; when it is not
set, the certificates that Jav [...]
type: string
+ connectionTimeout:
+ title: Connection Timeout
+ description: How long to wait for the connection to OpenSearch, in
milliseconds. When it is not set, it is 30000, which is 30 seconds.
+ type: integer
+ example: 5000
+ socketTimeout:
+ title: Socket Timeout
+ description: How long to wait for data from OpenSearch, in
milliseconds. When OpenSearch sends nothing for this long, for example while it
stores a document, the message fails. When it is not set, it is 30000, which is
30 seconds.
+ type: integer
+ example: 10000
dataTypes:
in:
headers:
@@ -170,4 +180,4 @@ spec:
- unmarshal:
json: {}
- to:
- uri:
"{{local-opensearch}}:{{clusterName}}?operation=INDEX&hostAddresses={{hostAddresses}}&enableSSL={{enableSSL}}&indexName={{?indexName}}&certificatePath=base64:{{?certificate}}&sslContextParameters={{?sslContextParameters}}"
+ uri:
"{{local-opensearch}}:{{clusterName}}?operation=INDEX&hostAddresses={{hostAddresses}}&enableSSL={{enableSSL}}&indexName={{?indexName}}&certificatePath=RAW(base64:{{?certificate}})&sslContextParameters={{?sslContextParameters}}&connectionTimeout={{?connectionTimeout}}&socketTimeout={{?socketTimeout}}"
diff --git
a/library/camel-kamelets/src/main/resources/kamelets/solr-sink.kamelet.yaml
b/library/camel-kamelets/src/main/resources/kamelets/solr-sink.kamelet.yaml
index c5078dc4f..f2fec6a0a 100644
--- a/library/camel-kamelets/src/main/resources/kamelets/solr-sink.kamelet.yaml
+++ b/library/camel-kamelets/src/main/resources/kamelets/solr-sink.kamelet.yaml
@@ -39,7 +39,7 @@ spec:
Output: when it is used inside a route, the body afterwards is the
answer of Solr, as a Map-like object with its `responseHeader`.
- Good to know: a body that is not JSON fails the message. With
`autocommit` false, the default, the documents become searchable only when Solr
commits them, as set in the configuration of the collection. It connects with
plain HTTP by default; set `enableSSL` to true for a Solr that is reached with
HTTPS, and give the certificate in `certificate` when Solr uses a self-signed
certificate.
+ Good to know: a body that is not JSON fails the message. With
`autocommit` false, the default, the documents become searchable only when Solr
commits them, as set in the configuration of the collection. It connects with
plain HTTP by default; set `enableSSL` to true for a Solr that is reached with
HTTPS, and give the certificate in `certificate` when Solr uses a self-signed
certificate. The documents go to the `/update` handler of the collection, or to
the one in `requestHandler`. [...]
required:
- servers
- collection
@@ -82,6 +82,22 @@ spec:
title: Certificate
description: The certificate of Solr, or of the certificate authority
(CA) that signed it, as a PEM or DER file encoded in base64 on one line, such
as the output of `base64 -w0 solr.crt` (`base64 -i solr.crt` on macOS). Needed
when Solr uses a self-signed certificate, or one that Java does not trust. When
it is not set, the certificates that Java trusts are used. Only used when
`enableSSL` is true.
type: string
+ requestHandler:
+ title: Request Handler
+ description: The path of the request handler of the collection to send
the documents to, such as `/update/catalog`, a handler defined in the
configuration of the collection, for example with its own defaults or update
chain. When it is not set, `/update` is used.
+ type: string
+ pattern: "^/[A-Za-z0-9_./-]+$"
+ example: "/update/catalog"
+ connectionTimeout:
+ title: Connection Timeout
+ description: How long to wait for the connection to Solr, in
milliseconds. When it is not set, it is 60000, which is one minute. Over plain
HTTP, the first message can wait twice as long, as the Solr client first checks
the server with a `HEAD` request.
+ type: integer
+ example: 5000
+ requestTimeout:
+ title: Request Timeout
+ description: How long to wait for the answer of Solr to each message,
in milliseconds. When Solr has not answered in time, the message fails. When it
is not set, it is 600000, which is 10 minutes. Over plain HTTP, the first
message can wait twice as long, as the Solr client first checks the server with
a `HEAD` request.
+ type: integer
+ example: 30000
types:
in:
mediaType: application/json
@@ -108,4 +124,4 @@ spec:
- removeHeader:
name: content-type
- to:
- uri:
"solr:{{servers}}/solr/?autocommit={{autocommit}}&password=RAW({{?password}})&username=RAW({{?username}})&enableSSL={{enableSSL}}&certificatePath=RAW(base64:{{?certificate}})"
+ uri:
"solr:{{servers}}/solr/?autocommit={{autocommit}}&password=RAW({{?password}})&username=RAW({{?username}})&enableSSL={{enableSSL}}&certificatePath=RAW(base64:{{?certificate}})&requestHandler={{?requestHandler}}&connectionTimeout={{?connectionTimeout}}&requestTimeout={{?requestTimeout}}"
diff --git
a/library/camel-kamelets/src/main/resources/kamelets/solr-source.kamelet.yaml
b/library/camel-kamelets/src/main/resources/kamelets/solr-source.kamelet.yaml
index 784ed1f0d..b3fad5885 100644
---
a/library/camel-kamelets/src/main/resources/kamelets/solr-source.kamelet.yaml
+++
b/library/camel-kamelets/src/main/resources/kamelets/solr-source.kamelet.yaml
@@ -37,7 +37,7 @@ spec:
Output: the body is JSON text, the answer of Solr: `responseHeader`, and
under `response` the list of the documents found, each with its fields. The
`Content-Type` header is `application/json`.
- Good to know: every query returns all the matching documents again, not
only the new ones; filter on a date in the query to get only new documents.
Each query returns only the first documents found: as many as `size`, or 10
with the usual Solr settings when `size` is not set. It connects with plain
HTTP by default; set `enableSSL` to true for a Solr that is reached with HTTPS,
and give the certificate in `certificate` when Solr uses a self-signed
certificate.
+ Good to know: every query returns all the matching documents again, not
only the new ones; filter on a date in the query to get only new documents.
Each query returns only the first documents found: as many as `size`, or 10
with the usual Solr settings when `size` is not set. It connects with plain
HTTP by default; set `enableSSL` to true for a Solr that is reached with HTTPS,
and give the certificate in `certificate` when Solr uses a self-signed
certificate. The query runs on the [...]
required:
- period
- servers
@@ -93,6 +93,22 @@ spec:
title: Certificate
description: The certificate of Solr, or of the certificate authority
(CA) that signed it, as a PEM or DER file encoded in base64 on one line, such
as the output of `base64 -w0 solr.crt` (`base64 -i solr.crt` on macOS). Needed
when Solr uses a self-signed certificate, or one that Java does not trust. When
it is not set, the certificates that Java trusts are used. Only used when
`enableSSL` is true.
type: string
+ requestHandler:
+ title: Request Handler
+ description: The path of the search handler of the collection that
runs the query, such as `/query` or `/newOrders`, a handler defined in the
configuration of the collection, for example with its own defaults or filters.
When it is not set, `/select` is used.
+ type: string
+ pattern: "^/[A-Za-z0-9_./-]+$"
+ example: "/query"
+ connectionTimeout:
+ title: Connection Timeout
+ description: How long to wait for the connection to Solr, in
milliseconds. When it is not set, it is 60000, which is one minute.
+ type: integer
+ example: 5000
+ requestTimeout:
+ title: Request Timeout
+ description: How long to wait for the answer of Solr to each query, in
milliseconds. When Solr has not answered in time, the query fails, the error is
logged, and the next query runs at the next period. When it is not set, it is
600000, which is 10 minutes.
+ type: integer
+ example: 30000
types:
out:
mediaType: application/json
@@ -126,7 +142,7 @@ spec:
- removeHeader:
name: content-type
- to:
- uri:
"solr:{{servers}}/solr/?password=RAW({{?password}})&username=RAW({{?username}})&enableSSL={{enableSSL}}&certificatePath=RAW(base64:{{?certificate}})&size={{?size}}"
+ uri:
"solr:{{servers}}/solr/?password=RAW({{?password}})&username=RAW({{?username}})&enableSSL={{enableSSL}}&certificatePath=RAW(base64:{{?certificate}})&size={{?size}}&requestHandler={{?requestHandler}}&connectionTimeout={{?connectionTimeout}}&requestTimeout={{?requestTimeout}}"
- marshal:
json:
library: Jackson