This is an automated email from the ASF dual-hosted git repository.
davsclaus pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/camel.git
The following commit(s) were added to refs/heads/master by this push:
new 5bd6d2a CAMEL-16205: camel-http - Optimize extract response body for
small payloads
5bd6d2a is described below
commit 5bd6d2a1f4161b31fa847fada51f75144973984e
Author: Claus Ibsen <claus.ib...@gmail.com>
AuthorDate: Mon Feb 15 17:39:51 2021 +0100
CAMEL-16205: camel-http - Optimize extract response body for small payloads
---
.../apache/camel/catalog/docs/http-component.adoc | 3 ++-
.../component/http/HttpComponentConfigurer.java | 6 ++++++
.../org/apache/camel/component/http/http.json | 1 +
.../org/apache/camel/component/http/https.json | 1 +
.../camel-http/src/main/docs/http-component.adoc | 3 ++-
.../apache/camel/component/http/HttpComponent.java | 12 ++++++++++++
.../apache/camel/component/http/HttpProducer.java | 22 ++++++++++++----------
.../component/dsl/HttpComponentBuilderFactory.java | 19 +++++++++++++++++++
.../dsl/HttpsComponentBuilderFactory.java | 19 +++++++++++++++++++
.../modules/ROOT/pages/http-component.adoc | 3 ++-
10 files changed, 76 insertions(+), 13 deletions(-)
diff --git
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/http-component.adoc
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/http-component.adoc
index 6afbfd0..66cfa4b 100644
---
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/http-component.adoc
+++
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/http-component.adoc
@@ -54,7 +54,7 @@ route, use the xref:jetty-component.adoc[Jetty Component]
instead.
// component options: START
-The HTTP component supports 26 options, which are listed below.
+The HTTP component supports 27 options, which are listed below.
@@ -63,6 +63,7 @@ The HTTP component supports 26 options, which are listed
below.
| Name | Description | Default | Type
| *cookieStore* (producer) | To use a custom
org.apache.http.client.CookieStore. By default the
org.apache.http.impl.client.BasicCookieStore is used which is an in-memory only
cookie store. Notice if bridgeEndpoint=true then the cookie store is forced to
be a noop cookie store as cookie shouldn't be stored as we are just bridging
(eg acting as a proxy). | | CookieStore
| *lazyStartProducer* (producer) | Whether the producer should be started lazy
(on the first message). By starting lazy you can use this to allow CamelContext
and routes to startup in situations where a producer may otherwise fail during
starting and cause the route to fail being started. By deferring this startup
to be lazy then the startup failure can be handled during routing messages via
Camel's routing error handlers. Beware that when the first message is processed
then creating and [...]
+| *responsePayloadStreaming{zwsp}Threshold* (producer) | This threshold in
bytes controls whether the response payload should be stored in memory as a
byte array or be streaming based. Set this to -1 to always use streaming mode.
| 8192 | int
| *allowJavaSerializedObject* (advanced) | Whether to allow java serialization
when a request uses context-type=application/x-java-serialized-object. This is
by default turned off. If you enable this then be aware that Java will
deserialize the incoming data from the request to Java and that can be a
potential security risk. | false | boolean
| *autowiredEnabled* (advanced) | Whether autowiring is enabled. This is used
for automatic autowiring options (the option must be marked as autowired) by
looking up in the registry to find if there is a single instance of matching
type, which then gets configured on the component. This can be used for
automatic configuring JDBC data sources, JMS connection factories, AWS Clients,
etc. | true | boolean
| *clientConnectionManager* (advanced) | To use a custom and shared
HttpClientConnectionManager to manage connections. If this has been configured
then this is always used for all endpoints created by this component. | |
HttpClientConnectionManager
diff --git
a/components/camel-http/src/generated/java/org/apache/camel/component/http/HttpComponentConfigurer.java
b/components/camel-http/src/generated/java/org/apache/camel/component/http/HttpComponentConfigurer.java
index da59e2a..c99b399 100644
---
a/components/camel-http/src/generated/java/org/apache/camel/component/http/HttpComponentConfigurer.java
+++
b/components/camel-http/src/generated/java/org/apache/camel/component/http/HttpComponentConfigurer.java
@@ -65,6 +65,8 @@ public class HttpComponentConfigurer extends
PropertyConfigurerSupport implement
case "proxyAuthPort": target.setProxyAuthPort(property(camelContext,
java.lang.Integer.class, value)); return true;
case "proxyauthusername":
case "proxyAuthUsername":
target.setProxyAuthUsername(property(camelContext, java.lang.String.class,
value)); return true;
+ case "responsepayloadstreamingthreshold":
+ case "responsePayloadStreamingThreshold":
target.setResponsePayloadStreamingThreshold(property(camelContext, int.class,
value)); return true;
case "sockettimeout":
case "socketTimeout": target.setSocketTimeout(property(camelContext,
int.class, value)); return true;
case "sslcontextparameters":
@@ -124,6 +126,8 @@ public class HttpComponentConfigurer extends
PropertyConfigurerSupport implement
case "proxyAuthPort": return java.lang.Integer.class;
case "proxyauthusername":
case "proxyAuthUsername": return java.lang.String.class;
+ case "responsepayloadstreamingthreshold":
+ case "responsePayloadStreamingThreshold": return int.class;
case "sockettimeout":
case "socketTimeout": return int.class;
case "sslcontextparameters":
@@ -184,6 +188,8 @@ public class HttpComponentConfigurer extends
PropertyConfigurerSupport implement
case "proxyAuthPort": return target.getProxyAuthPort();
case "proxyauthusername":
case "proxyAuthUsername": return target.getProxyAuthUsername();
+ case "responsepayloadstreamingthreshold":
+ case "responsePayloadStreamingThreshold": return
target.getResponsePayloadStreamingThreshold();
case "sockettimeout":
case "socketTimeout": return target.getSocketTimeout();
case "sslcontextparameters":
diff --git
a/components/camel-http/src/generated/resources/org/apache/camel/component/http/http.json
b/components/camel-http/src/generated/resources/org/apache/camel/component/http/http.json
index 5b7b7c7..c7ac369 100644
---
a/components/camel-http/src/generated/resources/org/apache/camel/component/http/http.json
+++
b/components/camel-http/src/generated/resources/org/apache/camel/component/http/http.json
@@ -26,6 +26,7 @@
"componentProperties": {
"cookieStore": { "kind": "property", "displayName": "Cookie Store",
"group": "producer", "label": "producer", "required": false, "type": "object",
"javaType": "org.apache.http.client.CookieStore", "deprecated": false,
"autowired": false, "secret": false, "description": "To use a custom
org.apache.http.client.CookieStore. By default the
org.apache.http.impl.client.BasicCookieStore is used which is an in-memory only
cookie store. Notice if bridgeEndpoint=true then the cookie store is f [...]
"lazyStartProducer": { "kind": "property", "displayName": "Lazy Start
Producer", "group": "producer", "label": "producer", "required": false, "type":
"boolean", "javaType": "boolean", "deprecated": false, "autowired": false,
"secret": false, "defaultValue": false, "description": "Whether the producer
should be started lazy (on the first message). By starting lazy you can use
this to allow CamelContext and routes to startup in situations where a producer
may otherwise fail during star [...]
+ "responsePayloadStreamingThreshold": { "kind": "property", "displayName":
"Response Payload Streaming Threshold", "group": "producer", "label":
"producer", "required": false, "type": "integer", "javaType": "int",
"deprecated": false, "autowired": false, "secret": false, "defaultValue": 8192,
"description": "This threshold in bytes controls whether the response payload
should be stored in memory as a byte array or be streaming based. Set this to
-1 to always use streaming mode." },
"allowJavaSerializedObject": { "kind": "property", "displayName": "Allow
Java Serialized Object", "group": "advanced", "label": "advanced", "required":
false, "type": "boolean", "javaType": "boolean", "deprecated": false,
"autowired": false, "secret": false, "defaultValue": false, "description":
"Whether to allow java serialization when a request uses
context-type=application\/x-java-serialized-object. This is by default turned
off. If you enable this then be aware that Java will des [...]
"autowiredEnabled": { "kind": "property", "displayName": "Autowired
Enabled", "group": "advanced", "label": "advanced", "required": false, "type":
"boolean", "javaType": "boolean", "deprecated": false, "autowired": false,
"secret": false, "defaultValue": true, "description": "Whether autowiring is
enabled. This is used for automatic autowiring options (the option must be
marked as autowired) by looking up in the registry to find if there is a single
instance of matching type, which t [...]
"clientConnectionManager": { "kind": "property", "displayName": "Client
Connection Manager", "group": "advanced", "label": "advanced", "required":
false, "type": "object", "javaType":
"org.apache.http.conn.HttpClientConnectionManager", "deprecated": false,
"autowired": false, "secret": false, "description": "To use a custom and shared
HttpClientConnectionManager to manage connections. If this has been configured
then this is always used for all endpoints created by this component." },
diff --git
a/components/camel-http/src/generated/resources/org/apache/camel/component/http/https.json
b/components/camel-http/src/generated/resources/org/apache/camel/component/http/https.json
index 9f574f5..39e9c0c 100644
---
a/components/camel-http/src/generated/resources/org/apache/camel/component/http/https.json
+++
b/components/camel-http/src/generated/resources/org/apache/camel/component/http/https.json
@@ -26,6 +26,7 @@
"componentProperties": {
"cookieStore": { "kind": "property", "displayName": "Cookie Store",
"group": "producer", "label": "producer", "required": false, "type": "object",
"javaType": "org.apache.http.client.CookieStore", "deprecated": false,
"autowired": false, "secret": false, "description": "To use a custom
org.apache.http.client.CookieStore. By default the
org.apache.http.impl.client.BasicCookieStore is used which is an in-memory only
cookie store. Notice if bridgeEndpoint=true then the cookie store is f [...]
"lazyStartProducer": { "kind": "property", "displayName": "Lazy Start
Producer", "group": "producer", "label": "producer", "required": false, "type":
"boolean", "javaType": "boolean", "deprecated": false, "autowired": false,
"secret": false, "defaultValue": false, "description": "Whether the producer
should be started lazy (on the first message). By starting lazy you can use
this to allow CamelContext and routes to startup in situations where a producer
may otherwise fail during star [...]
+ "responsePayloadStreamingThreshold": { "kind": "property", "displayName":
"Response Payload Streaming Threshold", "group": "producer", "label":
"producer", "required": false, "type": "integer", "javaType": "int",
"deprecated": false, "autowired": false, "secret": false, "defaultValue": 8192,
"description": "This threshold in bytes controls whether the response payload
should be stored in memory as a byte array or be streaming based. Set this to
-1 to always use streaming mode." },
"allowJavaSerializedObject": { "kind": "property", "displayName": "Allow
Java Serialized Object", "group": "advanced", "label": "advanced", "required":
false, "type": "boolean", "javaType": "boolean", "deprecated": false,
"autowired": false, "secret": false, "defaultValue": false, "description":
"Whether to allow java serialization when a request uses
context-type=application\/x-java-serialized-object. This is by default turned
off. If you enable this then be aware that Java will des [...]
"autowiredEnabled": { "kind": "property", "displayName": "Autowired
Enabled", "group": "advanced", "label": "advanced", "required": false, "type":
"boolean", "javaType": "boolean", "deprecated": false, "autowired": false,
"secret": false, "defaultValue": true, "description": "Whether autowiring is
enabled. This is used for automatic autowiring options (the option must be
marked as autowired) by looking up in the registry to find if there is a single
instance of matching type, which t [...]
"clientConnectionManager": { "kind": "property", "displayName": "Client
Connection Manager", "group": "advanced", "label": "advanced", "required":
false, "type": "object", "javaType":
"org.apache.http.conn.HttpClientConnectionManager", "deprecated": false,
"autowired": false, "secret": false, "description": "To use a custom and shared
HttpClientConnectionManager to manage connections. If this has been configured
then this is always used for all endpoints created by this component." },
diff --git a/components/camel-http/src/main/docs/http-component.adoc
b/components/camel-http/src/main/docs/http-component.adoc
index 6afbfd0..66cfa4b 100644
--- a/components/camel-http/src/main/docs/http-component.adoc
+++ b/components/camel-http/src/main/docs/http-component.adoc
@@ -54,7 +54,7 @@ route, use the xref:jetty-component.adoc[Jetty Component]
instead.
// component options: START
-The HTTP component supports 26 options, which are listed below.
+The HTTP component supports 27 options, which are listed below.
@@ -63,6 +63,7 @@ The HTTP component supports 26 options, which are listed
below.
| Name | Description | Default | Type
| *cookieStore* (producer) | To use a custom
org.apache.http.client.CookieStore. By default the
org.apache.http.impl.client.BasicCookieStore is used which is an in-memory only
cookie store. Notice if bridgeEndpoint=true then the cookie store is forced to
be a noop cookie store as cookie shouldn't be stored as we are just bridging
(eg acting as a proxy). | | CookieStore
| *lazyStartProducer* (producer) | Whether the producer should be started lazy
(on the first message). By starting lazy you can use this to allow CamelContext
and routes to startup in situations where a producer may otherwise fail during
starting and cause the route to fail being started. By deferring this startup
to be lazy then the startup failure can be handled during routing messages via
Camel's routing error handlers. Beware that when the first message is processed
then creating and [...]
+| *responsePayloadStreaming{zwsp}Threshold* (producer) | This threshold in
bytes controls whether the response payload should be stored in memory as a
byte array or be streaming based. Set this to -1 to always use streaming mode.
| 8192 | int
| *allowJavaSerializedObject* (advanced) | Whether to allow java serialization
when a request uses context-type=application/x-java-serialized-object. This is
by default turned off. If you enable this then be aware that Java will
deserialize the incoming data from the request to Java and that can be a
potential security risk. | false | boolean
| *autowiredEnabled* (advanced) | Whether autowiring is enabled. This is used
for automatic autowiring options (the option must be marked as autowired) by
looking up in the registry to find if there is a single instance of matching
type, which then gets configured on the component. This can be used for
automatic configuring JDBC data sources, JMS connection factories, AWS Clients,
etc. | true | boolean
| *clientConnectionManager* (advanced) | To use a custom and shared
HttpClientConnectionManager to manage connections. If this has been configured
then this is always used for all endpoints created by this component. | |
HttpClientConnectionManager
diff --git
a/components/camel-http/src/main/java/org/apache/camel/component/http/HttpComponent.java
b/components/camel-http/src/main/java/org/apache/camel/component/http/HttpComponent.java
index 03dee6d..65d4029 100644
---
a/components/camel-http/src/main/java/org/apache/camel/component/http/HttpComponent.java
+++
b/components/camel-http/src/main/java/org/apache/camel/component/http/HttpComponent.java
@@ -148,6 +148,10 @@ public class HttpComponent extends HttpCommonComponent
implements RestProducerFa
protected long connectionTimeToLive = -1;
@Metadata(label = "security", defaultValue = "false", description =
"Enable usage of global SSL context parameters.")
private boolean useGlobalSslContextParameters;
+ @Metadata(label = "producer", defaultValue = "8192",
+ description = "This threshold in bytes controls whether the
response payload"
+ + " should be stored in memory as a byte array or
be streaming based. Set this to -1 to always use streaming mode.")
+ protected int responsePayloadStreamingThreshold = 8192;
public HttpComponent() {
this(HttpEndpoint.class);
@@ -760,6 +764,14 @@ public class HttpComponent extends HttpCommonComponent
implements RestProducerFa
this.proxyAuthNtHost = proxyAuthNtHost;
}
+ public int getResponsePayloadStreamingThreshold() {
+ return responsePayloadStreamingThreshold;
+ }
+
+ public void setResponsePayloadStreamingThreshold(int
responsePayloadStreamingThreshold) {
+ this.responsePayloadStreamingThreshold =
responsePayloadStreamingThreshold;
+ }
+
@Override
public void doStart() throws Exception {
super.doStart();
diff --git
a/components/camel-http/src/main/java/org/apache/camel/component/http/HttpProducer.java
b/components/camel-http/src/main/java/org/apache/camel/component/http/HttpProducer.java
index 99ae025..2b954d1 100644
---
a/components/camel-http/src/main/java/org/apache/camel/component/http/HttpProducer.java
+++
b/components/camel-http/src/main/java/org/apache/camel/component/http/HttpProducer.java
@@ -448,18 +448,20 @@ public class HttpProducer extends DefaultProducer {
// ignore response
return null;
}
- long len = entity.getContentLength();
- if (len > 0 && len < IOHelper.DEFAULT_BUFFER_SIZE) {
+ int max =
getEndpoint().getComponent().getResponsePayloadStreamingThreshold();
+ if (max > 0) {
// optimize when we have content-length for small sizes to
avoid creating streaming objects
- int i = (int) len;
- byte[] arr = new byte[i];
- is.read(arr, 0, i);
- IOHelper.close(is);
- return arr;
- } else {
- // else for bigger payloads then wrap the response in a
stream cache so its re-readable
- return doExtractResponseBodyAsStream(is, exchange);
+ long len = entity.getContentLength();
+ if (len > 0 && len <= max) {
+ int i = (int) len;
+ byte[] arr = new byte[i];
+ is.read(arr, 0, i);
+ IOHelper.close(is);
+ return arr;
+ }
}
+ // else for bigger payloads then wrap the response in a stream
cache so its re-readable
+ return doExtractResponseBodyAsStream(is, exchange);
} else {
// use the response stream as-is
return is;
diff --git
a/core/camel-componentdsl/src/generated/java/org/apache/camel/builder/component/dsl/HttpComponentBuilderFactory.java
b/core/camel-componentdsl/src/generated/java/org/apache/camel/builder/component/dsl/HttpComponentBuilderFactory.java
index b569d34..aa03d71 100644
---
a/core/camel-componentdsl/src/generated/java/org/apache/camel/builder/component/dsl/HttpComponentBuilderFactory.java
+++
b/core/camel-componentdsl/src/generated/java/org/apache/camel/builder/component/dsl/HttpComponentBuilderFactory.java
@@ -92,6 +92,24 @@ public interface HttpComponentBuilderFactory {
return this;
}
/**
+ * This threshold in bytes controls whether the response payload should
+ * be stored in memory as a byte array or be streaming based. Set this
+ * to -1 to always use streaming mode.
+ *
+ * The option is a: <code>int</code> type.
+ *
+ * Default: 8192
+ * Group: producer
+ *
+ * @param responsePayloadStreamingThreshold the value to set
+ * @return the dsl builder
+ */
+ default HttpComponentBuilder responsePayloadStreamingThreshold(
+ int responsePayloadStreamingThreshold) {
+ doSetProperty("responsePayloadStreamingThreshold",
responsePayloadStreamingThreshold);
+ return this;
+ }
+ /**
* Whether to allow java serialization when a request uses
* context-type=application/x-java-serialized-object. This is by
default
* turned off. If you enable this then be aware that Java will
@@ -516,6 +534,7 @@ public interface HttpComponentBuilderFactory {
switch (name) {
case "cookieStore": ((HttpComponent)
component).setCookieStore((org.apache.http.client.CookieStore) value); return
true;
case "lazyStartProducer": ((HttpComponent)
component).setLazyStartProducer((boolean) value); return true;
+ case "responsePayloadStreamingThreshold": ((HttpComponent)
component).setResponsePayloadStreamingThreshold((int) value); return true;
case "allowJavaSerializedObject": ((HttpComponent)
component).setAllowJavaSerializedObject((boolean) value); return true;
case "autowiredEnabled": ((HttpComponent)
component).setAutowiredEnabled((boolean) value); return true;
case "clientConnectionManager": ((HttpComponent)
component).setClientConnectionManager((org.apache.http.conn.HttpClientConnectionManager)
value); return true;
diff --git
a/core/camel-componentdsl/src/generated/java/org/apache/camel/builder/component/dsl/HttpsComponentBuilderFactory.java
b/core/camel-componentdsl/src/generated/java/org/apache/camel/builder/component/dsl/HttpsComponentBuilderFactory.java
index ce4b940..4d20d49 100644
---
a/core/camel-componentdsl/src/generated/java/org/apache/camel/builder/component/dsl/HttpsComponentBuilderFactory.java
+++
b/core/camel-componentdsl/src/generated/java/org/apache/camel/builder/component/dsl/HttpsComponentBuilderFactory.java
@@ -93,6 +93,24 @@ public interface HttpsComponentBuilderFactory {
return this;
}
/**
+ * This threshold in bytes controls whether the response payload should
+ * be stored in memory as a byte array or be streaming based. Set this
+ * to -1 to always use streaming mode.
+ *
+ * The option is a: <code>int</code> type.
+ *
+ * Default: 8192
+ * Group: producer
+ *
+ * @param responsePayloadStreamingThreshold the value to set
+ * @return the dsl builder
+ */
+ default HttpsComponentBuilder responsePayloadStreamingThreshold(
+ int responsePayloadStreamingThreshold) {
+ doSetProperty("responsePayloadStreamingThreshold",
responsePayloadStreamingThreshold);
+ return this;
+ }
+ /**
* Whether to allow java serialization when a request uses
* context-type=application/x-java-serialized-object. This is by
default
* turned off. If you enable this then be aware that Java will
@@ -519,6 +537,7 @@ public interface HttpsComponentBuilderFactory {
switch (name) {
case "cookieStore": ((HttpComponent)
component).setCookieStore((org.apache.http.client.CookieStore) value); return
true;
case "lazyStartProducer": ((HttpComponent)
component).setLazyStartProducer((boolean) value); return true;
+ case "responsePayloadStreamingThreshold": ((HttpComponent)
component).setResponsePayloadStreamingThreshold((int) value); return true;
case "allowJavaSerializedObject": ((HttpComponent)
component).setAllowJavaSerializedObject((boolean) value); return true;
case "autowiredEnabled": ((HttpComponent)
component).setAutowiredEnabled((boolean) value); return true;
case "clientConnectionManager": ((HttpComponent)
component).setClientConnectionManager((org.apache.http.conn.HttpClientConnectionManager)
value); return true;
diff --git a/docs/components/modules/ROOT/pages/http-component.adoc
b/docs/components/modules/ROOT/pages/http-component.adoc
index 3086d81..3ed7bdb 100644
--- a/docs/components/modules/ROOT/pages/http-component.adoc
+++ b/docs/components/modules/ROOT/pages/http-component.adoc
@@ -56,7 +56,7 @@ route, use the xref:jetty-component.adoc[Jetty Component]
instead.
// component options: START
-The HTTP component supports 26 options, which are listed below.
+The HTTP component supports 27 options, which are listed below.
@@ -65,6 +65,7 @@ The HTTP component supports 26 options, which are listed
below.
| Name | Description | Default | Type
| *cookieStore* (producer) | To use a custom
org.apache.http.client.CookieStore. By default the
org.apache.http.impl.client.BasicCookieStore is used which is an in-memory only
cookie store. Notice if bridgeEndpoint=true then the cookie store is forced to
be a noop cookie store as cookie shouldn't be stored as we are just bridging
(eg acting as a proxy). | | CookieStore
| *lazyStartProducer* (producer) | Whether the producer should be started lazy
(on the first message). By starting lazy you can use this to allow CamelContext
and routes to startup in situations where a producer may otherwise fail during
starting and cause the route to fail being started. By deferring this startup
to be lazy then the startup failure can be handled during routing messages via
Camel's routing error handlers. Beware that when the first message is processed
then creating and [...]
+| *responsePayloadStreaming{zwsp}Threshold* (producer) | This threshold in
bytes controls whether the response payload should be stored in memory as a
byte array or be streaming based. Set this to -1 to always use streaming mode.
| 8192 | int
| *allowJavaSerializedObject* (advanced) | Whether to allow java serialization
when a request uses context-type=application/x-java-serialized-object. This is
by default turned off. If you enable this then be aware that Java will
deserialize the incoming data from the request to Java and that can be a
potential security risk. | false | boolean
| *autowiredEnabled* (advanced) | Whether autowiring is enabled. This is used
for automatic autowiring options (the option must be marked as autowired) by
looking up in the registry to find if there is a single instance of matching
type, which then gets configured on the component. This can be used for
automatic configuring JDBC data sources, JMS connection factories, AWS Clients,
etc. | true | boolean
| *clientConnectionManager* (advanced) | To use a custom and shared
HttpClientConnectionManager to manage connections. If this has been configured
then this is always used for all endpoints created by this component. | |
HttpClientConnectionManager
