Migration of EIP documentation
Project: http://git-wip-us.apache.org/repos/asf/camel/repo Commit: http://git-wip-us.apache.org/repos/asf/camel/commit/927244de Tree: http://git-wip-us.apache.org/repos/asf/camel/tree/927244de Diff: http://git-wip-us.apache.org/repos/asf/camel/diff/927244de Branch: refs/heads/master Commit: 927244de63583cd2eae451d6c14ce441b8de714c Parents: f6c27f9 Author: Sébastien COL <sebastien....@thalesgroup.com> Authored: Fri Feb 24 14:21:01 2017 +0100 Committer: Claus Ibsen <davscl...@apache.org> Committed: Fri Feb 24 14:30:24 2017 +0100 ---------------------------------------------------------------------- camel-core/src/main/docs/eips/loop-eip.adoc | 167 +++++ .../src/main/docs/eips/recipientList-eip.adoc | 415 +++++++++++ .../src/main/docs/eips/resequence-eip.adoc | 260 +++++++ .../src/main/docs/eips/routingSlip-eip.adoc | 114 +++ camel-core/src/main/docs/eips/sample-eip.adoc | 96 +++ camel-core/src/main/docs/eips/script-eip.adoc | 84 +++ camel-core/src/main/docs/eips/sort-eip.adoc | 98 +++ camel-core/src/main/docs/eips/split-eip.adoc | 688 +++++++++++++++++++ camel-core/src/main/docs/eips/throttle-eip.adoc | 95 +++ camel-core/src/main/docs/eips/validate-eip.adoc | 77 +++ 10 files changed, 2094 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/camel/blob/927244de/camel-core/src/main/docs/eips/loop-eip.adoc ---------------------------------------------------------------------- diff --git a/camel-core/src/main/docs/eips/loop-eip.adoc b/camel-core/src/main/docs/eips/loop-eip.adoc new file mode 100644 index 0000000..c1147ec --- /dev/null +++ b/camel-core/src/main/docs/eips/loop-eip.adoc @@ -0,0 +1,167 @@ +## Loop EIP +The Loop allows for processing a message a number of times, possibly in a different way for each iteration. Useful mostly during testing. + +[NOTE] +.Default mode +==== +Notice by default the loop uses the same exchange throughout the looping. So the result from the previous iteration will be used for the next (eg Pipes and Filters). From Camel 2.8 onwards you can enable copy mode instead. See the options table for more details. +==== + +### Options + +// eip options: START +The Loop EIP supports 2 options which are listed below: + +{% raw %} +[width="100%",cols="3,1m,6",options="header"] +|======================================================================= +| Name | Java Type | Description +| copy | Boolean | If the copy attribute is true a copy of the input Exchange is used for each iteration. That means each iteration will start from a copy of the same message. By default loop will loop the same exchange all over so each iteration may have different message content. +| doWhile | Boolean | Enables the while loop that loops until the predicate evaluates to false or null. +|======================================================================= +{% endraw %} +// eip options: END + + +### Exchange properties +For each iteration two properties are set on the Exchange. Processors can rely on these properties to process the Message in different ways. +[width="100%",cols="3,6",options="header"] +|======================================================================= +| Property | Description +| CamelLoopSize | Total number of loops. This is not available if running the loop in while loop mode. +| CamelLoopIndex | Index of the current iteration (0 based) +|======================================================================= + +### Examples +The following example shows how to take a request from the *direct:x* endpoint, then send the message repetitively to *mock:result*. The number of times the message is sent is either passed as an argument to `loop()`, or determined at runtime by evaluating an expression. The expression *must* evaluate to an int, otherwise a `RuntimeCamelException` is thrown. + +#### Using the Fluent Builders +Pass loop count as an argument +[source,java] +--------------------- +from("direct:a").loop(8).to("mock:result"); +--------------------- + +Use expression to determine loop count +[source,java] +--------------------- +from("direct:b").loop(header("loop")).to("mock:result"); +--------------------- + +Use expression to determine loop count +[source,java] +--------------------- +from("direct:c").loop().xpath("/hello/@times").to("mock:result"); +--------------------- + +#### Using the Spring XML Extensions +Pass loop count as an argument +[source,xml] +--------------------- +<route> + <from uri="direct:a"/> + <loop> + <constant>8</constant> + <to uri="mock:result"/> + </loop> +</route> +--------------------- + +Use expression to determine loop count +[source,xml] +--------------------- +<route> + <from uri="direct:b"/> + <loop> + <header>loop</header> + <to uri="mock:result"/> + </loop> +</route> +--------------------- + +For further examples of this pattern in use you could look at one of the junit test case. + +#### Using copy mode +*Available as of Camel 2.8* + +Now suppose we send a message to "direct:start" endpoint containing the letter A. + +The output of processing this route will be that, each "mock:loop" endpoint will receive "AB" as message. + +[source,java] +--------------------- +from("direct:start") + // instruct loop to use copy mode, which mean it will use a copy of the input exchange + // for each loop iteration, instead of keep using the same exchange all over + .loop(3).copy() + .transform(body().append("B")) + .to("mock:loop") + .end() + .to("mock:result"); +--------------------- + +However if we do *not* enable copy mode then "mock:loop" will receive "AB", "ABB", "ABBB", etc. messages. + +[source,java] +--------------------- +from("direct:start") + // by default loop will keep using the same exchange so on the 2nd and 3rd iteration its + // the same exchange that was previous used that are being looped all over + .loop(3) + .transform(body().append("B")) + .to("mock:loop") + .end() + .to("mock:result"); +--------------------- + +The equivalent example in XML DSL in copy mode is as follows: + +[source,xml] +--------------------- +<route> + <from uri="direct:start"/> + <!-- enable copy mode for loop eip --> + <loop copy="true"> + <constant>3</constant> + <transform> + <simple>${body}B</simple> + </transform> + <to uri="mock:loop"/> + </loop> + <to uri="mock:result"/> +</route> +--------------------- + +#### Using while mode +*Available as of Camel 2.17* + +The loop can act like a while loop that loops until the expression evaluates to false or null. + +For example the route below loops while the length of the message body is 5 or less characters. Notice that the DSL uses *loopDoWhile*. + +[source,java] +--------------------- +from("direct:start") + .loopDoWhile(simple("${body.length} <= 5")) + .to("mock:loop") + .transform(body().append("A")) + .end() + .to("mock:result"); +--------------------- + +And the same example in XML: +[source,xml] +--------------------- +<route> + <from uri="direct:start"/> + <loop doWhile="true"> + <simple>${body.length} <= 5</simple> + <to uri="mock:loop"/> + <transform> + <simple>A${body}</simple> + </transform> + </loop> + <to uri="mock:result"/> +</route> +--------------------- + +Notice in XML that the while loop is turned on using the *doWhile* attribute. + +### Using This Pattern +If you would like to use this EIP Pattern then please read the Getting Started, you may also find the Architecture useful particularly the description of Endpoint and URIs. Then you could try out some of the Examples first before trying this pattern out. http://git-wip-us.apache.org/repos/asf/camel/blob/927244de/camel-core/src/main/docs/eips/recipientList-eip.adoc ---------------------------------------------------------------------- diff --git a/camel-core/src/main/docs/eips/recipientList-eip.adoc b/camel-core/src/main/docs/eips/recipientList-eip.adoc new file mode 100644 index 0000000..ffc0e97 --- /dev/null +++ b/camel-core/src/main/docs/eips/recipientList-eip.adoc @@ -0,0 +1,415 @@ +## Recipient List EIP +The link:http://www.enterpriseintegrationpatterns.com/RecipientList.html[Recipient List] from the EIP patterns allows you to route messages to a number of dynamically specified recipients. + +image:http://www.enterpriseintegrationpatterns.com/img/RecipientList.gif[image] + +The recipients will receive a copy of the *same* Exchange, and Camel will execute them sequentially. + +### Options + +// eip options: START +The Recipient List EIP supports 15 options which are listed below: + +{% raw %} +[width="100%",cols="3,1m,6",options="header"] +|======================================================================= +| Name | Java Type | Description +| delimiter | String | Delimiter used if the Expression returned multiple endpoints. Can be turned off using the value false. The default value is +| parallelProcessing | Boolean | If enabled then sending messages to the recipients occurs concurrently. Note the caller thread will still wait until all messages has been fully processed before it continues. Its only the sending and processing the replies from the recipients which happens concurrently. +| strategyRef | String | Sets a reference to the AggregationStrategy to be used to assemble the replies from the recipients into a single outgoing message from the RecipientList. By default Camel will use the last reply as the outgoing message. You can also use a POJO as the AggregationStrategy +| strategyMethodName | String | This option can be used to explicit declare the method name to use when using POJOs as the AggregationStrategy. +| strategyMethodAllowNull | Boolean | If this option is false then the aggregate method is not used if there was no data to enrich. If this option is true then null values is used as the oldExchange (when no data to enrich) when using POJOs as the AggregationStrategy +| executorServiceRef | String | Refers to a custom Thread Pool to be used for parallel processing. Notice if you set this option then parallel processing is automatic implied and you do not have to enable that option as well. +| stopOnException | Boolean | Will now stop further processing if an exception or failure occurred during processing of an org.apache.camel.Exchange and the caused exception will be thrown. Will also stop if processing the exchange failed (has a fault message) or an exception was thrown and handled by the error handler (such as using onException). In all situations the recipient list will stop further processing. This is the same behavior as in pipeline which is used by the routing engine. The default behavior is to not stop but continue processing till the end +| ignoreInvalidEndpoints | Boolean | Ignore the invalidate endpoint exception when try to create a producer with that endpoint +| streaming | Boolean | If enabled then Camel will process replies out-of-order eg in the order they come back. If disabled Camel will process replies in the same order as defined by the recipient list. +| timeout | Long | Sets a total timeout specified in millis when using parallel processing. If the Recipient List hasn't been able to send and process all replies within the given timeframe then the timeout triggers and the Recipient List breaks out and continues. Notice if you provide a TimeoutAwareAggregationStrategy then the timeout method is invoked before breaking out. If the timeout is reached with running tasks still remaining certain tasks for which it is difficult for Camel to shut down in a graceful manner may continue to run. So use this option with a bit of care. +| onPrepareRef | String | Uses the Processor when preparing the org.apache.camel.Exchange to be send. This can be used to deep-clone messages that should be send or any custom logic needed before the exchange is send. +| shareUnitOfWork | Boolean | Shares the org.apache.camel.spi.UnitOfWork with the parent and each of the sub messages. Recipient List will by default not share unit of work between the parent exchange and each recipient exchange. This means each sub exchange has its own individual unit of work. +| cacheSize | Integer | Sets the maximum size used by the org.apache.camel.impl.ProducerCache which is used to cache and reuse producers when using this recipient list when uris are reused. +| parallelAggregate | Boolean | If enabled then the aggregate method on AggregationStrategy can be called concurrently. Notice that this would require the implementation of AggregationStrategy to be implemented as thread-safe. By default this is false meaning that Camel synchronizes the call to the aggregate method. Though in some use-cases this can be used to archive higher performance when the AggregationStrategy is implemented as thread-safe. +| stopOnAggregateException | Boolean | If enabled unwind exceptions occurring at aggregation time to the error handler when parallelProcessing is used. Currently aggregation time exceptions do not stop the route processing when parallelProcessing is used. Enabling this option allows to work around this behavior. The default value is false for the sake of backward compatibility. +|======================================================================= +{% endraw %} +// eip options: END + + +### Static Recipient List +The following example shows how to route a request from an input *queue:a* endpoint to a static list of destinations + +#### Using Annotations +You can use the RecipientList Annotation on a POJO to create a Dynamic Recipient List. For more details see the Bean Integration. + +#### Using the Fluent Builders + +[source,java] +--------------------- +RouteBuilder builder = new RouteBuilder() { + public void configure() { + errorHandler(deadLetterChannel("mock:error")); + + from("direct:a") + .multicast().to("direct:b", "direct:c", "direct:d"); + } +}; +--------------------- + +#### Using the Spring XML Extensions + +[source,xml] +--------------------- +<camelContext errorHandlerRef="errorHandler" xmlns="http://camel.apache.org/schema/spring";> + <route> + <from uri="direct:a"/> + <multicast> + <to uri="direct:b"/> + <to uri="direct:c"/> + <to uri="direct:d"/> + </multicast> + </route> +</camelContext> +--------------------- + +### Dynamic Recipient List +Usually one of the main reasons for using the Recipient List pattern is that the list of recipients is dynamic and calculated at runtime. The following example demonstrates how to create a dynamic recipient list using an Expression (which in this case extracts a named header value dynamically) to calculate the list of endpoints which are either of type Endpoint or are converted to a String and then resolved using the endpoint URIs. + +#### Using the Fluent Builders + +[source,java] +--------------------- +RouteBuilder builder = new RouteBuilder() { + public void configure() { + errorHandler(deadLetterChannel("mock:error")); + + from("direct:a") + .recipientList(header("foo")); + } +}; +--------------------- + +The above assumes that the header contains a list of endpoint URIs. The following takes a single string header and tokenizes it + +[source,java] +--------------------- +from("direct:a").recipientList( + header("recipientListHeader").tokenize(",")); +--------------------- + +##### Iteratable value +The dynamic list of recipients that are defined in the header must be iterable such as: + +* `java.util.Collection` +* `java.util.Iterator` +* arrays +* `org.w3c.dom.NodeList` +* a single String with values separated by comma +* any other type will be regarded as a single value + +#### Using the Spring XML Extensions +[source,xml] +--------------------- +<camelContext errorHandlerRef="errorHandler" xmlns="http://camel.apache.org/schema/spring";> + <route> + <from uri="direct:a"/> + <recipientList> + <xpath>$foo</xpath> + </recipientList> + </route> +</camelContext> +--------------------- + +For further examples of this pattern in action you could take a look at one of the junit test cases. + +##### Using delimiter in Spring XML +In Spring DSL you can set the delimiter attribute for setting a delimiter to be used if the header value is a single String with multiple separated endpoints. By default Camel uses comma as delimiter, but this option lets you specify a custom delimiter to use instead. + +[source,xml] +--------------------- +<route> + <from uri="direct:a" /> + <!-- use comma as a delimiter for String based values --> + <recipientList delimiter=","> + <header>myHeader</header> + </recipientList> +</route> +--------------------- + +So if *myHeader* contains a `String` with the value `"activemq:queue:foo, activemq:topic:hello , log:bar"` then Camel will split the `String` using the delimiter given in the XML that was comma, resulting into 3 endpoints to send to. You can use spaces between the endpoints as Camel will trim the value when it lookup the endpoint to send to. + +[NOTE] +In Java DSL you use the `tokenizer` to achieve the same. The route above in Java DSL: + +[source,java] +--------------------- +from("direct:a").recipientList(header("myHeader").tokenize(",")); +--------------------- + +In *Camel 2.1* its a bit easier as you can pass in the delimiter as 2nd parameter: + +[source,java] +--------------------- +from("direct:a").recipientList(header("myHeader"), "#"); +--------------------- + +### Sending to multiple recipients in parallel +*Available as of Camel 2.2* + +The Recipient List now supports `parallelProcessing` that for example Splitter also supports. You can use it to use a thread pool to have concurrent tasks sending the Exchange to multiple recipients concurrently. + +[source,java] +--------------------- +from("direct:a").recipientList(header("myHeader")).parallelProcessing(); +--------------------- + +And in Spring XML it is an attribute on the recipient list tag. + +[source,xml] +--------------------- +<route> + <from uri="direct:a"/> + <recipientList parallelProcessing="true"> + <header>myHeader</header> + </recipientList> +</route> +--------------------- + +### Stop continuing in case one recipient failed +*Available as of Camel 2.2* + +The Recipient List now supports `stopOnException` that for example Splitter also supports. You can use it to stop sending to any further recipients in case any recipient failed. + +[source,java] +--------------------- +from("direct:a").recipientList(header("myHeader")).stopOnException(); +--------------------- + +And in Spring XML its an attribute on the recipient list tag. + +[source,xml] +--------------------- +<route> + <from uri="direct:a"/> + <recipientList stopOnException="true"> + <header>myHeader</header> + </recipientList> +</route> +--------------------- + +[NOTE] +You can combine parallelProcessing and stopOnException and have them both true. + +### Ignore invalid endpoints +*Available as of Camel 2.3* + +The Recipient List now supports `ignoreInvalidEndpoints` (like the Routing Slip). You can use it to skip endpoints which are invalid. + +[source,java] +--------------------- +from("direct:a").recipientList(header("myHeader")).ignoreInvalidEndpoints(); +--------------------- + +And in Spring XML it is an attribute on the recipient list tag. + +[source,xml] +--------------------- +<route> + <from uri="direct:a"/> + <recipientList ignoreInvalidEndpoints="true"> + <header>myHeader</header> + </recipientList> +</route> +--------------------- + +Then let us say the `myHeader` contains the following two endpoints `direct:foo,xxx:bar`. The first endpoint is valid and works. However the second one is invalid and will just be ignored. Camel logs at INFO level about it, so you can see why the endpoint was invalid. + + +### Using custom `AggregationStrategy` +*Available as of Camel 2.2* + +You can now use your own `AggregationStrategy` with the Recipient List. However this is rarely needed. +What it is good for is that in case you are using Request Reply messaging then the replies from the recipients can be aggregated. +By default Camel uses `UseLatestAggregationStrategy` which just keeps that last received reply. If you must remember all the bodies that all the recipients sent back, +then you can use your own custom aggregator that keeps those. It is the same principle as with the Aggregator EIP so check it out for details. + +[source,java] +--------------------- +from("direct:a") + .recipientList(header("myHeader")).aggregationStrategy(new MyOwnAggregationStrategy()) + .to("direct:b"); +--------------------- + +And in Spring XML it is again an attribute on the recipient list tag. + +[source,xml] +--------------------- +<route> + <from uri="direct:a"/> + <recipientList strategyRef="myStrategy"> + <header>myHeader</header> + </recipientList> + <to uri="direct:b"/> +</route> + +<bean id="myStrategy" class="com.mycompany.MyOwnAggregationStrategy"/> +--------------------- + +### Knowing which endpoint when using custom `AggregationStrategy` +Available as of Camel 2.12 + +When using a custom `AggregationStrategy` then the `aggregate` method is always invoked in sequential order (also if parallel processing is enabled) of the endpoints the Recipient List is using. +However from Camel 2.12 onwards this is easier to know as the `newExchange` Exchange now has a property stored (key is `Exchange.RECIPIENT_LIST_ENDPOINT` with the uri of the Endpoint. +So you know which endpoint you are aggregating from. The code block shows how to access this property in your Aggregator. + +[source,java] +--------------------- +@Override +public Exchange aggregate(Exchange oldExchange, Exchange newExchange) { + String uri = newExchange.getProperty(Exchange.RECIPIENT_LIST_ENDPOINT, String.class); + ... +} +--------------------- + +### Using custom thread pool +*Available as of Camel 2.2* + +A thread pool is only used for `parallelProcessing`. You supply your own custom thread pool via the `ExecutorServiceStrategy` (see Camel's Threading Model), +the same way you would do it for the `aggregationStrategy`. By default Camel uses a thread pool with 10 threads (subject to change in future versions). + +### Using method call as recipient list +You can use a Bean to provide the recipients, for example: + +[source,java] +--------------------- +from("activemq:queue:test").recipientList().method(MessageRouter.class, "routeTo"); +--------------------- + +And then `MessageRouter`: + +[source,java] +--------------------- +public class MessageRouter { + + public String routeTo() { + String queueName = "activemq:queue:test2"; + return queueName; + } +} +--------------------- + +When you use a Bean then do *not* use the `@RecipientList` annotation as this will in fact add yet another recipient list, so you end up having two. Do *not* do the following. + +[source,java] +--------------------- +public class MessageRouter { + + @RecipientList + public String routeTo() { + String queueName = "activemq:queue:test2"; + return queueName; + } +} +--------------------- + +You should only use the snippet above (using `@RecipientList`) if you just route to a Bean which you then want to act as a recipient list. + +So the original route can be changed to: + +[source,java] +--------------------- +from("activemq:queue:test").bean(MessageRouter.class, "routeTo"); +--------------------- + +Which then would invoke the routeTo method and detect that it is annotated with `@RecipientList` and then act accordingly as if it was a recipient list EIP. + +### Using timeout +*Available as of Camel 2.5* + +If you use `parallelProcessing` then you can configure a total `timeout` value in millis. Camel will then process the messages in parallel until the timeout is hit. This allows you to continue processing if one message consumer is slow. For example you can set a timeout value of 20 sec. + +[WARNING] +.Tasks may keep running +==== +If the timeout is reached with running tasks still remaining, certain tasks for which it is difficult for Camel to shut down in a graceful manner may continue to run. So use this option with a bit of care. We may be able to improve this functionality in future Camel releases. +==== + +For example in the unit test below you can see that we multicast the message to 3 destinations. We have a timeout of 2 seconds, which means only the last two messages can be completed within the timeframe. This means we will only aggregate the last two which yields a result aggregation which outputs "BC". + +[source,java] +--------------------- +from("direct:start") + .multicast(new AggregationStrategy() { + public Exchange aggregate(Exchange oldExchange, Exchange newExchange) { + if (oldExchange == null) { + return newExchange; + } + + String body = oldExchange.getIn().getBody(String.class); + oldExchange.getIn().setBody(body + newExchange.getIn().getBody(String.class)); + return oldExchange; + } + }) + .parallelProcessing().timeout(250).to("direct:a", "direct:b", "direct:c") + // use end to indicate end of multicast route + .end() + .to("mock:result"); + +from("direct:a").delay(1000).to("mock:A").setBody(constant("A")); + +from("direct:b").to("mock:B").setBody(constant("B")); + +from("direct:c").to("mock:C").setBody(constant("C")); +--------------------- + +[NOTE] +.Timeout in other EIPs +==== +This timeout feature is also supported by Splitter and both multicast and recipientList. +==== + +By default if a timeout occurs the `AggregationStrategy` is not invoked. However you can implement a special version + +[source,java] +.TimeoutAwareAggregationStrategy +--------------------- +public interface TimeoutAwareAggregationStrategy extends AggregationStrategy { + + /** + * A timeout occurred + * + * @param oldExchange the oldest exchange (is <tt>null</tt> on first aggregation as we only have the new exchange) + * @param index the index + * @param total the total + * @param timeout the timeout value in millis + */ + void timeout(Exchange oldExchange, int index, int total, long timeout); +--------------------- + +This allows you to deal with the timeout in the `AggregationStrategy` if you really need to. + +[NOTE] +.Timeout is total +==== +The timeout is total, which means that after X time, Camel will aggregate the messages which have completed within the timeframe. +The remainders will be cancelled. Camel will also only invoke the `timeout` method in the `TimeoutAwareAggregationStrategy` once, for the first index which caused the timeout. +==== + +### Using onPrepare to execute custom logic when preparing messages +*Available as of Camel 2.8* + +See details at Multicast + +### Using ExchangePattern in recipients +*Available as of Camel 2.15* + +The recipient list will by default use the current Exchange Pattern. Though one can imagine use-cases where one wants to send a message to a recipient using a different exchange pattern. For example you may have a route that initiates as an InOnly route, but want to use InOut exchange pattern with a recipient list. To do this in earlier Camel releases, you would need to change the exchange pattern before the recipient list, or use onPrepare option to alter the pattern. From Camel 2.15 onwards, you can configure the exchange pattern directly in the recipient endpoints. + +For example in the route below we pick up new files (which will be started as InOnly) and then route to a recipient list. As we want to use InOut with the ActiveMQ (JMS) endpoint we can now specify this using the exchangePattern=InOut option. Then the response from the JMS request/reply will then be continued routed, and thus the response is what will be stored in as a file in the outbox directory. + +[source,java] +--------------------- +from("file:inbox") + // the exchange pattern is InOnly initially when using a file route + .recipientList().constant("activemq:queue:inbox?exchangePattern=InOut") + .to("file:outbox"); +--------------------- + +[WARNING] +==== +The recipient list will not alter the original exchange pattern. So in the example above the exchange pattern will still be InOnly when the message is routed to the file:outbox endpoint. + +If you want to alter the exchange pattern permanently then use the .setExchangePattern option. See more details at Request Reply and Event Message. +==== + +### Using This Pattern +If you would like to use this EIP Pattern then please read the Getting Started, you may also find the Architecture useful particularly the description of Endpoint and URIs. Then you could try out some of the Examples first before trying this pattern out. http://git-wip-us.apache.org/repos/asf/camel/blob/927244de/camel-core/src/main/docs/eips/resequence-eip.adoc ---------------------------------------------------------------------- diff --git a/camel-core/src/main/docs/eips/resequence-eip.adoc b/camel-core/src/main/docs/eips/resequence-eip.adoc new file mode 100644 index 0000000..9ac0995 --- /dev/null +++ b/camel-core/src/main/docs/eips/resequence-eip.adoc @@ -0,0 +1,260 @@ +## Resequence EIP +The link:http://www.enterpriseintegrationpatterns.com/Resequencer.html[Resequencer] from the link:https://camel.apache.org/enterprise-integration-patterns.html[EIP patterns] allows you to reorganise messages based on some comparator. + +By default in Camel we use an Expression to create the comparator; so that you can compare by a message header or the body or a piece of a message etc. + +image:http://www.enterpriseintegrationpatterns.com/img/Resequencer.gif[image] + +### Options + +// eip options: START +The Resequence EIP supports 1 options which are listed below: + +{% raw %} +[width="100%",cols="3,1m,6",options="header"] +|======================================================================= +| Name | Java Type | Description +| resequencerConfig | ResequencerConfig | To configure the resequencer in using either batch or stream configuration. Will by default use batch configuration. +|======================================================================= +{% endraw %} +// eip options: END + + +[NOTE] +.Change in Camel 2.7 +==== +The `<batch-config>` and `<stream-config>` tags in XML DSL in the Resequencer EIP must now be configured in the top, and not in the bottom. +So if you use those, then move them up just below the `<resequence>` EIP starts in the XML. If you are using Camel older than 2.7, then those configs should be at the bottom. +==== + +Camel supports two resequencing algorithms: + +* *Batch resequencing* collects messages into a batch, sorts the messages and sends them to their output. +* *Stream resequencing* re-orders (continuous) message streams based on the detection of gaps between messages. + +By default the Resequencer does not support duplicate messages and will only keep the last message, in case a message arrives with the same message expression. However in the batch mode you can enable it to allow duplicates. + +### Batch Resequencing +The following example shows how to use the batch-processing resequencer so that messages are sorted in order of the *body()* expression. That is messages are collected into a batch (either by a maximum number of messages per batch or using a timeout) then they are sorted in order and then sent out to their output. + +#### Using the Fluent Builders +[source,java] +--------------------- +from("direct:start") + .resequence().body() + .to("mock:result"); +--------------------- + +This is equivalent to +[source,java] +--------------------- +from("direct:start") + .resequence(body()).batch() + .to("mock:result"); +--------------------- + +The batch-processing resequencer can be further configured via the `size()` and `timeout()` methods. +[source,java] +--------------------- +from("direct:start") + .resequence(body()).batch().size(300).timeout(4000L) + .to("mock:result") +--------------------- + +This sets the batch size to 300 and the batch timeout to 4000 ms (by default, the batch size is 100 and the timeout is 1000 ms). Alternatively, you can provide a configuration object. + +[source,java] +--------------------- +from("direct:start") + .resequence(body()).batch(new BatchResequencerConfig(300, 4000L)) + .to("mock:result") +--------------------- + +So the above example will reorder messages from endpoint *direct:a* in order of their bodies, to the endpoint *mock:result*. + +Typically you'd use a header rather than the body to order things; or maybe a part of the body. So you could replace this expression with + +[source,java] +--------------------- +resequencer(header("mySeqNo")) +--------------------- + +for example to reorder messages using a custom sequence number in the header `mySeqNo`. + +You can of course use many different Expression languages such as XPath, XQuery, SQL or various Scripting Languages. + +#### Using the Spring XML Extensions +[source,xml] +--------------------- +<camelContext id="camel" xmlns="http://camel.apache.org/schema/spring";> + <route> + <from uri="direct:start" /> + <resequence> + <simple>body</simple> + <to uri="mock:result" /> + <!-- + batch-config can be ommitted for default (batch) resequencer settings + --> + <batch-config batchSize="300" batchTimeout="4000" /> + </resequence> + </route> +</camelContext> +--------------------- + +### Allow Duplicates +*Available as of Camel 2.4* + +In the `batch` mode, you can now allow duplicates. In Java DSL there is a `allowDuplicates()` method and in Spring XML there is an `allowDuplicates=true` attribute on the `<batch-config/>` you can use to enable it. + +### Reverse +*Available as of Camel 2.4* + +In the `batch` mode, you can now reverse the expression ordering. By default the order is based on 0..9,A..Z, which would let messages with low numbers be ordered first, and thus also also outgoing first. In some cases you want to reverse order, which is now possible. + +In Java DSL there is a `reverse()` method and in Spring XML there is an `reverse=true` attribute on the `<batch-config/>` you can use to enable it. + +### Resequence JMS messages based on JMSPriority +*Available as of Camel 2.4* + +It's now much easier to use the Resequencer to resequence messages from JMS queues based on JMSPriority. For that to work you need to use the two new options `allowDuplicates` and `reverse`. + +[source,java] +--------------------- +from("jms:queue:foo") + // sort by JMSPriority by allowing duplicates (message can have same JMSPriority) + // and use reverse ordering so 9 is first output (most important), and 0 is last + // use batch mode and fire every 3th second + .resequence(header("JMSPriority")).batch().timeout(3000).allowDuplicates().reverse() + .to("mock:result"); +--------------------- + +Notice this is *only* possible in the `batch` mode of the Resequencer. + +### Ignore invalid exchanges +*Available as of Camel 2.9* + +The Resequencer EIP will from Camel 2.9 onwards throw a `CamelExchangeException` if the incoming Exchange is not valid for the resequencer - ie. the expression cannot be evaluated, such as a missing header. +You can use the option `ignoreInvalidExchanges` to ignore these exceptions which means the Resequencer will then skip the invalid Exchange. + +[source,java] +--------------------- +from("direct:start") + .resequence(header("seqno")).batch().timeout(1000) + // ignore invalid exchanges (they are discarded) + .ignoreInvalidExchanges() + .to("mock:result"); +--------------------- + +This option is available for both batch and stream resequencer. + +### Reject Old Exchanges +*Available as of Camel 2.11* + +This option can be used to prevent out of order messages from being sent regardless of the event that delivered messages downstream (capacity, timeout, etc). If enabled using `rejectOld()`, the Resequencer will throw a `MessageRejectedException` when an incoming Exchange is "older" (based on the Comparator) than the last delivered message. This provides an extra level of control with regards to delayed message ordering. + +[source,java] +--------------------- +from("direct:start") + .onException(MessageRejectedException.class).handled(true).to("mock:error").end() + .resequence(header("seqno")).stream().timeout(1000).rejectOld() + .to("mock:result"); +--------------------- + +This option is available for the stream resequencer only. + +### Stream Resequencing +The next example shows how to use the stream-processing resequencer. Messages are re-ordered based on their sequence numbers given by a seqnum header using gap detection and timeouts on the level of individual messages. + +#### Using the Fluent Builders + +[source,java] +--------------------- +from("direct:start").resequence(header("seqnum")).stream().to("mock:result"); +--------------------- + +The stream-processing resequencer can be further configured via the `capacity()` and `timeout()` methods. +[source,java] +--------------------- +from("direct:start") + .resequence(header("seqnum")).stream().capacity(5000).timeout(4000L) + .to("mock:result") +--------------------- + +This sets the resequencer's capacity to 5000 and the timeout to 4000 ms (by default, the capacity is 1000 and the timeout is 1000 ms). Alternatively, you can provide a configuration object. +[source,java] +--------------------- +from("direct:start") + .resequence(header("seqnum")).stream(new StreamResequencerConfig(5000, 4000L)) + .to("mock:result") +--------------------- + +The stream-processing resequencer algorithm is based on the detection of gaps in a message stream rather than on a fixed batch size. +Gap detection in combination with timeouts removes the constraint of having to know the number of messages of a sequence (i.e. the batch size) in advance. Messages must contain a unique sequence number for which a predecessor and a successor is known. For example a message with the sequence number 3 has a predecessor message with the sequence number 2 and a successor message with the sequence number 4. The message sequence 2,3,5 has a gap because the successor of 3 is missing. The resequencer therefore has to retain message 5 until message 4 arrives (or a timeout occurs). + +If the maximum time difference between messages (with successor/predecessor relationship with respect to the sequence number) in a message stream is known, then the resequencer's timeout parameter should be set to this value. In this case it is guaranteed that all messages of a stream are delivered in correct order to the next processor. The lower the timeout value is compared to the out-of-sequence time difference the higher is the probability for out-of-sequence messages delivered by this resequencer. Large timeout values should be supported by sufficiently high capacity values. The capacity parameter is used to prevent the resequencer from running out of memory. + +By default, the stream resequencer expects long sequence numbers but other sequence numbers types can be supported as well by providing a custom expression. + +[source,java] +--------------------- +public class MyFileNameExpression implements Expression { + + public String getFileName(Exchange exchange) { + return exchange.getIn().getBody(String.class); + } + + public Object evaluate(Exchange exchange) { + // parser the file name with YYYYMMDD-DNNN pattern + String fileName = getFileName(exchange); + String[] files = fileName.split("-D"); + Long answer = Long.parseLong(files[0]) * 1000 + Long.parseLong(files[1]); + return answer; + } + + + public <T> T evaluate(Exchange exchange, Class<T> type) { + Object result = evaluate(exchange); + return exchange.getContext().getTypeConverter().convertTo(type, result); + } + +} +from("direct:start").resequence(new MyFileNameExpression()).stream().timeout(100).to("mock:result"); +--------------------- + +or custom comparator via the comparator() method + +[source,java] +--------------------- +ExpressionResultComparator<Exchange> comparator = new MyComparator(); +from("direct:start") + .resequence(header("seqnum")).stream().comparator(comparator) + .to("mock:result"); +--------------------- + +or via a StreamResequencerConfig object. + +[source,java] +--------------------- +ExpressionResultComparator<Exchange> comparator = new MyComparator(); +StreamResequencerConfig config = new StreamResequencerConfig(100, 1000L, comparator); + +from("direct:start") + .resequence(header("seqnum")).stream(config) + .to("mock:result"); +--------------------- + +Using the Spring XML Extensions + +[source,xml] +--------------------- +<camelContext id="camel" xmlns="http://camel.apache.org/schema/spring";> + <route> + <from uri="direct:start"/> + <resequence> + <simple>in.header.seqnum</simple> + <to uri="mock:result" /> + <stream-config capacity="5000" timeout="4000"/> + </resequence> + </route> +</camelContext> +--------------------- + +### Further Examples +For further examples of this pattern in use you could look at the batch-processing resequencer junit test case and the stream-processing resequencer junit test case + +### Using This Pattern +If you would like to use this EIP Pattern then please read the Getting Started, you may also find the Architecture useful particularly the description of Endpoint and URIs. Then you could try out some of the Examples first before trying this pattern out. http://git-wip-us.apache.org/repos/asf/camel/blob/927244de/camel-core/src/main/docs/eips/routingSlip-eip.adoc ---------------------------------------------------------------------- diff --git a/camel-core/src/main/docs/eips/routingSlip-eip.adoc b/camel-core/src/main/docs/eips/routingSlip-eip.adoc new file mode 100644 index 0000000..842c00e --- /dev/null +++ b/camel-core/src/main/docs/eips/routingSlip-eip.adoc @@ -0,0 +1,114 @@ +## Routing Slip EIP +The Routing Slip from the link:https://camel.apache.org/enterprise-integration-patterns.html[EIP patterns] allows you to route a message consecutively through a series of processing steps where the sequence of steps is not known at design time and can vary for each message. + +image:http://www.enterpriseintegrationpatterns.com/img/RoutingTableSimple.gif[image] + + +### Options + +// eip options: START +The Routing Slip EIP supports 3 options which are listed below: + +{% raw %} +[width="100%",cols="3,1m,6",options="header"] +|======================================================================= +| Name | Java Type | Description +| uriDelimiter | String | Sets the uri delimiter to use +| ignoreInvalidEndpoints | Boolean | Ignore the invalidate endpoint exception when try to create a producer with that endpoint +| cacheSize | Integer | Sets the maximum size used by the org.apache.camel.impl.ProducerCache which is used to cache and reuse producers when using this recipient list when uris are reused. +|======================================================================= +{% endraw %} +// eip options: END + +### Example +The following route will take any messages sent to the Apache ActiveMQ queue SomeQueue and pass them into the Routing Slip pattern. + +[source,java] +--------------------- +from("activemq:SomeQueue") + .routingSlip("aRoutingSlipHeader"); +--------------------- + +Messages will be checked for the existence of the `aRoutingSlipHeader` header. +The value of this header should be a comma-delimited list of endpoint URIs you wish the message to be routed to. +The Message will be routed in a pipeline fashion, i.e., one after the other. From *Camel 2.5* the Routing Slip will set a property, `Exchange.SLIP_ENDPOINT`, on the Exchange which contains the current endpoint as it advanced though the slip. This allows you to _know_ how far we have processed in the slip. + +The Routing Slip will compute the slip *beforehand* which means, the slip is only computed once. If you need to compute the slip _on-the-fly_ then use the Dynamic Router pattern instead. + +### Configuration Options +Here we set the header name and the URI delimiter to something different. + +#### Using the Fluent Builders +[source,java] +--------------------- +from("direct:c").routingSlip(header("aRoutingSlipHeader"), "#"); +--------------------- + +#### Using the Spring XML Extensions + +[source,xml] +--------------------- +<camelContext id="buildRoutingSlip" xmlns="http://activemq.apache.org/camel/schema/spring";> + <route> + <from uri="direct:c"/> + <routingSlip uriDelimiter="#"> + <header>aRoutingSlipHeader</header> + </routingSlip> + </route> +</camelContext> +--------------------- + +### Ignore Invalid Endpoints +*Available as of Camel 2.3* + +The Routing Slip now supports ignoreInvalidEndpoints which the Recipient List also supports. You can use it to skip endpoints which are invalid. +[source,java] +--------------------- +from("direct:a") + .routingSlip("myHeader") + .ignoreInvalidEndpoints(); +--------------------- + +And in Spring XML its an attribute on the recipient list tag: + +[source,xml] +--------------------- +<route> + <from uri="direct:a"/> + <routingSlip ignoreInvalidEndpoints="true"/> + <header>myHeader</header> + </routingSlip> +</route> +--------------------- + +Then let's say the myHeader contains the following two endpoints direct:foo,xxx:bar. The first endpoint is valid and works. However the second endpoint is invalid and will just be ignored. Camel logs at INFO level, so you can see why the endpoint was invalid. + +### Expression Support +*Available as of Camel 2.4* + +The Routing Slip now supports to take the expression parameter as the Recipient List does. You can tell Camel the expression that you want to use to get the routing slip. + +[source,java] +--------------------- +from("direct:a") + .routingSlip(header("myHeader")) + .ignoreInvalidEndpoints(); +--------------------- + +And in Spring XML its an attribute on the recipient list tag. +[source,xml] +--------------------- +<route> + <from uri="direct:a"/> + <!--NOTE from Camel 2.4.0, you need to specify the expression element inside of the routingSlip element --> + <routingSlip ignoreInvalidEndpoints="true"> + <header>myHeader</header> + </routingSlip> +</route> +--------------------- + +### Further Examples +For further examples of this pattern in use you could look at the routing slip test cases. + +### Using This Pattern +If you would like to use this EIP Pattern then please read the Getting Started, you may also find the Architecture useful particularly the description of Endpoint and URIs. Then you could try out some of the Examples first before trying this pattern out. http://git-wip-us.apache.org/repos/asf/camel/blob/927244de/camel-core/src/main/docs/eips/sample-eip.adoc ---------------------------------------------------------------------- diff --git a/camel-core/src/main/docs/eips/sample-eip.adoc b/camel-core/src/main/docs/eips/sample-eip.adoc new file mode 100644 index 0000000..1a551eb --- /dev/null +++ b/camel-core/src/main/docs/eips/sample-eip.adoc @@ -0,0 +1,96 @@ +## Sample EIP +*Available as of Camel 2.1* + +A sampling throttler allows you to extract a sample of the exchanges from the traffic through a route. + +It is configured with a sampling period during which only a single exchange is allowed to pass through. All other exchanges will be stopped. +Will by default use a sample period of 1 seconds. + +### Options +// eip options: START +The Sample EIP supports 3 options which are listed below: + +{% raw %} +[width="100%",cols="3,1m,6",options="header"] +|======================================================================= +| Name | Java Type | Description +| samplePeriod | Long | Sets the sample period during which only a single Exchange will pass through. +| messageFrequency | Long | Sets the sample message count which only a single Exchange will pass through after this many received. +| units | TimeUnit | Sets the time units for the sample period defaulting to seconds. +|======================================================================= +{% endraw %} +// eip options: END + + +### Samples +You use this EIP with the `sample` DSL as show in these samples. + +#### Using the Fluent Builders +These samples also show how you can use the different syntax to configure the sampling period: + +[source,java] +--------------------- +from("direct:sample") + .sample() + .to("mock:result"); + +from("direct:sample-configured") + .sample(1, TimeUnit.SECONDS) + .to("mock:result"); + +from("direct:sample-configured-via-dsl") + .sample().samplePeriod(1).timeUnits(TimeUnit.SECONDS) + .to("mock:result"); + +from("direct:sample-messageFrequency") + .sample(10) + .to("mock:result"); + +from("direct:sample-messageFrequency-via-dsl") + .sample().sampleMessageFrequency(5) + .to("mock:result"); +--------------------- + +#### Using the Spring XML Extensions +And the same example in Spring XML is: + +[source,xml] +--------------------- +<route> + <from uri="direct:sample"/> + <sample samplePeriod="1" units="seconds"> + <to uri="mock:result"/> + </sample> +</route> +<route> + <from uri="direct:sample-messageFrequency"/> + <sample messageFrequency="10"> + <to uri="mock:result"/> + </sample> +</route> +<route> + <from uri="direct:sample-messageFrequency-via-dsl"/> + <sample messageFrequency="5"> + <to uri="mock:result"/> + </sample> +</route> +--------------------- + +And since it uses a default of 1 second you can omit this configuration in case you also want to use 1 second +[source,xml] +--------------------- +<route> + <from uri="direct:sample"/> + <!-- will by default use 1 second period --> + <sample> + <to uri="mock:result"/> + </sample> +</route> +--------------------- + +### Using This Pattern +If you would like to use this EIP Pattern then please read the Getting Started, you may also find the Architecture useful particularly the description of Endpoint and URIs. Then you could try out some of the Examples first before trying this pattern out. + +### See Also + +* link:./throttle-eip.adoc[Throttler] +* link:/aggregator-eip.adoc[Aggregator] http://git-wip-us.apache.org/repos/asf/camel/blob/927244de/camel-core/src/main/docs/eips/script-eip.adoc ---------------------------------------------------------------------- diff --git a/camel-core/src/main/docs/eips/script-eip.adoc b/camel-core/src/main/docs/eips/script-eip.adoc new file mode 100644 index 0000000..ab9ed90 --- /dev/null +++ b/camel-core/src/main/docs/eips/script-eip.adoc @@ -0,0 +1,84 @@ +## Script EIP +*Available as of Camel 2.16* + +Is used to execute a script which does not change the message (by default). +This is useful when you need to invoke some logic that are not in Java code such as JavaScript, +Groovy or any of the other Languages. The message body is not changed (by default) however the scripting +context has access to the current Exchange and can essentially change the message or headers directly. +But the return value from the script is discarded and not used. +If the return value should be used as a changed message body then use link:./message-translator.adoc[Message Translator] EIP instead. + +### Options + +// eip options: START +The Script EIP supports 0 options which are listed below: + +{% raw %} +[width="100%",cols="3,1m,6",options="header"] +|======================================================================= +| Name | Java Type | Description +|======================================================================= +{% endraw %} +// eip options: END + +### Using from Java DSL +The route below will read the file contents and validate them against a regular expression. + +[source,java] +--------------------- +from("file://inbox") + .script().groovy("// some groovy code goes here") + .to("bean:MyServiceBean.processLine"); +--------------------- + +### Using from Spring DSL +And from XML its easy as well + +[source,xml] +--------------------- +<route> + <from uri="file://inbox"/> + <script> + <groovy>// some groovy code goes here</groovy> + </script> + <beanRef ref="myServiceBean" method="processLine"/> +</route> + +<bean id="myServiceBean" class="com.mycompany.MyServiceBean"/> +--------------------- + +Mind that you can use CDATA in XML if the groovy scrip uses < > etc + +[source,xml] +--------------------- +<route> + <from uri="file://inbox"/> + <script> + <groovy><![CDATA[ some groovy script here that can be multiple lines and whatnot ]]></groovy> + </script> + <beanRef ref="myServiceBean" method="processLine"/> +</route> + +<bean id="myServiceBean" class="com.mycompany.MyServiceBean"/> +--------------------- + +### Using external script files +You can refer to external script files instead of inlining the script. For example to load a groovy script from the classpath you need to prefix the value with *resource:* as shown: + +[source,xml] +--------------------- +<route> + <from uri="file://inbox"/> + <script> + <groovy>resource:classpath:com/foo/myscript.groovy</groovy> + </script> + <beanRef ref="myServiceBean" method="processLine"/> +</route> + +<bean id="myServiceBean" class="com.mycompany.MyServiceBean"/> +--------------------- + +You can also refer to the script from the file system with file: instead of classpath: such as file:/var/myscript.groovy + +### Using This Pattern +If you would like to use this EIP Pattern then please read the Getting Started, you may also find the Architecture useful particularly the description of Endpoint and URIs. Then you could try out some of the Examples first before trying this pattern out. http://git-wip-us.apache.org/repos/asf/camel/blob/927244de/camel-core/src/main/docs/eips/sort-eip.adoc ---------------------------------------------------------------------- diff --git a/camel-core/src/main/docs/eips/sort-eip.adoc b/camel-core/src/main/docs/eips/sort-eip.adoc new file mode 100644 index 0000000..38b980b --- /dev/null +++ b/camel-core/src/main/docs/eips/sort-eip.adoc @@ -0,0 +1,98 @@ +## Sort EIP +Sort can be used to sort a message. Imagine you consume text files and before processing each file you want to be sure the content is sorted. + +Sort will by default sort the body using a default comparator that handles numeric values or uses the string representation. You can provide your own comparator, and even an expression to return the value to be sorted. Sort requires the value returned from the expression evaluation is convertible to `java.util.List` as this is required by the JDK sort operation. + +### Options + +// eip options: START +The Sort EIP supports 1 options which are listed below: + +{% raw %} +[width="100%",cols="3,1m,6",options="header"] +|======================================================================= +| Name | Java Type | Description +| comparatorRef | String | Sets a reference to lookup for the comparator to use for sorting +|======================================================================= +{% endraw %} +// eip options: END + + +### Using from Java DSL +In the route below it will read the file content and tokenize by line breaks so each line can be sorted. +[source,java] +--------------------- +from("file://inbox").sort(body().tokenize("\n")).to("bean:MyServiceBean.processLine"); +--------------------- + +You can pass in your own comparator as a 2nd argument: +[source,java] +--------------------- +from("file://inbox").sort(body().tokenize("\n"), new MyReverseComparator()).to("bean:MyServiceBean.processLine"); +--------------------- + +### Using from Spring DSL +In the route below it will read the file content and tokenize by line breaks so each line can be sorted. + +*Camel 2.7 or better* +[source,xml] +--------------------- +<route> + <from uri="file://inbox"/> + <sort> + <simple>body</simple> + </sort> + <beanRef ref="myServiceBean" method="processLine"/> +</route> +--------------------- + +*Camel 2.6 or older* +[source,xml] +--------------------- +<route> + <from uri="file://inbox"/> + <sort> + <expression> + <simple>body</simple> + </expression> + </sort> + <beanRef ref="myServiceBean" method="processLine"/> +</route> +--------------------- + +And to use our own comparator we can refer to it as a spring bean: + +*Camel 2.7 or better* +[source,xml] +--------------------- +<route> + <from uri="file://inbox"/> + <sort comparatorRef="myReverseComparator"> + <simple>body</simple> + </sort> + <beanRef ref="MyServiceBean" method="processLine"/> +</route> + +<bean id="myReverseComparator" class="com.mycompany.MyReverseComparator"/> +--------------------- + +*Camel 2.6 or older* +[source,xml] +--------------------- +<route> + <from uri="file://inbox"/> + <sort comparatorRef="myReverseComparator"> + <expression> + <simple>body</simple> + </expression> + </sort> + <beanRef ref="MyServiceBean" method="processLine"/> +</route> + +<bean id="myReverseComparator" class="com.mycompany.MyReverseComparator"/> +--------------------- + +Besides `<simple>`, you can supply an expression using any language you like, so long as it returns a list. + +### Using This Pattern +If you would like to use this EIP Pattern then please read the Getting Started, you may also find the Architecture useful particularly the description of Endpoint and URIs. Then you could try out some of the Examples first before trying this pattern out.
