Added Dead Letter Channel docs to Gitbook
Project: http://git-wip-us.apache.org/repos/asf/camel/repo Commit: http://git-wip-us.apache.org/repos/asf/camel/commit/1489eb81 Tree: http://git-wip-us.apache.org/repos/asf/camel/tree/1489eb81 Diff: http://git-wip-us.apache.org/repos/asf/camel/diff/1489eb81 Branch: refs/heads/master Commit: 1489eb8124b48cc49df2b7aa4c3b84dc93c17f0c Parents: 91d0c70 Author: Andrea Cosentino <anco...@gmail.com> Authored: Wed Oct 26 13:29:05 2016 +0200 Committer: Andrea Cosentino <anco...@gmail.com> Committed: Wed Oct 26 13:29:05 2016 +0200 ---------------------------------------------------------------------- .../src/main/docs/dead-letter-channel.adoc | 507 +++++++++++++++++++ docs/user-manual/en/SUMMARY.md | 3 +- 2 files changed, 509 insertions(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/camel/blob/1489eb81/camel-core/src/main/docs/dead-letter-channel.adoc ---------------------------------------------------------------------- diff --git a/camel-core/src/main/docs/dead-letter-channel.adoc b/camel-core/src/main/docs/dead-letter-channel.adoc new file mode 100644 index 0000000..286a72c --- /dev/null +++ b/camel-core/src/main/docs/dead-letter-channel.adoc @@ -0,0 +1,507 @@ +[[DeadLetterChannel-DeadLetterChannel]] +Dead Letter Channel +~~~~~~~~~~~~~~~~~~~ + +Camel supports the +http://www.enterpriseintegrationpatterns.com/DeadLetterChannel.html[Dead +Letter Channel] from the link:enterprise-integration-patterns.html[EIP +patterns] using the +http://camel.apache.org/maven/current/camel-core/apidocs/org/apache/camel/processor/DeadLetterChannel.html[DeadLetterChannel] +processor which is an link:error-handler.html[Error Handler]. + +image:http://www.enterpriseintegrationpatterns.com/img/DeadLetterChannelSolution.gif[image] + +TIP:*Difference between Dead Letter Channel and Default Error +Handler* + +The Default Error Handler does very little: it ends the Exchange +immediately and propagates the thrown Exception back to the caller. + +The Dead Letter Channel lets you control behaviors including redelivery, +whether to propagate the thrown Exception to the caller (the *handled* +option), and where the (failed) Exchange should now be routed to. + +The Dead Letter Channel is also by default configured to not be verbose +in the logs, so when a message is handled and moved to the dead letter +endpoint, then there is nothing logged. If you want some level of +logging you can use the various options on the redelivery policy / dead +letter channel to configure this. For example if you want the message +history then set logExhaustedMessageHistory=true (and logHandled=true +for Camel 2.15.x or older). + +When the DeadLetterChannel moves a message to the dead letter endpoint, +any new Exception thrown is by default handled by the dead letter +channel as well. This ensures that the DeadLetterChannel will always +succeed. From *Camel 2.15* onwards this behavior can be changed by +setting the option deadLetterHandleNewException=false. Then if a new +Exception is thrown, then the dead letter channel will fail and +propagate back that new Exception (which is the behavior of the default +error handler). When a new Exception occurs then the dead letter channel +logs this at WARN level. This can be turned off by setting +logNewException=false. + +[[DeadLetterChannel-Redelivery]] +Redelivery +^^^^^^^^^^ + +It is common for a temporary outage or database deadlock to cause a +message to fail to process; but the chances are if its tried a few more +times with some time delay then it will complete fine. So we typically +wish to use some kind of redelivery policy to decide how many times to +try redeliver a message and how long to wait before redelivery attempts. + +The +http://camel.apache.org/maven/current/camel-core/apidocs/org/apache/camel/processor/RedeliveryPolicy.html[RedeliveryPolicy] +defines how the message is to be redelivered. You can customize things +like + +* how many times a message is attempted to be redelivered before it is +considered a failure and sent to the dead letter channel +* the initial redelivery timeout +* whether or not exponential backoff is used (i.e. the time between +retries increases using a backoff multiplier) +* whether to use collision avoidance to add some randomness to the +timings +* delay pattern (see below for details) +* *Camel 2.11:* whether to allow redelivery during stopping/shutdown + +Once all attempts at redelivering the message fails then the message is +forwarded to the dead letter queue. + +[[DeadLetterChannel-AboutmovingExchangetodeadletterqueueandusinghandled]] +About moving Exchange to dead letter queue and using handled +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +*Handled* on link:dead-letter-channel.html[Dead Letter Channel] + +When all attempts of redelivery have failed the +link:exchange.html[Exchange] is moved to the dead letter queue (the dead +letter endpoint). The exchange is then complete and from the client +point of view it was processed. As such the +link:dead-letter-channel.html[Dead Letter Channel] have handled the +link:exchange.html[Exchange]. + +For instance configuring the dead letter channel as: + +*Using the link:fluent-builders.html[Fluent Builders]* + +[source,java] +--------------------------------------------------- +errorHandler(deadLetterChannel("jms:queue:dead") + .maximumRedeliveries(3).redeliveryDelay(5000)); +--------------------------------------------------- + +*Using the link:spring-xml-extensions.html[Spring XML Extensions]* + +[source,xml] +---------------------------------------------------------------------------------------------- +<route errorHandlerRef="myDeadLetterErrorHandler"> + ... +</route> + +<bean id="myDeadLetterErrorHandler" class="org.apache.camel.builder.DeadLetterChannelBuilder"> + <property name="deadLetterUri" value="jms:queue:dead"/> + <property name="redeliveryPolicy" ref="myRedeliveryPolicyConfig"/> +</bean> + +<bean id="myRedeliveryPolicyConfig" class="org.apache.camel.processor.RedeliveryPolicy"> + <property name="maximumRedeliveries" value="3"/> + <property name="redeliveryDelay" value="5000"/> +</bean> +---------------------------------------------------------------------------------------------- + +The link:dead-letter-channel.html[Dead Letter Channel] above will clear +the caused exception (`setException(null)`), by moving the caused +exception to a property on the link:exchange.html[Exchange], with the +key `Exchange.EXCEPTION_CAUGHT`. Then the link:exchange.html[Exchange] +is moved to the `"jms:queue:dead"` destination and the client will not +notice the failure. + +[[DeadLetterChannel-AboutmovingExchangetodeadletterqueueandusingtheoriginalmessage]] +About moving Exchange to dead letter queue and using the original +message +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The option *useOriginalMessage* is used for routing the original input +message instead of the current message that potentially is modified +during routing. + +For instance if you have this route: + +[source,java] +--------------------------------- + from("jms:queue:order:input") + .to("bean:validateOrder") + .to("bean:transformOrder") + .to("bean:handleOrder"); +--------------------------------- + +The route listen for JMS messages and validates, transforms and handle +it. During this the link:exchange.html[Exchange] payload is +transformed/modified. So in case something goes wrong and we want to +move the message to another JMS destination, then we can configure our +link:dead-letter-channel.html[Dead Letter Channel] with the +*useOriginalMessage* option. But when we move the +link:exchange.html[Exchange] to this destination we do not know in which +state the message is in. Did the error happen in before the +transformOrder or after? So to be sure we want to move the original +input message we received from `jms:queue:order:input`. So we can do +this by enabling the *useOriginalMessage* option as shown below: + +[source,java] +------------------------------------------------------------------------- + // will use original body + errorHandler(deadLetterChannel("jms:queue:dead") + .useOriginalMessage().maximumRedeliveries(5).redeliverDelay(5000); +------------------------------------------------------------------------- + +Then the messages routed to the `jms:queue:dead` is the original input. +If we want to manually retry we can move the JMS message from the failed +to the input queue, with no problem as the message is the same as the +original we received. + +[[DeadLetterChannel-OnRedelivery]] +OnRedelivery +^^^^^^^^^^^^ + +When link:dead-letter-channel.html[Dead Letter Channel] is doing +redeliver its possible to configure a link:processor.html[Processor] +that is executed just *before* every redelivery attempt. This can be +used for the situations where you need to alter the message before its +redelivered. See below for sample. + +TIP:*onException and onRedeliver* +We also support for per link:exception-clause.html[*onException*] to set +a *onRedeliver*. That means you can do special on redelivery for +different exceptions, as opposed to onRedelivery set on +link:dead-letter-channel.html[Dead Letter Channel] can be viewed as a +global scope. + + +[[DeadLetterChannel-Redeliverydefaultvalues]] +Redelivery default values +^^^^^^^^^^^^^^^^^^^^^^^^^ + +Redelivery is disabled by default. + +The default redeliver policy will use the following values: + +* maximumRedeliveries=0 +* redeliverDelay=1000L (1 second) +* maximumRedeliveryDelay = 60 * 1000L (60 seconds) +* And the exponential backoff and collision avoidance is turned off. +* The retriesExhaustedLogLevel are set to LoggingLevel.ERROR +* The retryAttemptedLogLevel are set to LoggingLevel.DEBUG +* Stack traces is logged for exhausted messages from Camel 2.2 onwards. +* Handled exceptions is not logged from Camel 2.3 onwards +* logExhaustedMessageHistory is true for default error handler, and +false for dead letter channel. +* logExhaustedMessageBody *Camel 2.17:* is disabled by default to avoid +logging sensitive message body/header details. If this option is true, +then logExhaustedMessageHistory must also be true. + +The maximum redeliver delay ensures that a delay is never longer than +the value, default 1 minute. This can happen if you turn on the +exponential backoff. + +The maximum redeliveries is the number of *re* delivery attempts. By +default Camel will try to process the exchange 1 + 5 times. 1 time for +the normal attempt and then 5 attempts as redeliveries. + + Setting the maximumRedeliveries to a negative value such as -1 will +then always redelivery (unlimited). + + Setting the maximumRedeliveries to 0 will disable any re delivery +attempt. + +Camel will log delivery failures at the DEBUG logging level by default. +You can change this by specifying retriesExhaustedLogLevel and/or +retryAttemptedLogLevel. See +http://svn.apache.org/repos/asf/camel/trunk/camel-core/src/test/java/org/apache/camel/builder/ExceptionBuilderWithRetryLoggingLevelSetTest.java[ExceptionBuilderWithRetryLoggingLevelSetTest] +for an example. + +You can turn logging of stack traces on/off. If turned off Camel will +still log the redelivery attempt. Its just much less verbose. + +[[DeadLetterChannel-RedeliverDelayPattern]] +Redeliver Delay Pattern ++++++++++++++++++++++++ + +Delay pattern is used as a single option to set a range pattern for +delays. If used then the following options does not apply: (delay, +backOffMultiplier, useExponentialBackOff, useCollisionAvoidance, +maximumRedeliveryDelay). + +The idea is to set groups of ranges using the following syntax: +`limit:delay;limit 2:delay 2;limit 3:delay 3;...;limit N:delay N` + +Each group has two values separated with colon + +* limit = upper limit +* delay = delay in millis +And the groups is again separated with semi colon. +The rule of thumb is that the next groups should have a higher limit +than the previous group. + +Lets clarify this with an example: + +`delayPattern=5:1000;10:5000;20:20000` + +That gives us 3 groups: + +* 5:1000 +* 10:5000 +* 20:20000 + +Resulting in these delays for redelivery attempt: + +* Redelivery attempt number 1..4 = 0 millis (as the first group start +with 5) +* Redelivery attempt number 5..9 = 1000 millis (the first group) +* Redelivery attempt number 10..19 = 5000 millis (the second group) +* Redelivery attempt number 20.. = 20000 millis (the last group) + +Note: The first redelivery attempt is 1, so the first group should start +with 1 or higher. + +You can start a group with limit 1 to eg have a starting delay: +`delayPattern=1:1000;5:5000` + +* Redelivery attempt number 1..4 = 1000 millis (the first group) +* Redelivery attempt number 5.. = 5000 millis (the last group) + +There is no requirement that the next delay should be higher than the +previous. You can use any delay value you like. For example with +`delayPattern=1:5000;3:1000` we start with 5 sec delay and then later +reduce that to 1 second. + +[[DeadLetterChannel-Redeliveryheader]] +Redelivery header +^^^^^^^^^^^^^^^^^ + +When a message is redelivered the +http://camel.apache.org/maven/camel-core/apidocs/org/apache/camel/processor/DeadLetterChannel.html[DeadLetterChannel] +will append a customizable header to the message to indicate how many +times its been redelivered. +Before Camel 2.6: The header is *CamelRedeliveryCounter*, which is also +defined on the `Exchange.REDELIVERY_COUNTER`. +Starting with 2.6: The header *CamelRedeliveryMaxCounter*, which is +also defined on the `Exchange.REDELIVERY_MAX_COUNTER`, contains the +maximum redelivery setting. This header is absent if you use +`retryWhile` or have unlimited maximum redelivery configured. + +And a boolean flag whether it is being redelivered or not (first +attempt) +The header *CamelRedelivered* contains a boolean if the message is +redelivered or not, which is also defined on the `Exchange.REDELIVERED`. + +Dynamically calculated delay from the exchange +In Camel 2.9 and 2.8.2: The header is *CamelRedeliveryDelay*, which is +also defined on the `Exchange.REDELIVERY_DELAY`. +Is this header is absent, normal redelivery rules apply. + +[[DeadLetterChannel-Whichendpointfailed]] +Which endpoint failed ++++++++++++++++++++++ + +*Available as of Camel 2.1* + +When Camel routes messages it will decorate the +link:exchange.html[Exchange] with a property that contains the *last* +endpoint Camel send the link:exchange.html[Exchange] to: + +[source,java] +---------------------------------------------------------------------------------- +String lastEndpointUri = exchange.getProperty(Exchange.TO_ENDPOINT, String.class); +---------------------------------------------------------------------------------- + +The `Exchange.TO_ENDPOINT` have the constant value `CamelToEndpoint`. + +This information is updated when Camel sends a message to any endpoint. +So if it exists its the *last* endpoint which Camel send the Exchange +to. + +When for example processing the link:exchange.html[Exchange] at a given +link:endpoint.html[Endpoint] and the message is to be moved into the +dead letter queue, then Camel also decorates the Exchange with another +property that contains that *last* endpoint: + +[source,java] +----------------------------------------------------------------------------------------- +String failedEndpointUri = exchange.getProperty(Exchange.FAILURE_ENDPOINT, String.class); +----------------------------------------------------------------------------------------- + +The `Exchange.FAILURE_ENDPOINT` have the constant value +`CamelFailureEndpoint`. + +This allows for example you to fetch this information in your dead +letter queue and use that for error reporting. + + This is useable if the Camel route is a bit dynamic such as the dynamic +link:recipient-list.html[Recipient List] so you know which endpoints +failed. + +*Notice:* These information is kept on the Exchange even if the message +was successfully processed by a given endpoint, and then later fails for +example in a local link:bean.html[Bean] processing instead. So beware +that this is a hint that helps pinpoint errors. + +[source,java] +------------------------------------- +from("activemq:queue:foo") + .to("http://someserver/somepath";) + .beanRef("foo"); +------------------------------------- + +Now suppose the route above and a failure happens in the `foo` bean. +Then the `Exchange.TO_ENDPOINT` and `Exchange.FAILURE_ENDPOINT` will +still contain the value of `http://someserver/somepath`. + +[[DeadLetterChannel-OnPrepareFailure]] +OnPrepareFailure +^^^^^^^^^^^^^^^^ + +*Available as of Camel 2.16* + +Before the exchange is sent to the dead letter queue, you can use +onPrepare to allow a custom `Processor` to prepare the exchange, such as +adding information why the Exchange failed. For example the following +processor adds a header with the exception message + +[source,java] +----------------------------------------------------------------------------------------------- + public static class MyPrepareProcessor implements Processor { + @Override + public void process(Exchange exchange) throws Exception { + Exception cause = exchange.getProperty(Exchange.EXCEPTION_CAUGHT, Exception.class); + exchange.getIn().setHeader("FailedBecause", cause.getMessage()); + } + } +----------------------------------------------------------------------------------------------- + +Then configure the error handler to use the processor as follows: + +[source,java] +--------------------------------------------------------------------------------------- +errorHandler(deadLetterChannel("jms:dead").onPrepareFailure(new MyPrepareProcessor())); +--------------------------------------------------------------------------------------- + +Configuring this from XML DSL is as shown: + +[source,java] +-------------------------------------------------------------------------------------------------------------- + <bean id="myPrepare" + class="org.apache.camel.processor.DeadLetterChannelOnPrepareTest.MyPrepareProcessor"/> + + + <errorHandler id="dlc" type="DeadLetterChannel" deadLetterUri="jms:dead" onPrepareFailureRef="myPrepare"/> +-------------------------------------------------------------------------------------------------------------- + +The onPrepare is also available using the default error handler. + +[[DeadLetterChannel-Whichroutefailed]] +Which route failed +^^^^^^^^^^^^^^^^^^ + +*Available as of Camel 2.10.4/2.11* + +When Camel error handler handles an error such as +link:dead-letter-channel.html[Dead Letter Channel] or using +link:exception-clause.html[Exception Clause] with handled=true, then +Camel will decorate + + the link:exchange.html[Exchange] with the route id where the error +occurred. + +[source,java] +------------------------------------------------------------------------------------- +String failedRouteId = exchange.getProperty(Exchange.FAILURE_ROUTE_ID, String.class); +------------------------------------------------------------------------------------- + +The `Exchange.FAILURE_ROUTE_ID` have the constant value +`CamelFailureRouteId`. + +This allows for example you to fetch this information in your dead +letter queue and use that for error reporting. + +[[DeadLetterChannel-Controlifredeliveryisallowedduringstopping/shutdown]] +Control if redelivery is allowed during stopping/shutdown +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +*Available as of Camel 2.11* + +Prior to Camel 2.10, Camel will perform redelivery while stopping a +route, or shutting down Camel. This has improved a bit in Camel 2.10 +onwards, as Camel will not perform redelivery attempts when shutting +down aggressively (eg during link:graceful-shutdown.html[Graceful +Shutdown] and timeout hit). From Camel 2.11 onwards there is a new +option `allowRedeliveryWhileStopping` which you can use to control if +redelivery is allowed or not; notice that any in progress redelivery +will still be executed. This option can only disallow any redelivery to +be executed *after* the stopping of a route/shutdown of Camel has been +triggered. If a redelivery is dissallowed then a +`RejectedExcutionException` is set on the link:exchange.html[Exchange] +and the processing of the link:exchange.html[Exchange] stops. This means +any consumer will see the link:exchange.html[Exchange] as failed due the +`RejectedExecutionException`. + +The default value is `true` to be backwards compatible as before. For +example the following sample shows how to do this with Java DSL and XML +DSL + +And the sample sample with XML DSL + +[[DeadLetterChannel-Samples]] +Samples +^^^^^^^ + +The following example shows how to configure the Dead Letter Channel +configuration using the link:dsl.html[DSL] + +You can also configure the +http://camel.apache.org/maven/current/camel-core/apidocs/org/apache/camel/processor/RedeliveryPolicy.html[RedeliveryPolicy] +as this example shows + +[[DeadLetterChannel-HowcanImodifytheExchangebeforeredelivery?]] +How can I modify the Exchange before redelivery? +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +We support directly in link:dead-letter-channel.html[Dead Letter +Channel] to set a link:processor.html[Processor] that is executed +*before* each redelivery attempt. + +When link:dead-letter-channel.html[Dead Letter Channel] is doing +redeliver its possible to configure a link:processor.html[Processor] +that is executed just *before* every redelivery attempt. This can be +used for the situations where you need to alter the message before its +redelivered. + +Here we configure the link:dead-letter-channel.html[Dead Letter Channel] +to use our processor `MyRedeliveryProcessor` to be executed before each +redelivery. + +And this is the processor `MyRedeliveryProcessor` where we alter the +message. + +[[DeadLetterChannel-HowcanIlogwhatcausedtheDeadLetterChanneltobeinvoked?]] +How can I log what caused the Dead Letter Channel to be invoked? +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +You often need to know what went wrong that caused the Dead Letter +Channel to be used and it does not offer logging for this purpose. So +the Dead Letter Channel's endpoint can be set to a endpoint of our own +(such as `direct:deadLetterChannel`). We write a route to accept this +Exchange and log the Exception, then forward on to where we want the +failed Exchange moved to (which might be a DLQ queue for instance). See +also http://stackoverflow.com/questions/13711462/logging-camel-exceptions-and-sending-to-the-dead-letter-channel[http://stackoverflow.com/questions/13711462/logging-camel-exceptions-and-sending-to-the-dead-letter-channel] + +[[DeadLetterChannel-UsingThisPattern]] +Using This Pattern +++++++++++++++++++ + +If you would like to use this EIP Pattern then please read the +link:getting-started.html[Getting Started], you may also find the +link:architecture.html[Architecture] useful particularly the description +of link:endpoint.html[Endpoint] and link:uris.html[URIs]. Then you could +try out some of the link:examples.html[Examples] first before trying +this pattern out. + +* link:error-handler.html[Error Handler] +* link:exception-clause.html[Exception Clause] + http://git-wip-us.apache.org/repos/asf/camel/blob/1489eb81/docs/user-manual/en/SUMMARY.md ---------------------------------------------------------------------- diff --git a/docs/user-manual/en/SUMMARY.md b/docs/user-manual/en/SUMMARY.md index d04baee..a7d5631 100644 --- a/docs/user-manual/en/SUMMARY.md +++ b/docs/user-manual/en/SUMMARY.md @@ -83,7 +83,8 @@ * [Pipes and Filter](pipes-and-filters.adoc) * Messaging Channels * [Point to Point Channel](point-to-point-channel.adoc) - * [Publish Subscribe Channel](point-to-point-channel.adoc) + * [Publish Subscribe Channel](publish-subscribe-channel.adoc) + * [Dead Letter Channel](dead-letter-channel.adoc) * Message Construction * [Correlation Identifier](correlation-identifier.adoc) * [Event Message](event-message.adoc)
