davsclaus commented on code in PR #1295: URL: https://github.com/apache/camel-website/pull/1295#discussion_r1959925886
########## content/blog/2025/02/connecting-edi-to-the-enterprise-with-camel-and-smooks/index.md: ########## @@ -0,0 +1,452 @@ +--- +title: "Connecting EDI to the Enterprise with Camel & Smooks" +date: 2025-02-13 +draft: false +authors: [cjmamo] +categories: ["Usecases", "Howtos", "Camel", "JBang", "Transformation"] +preview: "Demonstrating EDI X12 integration with Camel and Smooks" +--- + +[Electronic Data Interchange (EDI)](https://en.wikipedia.org/wiki/Electronic_data_interchange) underpins the flow of information in numerous industries. From healthcare, retail, and aviation, to finance, manufacturing, and logistics, EDI is the workhorse carrying billions of transactions across applications in these industries. + +Historically viewed as a long, complex and costly journey, connecting EDI to the enterprise is traditionally thought to belong in the realm of expensive proprietary software or organisations with sizeable in-house IT teams. The goal of this blog post is to dispel this perception. I will illustrate how sustainable, mature open-source software can stand up the advanced infrastructure required to integrate EDI with an organisation’s downstream systems. Despite the complexities and nuances of EDI, the end result is a low-code flexible solution that can be run with a single keystroke thanks to [Camel JBang](/manual/camel-jbang.html). + +The open-source frameworks forming the backbone of this EDI integration are Camel and Smooks. [Smooks](https://www.smooks.org/) is the engine powering EDI implementations around the world. It is an extensible framework used to build event-driven data integration applications that target structured data (XML, [CSV](https://www.smooks.org/documentation/#csv), [EDI](https://www.smooks.org/documentation/#edi), [binary formats](https://www.smooks.org/documentation/#dfdl), etc…) with [XPath-like selectors](https://www.smooks.org/documentation/#selectors) to break it up into chunks. Smooks's ability to transform as well as slice and dice EDI documents together with [Camel’s rich palette of connectors](/components/next/index.html) means that we have a solid foundation to the integration described next. + +Depicted below is a common supply chain transaction flow where (1) a customer transmits an EDI purchase order to a supplier requesting goods or services followed by (2) the supplier acknowledging the transaction. Between receiving the purchase order and acknowledging it, the supplier exchanges bits and pieces of the order with its internal systems. + + + +The Camel application I will showcase exchanges [X12](https://x12.org/) documents, a flavour of EDI, to implement the supplier-side flow. While Smooks ships with out-of-the-box support for [EDIFACT](https://www.smooks.org/documentation/#edifact), an EDI dialect popular within Europe, X12 is broadly adopted in North America. It is considered one of the more challenging standards to follow, in small part due to its licensed approach in acquiring the guides governing the data exchange. Later on, we will see how one can parse the X12 purchase orders and generate acknowledgements without these implementation guides in hand. + +I will break down the above integration into stages in order to comprehend the overall flow (for those hungry to see the solution in action, you can skip the in-depth explanation and follow the [repo README instructions](https://github.com/apache/camel-jbang-examples/blob/main/edi-x12-as2/README.adoc) to run and test the JBang app): + +1. The customer's system transmits an EDI X12 purchase order in an [AS2](/components/next/as2-component.html) message sent over HTTP (HTTP is used for illustration purposes: a production system would typically transmit the AS2 message over HTTPS). The EDI capturing the information in the purchase order would resemble: + + ```edi + ISA*00* *00* *ZZ*ACME *ZZ*MYCORP *071216*1406*U*00204*000000263*1*T*>~ + GS*IN*ACME*MYCORP*20071216*1406*000000001*X*004010~ + ST*850*0001~ + BEG*00*SA*XX-1234**20170301**NA~ + PER*BD*ED SMITH*TE*8001234567~ + TAX*53247765*SP*CA*********9~ + N1*BY*ABC AEROSPACE*9*1234567890101~ + N2*AIRCRAFT DIVISION~ + N3*2000 JET BLVD~ + N4*FIGHTER TOWN*CA*98898~ + PO1*1*25*EA*36*PE*MG*XYZ-1234~ + MEA*WT*WT*10*OZ~ + IT8*******B0~ + SCH*25*EA***106*20170615~ + CTT*1~ + AMT*TT*900~ + SE*15*0001~ + GE*1*000000001~ + IEA*1*000000263~ + ``` + + Transporting EDI with AS2, in contrast with transporting EDI with plain HTTPS or FTPS, is appealing for business-to-business exchanges because AS2 offers strong security guarantees such as non-repudiation of receipts. + +2. Focusing on the supplier side, the _Receive Purchase Order_ Camel route listening for AS2 messages accepts the message containing the purchase order. This route pushes the X12 document onto a [JMS](/components/next/jms-component.html) queue. Immediately after, the AS2 listener endpoint delivers a notification to the AS2 client (i.e., the customer) confirming receipt of the message. + +3. The _Ingest Purchase Order_ route picks up the X12 document from the queue and sends it to a [Smooks endpoint](/components/next/smooks-component.html) in order to ingest the EDI. The Smooks endpoint references the file `ingest-x12-config.xml` which configures Smooks to turn the EDI into a stream of events. Under the hood, Smooks is taking advantage of DFDL for parsing and writing the EDI. [DFDL (Data Format Description Language)](https://ogf.org/ogf/doku.php/standards/dfdl/dfdl) is an open standard modeling language for describing general text and binary data. Smooks uses DFDL to emit events from EDI. For instance, given the EDI sample shown earlier, Smooks leverages DFDL to emit, conceptually: + + ```xml + <interchange> + <segment> + <segmentId>ISA</segmentId> + <dataElement>00</dataElement> + <dataElement></dataElement> + <dataElement>00</dataElement> + <dataElement></dataElement> + <dataElement>ZZ</dataElement> + <dataElement>ACME</dataElement> + <dataElement>ZZ</dataElement> + <dataElement>MYCORP</dataElement> + <dataElement>071216</dataElement> + <dataElement>1406</dataElement> + <dataElement>U</dataElement> + <dataElement>00204</dataElement> + <dataElement>000000263</dataElement> + <dataElement>1</dataElement> + <dataElement>T</dataElement> + <dataElement>></dataElement> + </segment> + ... + <segment> + <segmentId>IEA</segmentId> + <dataElement>1</dataElement> + <dataElement>000000263</dataElement> + </segment> + </interchange> + ``` + + Each emitted **event** corresponds to a segment or data element within the EDI document. Within `ingest-x12-config.xml`, Smooks is configured to **bind** these events to POJOs using **visitors** which target the events with **selectors**. Different visitors then **route** the POJOs via Camel to (a) verify the document identifiers with a Trading Partner Management system, AND (b) send them to an ERP for purchase order fulfillment. + +4. The Smooks endpoint produces an XML representation of the EDI purchase order, even if a non-fatal error happens while binding the events or routing the beans. The _Ingest Purchase Order_ route goes on to post this XML to a tracking system for transaction visibility and monitoring purposes. + +5. To complete the circle, the _Generate Functional Ack_ route transmits an X12 acknowledgement of the transaction over AS2 to the customer. In particular, the route takes the POJOs, which were derserialised from EDI segments in the previous Smooks execution, and sends them to another Smooks endpoint. This endpoint calls Smooks with the `gen-x12-ack-config.xml` config to: + 1. Generate the acknowledgment event stream by materialising an XML template with [FreeMarker](https://freemarker.apache.org/) (a popular template engine), + 2. Record the acknowledgement via Camel, and + 3. Serialise the acknowledgement to EDI + +--- + +We will now dive into the individual YAML routes and Smooks XML configs orchestrating the B2B exchange. + +#### Receive Purchase Order Route + +The following route receives the purchase order from the customer: + +```yaml +- routeConfiguration: + id: as2Error + onException: + - onException: + handled: + constant: true + exception: + - java.lang.Exception + steps: + - script: + groovy: | + httpContext = exchangeProperties['CamelAs2.interchange'] + httpContext.setAttribute(org.apache.camel.component.as2.api.AS2Header.DISPOSITION_TYPE, + org.apache.camel.component.as2.api.entity.AS2DispositionType.FAILED) Review Comment: doh somehow I did not grep that file, thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@camel.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
