FTP2Page edited by Claus IbsenFTP/SFTP Component - Camel 2.0 onwardsThis component provides access to remote file systems over the FTP and SFTP protocols.
URI format
ftp://[usern...@]hostname[:port]/directoryname[?options]
sftp://[usern...@]hostname[:port]/directoryname[?options]
Where directoryname represents the underlying directory. Can contain nested folders. If no username is provided, then anonymous login is attempted using no password. You can append query options to the URI in the following format, ?option=value&option=value&... URI OptionsThe options below are exclusive for the FTP2 component.
More URI options
Examplesftp://some...@someftpserver.com/public/upload/images/holiday2008?password=secret&binary=true
limitationsThe option readLock can be used to force Camel not to consume files that is currently in the progress of being written. However, this option is turned off by default, as it requires that the user has write access. There are other solutions to avoid consuming files that are currently being written over FTP; for instance, you can write to a temporary destination and move the file after it has been written. The ftp producer does not support appending to existing files. Any existing files on the remote server will be deleted before the file is written. Message HeadersThe following message headers can be used to affect the behavior of the component
Using Local Work DirectoryCamel supports consuming from remote FTP servers and downloading the files directly into a local work directory. This avoids reading the entire remote file content into memory as it is streamed directly into the local file using FileOutputStream. Camel will store to a local file with the same name as the remote file, though with .progress as extension while the file is being downloaded. Afterwards, the file is renamed to remove the .inprogress suffix. And finally, when the Exchange is complete the local file is deleted. So if you want to download files from a remote FTP server and store it as files then you need to route to a file endpoint such as:
from("ftp://some...@someserver.com?password=secret&localWorkDirectory=/tmp").to("file://inbox");
SamplesIn the sample below we set up Camel to download all the reports from the FTP server once every hour (60 min) as BINARY content and store it as files on the local file system.
protected RouteBuilder createRouteBuilder() throws Exception {
return new RouteBuilder() {
public void configure() throws Exception {
// we use a delay of 60 minutes (eg. once pr. hour we poll the FTP server
long delay = 60 * 60 * 1000L;
// from the given FTP server we poll (= download) all the files
// from the public/reports folder as BINARY types and store this as files
// in a local directory. Camel will use the filenames from the FTPServer
// notice that the FTPConsumer properties must be prefixed with "consumer." in the URL
// the delay parameter is from the FileConsumer component so we should use consumer.delay as
// the URI parameter name. The FTP Component is an extension of the File Component.
from("ftp://sc...@localhost/public/reports?password=tiger&binary=true&consumer.delay=" + delay).
to("file://target/test-reports");
}
};
}
And the route using Spring DSL:
<route>
<from uri="ftp://sc...@localhost/public/reports?password=tiger&binary=true&delay=60000"/>
<to uri="file://target/test-reports"/>
</route>
Consuming a remote FTP server triggered by a routeThe FTP consumer is built as a scheduled consumer to be used in the from route. However, if you want to start consuming from an FTP server triggered within a route, it's a bit cumbersome to do this in Camel 1.x as opposed to Camel 2.0 where its supported directly in the DSL. However it's possible to do so in Camel 1.x, as this code below demonstrates. Camel 1.xIn the sample we have a SEDA queue where a message arrives that holds a message containing a filename to poll from a remote FTP server. So we set up a basic FTP URL as:
// we use delay=5000 to use 5 sec delay between pools to avoid polling a second time before we stop the consumer
// this is because we only want to run a single poll and get the file
private String getFtpUrl() {
return "ftp://ad...@localhost:" + getPort() + "/getme?password=admin&binary=false&delay=5000";
}
And then we have the route where we use a Processor within the route, so we can use Java code. In this Java code, we create the ftp consumer that downloads the file we want. And after the download we can get the content of the file and put it in the original exchange that continues being routed. As this is based on an unit test it routes to a Mock endpoint.
from("seda:start").process(new Processor() {
public void process(final Exchange exchange) throws Exception {
// get the filename from our custome header we want to get from a remote server
String filename = exchange.getIn().getHeader("myfile", String.class);
// construct the total url for the ftp consumer
// add the fileName option with the file we want to consume
String url = "" + "&fileName=" + filename;
// create a ftp endpoint
Endpoint ftp = context.getEndpoint(url);
// create a polling consumer where we can poll the myfile from the ftp server
PollingConsumer consumer = ftp.createPollingConsumer();
// must start the consumer before we can receive
consumer.start();
// poll the file from the ftp server
Exchange result = consumer.receive();
// the result is the response from the FTP consumer (the downloaded file)
// replace the outher exchange with the content from the downloaded file
exchange.getIn().setBody(result.getIn().getBody());
// stop the consumer
consumer.stop();
}
}).to("mock:result");
Camel 2.0The same sample would be like this in Camel 2.0 where we can use the Content Enricher EIP with the pollEnrich DSL:
from("seda:start")
// set the filename in FILE_NAME header so Camel know the name of the remote file to poll
.setHeader(Exchange.FILE_NAME, header("myfile"))
.pollEnrich("ftp://ad...@localhost:21/getme?password=admin&binary=false")
.to("mock:result");
Filter using org.apache.camel.component.file.GenericFileFilterCamel supports pluggable filtering strategies. This strategy it to use the build in org.apache.camel.component.file.GenericFileFilter in Java. You can then configure the endpoint with such a filter to skip certain filters before being processed. In the sample we have build our own filter that only accepts files starting with report in the filename.
public class MyFileFilter implements GenericFileFilter {
public boolean accept(GenericFile file) {
// we only want report files
return file.getFileName().startsWith("report");
}
}
And then we can configure our route using the filter attribute to reference our filter (using # notation) that we have defines in the spring XML file:
<!-- define our sorter as a plain spring bean -->
<bean id="myFilter" class="com.mycompany.MyFileFilter"/>
<route>
<from uri="ftp://someu...@someftpserver.com?password=secret&filter=#myFilter"/>
<to uri="bean:processInbox"/>
</route>
Filtering using ANT path matcherThe ANT path matcher is a filter that is shipped out-of-the-box in the camel-spring jar. So you need to depend on camel-spring if you are using Maven. The file paths are matched with the following rules:
The sample below demonstrates how to use it:
<camelContext xmlns="http://camel.apache.org/schema/spring">
<template id="camelTemplate"/>
<!-- use myFilter as filter to allow setting ANT paths for which files to scan for -->
<endpoint id="myFTPEndpoint" uri="ftp://ad...@localhost:20123/antpath?password=admin&recursive=true&delay=10000&initialDelay=2000&filter=#myAntFilter"/>
<route>
<from ref="myFTPEndpoint"/>
<to uri="mock:result"/>
</route>
</camelContext>
<!-- we use the AntPathMatcherRemoteFileFilter to use ant paths for includes and exlucde -->
<bean id="myAntFilter" class="org.apache.camel.component.file.AntPathMatcherGenericFileFilter">
<!-- include and file in the subfolder that has day in the name -->
<property name="includes" value="**/subfolder/**/*day*"/>
<!-- exclude all files with bad in name or .xml files. Use comma to seperate multiple excludes -->
<property name="excludes" value="**/*bad*,**/*.xml"/>
</bean>
Debug loggingThis component has log level TRACE that can be helpful if you have problems. See Also
Change Notification Preferences
View Online
|
View Change
|
Add Comment
|
- [CONF] Apache Camel > FTP2 confluence
- [CONF] Apache Camel > FTP2 confluence
- [CONF] Apache Camel > FTP2 confluence
- [CONF] Apache Camel > FTP2 confluence
- [CONF] Apache Camel > FTP2 confluence
- [CONF] Apache Camel > FTP2 confluence
- [CONF] Apache Camel > FTP2 confluence
- [CONF] Apache Camel > FTP2 confluence
- [CONF] Apache Camel > FTP2 confluence
- [CONF] Apache Camel > FTP2 confluence
- [CONF] Apache Camel > FTP2 confluence
