FTPPage edited by willem jiangFTP/SFTP Component - Camel 1.x onlyThis component provides access to remote file systems over the FTP and SFTP protocols. Maven users will need to add the following dependency to their pom.xml for this component:
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-cometd</artifactId>
<version>x.x.x</version>
<!-- use the same version as your Camel core version -->
</dependency>
{info:title=Using Camel 2.x}
If you are using Camel 2.x then see the FTP documentation at his [link|FTP2].
This documentation is only for Camel 1.x.
{info}
h3. URI format
[username@]hostname:port/filename?options" rel="nofollow">ftp://[username@]hostname:port/filename?options
Where *filename* represents the underlying file name or directory. Can contain nested folders.
The *username* is currently only possible to provide in the hostname parameter.
If no *username* is provided then {{anonymous}} login is attempted using no password.
If no *port* number is provided, Camel will provide default values according to the protocol (ftp = 21, sftp = 22).
You can append query options to the URI in the following format, {{?option=value&option=value&...}}
h3. URI Options
{div:class=confluenceTableSmall}
|| Name || Default Value || Description ||
| {{directory}} | {{true}} | Indicates whether or not the given file name should be interpreted by default as a directory or file (as it sometimes hard to be sure with some FTP servers). |
| {{password}} | {{null}} | Specifies the password to use to log in to the remote file system. |
| {{binary}} | {{false}} | Specifies the file transfer mode {{BINARY}} or {{ASCII}}. Default is {{ASCII}}. |
| {{ftpClientConfig}} | {{null}} | Camel 1.5: Reference to a bean in the registry as a {{[org.apache.commons.net.ftp.FTPClientConfig|http://commons.apache.org/net/apidocs/org/apache/commons/net/ftp/FTPClientConfig.html]}} class. Use this option if you need to configure the client according to the FTP Server date format, locale, timezone, platform etc. See the javadoc {{[FTPClientConfig|http://commons.apache.org/net/apidocs/org/apache/commons/net/ftp/FTPClientConfig.html]}} for more documentation.
| {{consumer.recursive}} | {{true}}/{{false}} | If a directory, will look for changes in files in all the sub-directories. Is {{true}} by default for Camel 1.4 or older. Will change to {{false}} by default value as of Camel 1.5. |
| {{consumer.setNames}} | {{true}} | *...@deprecated* Used by FTPConsumer. If {{true}}, Camel will use the filename the file has on the FTP server. The filename is stored on the In message in the header, {{FileComponent.HEADER_FILE_NAME}}. *Note:* In Camel 1.4 the default value has changed to {{true}}. |
| {{consumer.delay}} | {{500}} | Delay in milliseconds between each poll. |
| {{consumer.initialDelay}} | {{1000}} | Milliseconds before polling starts. |
| {{consumer.userFixedDelay}} | {{false}} | Set to {{true}} to use fixed delay between polls, otherwise fixed rate is used. See [ScheduledExecutorService|http://java.sun.com/j2se/1.5.0/docs/api/java/util/concurrent/ScheduledExecutorService.html] in JDK for details. |
| {{consumer.regexPattern}} | {{null}} | Used by FTPConsumer. Regular _expression_ to use for matching files when consuming. |
| {{consumer.exclusiveReadLock}} | {{false}}| Camel 1.5: Used by FTPConsumer. If set to {{true}}, Camel will only poll the ftp files if it has exclusive read access to the file (that is, the file is not in the process of being written). Camel will wait until exclusive access is granted, testing once every second (where the test consists of Camel attempting to rename the file). If set to {{false}}, Camel will poll the file even if it is in the process of being written. |
| {{consumer.deleteFile}} | {{false}} | Camel 1.5: Used by FTPConsumer. Flag to set if the consumed file should be deleted after it has been downloaded. |
| {{consumer.moveNamePrefix}} | {{null}} | Camel 1.5: Used by FTPConsumer. The prefix string prepended to the filename when moving it. For example, to move processed files into the {{done}} directory, set this value to {{done/}} |
| {{consumer.moveNamePostfix}} | {{null}} | Camel 1.5: Used by FTPConsumer. The postfix string appended to the filename when moving it. For example to rename processed files from {{foo}} to {{foo.old}}, set this value to {{.old}}. |
| {{consumer.excludedNamePrefix}} | {{null}} | Camel 1.5: Used by FTPConsumer. Used to exclude files, if filename starts with the given prefix. |
| {{consumer.excludedNamePostfix}} | {{null}} | Camel 1.5: Used by FTPConsumer. Used to exclude files, if filename ends with the given postfix. |
| {{consumer.timestamp}} | {{false}} | Camel 1.5: *...@deprecated* will be removed in Camel 2.0. This option is only for backwards compatability. |
| {{_expression_}} | {{null}} | Camel 1.5: Use an _expression_ to dynamically set the filename. This allows you very easily to set dynamic pattern style filenames. If an _expression_ is set, it takes precedence over the {{org.apache.camel.file.name}} header. (Note: The header can itself also be an _expression_.) The {{_expression_}} option supports both {{String}} and {{_expression_}} types. If the _expression_ is a {{String}} type, it is *always* evaluated using the [File Language]. If the _expression_ is an {{_expression_}} type, then the specified {{_expression_}} type is used. This allows you, for instance, to use [OGNL] expressions. |
| {{passiveMode}} | {{false}} | Camel 1.6.0: Specifies whether to use a passive mode connections. Default is active mode ({{false}}). This feature is only for regular FTP, not SFTP. |
| {{knownHosts}} | {{null}} | Camel 1.6.0: Sets the {{known_hosts}} file, so that the SFTP endpoint can do host key verification. |
| {{privateKeyFile}} | {{null}} | Camel 1.6.0: Set the private key file to that the SFTP endpoint can do private key verification. |
| {{privateKeyFilePassphrase}} | {{null}} | Camel 1.6.0: Set the private key file passphrase to that the SFTP endpoint can do private key verification. |
{div}
h4. Examples
{{ftp://some...@someftpserver.com/public/upload/images/holiday2008?password=secret&binary=true}}
{{ftp://someonee...@someotherftpserver.co.uk:12049/reports/2008/budget.txt?password=secret&binary=false&directory=false}}
{{ftp://publicftpserver.com/download}}
{warning:title=Timestamp}
In Camel 1.4 or older the FTP consumer uses an internal timestamp for last polling. This timestamp is used to match for new remote files: if remote file modified timestamp > last poll timestamp => file can be consumed.
In Camel 1.5 this algorithm has been disabled by default, as it is not reliable over the FTP protocol. FTP Servers only return file modified timestamps using {{hh:mm}} (not seconds). And of course the clocks between the client and server can also be out of sync. Bottom line is that timestamp check for FTP protocol should *not* be used. That is why this feature is marked as *...@deprecated* and will be removed in Camel 2.0.
We encourage you to use a different strategy for matching new remote files: such as deleting or moving the file after download.
{warning}
{warning:title=FTP does not support concurrency for same endpoint}
In Camel 1.x both the FTP consumer and FTP producer (created from the same endpoint) does not support concurrency (the backing FTP client is not thread safe).
{warning}
{tip:title=More examples}
This component is an extension of the [File] component. So there could be more samples and details on the [File] component page as well.
{tip}
h3. New default behavior for FTP/SFTP-Consumers in Camel 1.5
The consumer will always skip any file which name starts with a dot, such as {{".", ".camel", ".m2" or ".groovy"}}. Only files (not directories) is matched for valid filename if options such as: {{consumer.regexPattern, consumer.excludeNamePrefix, consumer.excludeNamePostfix}} is used.
The consumer recursive option will be changed from *true* to *false* as the default value. We don't feel that Camel out-of-the-box should recursive poll.
The consumer will *not* use timestamp algorithm for determine if a remote file is a new file - see warning section above. To use the old behavior of Camel 1.4 or older you can use the option {{consumer.timestamp=true}}.
h3. Exclusive Read Lock
The option *readLock* can be used to force Camel *not* to consume files that is currently in the progress of being written. However this option is default turned off, as it requires that the user has write access. There are other solutions to avoid consuming files that are currently being written over FTP, for instance you can write the a temporary destination and move the file after it has been written.
h3. Message Headers
The following message headers can be used to affect the behavior of the component
|| Header || Description ||
| org.apache.camel.file.name | Specifies the output file name (relative to the endpoint directory) to be used for the output message when sending to the endpoint. If this is not present and no _expression_ either then a generated message Id is used as filename instead. |
| org.apache.camel.file.name.produced | New in Camel 1.5: The actual absolute filepath (path + name) for the output file that was written. This header is set by Camel and its purpose is providing end-users the name of the file that was written. |
| file.remote.host | The hostname of the remote server |
| file.remote.name | The name of the file consumed from the remote server |
| file.remote.fullName | The fullname of the file consumed from the remote server |
h3. Consumer properties
When using FTPConsumer (downloading files from a FTP Server) the consumer specific properties from the [File] component should be prefixed with "consumer.". For example the delay option from File Component should be specified as "consumer.delay=30000" in the URI. See the samples or some of the unit tests of this component.
h3. Filename _expression_
In Camel 1.5 we have support for setting the filename using an _expression_. This can be set either using the *_expression_* option or as a string based [File Language] _expression_ in the {{org.apache.camel.file.name}} header. See the [File Language] for some samples.
h3. Camel 1.x Known issues
See the timestamp warning.
When consuming files (downloading) you must use type conversation to either String or to InputStream for ASCII and BINARY file types.
In Camel 1.4 this is fixed, as there are build in type converters for the ASCII and BINARY file types, meaning that you do not need the convertBodyTo _expression_.
In Camel 1.4 or below Camel FTPConsumer will poll files regardless if the file is currently being written. See the *consumer.exclusiveReadLock* option.
Also in Camel 1.3 since setNames is default *false* then you must explicitly set the filename using the setHeader _expression_ when consuming from FTP directly to File.
The code below illustrates this:
private String ftpUrl = "ftp://camelri...@localhost:21/public/downloads?password=admin&binary=false"; return new RouteBuilder() { Unknown macro: {
from(ftpUrl).setHeader(FileComponent.HEADER_FILE_NAME, constant("downloaded.txt")).convertBodyTo(String.class).to(fileUrl);
} };
Or you can set the option to *true* as illustrated below:
private String ftpUrl = "ftp://camelri...@localhost:21/public/downloads?password=admin&binary=false&consumer.setNames=true"; return new RouteBuilder() { Unknown macro: {
from(ftpUrl).convertBodyTo(String.class).to(fileUrl);
} };
h3. Sample
In the sample below we setup Camel to download all the reports from the FTP server once every hour (60 min) as BINARY content and store it as files on the local file system.
{snippet:id=e1|lang=java|url=""
And the route using Spring DSL:
{code:xml}
<route>
<from uri="ftp://sc...@localhost/public/reports?password=tiger&binary=true&consumer.delay=60000"/>
<to uri="file://target/test-reports"/>
</route>
Using _expression_ for filenamesIn this sample we want to move consumed files to a backup folder using today's date as a sub foldername. Notice that the move happens on the remote FTP server. If you want to store the downloaded file on your local disk then route it to the File component as the sample above illustrates. from(ftpUrl + "&_expression_=backup/${date:now:yyyyMMdd}/${file:name}").to("..."); See File Language for more samples. Consuming a remote FTP server triggered by a routeThe FTP consumer is built as a scheduled consumer to be used in the from route. However if you want to start consuming from a FTP server triggered within a route it's a bit cumbersome to do this in Camel 1.x (we plan to improve this in Camel 2.x). However it's possible as this code below demonstrates. In the sample we have a SEDA queue where a message arrives that holds a message containing a filename to poll from a remote FTP server. So we setup a basic FTP url as: // we use directory=false to indicate we only want to consume a single file // we use delay=5000 to use 5 sec delay between pools to avoid polling a second time before we stop the consumer // this is because we only want to run a single poll and get the file // file=getme/ is the path to the folder where the file is private String getUrl = "ftp://ad...@localhost:" + port + "?password=admin&binary=false&directory=false&consumer.delay=5000&file=getme/"; And then we have the route where we use Processor within the route so we can use Java code. In this Java code we create the ftp consumer that downloads the file we want. And after the download we can get the content of the file and put it in the original exchange that continues being routed. As this is based on an unit test it routes to a Mock endpoint. from("seda:start").process(new Processor() { public void process(final Exchange exchange) throws Exception { // get the filename from our custome header we want to get from a remote server String filename = exchange.getIn().getHeader("myfile", String.class); // construct the total url for the ftp consumer String url = "" + filename; // create a ftp endpoint Endpoint ftp = context.getEndpoint(url); // create a polling consumer so we can poll the remote ftp file PollingConsumer consumer = ftp.createPollingConsumer(); consumer.start(); // receive the remote ftp without timeout Exchange result = consumer.receive(); // we must stop the consumer consumer.stop(); // the result is the response from the FTP consumer (the downloaded file) // replace the outher exchange with the content from the downloaded file exchange.getIn().setBody(result.getIn().getBody()); } }).to("mock:result"); Debug loggingThis component has log level TRACE that can be helpful if you have problems. See Also
Change Notification Preferences
View Online
|
View Change
|
Add Comment
|
- [CONF] Apache Camel > FTP confluence
- [CONF] Apache Camel > FTP confluence
- [CONF] Apache Camel > FTP confluence
- [CONF] Apache Camel > FTP confluence
- [CONF] Apache Camel > FTP confluence
- [CONF] Apache Camel > FTP confluence
