FTP2Page edited by Richard Kettelerij
Comment:
corrected FTPS in SFTP after patch applied by Willem
Changes (2)
Full ContentFTP/SFTP/FTPS Component - Camel 2.0 onwardsThis component provides access to remote file systems over the FTP and SFTP protocols. Maven users will need to add the following dependency to their pom.xml for this component: <dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-ftp</artifactId> <version>x.x.x</version> <!-- use the same version as your Camel core version --> </dependency>
You may want to pick either one depending on your situation and security issues. Some Camel end users can only download files if they use stepwise, while others can only download if they do not. At least you have the choice to pick (from Camel 2.6 onwards). In Came 2.0 - 2.5 there is only one mode and it is:
From Camel 2.6 onwards there is now an option stepwise you can use to control the behavior. Note that stepwise changing of directory will in most cases only work when the user is confined to it's home directory and when the home directory is reported as "/". The difference between the two of them is best illustrated with an example. Suppose we have the following directory structure on the remote FTP server we need to traverse and download files: / /one /one/two /one/two/sub-a /one/two/sub-b And that we have a file in each of sub-a (a.txt) and sub-b (b.txt) folder. Using stepwise=true (default mode)TYPE A 200 Type set to A PWD 257 "/" is current directory. CWD one 250 CWD successful. "/one" is current directory. CWD two 250 CWD successful. "/one/two" is current directory. SYST 215 UNIX emulated by FileZilla PORT 127,0,0,1,17,94 200 Port command successful LIST 150 Opening data channel for directory list. 226 Transfer OK CWD sub-a 250 CWD successful. "/one/two/sub-a" is current directory. PORT 127,0,0,1,17,95 200 Port command successful LIST 150 Opening data channel for directory list. 226 Transfer OK CDUP 200 CDUP successful. "/one/two" is current directory. CWD sub-b 250 CWD successful. "/one/two/sub-b" is current directory. PORT 127,0,0,1,17,96 200 Port command successful LIST 150 Opening data channel for directory list. 226 Transfer OK CDUP 200 CDUP successful. "/one/two" is current directory. CWD / 250 CWD successful. "/" is current directory. PWD 257 "/" is current directory. CWD one 250 CWD successful. "/one" is current directory. CWD two 250 CWD successful. "/one/two" is current directory. PORT 127,0,0,1,17,97 200 Port command successful RETR foo.txt 150 Opening data channel for file transfer. 226 Transfer OK CWD / 250 CWD successful. "/" is current directory. PWD 257 "/" is current directory. CWD one 250 CWD successful. "/one" is current directory. CWD two 250 CWD successful. "/one/two" is current directory. CWD sub-a 250 CWD successful. "/one/two/sub-a" is current directory. PORT 127,0,0,1,17,98 200 Port command successful RETR a.txt 150 Opening data channel for file transfer. 226 Transfer OK CWD / 250 CWD successful. "/" is current directory. PWD 257 "/" is current directory. CWD one 250 CWD successful. "/one" is current directory. CWD two 250 CWD successful. "/one/two" is current directory. CWD sub-b 250 CWD successful. "/one/two/sub-b" is current directory. PORT 127,0,0,1,17,99 200 Port command successful RETR b.txt 150 Opening data channel for file transfer. 226 Transfer OK CWD / 250 CWD successful. "/" is current directory. QUIT 221 Goodbye disconnected. As you can see when stepwise is enabled, it will traverse the directory structure using CD xxx. Using stepwise=false230 Logged on TYPE A 200 Type set to A SYST 215 UNIX emulated by FileZilla PORT 127,0,0,1,4,122 200 Port command successful LIST one/two 150 Opening data channel for directory list 226 Transfer OK PORT 127,0,0,1,4,123 200 Port command successful LIST one/two/sub-a 150 Opening data channel for directory list 226 Transfer OK PORT 127,0,0,1,4,124 200 Port command successful LIST one/two/sub-b 150 Opening data channel for directory list 226 Transfer OK PORT 127,0,0,1,4,125 200 Port command successful RETR one/two/foo.txt 150 Opening data channel for file transfer. 226 Transfer OK PORT 127,0,0,1,4,126 200 Port command successful RETR one/two/sub-a/a.txt 150 Opening data channel for file transfer. 226 Transfer OK PORT 127,0,0,1,4,127 200 Port command successful RETR one/two/sub-b/b.txt 150 Opening data channel for file transfer. 226 Transfer OK QUIT 221 Goodbye disconnected. As you can see when not using stepwise, there are no CD operation invoked at all. Examplesftp://some...@someftpserver.com/public/upload/images/holiday2008?password=secret&binary=true
Default when consuming filesThe FTP consumer will by default leave the consumed files untouched on the remote FTP server. You have to configure it explicit if you want it to delete the files or move them to another location. For example you can use delete=true to delete the files, or use move=.done to move the files into a hidden done sub directory. The regular File consumer is different as it will by default move files to a .camel sub directory. The reason Camel does not do this by default for the FTP consumer is that it may lack permissions by default to be able to move or delete files. limitationsThe option readLock can be used to force Camel not to consume files that is currently in the progress of being written. However, this option is turned off by default, as it requires that the user has write access. And there is only a few options supported for FTP. When moving files using move or preMove option the files are restricted to the FTP_ROOT folder. That prevents you from moving files outside the FTP area. If you want to move files to another area you can use soft links and move files into a soft linked folder. Message HeadersThe following message headers can be used to affect the behavior of the component
About timeoutsThe two set of libraries (see top) has different API for setting timeout. You can use the connectTimeout option for both of them to set a timeout in millis to establish a network connection. An individual soTimeout can also be set on the FTP/FTPS, which corresponds to using ftpClient.soTimeout. Notice SFTP will automatically use connectTimeout as its soTimeout. The timeout option only applies for FTP/FTSP as the data timeout, which corresponds to the ftpClient.dataTimeout value. All timeout values are in millis. Using Local Work DirectoryCamel supports consuming from remote FTP servers and downloading the files directly into a local work directory. This avoids reading the entire remote file content into memory as it is streamed directly into the local file using FileOutputStream. Camel will store to a local file with the same name as the remote file, though with .inprogress as extension while the file is being downloaded. Afterwards, the file is renamed to remove the .inprogress suffix. And finally, when the Exchange is complete the local file is deleted. So if you want to download files from a remote FTP server and store it as files then you need to route to a file endpoint such as:
from("ftp://some...@someserver.com?password=secret&localWorkDirectory=/tmp").to("file://inbox");
SamplesIn the sample below we set up Camel to download all the reports from the FTP server once every hour (60 min) as BINARY content and store it as files on the local file system. protected RouteBuilder createRouteBuilder() throws Exception { return new RouteBuilder() { public void configure() throws Exception { // we use a delay of 60 minutes (eg. once pr. hour we poll the FTP server long delay = 60 * 60 * 1000L; // from the given FTP server we poll (= download) all the files // from the public/reports folder as BINARY types and store this as files // in a local directory. Camel will use the filenames from the FTPServer // notice that the FTPConsumer properties must be prefixed with "consumer." in the URL // the delay parameter is from the FileConsumer component so we should use consumer.delay as // the URI parameter name. The FTP Component is an extension of the File Component. from("ftp://tiger:scott@localhost/public/reports?binary=true&consumer.delay=" + delay). to("file://target/test-reports"); } }; } And the route using Spring DSL: <route> <from uri="ftp://scott@localhost/public/reports?password=tiger&binary=true&delay=60000"/> <to uri="file://target/test-reports"/> </route> Consuming a remote FTPS server (implicit SSL) and client authentication
from("ftps://admin@localhost:2222/public/camel?password=admin&securityProtocol=SSL&isImplicit=true
&ftpClient.keyStore.file=./src/test/resources/server.jks
&ftpClient.keyStore.password=password&ftpClient.keyStore.keyPassword=password")
.to("bean:foo");
Consuming a remote FTPS server (explicit TLS) and a custom trust store configurationfrom("ftps://admin@localhost:2222/public/camel?password=admin&ftpClient.trustStore.file=./src/test/resources/server.jks&ftpClient.trustStore.password=password") .to("bean:foo"); Filter using org.apache.camel.component.file.GenericFileFilterCamel supports pluggable filtering strategies. This strategy it to use the build in org.apache.camel.component.file.GenericFileFilter in Java. You can then configure the endpoint with such a filter to skip certain filters before being processed. In the sample we have build our own filter that only accepts files starting with report in the filename. public class MyFileFilter implements GenericFileFilter { public boolean accept(GenericFile file) { // we only want report files return file.getFileName().startsWith("report"); } } And then we can configure our route using the filter attribute to reference our filter (using # notation) that we have defines in the spring XML file: <!-- define our sorter as a plain spring bean --> <bean id="myFilter" class="com.mycompany.MyFileFilter"/> <route> <from uri="ftp://someu...@someftpserver.com?password=secret&filter=#myFilter"/> <to uri="bean:processInbox"/> </route> Filtering using ANT path matcherThe ANT path matcher is a filter that is shipped out-of-the-box in the camel-spring jar. So you need to depend on camel-spring if you are using Maven. The file paths are matched with the following rules:
The sample below demonstrates how to use it: <camelContext xmlns="http://camel.apache.org/schema/spring"> <template id="camelTemplate"/> <!-- use myFilter as filter to allow setting ANT paths for which files to scan for --> <endpoint id="myFTPEndpoint" uri="ftp://admin@localhost:20123/antpath?password=admin&recursive=true&delay=10000&initialDelay=2000&filter=#myAntFilter"/> <route> <from ref="myFTPEndpoint"/> <to uri="mock:result"/> </route> </camelContext> <!-- we use the AntPathMatcherRemoteFileFilter to use ant paths for includes and exlucde --> <bean id="myAntFilter" class="org.apache.camel.component.file.AntPathMatcherGenericFileFilter"> <!-- include and file in the subfolder that has day in the name --> <property name="includes" value="**/subfolder/**/*day*"/> <!-- exclude all files with bad in name or .xml files. Use comma to seperate multiple excludes --> <property name="excludes" value="**/*bad*,**/*.xml"/> </bean> Debug loggingThis component has log level TRACE that can be helpful if you have problems. See Also
Change Notification Preferences
View Online
|
View Changes
|
Add Comment
|
- [CONF] Apache Camel > FTP2 confluence
- [CONF] Apache Camel > FTP2 confluence
- [CONF] Apache Camel > FTP2 confluence
- [CONF] Apache Camel > FTP2 confluence
- [CONF] Apache Camel > FTP2 confluence
- [CONF] Apache Camel > FTP2 confluence
- [CONF] Apache Camel > FTP2 confluence
- [CONF] Apache Camel > FTP2 confluence
- [CONF] Apache Camel > FTP2 confluence
- [CONF] Apache Camel > FTP2 confluence
- [CONF] Apache Camel > FTP2 confluence
