This is an automated email from the ASF dual-hosted git repository.
Croway pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel.git
The following commit(s) were added to refs/heads/main by this push:
new a91ba7e7c5c6 CAMEL-26928: camel-file - the consumer's move, preMove
and moveFailed honour jailStartingDirectory for symbolic links (#29855)
a91ba7e7c5c6 is described below
commit a91ba7e7c5c62a8e2839b15f0ba5b69c195b404b
Author: Claus Ibsen <[email protected]>
AuthorDate: Sun Nov 15 15:26:25 2026 +0100
CAMEL-26928: camel-file - the consumer's move, preMove and moveFailed
honour jailStartingDirectory for symbolic links (#29855)
* CAMEL-26928: camel-file - the consumer's move, preMove and moveFailed
honour jailStartingDirectory for symbolic links
With jailStartingDirectory enabled (the default), the target of a consumer
move (move, preMove, moveFailed and the default .camel folder) that is
inside the starting directory is now checked, with symbolic links resolved,
before the existing target is deleted, its parent directories are created
and the file is renamed, as the producer already does. A target that
resolves outside the starting directory fails the move like any other
failed move. Targets configured outside the starting directory keep working,
and jailStartingDirectory=false keeps the previous behaviour.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Claus Ibsen <[email protected]>
* CAMEL-26928: camel-file - upgrade notes for 4.22.3 and 4.18.6
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Claus Ibsen <[email protected]>
* CAMEL-26928: camel-file - split a long sentence in the symbolic links
section
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Claus Ibsen <[email protected]>
* CAMEL-26928: camel-file - skip a file whose move target resolves outside
the starting directory, and check every delete and rename onto a target
The consumer evaluates the preMove, move and moveFailed targets (and the
default .camel folder) in begin, before the file is processed. When one
resolves outside the starting directory through a symbolic link, the file
is skipped (WARN the first time), instead of being processed and failing
its move on every poll.
FileOperations checks the target in renameFile, deleteFile and before
buildDirectory creates directories, so the producer's moveExisting and
tempFileName renames are covered too.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Claus Ibsen <[email protected]>
* CAMEL-26928: camel-file - document that such files are skipped, and the
producer moveExisting and tempFileName checks
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Claus Ibsen <[email protected]>
* CAMEL-26928: camel-file - the move check costs little without a link,
runs before the read lock, and the docs recommend pointing the move at the
folder a link resolves to
Review of #29855 by Federico Mariani.
- GenericFileHelper.isWithinDirectoryResolvingLinks first looks at the
segments below the starting directory without following links. When
none is a link (and there is no . or .. segment) the path stays
within, and neither the path nor the starting directory is resolved
with toRealPath. The listing check of FileConsumer uses the same
shortcut. deleteFile checks only a file that exists.
- The default .camel target is computed without evaluating an
expression on a copy of the exchange (DefaultMoveRenamer).
- The rename and delete strategies check the move targets before the
read lock is acquired, so a skipped file does not wait for the lock on
every poll.
- The upgrade guides and the component doc recommend pointing move,
preMove or moveFailed at the folder a link resolves to, say that
jailStartingDirectory=false also lets the consumer read links to files
outside, and that the expressions are evaluated one extra time.
10000 empty files per round: merge-base 0.023 ms per file, 9e6be9db
0.046 ms, now 0.030 ms (Linux).
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Signed-off-by: Claus Ibsen <[email protected]>
---------
Signed-off-by: Claus Ibsen <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
---
.../apache/camel/catalog/docs/file-component.adoc | 24 +-
.../camel-file/src/main/docs/file-component.adoc | 24 +-
.../apache/camel/component/file/FileConsumer.java | 6 +-
.../camel/component/file/FileOperations.java | 61 +++-
.../camel/component/file/GenericFileHelper.java | 64 +++++
.../component/file/GenericFileOperations.java | 13 +
.../file/strategy/DefaultMoveRenamer.java | 44 +++
.../file/strategy/FileProcessStrategyFactory.java | 2 +-
.../strategy/GenericFileDeleteProcessStrategy.java | 6 +
.../GenericFileProcessStrategySupport.java | 93 ++++++
.../strategy/GenericFileRenameProcessStrategy.java | 6 +
.../component/file/GenericFileHelperTest.java | 46 +++
...nsumerMoveJailStartingDirectorySymlinkTest.java | 315 +++++++++++++++++++++
...oducerJailStartingDirectorySymlinkMoveTest.java | 163 +++++++++++
.../ROOT/pages/camel-4x-upgrade-guide-4_18.adoc | 25 ++
.../ROOT/pages/camel-4x-upgrade-guide-4_22.adoc | 25 ++
.../ROOT/pages/camel-4x-upgrade-guide-4_24.adoc | 25 ++
17 files changed, 924 insertions(+), 18 deletions(-)
diff --git
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/file-component.adoc
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/file-component.adoc
index f695acd43a25..0799687de173 100644
---
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/file-component.adoc
+++
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/file-component.adoc
@@ -719,9 +719,27 @@ done and ready to be consumed.
With `jailStartingDirectory` enabled (the default), symbolic links are
resolved when checking that a file stays within
the starting directory. The consumer skips a file, or with `recursive=true` a
directory, that is a link resolving
outside the starting directory (logged at WARN the first time), and the
producer fails to write through such a link.
-Links that resolve inside the starting directory, and a starting directory
that is itself a link, work as usual. Set
-`jailStartingDirectory=false` if the application relies on following links to
other directories. The check and the
-read or write that follows are not atomic, so the permissions of the
directories still matter.
+
+The consumer also checks where it would move a file: the targets of `preMove`,
`move` and `moveFailed`, including the
+default `.camel` folder. It checks them before the file is processed (and
before the read lock is acquired), so the
+expressions of these options are evaluated one extra time per file: for a file
processed without error, `move` is
+evaluated twice and `moveFailed` once. Keep this in mind for an expression
with side effects, such as a bean that hands
+out a sequence number. When a target is inside the starting directory but
resolves outside of it through a link, the
+consumer skips the file (logged at WARN the first time). The file is not
processed and stays in the folder. This applies to
+`moveFailed` too, even when the file would be processed without errors. A
target that depends on what the route sets,
+such as an exchange property, can only be checked when the file is moved. When
such a target resolves outside, the move
+fails like any other failed move, and the file is consumed again on the next
poll.
+
+The producer checks the files it deletes or renames onto in the same way: with
`fileExist=Move` and `moveExisting`, and
+when it renames a file written with `tempFileName`. Such a write fails.
+
+A target configured outside the starting directory (such as `move=../archive`)
is not checked. Links that resolve inside
+the starting directory, and a starting directory that is itself a link, work
as usual. When a folder such as `.camel`
+is linked to another volume on purpose, point the option at the folder the
link resolves to, such as
+`move=/mnt/volume2/${file:name}`. Setting `jailStartingDirectory=false` also
keeps following such links, but then the
+consumer also reads a file in the starting directory that is a link to a file
outside of it, and the producer writes
+through such links. The check and the read or write that follows are not
atomic, so the permissions of the directories
+still matter.
=== Done files
diff --git a/components/camel-file/src/main/docs/file-component.adoc
b/components/camel-file/src/main/docs/file-component.adoc
index f695acd43a25..0799687de173 100644
--- a/components/camel-file/src/main/docs/file-component.adoc
+++ b/components/camel-file/src/main/docs/file-component.adoc
@@ -719,9 +719,27 @@ done and ready to be consumed.
With `jailStartingDirectory` enabled (the default), symbolic links are
resolved when checking that a file stays within
the starting directory. The consumer skips a file, or with `recursive=true` a
directory, that is a link resolving
outside the starting directory (logged at WARN the first time), and the
producer fails to write through such a link.
-Links that resolve inside the starting directory, and a starting directory
that is itself a link, work as usual. Set
-`jailStartingDirectory=false` if the application relies on following links to
other directories. The check and the
-read or write that follows are not atomic, so the permissions of the
directories still matter.
+
+The consumer also checks where it would move a file: the targets of `preMove`,
`move` and `moveFailed`, including the
+default `.camel` folder. It checks them before the file is processed (and
before the read lock is acquired), so the
+expressions of these options are evaluated one extra time per file: for a file
processed without error, `move` is
+evaluated twice and `moveFailed` once. Keep this in mind for an expression
with side effects, such as a bean that hands
+out a sequence number. When a target is inside the starting directory but
resolves outside of it through a link, the
+consumer skips the file (logged at WARN the first time). The file is not
processed and stays in the folder. This applies to
+`moveFailed` too, even when the file would be processed without errors. A
target that depends on what the route sets,
+such as an exchange property, can only be checked when the file is moved. When
such a target resolves outside, the move
+fails like any other failed move, and the file is consumed again on the next
poll.
+
+The producer checks the files it deletes or renames onto in the same way: with
`fileExist=Move` and `moveExisting`, and
+when it renames a file written with `tempFileName`. Such a write fails.
+
+A target configured outside the starting directory (such as `move=../archive`)
is not checked. Links that resolve inside
+the starting directory, and a starting directory that is itself a link, work
as usual. When a folder such as `.camel`
+is linked to another volume on purpose, point the option at the folder the
link resolves to, such as
+`move=/mnt/volume2/${file:name}`. Setting `jailStartingDirectory=false` also
keeps following such links, but then the
+consumer also reads a file in the starting directory that is a link to a file
outside of it, and the producer writes
+through such links. The check and the read or write that follows are not
atomic, so the permissions of the directories
+still matter.
=== Done files
diff --git
a/components/camel-file/src/main/java/org/apache/camel/component/file/FileConsumer.java
b/components/camel-file/src/main/java/org/apache/camel/component/file/FileConsumer.java
index 594de4d99cb5..0c5a687328f9 100644
---
a/components/camel-file/src/main/java/org/apache/camel/component/file/FileConsumer.java
+++
b/components/camel-file/src/main/java/org/apache/camel/component/file/FileConsumer.java
@@ -86,7 +86,11 @@ public class FileConsumer extends GenericFileConsumer<File>
implements ResumeAwa
protected boolean isWithinStartingDirectory(String absoluteFilePath) {
// a local listing entry is a single path segment, but it can be a
symbolic link to a file or to a directory
// (entered with recursive) whose target lies outside the starting
directory, so compare the resolved paths
- // the starting directory is resolved once per poll, each listed file
on every poll (one toRealPath per file)
+ // a file that is not a link (nor below a link) needs no resolving;
otherwise the starting directory is resolved
+ // once per poll, and the file on every poll (one toRealPath per file)
+ if
(GenericFileHelper.isExistingBelowWithoutLinks(Path.of(absoluteFilePath),
getEndpoint().getFile().toPath())) {
+ return true;
+ }
try {
Path startingDirectory = resolvedStartingDirectory;
if (startingDirectory == null) {
diff --git
a/components/camel-file/src/main/java/org/apache/camel/component/file/FileOperations.java
b/components/camel-file/src/main/java/org/apache/camel/component/file/FileOperations.java
index 2d99e5f682db..e24b4d0ccbbc 100644
---
a/components/camel-file/src/main/java/org/apache/camel/component/file/FileOperations.java
+++
b/components/camel-file/src/main/java/org/apache/camel/component/file/FileOperations.java
@@ -88,6 +88,13 @@ public class FileOperations implements
GenericFileOperations<File> {
@Override
public boolean deleteFile(String name) throws
GenericFileOperationFailedException {
File file = new File(name);
+ if (!file.exists()) {
+ // nothing to delete, also not through a symbolic link, such as
the target of a move (renameFile checks it)
+ return false;
+ }
+ // such as an existing file in a folder the producer moves existing
files to (moveExisting), or the existing
+ // target of a consumer move, which must not be deleted through a
symbolic link resolving outside
+ jailToStartingDirectory(file, "Cannot delete file: ");
return FileUtil.deleteFile(file);
}
@@ -96,6 +103,9 @@ public class FileOperations implements
GenericFileOperations<File> {
boolean renamed;
File file = new File(from);
File target = new File(to);
+ // the consumer moves (move, preMove, moveFailed and the default
.camel folder), and the producer renames
+ // (tempFileName and moveExisting) a file onto the target, so check it
before anything is renamed or copied
+ jailToStartingDirectory(target, "Cannot rename file: " + from + " to:
");
try {
if (endpoint.isRenameUsingCopy()) {
renamed = FileUtil.renameFileUsingCopy(file, target);
@@ -213,6 +223,8 @@ public class FileOperations implements
GenericFileOperations<File> {
// the directory already exists
return true;
} else {
+ // the missing directories must not be created through a
symbolic link resolving outside
+ jailToStartingDirectory(path, "Cannot create directory: ");
LOG.trace("Building directory: {}", path);
return buildDirectory(path,
endpoint.getDirectoryPermissions(), absolute, endpoint.isAutoCreateStepwise());
}
@@ -270,25 +282,54 @@ public class FileOperations implements
GenericFileOperations<File> {
* before it creates missing parent directories of the target.
*/
void jailToStartingDirectory(File target) {
- if (!endpoint.isJailStartingDirectory()) {
- return;
- }
- Path startingDirectory = endpoint.getFile().toPath();
- if
(!target.toPath().toAbsolutePath().normalize().startsWith(startingDirectory.toAbsolutePath().normalize()))
{
- return;
- }
+ jailToStartingDirectory(target, "Cannot write file: ");
+ }
+
+ /**
+ * Checks a path before it is written, deleted, renamed onto or created as
a directory.
+ *
+ * @throws GenericFileOperationFailedException if the path is inside the
starting directory but resolves outside of
+ * it through a symbolic link
+ */
+ private void jailToStartingDirectory(File target, String failure) {
boolean within;
try {
- within =
GenericFileHelper.isWithinDirectoryResolvingLinks(target.toPath(),
startingDirectory);
+ within = isWithinStartingDirectoryResolvingLinks(target);
} catch (IOException e) {
throw new GenericFileOperationFailedException(
"Cannot verify file: " + target + " is within the starting
directory: " + endpoint.getFile(), e);
}
if (!within) {
throw new GenericFileOperationFailedException(
- "Cannot write file: " + target + " as it resolves outside
the starting directory: "
- +
endpoint.getFile());
+ failure + target + " as it resolves outside the starting
directory: " + endpoint.getFile());
+ }
+ }
+
+ /**
+ * Whether the path stays within the starting directory once symbolic
links are resolved. Always true when
+ * jailStartingDirectory is disabled, and for a path that is lexically
outside the starting directory (such as a
+ * move to ../archive the route author configured).
+ */
+ @Override
+ public boolean isWithinStartingDirectory(String name) {
+ try {
+ return isWithinStartingDirectoryResolvingLinks(new File(name));
+ } catch (IOException e) {
+ // such as a dangling symbolic link
+ LOG.debug("Cannot resolve file: {} against the starting directory
due to: {}", name, e.getMessage());
+ return false;
+ }
+ }
+
+ private boolean isWithinStartingDirectoryResolvingLinks(File target)
throws IOException {
+ if (endpoint == null || !endpoint.isJailStartingDirectory()) {
+ return true;
+ }
+ Path startingDirectory = endpoint.getFile().toPath();
+ if
(!target.toPath().toAbsolutePath().normalize().startsWith(startingDirectory.toAbsolutePath().normalize()))
{
+ return true;
}
+ return
GenericFileHelper.isWithinDirectoryResolvingLinks(target.toPath(),
startingDirectory);
}
@Override
diff --git
a/components/camel-file/src/main/java/org/apache/camel/component/file/GenericFileHelper.java
b/components/camel-file/src/main/java/org/apache/camel/component/file/GenericFileHelper.java
index 116c7b42b248..e644dffb93b0 100644
---
a/components/camel-file/src/main/java/org/apache/camel/component/file/GenericFileHelper.java
+++
b/components/camel-file/src/main/java/org/apache/camel/component/file/GenericFileHelper.java
@@ -24,7 +24,9 @@ import java.io.Reader;
import java.nio.charset.Charset;
import java.nio.file.Files;
import java.nio.file.LinkOption;
+import java.nio.file.NoSuchFileException;
import java.nio.file.Path;
+import java.nio.file.attribute.BasicFileAttributes;
import java.util.function.Supplier;
import org.apache.camel.Exchange;
@@ -93,9 +95,71 @@ public final class GenericFileHelper {
* @throws IOException if the path cannot be resolved, for example a
dangling symbolic link
*/
public static boolean isWithinDirectoryResolvingLinks(Path target, Path
directory) throws IOException {
+ if (isBelowWithoutLinks(target, directory)) {
+ return true;
+ }
return
resolveExistingPathSegments(target).startsWith(resolveExistingPathSegments(directory));
}
+ /**
+ * Whether the path is below the directory, segment by segment as written,
and no existing segment below the
+ * directory is a symbolic link (or another kind of link, such as a
junction on Windows). Such a path resolves
+ * within the directory, wherever the directory itself resolves to, so
neither needs a {@link Path#toRealPath}: this
+ * costs one lookup of the attributes per existing segment below the
directory. A path with a {@code .} or
+ * {@code ..} segment below the directory, or a segment whose attributes
cannot be read, needs the links resolved.
+ *
+ * @param target the local path (need not exist yet)
+ * @param directory the directory
+ * @return {@code true} if the path is known to stay within the
directory, {@code false} if the links must
+ * be resolved to tell
+ */
+ static boolean isBelowWithoutLinks(Path target, Path directory) {
+ return isBelowWithoutLinks(target, directory, false);
+ }
+
+ /**
+ * As {@link #isBelowWithoutLinks(Path, Path)}, for a path that is
expected to exist (such as a listed file), which
+ * saves a lookup of the last segment.
+ */
+ static boolean isExistingBelowWithoutLinks(Path target, Path directory) {
+ return isBelowWithoutLinks(target, directory, true);
+ }
+
+ private static boolean isBelowWithoutLinks(Path target, Path directory,
boolean expectExisting) {
+ final Path absoluteDirectory = directory.toAbsolutePath();
+ final Path absoluteTarget = target.toAbsolutePath();
+ if (!absoluteTarget.startsWith(absoluteDirectory)) {
+ return false;
+ }
+ final int count = absoluteTarget.getNameCount();
+ Path current = absoluteDirectory;
+ for (int i = absoluteDirectory.getNameCount(); i < count; i++) {
+ Path segment = absoluteTarget.getName(i);
+ String name = segment.toString();
+ if (".".equals(name) || "..".equals(name)) {
+ return false;
+ }
+ current = current.resolve(segment);
+ // the last segment of a move target usually does not exist yet,
and Files.exists tells without throwing
+ if (i == count - 1 && !expectExisting && !Files.exists(current,
LinkOption.NOFOLLOW_LINKS)) {
+ return true;
+ }
+ BasicFileAttributes attributes;
+ try {
+ attributes = Files.readAttributes(current,
BasicFileAttributes.class, LinkOption.NOFOLLOW_LINKS);
+ } catch (NoSuchFileException e) {
+ // this segment does not exist, nor do the ones below it, so
none of them is a link
+ return true;
+ } catch (IOException e) {
+ return false;
+ }
+ if (attributes.isSymbolicLink() || attributes.isOther()) {
+ return false;
+ }
+ }
+ return true;
+ }
+
/**
* Resolves the longest existing prefix of the path on the filesystem
(following symbolic links) and appends the
* remaining, not yet existing, segments.
diff --git
a/components/camel-file/src/main/java/org/apache/camel/component/file/GenericFileOperations.java
b/components/camel-file/src/main/java/org/apache/camel/component/file/GenericFileOperations.java
index 2e875d0f936a..23e892166f81 100644
---
a/components/camel-file/src/main/java/org/apache/camel/component/file/GenericFileOperations.java
+++
b/components/camel-file/src/main/java/org/apache/camel/component/file/GenericFileOperations.java
@@ -71,6 +71,19 @@ public interface GenericFileOperations<T> {
*/
boolean buildDirectory(String directory, boolean absolute) throws
GenericFileOperationFailedException;
+ /**
+ * Whether a path that a file is about to be renamed or moved onto, or
that is about to be deleted or created as a
+ * directory, stays within the starting directory of the endpoint.
Operations that refuse such a path in
+ * {@link #renameFile(String, String)}, {@link #deleteFile(String)} and
{@link #buildDirectory(String, boolean)}
+ * override this, so the consumer can skip a file whose move target would
be refused before the file is processed.
+ *
+ * @param name the path
+ * @return <tt>true</tt> if the path can be used, which is the default
+ */
+ default boolean isWithinStartingDirectory(String name) {
+ return true;
+ }
+
/**
* Retrieves the file
*
diff --git
a/components/camel-file/src/main/java/org/apache/camel/component/file/strategy/DefaultMoveRenamer.java
b/components/camel-file/src/main/java/org/apache/camel/component/file/strategy/DefaultMoveRenamer.java
new file mode 100644
index 000000000000..00392962cdd4
--- /dev/null
+++
b/components/camel-file/src/main/java/org/apache/camel/component/file/strategy/DefaultMoveRenamer.java
@@ -0,0 +1,44 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.component.file.strategy;
+
+import java.nio.file.Path;
+
+import org.apache.camel.Expression;
+import org.apache.camel.component.file.FileConstants;
+import org.apache.camel.component.file.GenericFile;
+
+/**
+ * The renamer of the default move, into the .camel folder next to the file.
The consumer checks where a file would be
+ * moved before it processes the file, and for this renamer it can tell
without evaluating the expression.
+ */
+final class DefaultMoveRenamer<T> extends GenericFileExpressionRenamer<T> {
+
+ DefaultMoveRenamer(Expression expression) {
+ super(expression);
+ }
+
+ /**
+ * The path the file is moved to: ${file:parent}/.camel/${file:onlyname}
+ */
+ static String target(GenericFile<?> file) {
+ Path path = Path.of(file.getAbsoluteFilePath());
+ Path folder = path.getParent() != null
+ ? path.getParent().resolve(FileConstants.DEFAULT_SUB_FOLDER) :
Path.of(FileConstants.DEFAULT_SUB_FOLDER);
+ return folder.resolve(file.getFileNameOnly()).toString();
+ }
+}
diff --git
a/components/camel-file/src/main/java/org/apache/camel/component/file/strategy/FileProcessStrategyFactory.java
b/components/camel-file/src/main/java/org/apache/camel/component/file/strategy/FileProcessStrategyFactory.java
index a1920ea58021..f54091ceb47c 100644
---
a/components/camel-file/src/main/java/org/apache/camel/component/file/strategy/FileProcessStrategyFactory.java
+++
b/components/camel-file/src/main/java/org/apache/camel/component/file/strategy/FileProcessStrategyFactory.java
@@ -116,7 +116,7 @@ public final class FileProcessStrategyFactory implements
GenericFileProcessStrat
Language language = context.resolveLanguage("file");
Expression expression
= language.createExpression("${file:parent}/" +
FileConstants.DEFAULT_SUB_FOLDER + "/${file:onlyname}");
- return new GenericFileExpressionRenamer<>(expression);
+ return new DefaultMoveRenamer<>(expression);
}
@SuppressWarnings("unchecked")
diff --git
a/components/camel-file/src/main/java/org/apache/camel/component/file/strategy/GenericFileDeleteProcessStrategy.java
b/components/camel-file/src/main/java/org/apache/camel/component/file/strategy/GenericFileDeleteProcessStrategy.java
index 55b1c0ebb774..adacecd98bc5 100644
---
a/components/camel-file/src/main/java/org/apache/camel/component/file/strategy/GenericFileDeleteProcessStrategy.java
+++
b/components/camel-file/src/main/java/org/apache/camel/component/file/strategy/GenericFileDeleteProcessStrategy.java
@@ -37,6 +37,12 @@ public class GenericFileDeleteProcessStrategy<T> extends
GenericFileProcessStrat
GenericFileOperations<T> operations, GenericFileEndpoint<T>
endpoint, Exchange exchange, GenericFile<T> file)
throws Exception {
+ // skip the file before it is processed if it cannot be moved where it
would be moved to, and before the read
+ // lock is acquired, so a skipped file does not wait for the read lock
on every poll
+ if (!acceptMoveTargets(operations, endpoint, exchange, file,
beginRenamer, null, failureRenamer)) {
+ return false;
+ }
+
// must invoke super
boolean result = super.begin(operations, endpoint, exchange, file);
if (!result) {
diff --git
a/components/camel-file/src/main/java/org/apache/camel/component/file/strategy/GenericFileProcessStrategySupport.java
b/components/camel-file/src/main/java/org/apache/camel/component/file/strategy/GenericFileProcessStrategySupport.java
index 4eeaaa2bb8ea..913abbf93abb 100644
---
a/components/camel-file/src/main/java/org/apache/camel/component/file/strategy/GenericFileProcessStrategySupport.java
+++
b/components/camel-file/src/main/java/org/apache/camel/component/file/strategy/GenericFileProcessStrategySupport.java
@@ -17,17 +17,21 @@
package org.apache.camel.component.file.strategy;
import java.io.File;
+import java.util.Map;
import org.apache.camel.CamelContext;
import org.apache.camel.CamelContextAware;
import org.apache.camel.Exchange;
import org.apache.camel.component.file.FileConstants;
+import org.apache.camel.component.file.FileEndpoint;
import org.apache.camel.component.file.GenericFile;
import org.apache.camel.component.file.GenericFileEndpoint;
import org.apache.camel.component.file.GenericFileExclusiveReadLockStrategy;
import org.apache.camel.component.file.GenericFileOperationFailedException;
import org.apache.camel.component.file.GenericFileOperations;
import org.apache.camel.component.file.GenericFileProcessStrategy;
+import org.apache.camel.support.ExchangeHelper;
+import org.apache.camel.support.LRUCacheFactory;
import org.apache.camel.support.service.ServiceHelper;
import org.apache.camel.support.service.ServiceSupport;
import org.apache.camel.util.FileUtil;
@@ -44,6 +48,8 @@ public abstract class GenericFileProcessStrategySupport<T>
extends ServiceSuppor
protected GenericFileExclusiveReadLockStrategy<T>
exclusiveReadLockStrategy;
protected CamelContext camelContext;
+ // files already reported as skipped as their move target resolves outside
the starting directory (bounded)
+ private final Map<String, Boolean>
skippedMoveTargetOutsideStartingDirectory = LRUCacheFactory.newLRUCache(1000);
@Override
public CamelContext getCamelContext() {
@@ -127,6 +133,9 @@ public abstract class GenericFileProcessStrategySupport<T>
extends ServiceSuppor
}
protected GenericFile<T> renameFile(GenericFileOperations<T> operations,
GenericFile<T> from, GenericFile<T> to) {
+ // the operations refuse a target that resolves outside the starting
directory (see
+ // GenericFileOperations.isWithinStartingDirectory) before anything is
deleted, created or renamed
+
// deleting any existing files before renaming
try {
operations.deleteFile(to.getAbsoluteFilePath());
@@ -151,6 +160,89 @@ public abstract class GenericFileProcessStrategySupport<T>
extends ServiceSuppor
return to;
}
+ /**
+ * Evaluates the targets the file is moved to before the file is
processed, and checks them with
+ * {@link GenericFileOperations#isWithinStartingDirectory(String)}: the
preMove target, then the move and moveFailed
+ * targets of the file where preMove puts it. A file with a target that is
refused is skipped, rather than processed
+ * and then failing its move, which would leave the file in place to be
consumed again on every poll. A target that
+ * cannot be evaluated yet is checked again when the file is moved. Only
done for the local file endpoint with
+ * jailStartingDirectory enabled.
+ *
+ * @return <tt>true</tt> to process the file, <tt>false</tt> to skip it
+ */
+ protected boolean acceptMoveTargets(
+ GenericFileOperations<T> operations, GenericFileEndpoint<T>
endpoint, Exchange exchange, GenericFile<T> file,
+ GenericFileRenamer<T> beginRenamer, GenericFileRenamer<T>
commitRenamer, GenericFileRenamer<T> failureRenamer) {
+ if (!endpoint.isJailStartingDirectory() || !(endpoint instanceof
FileEndpoint)) {
+ // only the local file operations resolve symbolic links, so do
not evaluate the targets for nothing
+ return true;
+ }
+ GenericFile<T> current = file;
+ if (beginRenamer != null) {
+ GenericFile<T> to = evaluateMoveTarget(operations, endpoint,
exchange, file, beginRenamer);
+ if (to == null) {
+ // the move and moveFailed targets depend on the preMove target
+ return true;
+ }
+ if (!acceptMoveTarget(operations, endpoint, file,
to.getAbsoluteFilePath())) {
+ return false;
+ }
+ current = to;
+ }
+ return evaluateAndAcceptMoveTarget(operations, endpoint, exchange,
file, current, commitRenamer)
+ && evaluateAndAcceptMoveTarget(operations, endpoint, exchange,
file, current, failureRenamer);
+ }
+
+ private boolean evaluateAndAcceptMoveTarget(
+ GenericFileOperations<T> operations, GenericFileEndpoint<T>
endpoint, Exchange exchange, GenericFile<T> file,
+ GenericFile<T> current, GenericFileRenamer<T> renamer) {
+ if (renamer == null) {
+ return true;
+ }
+ if (renamer instanceof DefaultMoveRenamer) {
+ // the default .camel folder next to the file has no expression to
evaluate
+ return acceptMoveTarget(operations, endpoint, file,
DefaultMoveRenamer.target(current));
+ }
+ GenericFile<T> to = evaluateMoveTarget(operations, endpoint, exchange,
current, renamer);
+ return to == null || acceptMoveTarget(operations, endpoint, file,
to.getAbsoluteFilePath());
+ }
+
+ private GenericFile<T> evaluateMoveTarget(
+ GenericFileOperations<T> operations, GenericFileEndpoint<T>
endpoint, Exchange exchange, GenericFile<T> file,
+ GenericFileRenamer<T> renamer) {
+ try {
+ // evaluate with a copy, the same way as when the file is moved
+ Exchange copy = ExchangeHelper.createCopy(exchange, true);
+ if (endpoint instanceof FileEndpoint fe) {
+ file.bindToExchange(copy, fe.isProbeContentType());
+ } else {
+ file.bindToExchange(copy);
+ }
+
copy.getMessage().setMessageId(exchange.getMessage().getMessageId());
+ copy.setExchangeId(exchange.getExchangeId());
+ return renamer.renameFile(operations, copy, file);
+ } catch (Exception e) {
+ LOG.debug("Cannot evaluate the move target of file: {} before it
is processed due to: {}", file, e.getMessage());
+ return null;
+ }
+ }
+
+ private boolean acceptMoveTarget(
+ GenericFileOperations<T> operations, GenericFileEndpoint<T>
endpoint, GenericFile<T> file, String target) {
+ if (operations.isWithinStartingDirectory(target)) {
+ return true;
+ }
+ String source = file.getAbsoluteFilePath();
+ String message = "Skipping file: {} as the target it would be moved
to: {} resolves outside the starting directory: {}";
+ // such a file stays where it is and is listed again on every poll, so
only warn the first time
+ if (skippedMoveTargetOutsideStartingDirectory.put(source,
Boolean.TRUE) == null) {
+ LOG.warn(message, source, target,
endpoint.getConfiguration().getDirectory());
+ } else {
+ LOG.debug(message, source, target,
endpoint.getConfiguration().getDirectory());
+ }
+ return false;
+ }
+
protected void deleteLocalWorkFile(Exchange exchange) {
// delete local work file, if it was used (eg by ftp component)
String local =
exchange.getIn().getHeader(FileConstants.FILE_LOCAL_WORK_PATH, String.class);
@@ -173,6 +265,7 @@ public abstract class GenericFileProcessStrategySupport<T>
extends ServiceSuppor
@Override
protected void doStop() throws Exception {
+ skippedMoveTargetOutsideStartingDirectory.clear();
ServiceHelper.stopService(exclusiveReadLockStrategy);
}
diff --git
a/components/camel-file/src/main/java/org/apache/camel/component/file/strategy/GenericFileRenameProcessStrategy.java
b/components/camel-file/src/main/java/org/apache/camel/component/file/strategy/GenericFileRenameProcessStrategy.java
index b15d271e35e6..a6636c1bd93e 100644
---
a/components/camel-file/src/main/java/org/apache/camel/component/file/strategy/GenericFileRenameProcessStrategy.java
+++
b/components/camel-file/src/main/java/org/apache/camel/component/file/strategy/GenericFileRenameProcessStrategy.java
@@ -35,6 +35,12 @@ public class GenericFileRenameProcessStrategy<T> extends
GenericFileProcessStrat
public boolean begin(
GenericFileOperations<T> operations, GenericFileEndpoint<T>
endpoint, Exchange exchange, GenericFile<T> file)
throws Exception {
+ // skip the file before it is processed if it cannot be moved where it
would be moved to, and before the read
+ // lock is acquired, so a skipped file does not wait for the read lock
on every poll
+ if (!acceptMoveTargets(operations, endpoint, exchange, file,
beginRenamer, commitRenamer, failureRenamer)) {
+ return false;
+ }
+
// must invoke super
boolean result = super.begin(operations, endpoint, exchange, file);
if (!result) {
diff --git
a/components/camel-file/src/test/java/org/apache/camel/component/file/GenericFileHelperTest.java
b/components/camel-file/src/test/java/org/apache/camel/component/file/GenericFileHelperTest.java
index 70ec1864acf7..dbe7f721895d 100644
---
a/components/camel-file/src/test/java/org/apache/camel/component/file/GenericFileHelperTest.java
+++
b/components/camel-file/src/test/java/org/apache/camel/component/file/GenericFileHelperTest.java
@@ -172,4 +172,50 @@ public class GenericFileHelperTest {
}
}
}
+
+ @Test
+ public void pathWithoutLinksIsBelowTheDirectoryWithoutResolving(@TempDir
Path tmp) throws IOException {
+ Path inbox = Files.createDirectories(tmp.resolve("inbox"));
+ Files.createFile(inbox.resolve("hello.txt"));
+ Files.createDirectories(inbox.resolve("sub"));
+
+ assertTrue(GenericFileHelper.isBelowWithoutLinks(inbox, inbox));
+
assertTrue(GenericFileHelper.isBelowWithoutLinks(inbox.resolve("hello.txt"),
inbox));
+
assertTrue(GenericFileHelper.isBelowWithoutLinks(inbox.resolve("sub/hello.txt"),
inbox));
+ // the default move folder that does not exist yet
+
assertTrue(GenericFileHelper.isBelowWithoutLinks(inbox.resolve(".camel/hello.txt"),
inbox));
+
+ // not below the directory as written, or with a . or .. segment: the
links must be resolved to tell
+
assertFalse(GenericFileHelper.isBelowWithoutLinks(tmp.resolve("outside/hello.txt"),
inbox));
+
assertFalse(GenericFileHelper.isBelowWithoutLinks(inbox.resolve("sub/../hello.txt"),
inbox));
+
assertFalse(GenericFileHelper.isBelowWithoutLinks(inbox.resolve("./hello.txt"),
inbox));
+ }
+
+ @Test
+ public void pathThroughLinkIsResolved(@TempDir Path tmp) throws
IOException {
+ Path inbox = Files.createDirectories(tmp.resolve("inbox"));
+ Path sub = Files.createDirectories(inbox.resolve("sub"));
+ Path outside = Files.createDirectories(tmp.resolve("outside"));
+ Path camel = inbox.resolve(".camel");
+ Path inner = inbox.resolve("inner");
+ Path startLink = tmp.resolve("start");
+ try {
+ Files.createSymbolicLink(camel, outside);
+ Files.createSymbolicLink(inner, sub);
+ Files.createSymbolicLink(startLink, inbox);
+ } catch (UnsupportedOperationException | IOException e) {
+ Assumptions.abort("Symbolic links are not supported on this
platform: " + e.getMessage());
+ }
+
+ // a link below the directory needs resolving: one resolving outside
is refused, one resolving inside is not
+
assertFalse(GenericFileHelper.isBelowWithoutLinks(camel.resolve("hello.txt"),
inbox));
+
assertFalse(GenericFileHelper.isWithinDirectoryResolvingLinks(camel.resolve("hello.txt"),
inbox));
+
assertFalse(GenericFileHelper.isBelowWithoutLinks(inner.resolve("hello.txt"),
inbox));
+
assertTrue(GenericFileHelper.isWithinDirectoryResolvingLinks(inner.resolve("hello.txt"),
inbox));
+
+ // a directory that is itself a link: the paths below it resolve
within it
+
assertTrue(GenericFileHelper.isBelowWithoutLinks(startLink.resolve("hello.txt"),
startLink));
+
assertTrue(GenericFileHelper.isWithinDirectoryResolvingLinks(startLink.resolve("hello.txt"),
startLink));
+
assertFalse(GenericFileHelper.isWithinDirectoryResolvingLinks(startLink.resolve(".camel/hello.txt"),
startLink));
+ }
}
diff --git
a/core/camel-core/src/test/java/org/apache/camel/component/file/FileConsumerMoveJailStartingDirectorySymlinkTest.java
b/core/camel-core/src/test/java/org/apache/camel/component/file/FileConsumerMoveJailStartingDirectorySymlinkTest.java
new file mode 100644
index 000000000000..2567c7eeb24e
--- /dev/null
+++
b/core/camel-core/src/test/java/org/apache/camel/component/file/FileConsumerMoveJailStartingDirectorySymlinkTest.java
@@ -0,0 +1,315 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.component.file;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.FileSystemException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.Queue;
+import java.util.concurrent.ConcurrentLinkedQueue;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import org.apache.camel.Consumer;
+import org.apache.camel.ContextTestSupport;
+import org.apache.camel.Endpoint;
+import org.apache.camel.Exchange;
+import org.apache.camel.builder.RouteBuilder;
+import
org.apache.camel.component.file.strategy.MarkerFileExclusiveReadLockStrategy;
+import org.apache.camel.component.mock.MockEndpoint;
+import org.apache.camel.spi.ExceptionHandler;
+import org.apache.camel.support.DefaultPollingConsumerPollStrategy;
+import org.junit.jupiter.api.Assumptions;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+import static org.awaitility.Awaitility.await;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+/**
+ * With jailStartingDirectory (the default) the file consumer must not move a
consumed file (move, preMove, moveFailed
+ * and the default .camel folder) through a symbolic link that lives inside
the starting directory and resolves outside
+ * of it. Such a file is skipped before it is processed, so it is not
processed again on every poll.
+ */
+class FileConsumerMoveJailStartingDirectorySymlinkTest extends
ContextTestSupport {
+
+ private static final String QUERY =
"?initialDelay=0&delay=100&pollStrategy=#countPolls"
+ +
"&exceptionHandler=#moveErrors&onCompletionExceptionHandler=#moveErrors";
+
+ private final Queue<Throwable> moveErrors = new ConcurrentLinkedQueue<>();
+ private final AtomicInteger polls = new AtomicInteger();
+ private Path inbox;
+ private Path outside;
+ private Path outsideFile;
+
+ @Override
+ public boolean isUseRouteBuilder() {
+ return false;
+ }
+
+ @BeforeEach
+ void createDirectories() throws IOException {
+ inbox = Files.createDirectories(testDirectory("inbox"));
+ outside = Files.createDirectories(testDirectory("outside"));
+ outsideFile = Files.writeString(outside.resolve("hello.txt"),
"Outside");
+ context.getRegistry().bind("moveErrors", new
CollectingExceptionHandler());
+ context.getRegistry().bind("countPolls", new
DefaultPollingConsumerPollStrategy() {
+ @Override
+ public void commit(Consumer consumer, Endpoint endpoint, int
polledMessages) {
+ polls.incrementAndGet();
+ }
+ });
+ }
+
+ @Test
+ void defaultMoveFolderLinkingOutsideStartingDirectoryIsNotUsed() throws
Exception {
+ Path file = Files.writeString(inbox.resolve("hello.txt"), "Hello
World");
+ createSymbolicLink(inbox.resolve(".camel"), outside);
+
+ consume(QUERY, false);
+
+ assertSkipped(file);
+ }
+
+ @Test
+ void skippedFileDoesNotTakeTheReadLock() throws Exception {
+ // a read lock such as changed waits before it is granted, so a
skipped file must not take it on every poll
+ AtomicInteger locks = new AtomicInteger();
+ context.getRegistry().bind("countLocks", new
MarkerFileExclusiveReadLockStrategy() {
+ @Override
+ public boolean acquireExclusiveReadLock(
+ GenericFileOperations<File> operations, GenericFile<File>
file, Exchange exchange)
+ throws Exception {
+ locks.incrementAndGet();
+ return super.acquireExclusiveReadLock(operations, file,
exchange);
+ }
+ });
+ Path file = Files.writeString(inbox.resolve("hello.txt"), "Hello
World");
+ createSymbolicLink(inbox.resolve(".camel"), outside);
+
+ consume(QUERY + "&exclusiveReadLockStrategy=#countLocks", false);
+
+ assertSkipped(file);
+ assertEquals(0, locks.get(), "the skipped file must not take the read
lock");
+ }
+
+ @Test
+ void moveFolderLinkingOutsideStartingDirectoryIsNotUsed() throws Exception
{
+ Path file = Files.writeString(inbox.resolve("hello.txt"), "Hello
World");
+ createSymbolicLink(inbox.resolve("done"), outside);
+
+ consume(QUERY + "&move=done", false);
+
+ assertSkipped(file);
+ }
+
+ @Test
+ void preMoveFolderLinkingOutsideStartingDirectoryIsNotUsed() throws
Exception {
+ Path file = Files.writeString(inbox.resolve("hello.txt"), "Hello
World");
+ createSymbolicLink(inbox.resolve("inprogress"), outside);
+
+ consume(QUERY + "&preMove=inprogress&noop=true", false);
+
+ assertSkipped(file);
+ }
+
+ @Test
+ void
moveFolderBelowPreMoveFolderLinkingOutsideStartingDirectoryIsNotUsed() throws
Exception {
+ // the default .camel folder is relative to where preMove puts the file
+ Path file = Files.writeString(inbox.resolve("hello.txt"), "Hello
World");
+ Files.createDirectories(inbox.resolve("inprogress"));
+ createSymbolicLink(inbox.resolve("inprogress/.camel"), outside);
+
+ consume(QUERY + "&preMove=inprogress", false);
+
+ assertSkipped(file);
+ }
+
+ @Test
+ void moveFailedFolderLinkingOutsideStartingDirectoryIsNotUsed() throws
Exception {
+ Path file = Files.writeString(inbox.resolve("hello.txt"), "Hello
World");
+ createSymbolicLink(inbox.resolve("failed"), outside);
+
+ consume(QUERY + "&moveFailed=failed", true);
+
+ assertSkipped(file);
+ }
+
+ @Test
+ void moveFailedFolderLinkingOutsideStartingDirectoryIsNotUsedWithDelete()
throws Exception {
+ Path file = Files.writeString(inbox.resolve("hello.txt"), "Hello
World");
+ createSymbolicLink(inbox.resolve("failed"), outside);
+
+ consume(QUERY + "&delete=true&moveFailed=failed", true);
+
+ assertSkipped(file);
+ }
+
+ @Test
+ void recursiveSubDirectoryMoveFolderLinkingOutsideIsNotUsed() throws
Exception {
+ Path sub = Files.createDirectories(inbox.resolve("sub"));
+ Path file = Files.writeString(sub.resolve("hello.txt"), "Hello World");
+ createSymbolicLink(sub.resolve(".camel"), outside);
+
+ consume(QUERY + "&recursive=true", false);
+
+ assertSkipped(file);
+ }
+
+ @Test
+ void renameUsingCopyMoveFolderLinkingOutsideIsNotUsed() throws Exception {
+ Path file = Files.writeString(inbox.resolve("hello.txt"), "Hello
World");
+ createSymbolicLink(inbox.resolve(".camel"), outside);
+
+ consume(QUERY + "&renameUsingCopy=true", false);
+
+ assertSkipped(file);
+ }
+
+ @Test
+ void moveTargetKnownOnlyWhenProcessedIsCheckedWhenMoved() throws Exception
{
+ // the route sets the folder, so the target is only known once the
file is processed
+ Path file = Files.writeString(inbox.resolve("hello.txt"), "Hello
World");
+ createSymbolicLink(inbox.resolve("done"), outside);
+
+ context.addRoutes(new RouteBuilder() {
+ @Override
+ public void configure() {
+ from(fileUri(inbox, QUERY +
"&move=${exchangeProperty.folder}/${file:onlyname}"))
+ .setProperty("folder", constant("done"))
+ .convertBodyTo(String.class).to("mock:result");
+ }
+ });
+ context.start();
+
+ await().atMost(10, TimeUnit.SECONDS).until(() ->
!moveErrors.isEmpty());
+
+ assertEquals("Outside", readString(outsideFile), "the file outside the
starting directory must not be replaced");
+ assertTrue(Files.exists(file), "the consumed file must stay where it
is");
+ assertTrue(moveErrors.stream().anyMatch(e ->
String.valueOf(e.getMessage())
+ .contains("as it resolves outside the starting directory")),
moveErrors.toString());
+ }
+
+ @Test
+ void moveFolderLinkingInsideStartingDirectoryIsUsed() throws Exception {
+ Path file = Files.writeString(inbox.resolve("hello.txt"), "Hello
World");
+ Path archive = Files.createDirectories(inbox.resolve("archive"));
+ createSymbolicLink(inbox.resolve("done"), archive);
+
+ consume(QUERY + "&move=done", false);
+
+ await().atMost(10, TimeUnit.SECONDS).until(() ->
Files.exists(archive.resolve("hello.txt")));
+ assertFalse(Files.exists(file));
+ assertEquals("Hello World",
Files.readString(archive.resolve("hello.txt")));
+ assertTrue(moveErrors.isEmpty(), moveErrors.toString());
+ }
+
+ @Test
+ void moveOutsideStartingDirectoryConfiguredByRouteStillWorks() throws
Exception {
+ Path file = Files.writeString(inbox.resolve("hello.txt"), "Hello
World");
+ Path archive = testDirectory("archive");
+
+ consume(QUERY + "&move=../archive/${file:onlyname}", false);
+
+ await().atMost(10, TimeUnit.SECONDS).until(() ->
Files.exists(archive.resolve("hello.txt")));
+ assertFalse(Files.exists(file));
+ assertEquals("Hello World",
Files.readString(archive.resolve("hello.txt")));
+ assertTrue(moveErrors.isEmpty(), moveErrors.toString());
+ }
+
+ @Test
+ void jailStartingDirectoryDisabledMovesThroughLink() throws Exception {
+ Path file = Files.writeString(inbox.resolve("hello.txt"), "Hello
World");
+ createSymbolicLink(inbox.resolve(".camel"), outside);
+
+ consume(QUERY + "&jailStartingDirectory=false", false);
+
+ await().atMost(10, TimeUnit.SECONDS).until(() -> !Files.exists(file));
+ assertEquals("Hello World", Files.readString(outsideFile));
+ assertTrue(moveErrors.isEmpty(), moveErrors.toString());
+ }
+
+ private void assertSkipped(Path file) throws Exception {
+ MockEndpoint mock = getMockEndpoint("mock:result");
+ mock.expectedMessageCount(0);
+ mock.setAssertPeriod(200);
+
+ // the file is listed on every poll, and skipped each time
+ await().atMost(10, TimeUnit.SECONDS).until(() -> polls.get() >= 5);
+
+ mock.assertIsSatisfied();
+ assertEquals("Outside", readString(outsideFile), "the file outside the
starting directory must not be replaced");
+ assertTrue(Files.exists(file), "the skipped file must stay where it
is");
+ assertTrue(moveErrors.isEmpty(), moveErrors.toString());
+ }
+
+ private void consume(String query, boolean fail) throws Exception {
+ context.addRoutes(new RouteBuilder() {
+ @Override
+ public void configure() {
+ if (fail) {
+ from(fileUri(inbox,
query)).convertBodyTo(String.class).to("mock:result")
+ .throwException(new
IllegalArgumentException("Forced"));
+ } else {
+ from(fileUri(inbox,
query)).convertBodyTo(String.class).to("mock:result");
+ }
+ }
+ });
+ context.start();
+ }
+
+ private static String readString(Path path) {
+ try {
+ return Files.exists(path) ? Files.readString(path) : null;
+ } catch (IOException e) {
+ throw new IllegalStateException(e);
+ }
+ }
+
+ private static void createSymbolicLink(Path link, Path target) {
+ try {
+ Files.createSymbolicLink(link, target);
+ } catch (UnsupportedOperationException | FileSystemException e) {
+ // such as Windows without the privilege to create symbolic links
+ Assumptions.abort("Symbolic links cannot be created on this
platform: " + e.getMessage());
+ } catch (IOException e) {
+ throw new IllegalStateException(e);
+ }
+ }
+
+ private final class CollectingExceptionHandler implements ExceptionHandler
{
+
+ @Override
+ public void handleException(Throwable exception) {
+ moveErrors.add(exception);
+ }
+
+ @Override
+ public void handleException(String message, Throwable exception) {
+ moveErrors.add(exception);
+ }
+
+ @Override
+ public void handleException(String message, Exchange exchange,
Throwable exception) {
+ moveErrors.add(exception);
+ }
+ }
+}
diff --git
a/core/camel-core/src/test/java/org/apache/camel/component/file/FileProducerJailStartingDirectorySymlinkMoveTest.java
b/core/camel-core/src/test/java/org/apache/camel/component/file/FileProducerJailStartingDirectorySymlinkMoveTest.java
new file mode 100644
index 000000000000..7f2a8699c205
--- /dev/null
+++
b/core/camel-core/src/test/java/org/apache/camel/component/file/FileProducerJailStartingDirectorySymlinkMoveTest.java
@@ -0,0 +1,163 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.component.file;
+
+import java.io.IOException;
+import java.nio.file.FileSystemException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+
+import org.apache.camel.CamelExecutionException;
+import org.apache.camel.ContextTestSupport;
+import org.apache.camel.Exchange;
+import org.junit.jupiter.api.Assumptions;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+/**
+ * With jailStartingDirectory (the default) the file producer must not delete
or rename onto a file outside the starting
+ * directory through a symbolic link that lives inside it, when it moves an
existing file (moveExisting) or renames the
+ * file it wrote with a temporary name (tempFileName).
+ */
+class FileProducerJailStartingDirectorySymlinkMoveTest extends
ContextTestSupport {
+
+ private static final String MOVE_EXISTING =
"?fileExist=Move&moveExisting=archive/${file:onlyname}";
+ private static final String TEMP_FILE_NAME =
"?tempFileName=../tmp-${file:onlyname}";
+
+ private Path out;
+ private Path elsewhere;
+
+ @BeforeEach
+ void createDirectories() throws IOException {
+ out = Files.createDirectories(testDirectory("out"));
+ elsewhere = Files.createDirectories(testDirectory("elsewhere"));
+ }
+
+ @Test
+ void moveExistingOntoFileThroughLinkToOutsideIsRefused() throws Exception {
+ Path existing = Files.writeString(out.resolve("cfg.txt"), "Old");
+ Path outsideFile = Files.writeString(elsewhere.resolve("cfg.txt"),
"Outside");
+ createSymbolicLink(out.resolve("archive"), elsewhere);
+
+ assertRefused(fileUri(out, MOVE_EXISTING), "cfg.txt");
+
+ assertEquals("Outside", Files.readString(outsideFile));
+ assertEquals("Old", Files.readString(existing));
+ }
+
+ @Test
+ void moveExistingThroughLinkToOutsideIsRefused() throws Exception {
+ Path existing = Files.writeString(out.resolve("cfg.txt"), "Old");
+ createSymbolicLink(out.resolve("archive"), elsewhere);
+
+ assertRefused(fileUri(out, MOVE_EXISTING), "cfg.txt");
+
+ assertFalse(Files.exists(elsewhere.resolve("cfg.txt")));
+ assertEquals("Old", Files.readString(existing));
+ }
+
+ @Test
+ void moveExistingThroughLinkInsideStartingDirectoryIsUsed() throws
Exception {
+ Files.writeString(out.resolve("cfg.txt"), "Old");
+ Path archive = Files.createDirectories(out.resolve("real-archive"));
+ createSymbolicLink(out.resolve("archive"), archive);
+
+ template.sendBodyAndHeader(fileUri(out, MOVE_EXISTING), "New",
Exchange.FILE_NAME, "cfg.txt");
+
+ assertEquals("Old", Files.readString(archive.resolve("cfg.txt")));
+ assertEquals("New", Files.readString(out.resolve("cfg.txt")));
+ }
+
+ @Test
+ void moveExistingThroughLinkWhenJailStartingDirectoryIsDisabled() throws
Exception {
+ Files.writeString(out.resolve("cfg.txt"), "Old");
+ Path outsideFile = Files.writeString(elsewhere.resolve("cfg.txt"),
"Outside");
+ createSymbolicLink(out.resolve("archive"), elsewhere);
+
+ template.sendBodyAndHeader(fileUri(out, MOVE_EXISTING +
"&jailStartingDirectory=false"), "New",
+ Exchange.FILE_NAME, "cfg.txt");
+
+ assertEquals("Old", Files.readString(outsideFile));
+ assertEquals("New", Files.readString(out.resolve("cfg.txt")));
+ }
+
+ @Test
+ void tempFileNameRenameOntoFileThroughLinkToOutsideIsRefused() throws
Exception {
+ Path outsideFile = Files.writeString(elsewhere.resolve("hello.txt"),
"Outside");
+ createSymbolicLink(out.resolve("sub"), elsewhere);
+
+ assertRefused(fileUri(out, TEMP_FILE_NAME), "sub/hello.txt");
+
+ assertEquals("Outside", Files.readString(outsideFile));
+ }
+
+ @Test
+ void tempFileNameRenameThroughLinkToOutsideIsRefused() throws Exception {
+ createSymbolicLink(out.resolve("sub"), elsewhere);
+
+ assertRefused(fileUri(out, TEMP_FILE_NAME), "sub/hello.txt");
+
+ assertFalse(Files.exists(elsewhere.resolve("hello.txt")));
+ }
+
+ @Test
+ void tempFileNameRenameThroughLinkInsideStartingDirectoryIsUsed() throws
Exception {
+ Path real = Files.createDirectories(out.resolve("real"));
+ Files.writeString(real.resolve("hello.txt"), "Old");
+ createSymbolicLink(out.resolve("sub"), real);
+
+ template.sendBodyAndHeader(fileUri(out, TEMP_FILE_NAME), "Hello
World", Exchange.FILE_NAME, "sub/hello.txt");
+
+ assertEquals("Hello World",
Files.readString(real.resolve("hello.txt")));
+ }
+
+ @Test
+ void tempFileNameRenameThroughLinkWhenJailStartingDirectoryIsDisabled()
throws Exception {
+ Path outsideFile = Files.writeString(elsewhere.resolve("hello.txt"),
"Outside");
+ createSymbolicLink(out.resolve("sub"), elsewhere);
+
+ template.sendBodyAndHeader(fileUri(out, TEMP_FILE_NAME +
"&jailStartingDirectory=false"), "Hello World",
+ Exchange.FILE_NAME, "sub/hello.txt");
+
+ assertEquals("Hello World", Files.readString(outsideFile));
+ }
+
+ private void assertRefused(String uri, String fileName) {
+ CamelExecutionException e = assertThrows(CamelExecutionException.class,
+ () -> template.sendBodyAndHeader(uri, "New",
Exchange.FILE_NAME, fileName));
+
+ GenericFileOperationFailedException cause
+ =
assertIsInstanceOf(GenericFileOperationFailedException.class, e.getCause());
+ assertTrue(cause.getMessage().contains("as it resolves outside the
starting directory"), cause.getMessage());
+ }
+
+ private static void createSymbolicLink(Path link, Path target) {
+ try {
+ Files.createSymbolicLink(link, target);
+ } catch (UnsupportedOperationException | FileSystemException e) {
+ // such as Windows without the privilege to create symbolic links
+ Assumptions.abort("Symbolic links cannot be created on this
platform: " + e.getMessage());
+ } catch (IOException e) {
+ throw new IllegalStateException(e);
+ }
+ }
+}
diff --git
a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_18.adoc
b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_18.adoc
index e940d28f4784..ebae119e33fa 100644
--- a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_18.adoc
+++ b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_18.adoc
@@ -86,6 +86,31 @@ wildcard to select a file, use the real name of the file in
the producer, and th
consumer that lists the directory instead of a wildcard in `fileName` with
`useList=false`. See the
xref:components::sftp-component.adoc[SFTP component].
+=== camel-file - the consumer skips a file it would move through a symbolic
link resolving outside the starting directory
+
+With `jailStartingDirectory` enabled (the default), the file consumer now
resolves symbolic links on the targets of
+`preMove`, `move` and `moveFailed`, including the default `.camel` folder. It
checks these targets before the file is
+processed. When a target is inside the starting directory but resolves outside
of it through a link, the consumer skips
+the file. It logs a WARN the first time. The file is not processed and not
moved, and it stays in the folder. Before,
+the existing target was deleted and the file was moved wherever the link
pointed. A target that depends on what the
+route sets can only be checked when the file is moved. Such a move fails, and
the file is consumed again on the next
+poll.
+
+The file producer now refuses to delete or rename onto a file through such a
link. This covers `fileExist=Move` with
+`moveExisting`, and the rename of a file written with `tempFileName`. The
write fails.
+
+The expressions of `preMove`, `move` and `moveFailed` are evaluated before the
file is processed, so each of them is
+evaluated one extra time per file: for a file processed without error, `move`
is now evaluated twice and `moveFailed`
+once, where before `move` was evaluated once and `moveFailed` not at all. This
matters for an expression with side
+effects, such as a bean that hands out a sequence number. The default `.camel`
folder has no expression to evaluate.
+
+Targets configured outside the starting directory (such as `move=../archive`)
and links resolving inside it work as
+before. A deployment that links one of these folders to another directory on
purpose, such as `.camel` on another
+volume, should point the option at the folder the link resolves to, such as
`move=/mnt/volume2/${file:name}`, as a
+target configured outside the starting directory is not checked. Setting
`jailStartingDirectory=false` also keeps
+following those links, but then the consumer also reads a file in the starting
directory that is a link to a file
+outside of it, and the producer writes through such links.
+
=== camel-hazelcast - the hazelcast-seda consumer restores a transferred
exchange only with transferExchange=true
A `hazelcast-seda` consumer now restores the headers and exchange properties
of an exchange that a producer with
diff --git
a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_22.adoc
b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_22.adoc
index 08770f5ee83e..19ed65878e43 100644
--- a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_22.adoc
+++ b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_22.adoc
@@ -107,6 +107,31 @@ wildcard to select a file, use the real name of the file
in the producer, and th
consumer that lists the directory instead of a wildcard in `fileName` with
`useList=false`. See the
xref:components::sftp-component.adoc[SFTP component].
+=== camel-file - the consumer skips a file it would move through a symbolic
link resolving outside the starting directory
+
+With `jailStartingDirectory` enabled (the default), the file consumer now
resolves symbolic links on the targets of
+`preMove`, `move` and `moveFailed`, including the default `.camel` folder. It
checks these targets before the file is
+processed. When a target is inside the starting directory but resolves outside
of it through a link, the consumer skips
+the file. It logs a WARN the first time. The file is not processed and not
moved, and it stays in the folder. Before,
+the existing target was deleted and the file was moved wherever the link
pointed. A target that depends on what the
+route sets can only be checked when the file is moved. Such a move fails, and
the file is consumed again on the next
+poll.
+
+The file producer now refuses to delete or rename onto a file through such a
link. This covers `fileExist=Move` with
+`moveExisting`, and the rename of a file written with `tempFileName`. The
write fails.
+
+The expressions of `preMove`, `move` and `moveFailed` are evaluated before the
file is processed, so each of them is
+evaluated one extra time per file: for a file processed without error, `move`
is now evaluated twice and `moveFailed`
+once, where before `move` was evaluated once and `moveFailed` not at all. This
matters for an expression with side
+effects, such as a bean that hands out a sequence number. The default `.camel`
folder has no expression to evaluate.
+
+Targets configured outside the starting directory (such as `move=../archive`)
and links resolving inside it work as
+before. A deployment that links one of these folders to another directory on
purpose, such as `.camel` on another
+volume, should point the option at the folder the link resolves to, such as
`move=/mnt/volume2/${file:name}`, as a
+target configured outside the starting directory is not checked. Setting
`jailStartingDirectory=false` also keeps
+following those links, but then the consumer also reads a file in the starting
directory that is a link to a file
+outside of it, and the producer writes through such links.
+
=== camel-hazelcast - the hazelcast-seda consumer restores a transferred
exchange only with transferExchange=true
A `hazelcast-seda` consumer now restores the headers and exchange properties
of an exchange that a producer with
diff --git
a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
index bd8b466e92fc..cb5d5d74ece5 100644
--- a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
+++ b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
@@ -2475,6 +2475,31 @@ which could cause the producer to create a spurious
directory with a backslash i
consumer to miss files in a backslash-named directory. On Windows, where
`java.io.File` already
normalizes both separators to `\`, this change has no effect.
+=== camel-file - the consumer skips a file it would move through a symbolic
link resolving outside the starting directory
+
+With `jailStartingDirectory` enabled (the default), the file consumer now
resolves symbolic links on the targets of
+`preMove`, `move` and `moveFailed`, including the default `.camel` folder. It
checks these targets before the file is
+processed. When a target is inside the starting directory but resolves outside
of it through a link, the consumer skips
+the file. It logs a WARN the first time. The file is not processed and not
moved, and it stays in the folder. Before,
+the existing target was deleted and the file was moved wherever the link
pointed. A target that depends on what the
+route sets can only be checked when the file is moved. Such a move fails, and
the file is consumed again on the next
+poll.
+
+The file producer now refuses to delete or rename onto a file through such a
link. This covers `fileExist=Move` with
+`moveExisting`, and the rename of a file written with `tempFileName`. The
write fails.
+
+The expressions of `preMove`, `move` and `moveFailed` are evaluated before the
file is processed, so each of them is
+evaluated one extra time per file: for a file processed without error, `move`
is now evaluated twice and `moveFailed`
+once, where before `move` was evaluated once and `moveFailed` not at all. This
matters for an expression with side
+effects, such as a bean that hands out a sequence number. The default `.camel`
folder has no expression to evaluate.
+
+Targets configured outside the starting directory (such as `move=../archive`)
and links resolving inside it work as
+before. A deployment that links one of these folders to another directory on
purpose, such as `.camel` on another
+volume, should point the option at the folder the link resolves to, such as
`move=/mnt/volume2/${file:name}`, as a
+target configured outside the starting directory is not checked. Setting
`jailStartingDirectory=false` also keeps
+following those links, but then the consumer also reads a file in the starting
directory that is a link to a file
+outside of it, and the producer writes through such links.
+
=== camel-flatpack - with splitRows each message has its own row, and a
failing row fails the producer
With `splitRows=true` (the default), every row message shared the one parsed
`DataSet`, so the next row moved the row