davsclaus commented on code in PR #29855:
URL: https://github.com/apache/camel/pull/29855#discussion_r4575758874
##########
components/camel-file/src/main/java/org/apache/camel/component/file/FileOperations.java:
##########
@@ -270,25 +278,54 @@ public void releaseRetrievedFileResources(Exchange
exchange) throws GenericFileO
* before it creates missing parent directories of the target.
*/
void jailToStartingDirectory(File target) {
- if (!endpoint.isJailStartingDirectory()) {
- return;
- }
- Path startingDirectory = endpoint.getFile().toPath();
- if
(!target.toPath().toAbsolutePath().normalize().startsWith(startingDirectory.toAbsolutePath().normalize()))
{
- return;
- }
+ jailToStartingDirectory(target, "Cannot write file: ");
+ }
+
+ /**
+ * Checks a path before it is written, deleted, renamed onto or created as
a directory.
+ *
+ * @throws GenericFileOperationFailedException if the path is inside the
starting directory but resolves outside of
+ * it through a symbolic link
+ */
+ private void jailToStartingDirectory(File target, String failure) {
boolean within;
try {
- within =
GenericFileHelper.isWithinDirectoryResolvingLinks(target.toPath(),
startingDirectory);
+ within = isWithinStartingDirectoryResolvingLinks(target);
} catch (IOException e) {
throw new GenericFileOperationFailedException(
"Cannot verify file: " + target + " is within the starting
directory: " + endpoint.getFile(), e);
}
if (!within) {
throw new GenericFileOperationFailedException(
- "Cannot write file: " + target + " as it resolves outside
the starting directory: "
- +
endpoint.getFile());
+ failure + target + " as it resolves outside the starting
directory: " + endpoint.getFile());
+ }
+ }
+
+ /**
+ * Whether the path stays within the starting directory once symbolic
links are resolved. Always true when
+ * jailStartingDirectory is disabled, and for a path that is lexically
outside the starting directory (such as a
+ * move to ../archive the route author configured).
+ */
+ @Override
+ public boolean isWithinStartingDirectory(String name) {
+ try {
+ return isWithinStartingDirectoryResolvingLinks(new File(name));
+ } catch (IOException e) {
+ // such as a dangling symbolic link
+ LOG.debug("Cannot resolve file: {} against the starting directory
due to: {}", name, e.getMessage());
+ return false;
+ }
+ }
+
+ private boolean isWithinStartingDirectoryResolvingLinks(File target)
throws IOException {
+ if (endpoint == null || !endpoint.isJailStartingDirectory()) {
+ return true;
+ }
+ Path startingDirectory = endpoint.getFile().toPath();
+ if
(!target.toPath().toAbsolutePath().normalize().startsWith(startingDirectory.toAbsolutePath().normalize()))
{
+ return true;
}
+ return
GenericFileHelper.isWithinDirectoryResolvingLinks(target.toPath(),
startingDirectory);
Review Comment:
Thanks for the measurements. Changed in b37495e6:
- `GenericFileHelper.isWithinDirectoryResolvingLinks` first looks at the
segments below the starting directory without following links (one attribute
lookup each). When none is a link and there is no `.` or `..` segment, the path
cannot resolve outside, so neither the path nor the starting directory goes
through `toRealPath`. Only a path through a link takes the resolving check. The
listing check of `FileConsumer` (CAMEL-25412) uses the same shortcut, so it no
longer resolves every listed file on every poll either.
- The check before processing computes the default `.camel` target directly,
without a copy of the exchange, a bind and an expression
(`DefaultMoveRenamer`). A configured `move`/`moveFailed` is still evaluated, as
before.
- `deleteFile` checks only a file that exists.
Same harness shape (10000 empty files, `initialDelay=0&delay=10`, counting
processor, steady rounds), Linux, btrfs, JDK 27:
| | per file |
|---|---|
| merge-base f07a0e80 | 0.023 ms |
| 9e6be9db | 0.046 ms |
| b37495e6 | 0.030 ms |
The rest over the merge-base is the attribute lookups of the two checks
(before processing, and of the rename). I could not measure on macOS; there the
merge-base already paid a `toRealPath` per listed file, which is now gone
without a link, so the gap should be smaller than on Linux. A rerun of your
harness would tell.
_Claude Code on behalf of davsclaus_
##########
docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc:
##########
@@ -2336,6 +2336,23 @@ which could cause the producer to create a spurious
directory with a backslash i
consumer to miss files in a backslash-named directory. On Windows, where
`java.io.File` already
normalizes both separators to `\`, this change has no effect.
+=== camel-file - the consumer skips a file it would move through a symbolic
link resolving outside the starting directory
+
+With `jailStartingDirectory` enabled (the default), the file consumer now
resolves symbolic links on the targets of
+`preMove`, `move` and `moveFailed`, including the default `.camel` folder. It
checks these targets before the file is
+processed. When a target is inside the starting directory but resolves outside
of it through a link, the consumer skips
+the file. It logs a WARN the first time. The file is not processed and not
moved, and it stays in the folder. Before,
+the existing target was deleted and the file was moved wherever the link
pointed. A target that depends on what the
+route sets can only be checked when the file is moved. Such a move fails, and
the file is consumed again on the next
+poll.
+
+The file producer now refuses to delete or rename onto a file through such a
link. This covers `fileExist=Move` with
+`moveExisting`, and the rename of a file written with `tempFileName`. The
write fails.
+
+Targets configured outside the starting directory (such as `move=../archive`)
and links resolving inside it work as
+before. A deployment that links these folders to another directory on purpose,
such as `.camel` on another volume, must
+set `jailStartingDirectory=false` to keep following those links.
Review Comment:
Agreed, changed in b37495e6, in the 4.18, 4.22 and 4.24 guides and in the
component doc:
- A deployment that links such a folder elsewhere on purpose is told to
point `move` (or `preMove`, `moveFailed`) at the folder the link resolves to,
such as `move=/mnt/volume2/${file:name}`, as a target configured outside the
starting directory is not checked.
- `jailStartingDirectory=false` stays as the alternative, saying that the
consumer then also reads a file in the starting directory that links to a file
outside, and the producer writes through such links.
- A new paragraph says the `preMove`, `move` and `moveFailed` expressions
are evaluated one extra time per file: for a file processed without error
`move` twice and `moveFailed` once, where before `move` once and `moveFailed`
never, which matters for an expression with side effects such as a sequence.
The default `.camel` folder has no expression to evaluate.
_Claude Code on behalf of davsclaus_
##########
components/camel-file/src/main/java/org/apache/camel/component/file/strategy/GenericFileRenameProcessStrategy.java:
##########
@@ -41,6 +41,11 @@ public boolean begin(
return false;
}
+ // skip the file before it is processed if it cannot be moved where it
would be moved to
+ if (!acceptMoveTargets(operations, endpoint, exchange, file,
beginRenamer, commitRenamer, failureRenamer)) {
Review Comment:
Changed in b37495e6: both `GenericFileRenameProcessStrategy` and
`GenericFileDeleteProcessStrategy` call `acceptMoveTargets` before
`super.begin`, so a skipped file does not take (or wait for) the read lock.
`abort` still runs for the skipped file; the read lock strategies release only
a lock they acquired (marker file and idempotent strategies check their
acquired property), the same as when a lock is not granted.
New test
`FileConsumerMoveJailStartingDirectorySymlinkTest.skippedFileDoesNotTakeTheReadLock`:
a counting `MarkerFileExclusiveReadLockStrategy` is never asked for the lock
over at least 5 polls of a skipped file.
_Claude Code on behalf of davsclaus_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]