[
https://issues.apache.org/jira/browse/MBUILDCACHE-64?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17760935#comment-17760935
]
ASF GitHub Bot commented on MBUILDCACHE-64:
-------------------------------------------
AlexanderAshitkin commented on code in PR #91:
URL:
https://github.com/apache/maven-build-cache-extension/pull/91#discussion_r1311038794
##########
src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java:
##########
@@ -165,45 +184,94 @@ public MavenProjectInput(
this.repoSystem = repoSystem;
this.remoteCache = remoteCache;
Properties properties = project.getProperties();
- this.dirGlob = properties.getProperty(CACHE_INPUT_GLOB_NAME,
config.getDefaultGlob());
+ this.defaultFilenameGlob =
properties.getProperty(CACHE_INPUT_GLOB_NAME, config.getDefaultGlob());
this.processPlugins =
Boolean.parseBoolean(properties.getProperty(CACHE_PROCESS_PLUGINS,
config.isProcessPlugins()));
this.tmpDir = System.getProperty("java.io.tmpdir");
+ this.baseDirectoryGlob = baseDirPath.toString().replace("\\", "/") +
"/";
+
org.apache.maven.model.Build build = project.getBuild();
- filteredOutPaths = new ArrayList<>(Arrays.asList(
- normalizedPath(build.getDirectory()), // target by default
- normalizedPath(build.getOutputDirectory()),
- normalizedPath(build.getTestOutputDirectory())));
+ addToExcludedSection(
+ convertToPathMatcherFileSeperator(
+
normalizedPath(build.getDirectory()).toString())
+ + GLOB_SX_ALL_SUB_FILES,
+ false); // target by default
+ addToExcludedSection(
+ convertToPathMatcherFileSeperator(
+
normalizedPath(build.getOutputDirectory()).toString())
+ + GLOB_SX_ALL_SUB_FILES,
+ false); // target/classes by default
+ addToExcludedSection(
+ convertToPathMatcherFileSeperator(
+
normalizedPath(build.getTestOutputDirectory()).toString())
+ + GLOB_SX_ALL_SUB_FILES,
+ false); // target/test-classes by default
List<Exclude> excludes = config.getGlobalExcludePaths();
for (Exclude excludePath : excludes) {
- filteredOutPaths.add(Paths.get(excludePath.getValue()));
+ addToExcludedSection(excludePath.getValue(), true);
}
for (String propertyName : properties.stringPropertyNames()) {
if (propertyName.startsWith(CACHE_EXCLUDE_NAME)) {
String propertyValue = properties.getProperty(propertyName);
- Path path = Paths.get(propertyValue);
- filteredOutPaths.add(path);
+ addToExcludedSection(propertyValue, true);
+
if (LOGGER.isDebugEnabled()) {
LOGGER.debug(
- "Adding an excludePath from property '{}', values
is '{}', path is '{}' ",
- propertyName,
- propertyValue,
- path);
+ "Adding an excludePath from property '{}', value
is '{}'", propertyName, propertyValue);
}
}
}
CacheUtils.debugPrintCollection(
- LOGGER,
- filteredOutPaths,
- "List of excluded paths (checked either by fileName or by
startsWith prefix)",
- "Path entry");
+ LOGGER, inputExcludePathMatcherString, "List of excluded glob
patterns", "Pattern");
this.fileComparator = new PathIgnoringCaseComparator();
}
+ private String convertToPathMatcherFileSeperator(String path) {
+ return path.replace("\\", "/");
+ }
+
+ /**
+ * Add a value from the excluded section list to the directories and/or
the filenames ban list.
+ * @param excludedValue a value from the exclude list
+ */
+ private void addToExcludedSection(String excludedValue, boolean
addProjectBaseDir) {
+
+ String pathMatcherGlob = GLOB_PX
+ +
+ // Add the base directory to any input directly coming from
user configuration
+ (addProjectBaseDir ? baseDirectoryGlob : "")
+ +
+ // If the glob start with "/", we remove it since it's already
added in the added basedir glob
+ (excludedValue.startsWith("/") ? excludedValue.substring(1) :
excludedValue);
+
+ // In order to skip unnecessary subtree dir walking, we use a
different PathMatcher list for "directories" or
+ // "files + directories"
+ inputExcludePathMatchers.add(new
TreeWalkerPathMatcher(pathMatcherGlob, false));
+
+ // Globs ending with "**" should end any sub-directory inspection
+ if (pathMatcherGlob.endsWith("**")) {
Review Comment:
Well, there are different semantics now and before for project-level
properties.
Before: declaring `<exclude.me>mydir/mysubdir</exclude.me>` exclusion was
filtering out the whole subtree (through the `Path#startsWith`)
Now: declaring `mydir/mysubdir` doesn't exclude anything.
```
FileSystems.getDefault().getPathMatcher("glob:dir");
System.out.println(pathMatcher.matches(Paths.get("dir/file.txt")));`
-----OUT-----
false
```
For consistency, we must keep the path semantics of properties or migrate
them to the globs/regex semantics, delegating filtering expressions to the
users. But the main point is that it should be consistent, transparent, and
determenistic.
##########
src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java:
##########
@@ -165,45 +184,94 @@ public MavenProjectInput(
this.repoSystem = repoSystem;
this.remoteCache = remoteCache;
Properties properties = project.getProperties();
- this.dirGlob = properties.getProperty(CACHE_INPUT_GLOB_NAME,
config.getDefaultGlob());
+ this.defaultFilenameGlob =
properties.getProperty(CACHE_INPUT_GLOB_NAME, config.getDefaultGlob());
this.processPlugins =
Boolean.parseBoolean(properties.getProperty(CACHE_PROCESS_PLUGINS,
config.isProcessPlugins()));
this.tmpDir = System.getProperty("java.io.tmpdir");
+ this.baseDirectoryGlob = baseDirPath.toString().replace("\\", "/") +
"/";
+
org.apache.maven.model.Build build = project.getBuild();
- filteredOutPaths = new ArrayList<>(Arrays.asList(
- normalizedPath(build.getDirectory()), // target by default
- normalizedPath(build.getOutputDirectory()),
- normalizedPath(build.getTestOutputDirectory())));
+ addToExcludedSection(
+ convertToPathMatcherFileSeperator(
+
normalizedPath(build.getDirectory()).toString())
+ + GLOB_SX_ALL_SUB_FILES,
+ false); // target by default
+ addToExcludedSection(
+ convertToPathMatcherFileSeperator(
+
normalizedPath(build.getOutputDirectory()).toString())
+ + GLOB_SX_ALL_SUB_FILES,
+ false); // target/classes by default
+ addToExcludedSection(
+ convertToPathMatcherFileSeperator(
+
normalizedPath(build.getTestOutputDirectory()).toString())
+ + GLOB_SX_ALL_SUB_FILES,
+ false); // target/test-classes by default
List<Exclude> excludes = config.getGlobalExcludePaths();
for (Exclude excludePath : excludes) {
- filteredOutPaths.add(Paths.get(excludePath.getValue()));
+ addToExcludedSection(excludePath.getValue(), true);
}
for (String propertyName : properties.stringPropertyNames()) {
if (propertyName.startsWith(CACHE_EXCLUDE_NAME)) {
String propertyValue = properties.getProperty(propertyName);
- Path path = Paths.get(propertyValue);
- filteredOutPaths.add(path);
+ addToExcludedSection(propertyValue, true);
+
if (LOGGER.isDebugEnabled()) {
LOGGER.debug(
- "Adding an excludePath from property '{}', values
is '{}', path is '{}' ",
- propertyName,
- propertyValue,
- path);
+ "Adding an excludePath from property '{}', value
is '{}'", propertyName, propertyValue);
}
}
}
CacheUtils.debugPrintCollection(
- LOGGER,
- filteredOutPaths,
- "List of excluded paths (checked either by fileName or by
startsWith prefix)",
- "Path entry");
+ LOGGER, inputExcludePathMatcherString, "List of excluded glob
patterns", "Pattern");
this.fileComparator = new PathIgnoringCaseComparator();
}
+ private String convertToPathMatcherFileSeperator(String path) {
+ return path.replace("\\", "/");
+ }
+
+ /**
+ * Add a value from the excluded section list to the directories and/or
the filenames ban list.
+ * @param excludedValue a value from the exclude list
+ */
+ private void addToExcludedSection(String excludedValue, boolean
addProjectBaseDir) {
+
+ String pathMatcherGlob = GLOB_PX
+ +
+ // Add the base directory to any input directly coming from
user configuration
+ (addProjectBaseDir ? baseDirectoryGlob : "")
+ +
+ // If the glob start with "/", we remove it since it's already
added in the added basedir glob
+ (excludedValue.startsWith("/") ? excludedValue.substring(1) :
excludedValue);
+
+ // In order to skip unnecessary subtree dir walking, we use a
different PathMatcher list for "directories" or
+ // "files + directories"
+ inputExcludePathMatchers.add(new
TreeWalkerPathMatcher(pathMatcherGlob, false));
+
+ // Globs ending with "**" should end any sub-directory inspection
+ if (pathMatcherGlob.endsWith("**")) {
Review Comment:
Well, there are different semantics now and before for project-level
properties.
Before: declaring `<exclude.me>mydir/mysubdir</exclude.me>` exclusion was
filtering out the whole subtree (through the `Path#startsWith`)
Now: declaring `mydir/mysubdir` doesn't exclude anything.
```
FileSystems.getDefault().getPathMatcher("glob:dir");
System.out.println(pathMatcher.matches(Paths.get("dir/file.txt")));`
-----OUT-----
false
```
For consistency, we must keep the path semantics of properties or migrate
them to the globs/regex semantics, delegating filtering expressions to the
users. But the main point is that it should be consistent, transparent, and
deterministic.
> Apply global exclusions to folder names
> ---------------------------------------
>
> Key: MBUILDCACHE-64
> URL: https://issues.apache.org/jira/browse/MBUILDCACHE-64
> Project: Maven Build Cache Extension
> Issue Type: Bug
> Affects Versions: 1.0.1
> Reporter: Frank Wagner
> Assignee: Olivier Lamy
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.1.0
>
>
> It is currently not possible to exclude folders by their name, like
> {quote}<input>
> <global>
> <excludes>
> <exclude>node_modules</exclude>
> <exclude>dist</exclude>
> <exclude>build</exclude>
> </excludes>
> </global>
> ...
> {quote}
> That's because isFilteredOutSubpath(),
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L638,]
> uses startWith on normalized absolute paths.
> That function could be enhanced with an additional criterion like in
> [https://github.com/apache/maven-build-cache-extension/blob/master/src/main/java/org/apache/maven/buildcache/checksum/MavenProjectInput.java#L510]
> {{filteredOutPaths.stream().anyMatch(it ->
> it.getFileName().equals(entry.getFileName()))}}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
