svn commit: r1886352 - in /tomcat/site/trunk: docs/migration-10.html xdocs/migration-10.xml
Author: markt Date: Tue Feb 9 08:22:38 2021 New Revision: 1886352 URL: http://svn.apache.org/viewvc?rev=1886352&view=rev Log: Fix copy/paste error. 10.0.x is now stable. Modified: tomcat/site/trunk/docs/migration-10.html tomcat/site/trunk/xdocs/migration-10.xml Modified: tomcat/site/trunk/docs/migration-10.html URL: http://svn.apache.org/viewvc/tomcat/site/trunk/docs/migration-10.html?rev=1886352&r1=1886351&r2=1886352&view=diff == --- tomcat/site/trunk/docs/migration-10.html (original) +++ tomcat/site/trunk/docs/migration-10.html Tue Feb 9 08:22:38 2021 @@ -14,9 +14,6 @@ of Apache Tomcat. This section lists all the known changes between 9.0.x and 10.0.x which may cause backwards compatibility problems when upgrading. - Until Tomcat 10 is declared stable, there may be additional changes made - that cause backwards compatibility problems when upgrading. - Java 8 required Apache Tomcat 10.0.x requires Java 8 or later. This is unchanged from @@ -118,13 +115,13 @@ of Apache Tomcat. Upgrading 10.0.x - When upgrading instances of Apache Tomcat from one version of Tomcat 9 to + When upgrading instances of Apache Tomcat from one version of Tomcat 10 to another, particularly when using separate locations for $CATALINA_HOME and $CATALINA_BASE, it is necessary to ensure that any changes in the configuration files such as new attributes and changes to defaults are applied as part of the upgrade. To assist with the identification of these changes, the form below may be used to view the differences between the configuration - files in different versions of Tomcat 9. + files in different versions of Tomcat 10. Tomcat 10.0.x noteable changes The Tomcat developers aim for each patch release to be fully backwards Modified: tomcat/site/trunk/xdocs/migration-10.xml URL: http://svn.apache.org/viewvc/tomcat/site/trunk/xdocs/migration-10.xml?rev=1886352&r1=1886351&r2=1886352&view=diff == --- tomcat/site/trunk/xdocs/migration-10.xml (original) +++ tomcat/site/trunk/xdocs/migration-10.xml Tue Feb 9 08:22:38 2021 @@ -25,9 +25,6 @@ of Apache Tomcat. This section lists all the known changes between 9.0.x and 10.0.x which may cause backwards compatibility problems when upgrading. - Until Tomcat 10 is declared stable, there may be additional changes made - that cause backwards compatibility problems when upgrading. - Apache Tomcat 10.0.x requires Java 8 or later. This is unchanged from @@ -131,13 +128,13 @@ of Apache Tomcat. - When upgrading instances of Apache Tomcat from one version of Tomcat 9 to + When upgrading instances of Apache Tomcat from one version of Tomcat 10 to another, particularly when using separate locations for $CATALINA_HOME and $CATALINA_BASE, it is necessary to ensure that any changes in the configuration files such as new attributes and changes to defaults are applied as part of the upgrade. To assist with the identification of these changes, the form below may be used to view the differences between the configuration - files in different versions of Tomcat 9. + files in different versions of Tomcat 10. The Tomcat developers aim for each patch release to be fully backwards - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 65131] OpenSSLEngine errors on a connection affect other connections
https://bz.apache.org/bugzilla/show_bug.cgi?id=65131 --- Comment #1 from Remy Maucherat --- Ok, I can see the error check could still be a bit shaky, a loop could possibly be used in clearLastError (documentation of the OpenSSL API is here: https://www.openssl.org/docs/manmaster/man3/ERR_get_error.html ). Not sure why clearLastError is called in unwrap however, maybe Mark can comment on that one. The TODO is accurate: ideally every SSL call should check the error. BTW, I doubt this reproduces anything, ab works just fine, so curl isn't going to be doing anything special beyond that. The NPE in SecureNioChannel.close is possible, it would be a cosmetic error but the code could be more defensive. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] branch master updated: Avoid possible NPE
This is an automated email from the ASF dual-hosted git repository. remm pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/tomcat.git The following commit(s) were added to refs/heads/master by this push: new eb3dfbf Avoid possible NPE eb3dfbf is described below commit eb3dfbfff450864c558b03e2e4ef3e707b62f85d Author: remm AuthorDate: Tue Feb 9 13:19:11 2021 +0100 Avoid possible NPE Very early unexpected exceptions could lead to a null sslEngine, nothing to do in that case on close. Seen in BZ65131. --- java/org/apache/tomcat/util/net/SecureNio2Channel.java | 9 - java/org/apache/tomcat/util/net/SecureNioChannel.java | 5 + webapps/docs/changelog.xml | 4 3 files changed, 17 insertions(+), 1 deletion(-) diff --git a/java/org/apache/tomcat/util/net/SecureNio2Channel.java b/java/org/apache/tomcat/util/net/SecureNio2Channel.java index 611038e..b2d94d1 100644 --- a/java/org/apache/tomcat/util/net/SecureNio2Channel.java +++ b/java/org/apache/tomcat/util/net/SecureNio2Channel.java @@ -596,8 +596,15 @@ public class SecureNio2Channel extends Nio2Channel { */ @Override public void close() throws IOException { -if (closing) return; +if (closing) { +return; +} closing = true; +if (sslEngine == null) { +netOutBuffer.clear(); +closed = true; +return; +} sslEngine.closeOutbound(); long timeout = endpoint.getConnectionTimeout(); diff --git a/java/org/apache/tomcat/util/net/SecureNioChannel.java b/java/org/apache/tomcat/util/net/SecureNioChannel.java index 1ac2061..1f13338 100644 --- a/java/org/apache/tomcat/util/net/SecureNioChannel.java +++ b/java/org/apache/tomcat/util/net/SecureNioChannel.java @@ -525,6 +525,11 @@ public class SecureNioChannel extends NioChannel { return; } closing = true; +if (sslEngine == null) { +netOutBuffer.clear(); +closed = true; +return; +} sslEngine.closeOutbound(); if (!flush(netOutBuffer)) { diff --git a/webapps/docs/changelog.xml b/webapps/docs/changelog.xml index f8057a2..a71b7c8 100644 --- a/webapps/docs/changelog.xml +++ b/webapps/docs/changelog.xml @@ -123,6 +123,10 @@ 65118: Fix a potential NullPointerException when pruning closed HTTP/2 streams from the connection. (markt) + +Avoid NullPointerException when a secure channel is closed before the +SSL engine was initialized. (remm) + - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] branch 9.0.x updated: Avoid possible NPE
This is an automated email from the ASF dual-hosted git repository. remm pushed a commit to branch 9.0.x in repository https://gitbox.apache.org/repos/asf/tomcat.git The following commit(s) were added to refs/heads/9.0.x by this push: new 1a9b9ac Avoid possible NPE 1a9b9ac is described below commit 1a9b9acaa4ac16e9a8de68508eac5a43eb217563 Author: remm AuthorDate: Tue Feb 9 13:19:11 2021 +0100 Avoid possible NPE Very early unexpected exceptions could lead to a null sslEngine, nothing to do in that case on close. Seen in BZ65131. --- java/org/apache/tomcat/util/net/SecureNio2Channel.java | 9 - java/org/apache/tomcat/util/net/SecureNioChannel.java | 5 + webapps/docs/changelog.xml | 4 3 files changed, 17 insertions(+), 1 deletion(-) diff --git a/java/org/apache/tomcat/util/net/SecureNio2Channel.java b/java/org/apache/tomcat/util/net/SecureNio2Channel.java index 611038e..b2d94d1 100644 --- a/java/org/apache/tomcat/util/net/SecureNio2Channel.java +++ b/java/org/apache/tomcat/util/net/SecureNio2Channel.java @@ -596,8 +596,15 @@ public class SecureNio2Channel extends Nio2Channel { */ @Override public void close() throws IOException { -if (closing) return; +if (closing) { +return; +} closing = true; +if (sslEngine == null) { +netOutBuffer.clear(); +closed = true; +return; +} sslEngine.closeOutbound(); long timeout = endpoint.getConnectionTimeout(); diff --git a/java/org/apache/tomcat/util/net/SecureNioChannel.java b/java/org/apache/tomcat/util/net/SecureNioChannel.java index 0dedafe..bfaaa8f 100644 --- a/java/org/apache/tomcat/util/net/SecureNioChannel.java +++ b/java/org/apache/tomcat/util/net/SecureNioChannel.java @@ -551,6 +551,11 @@ public class SecureNioChannel extends NioChannel { return; } closing = true; +if (sslEngine == null) { +netOutBuffer.clear(); +closed = true; +return; +} sslEngine.closeOutbound(); if (!flush(netOutBuffer)) { diff --git a/webapps/docs/changelog.xml b/webapps/docs/changelog.xml index ac585da..9dbf03a 100644 --- a/webapps/docs/changelog.xml +++ b/webapps/docs/changelog.xml @@ -118,6 +118,10 @@ 65118: Fix a potential NullPointerException when pruning closed HTTP/2 streams from the connection. (markt) + +Avoid NullPointerException when a secure channel is closed before the +SSL engine was initialized. (remm) + - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat] branch 8.5.x updated: Avoid possible NPE
This is an automated email from the ASF dual-hosted git repository. remm pushed a commit to branch 8.5.x in repository https://gitbox.apache.org/repos/asf/tomcat.git The following commit(s) were added to refs/heads/8.5.x by this push: new e5c1075 Avoid possible NPE e5c1075 is described below commit e5c1075e06618b36f89b5cae2c467e236e0d Author: remm AuthorDate: Tue Feb 9 13:19:11 2021 +0100 Avoid possible NPE Very early unexpected exceptions could lead to a null sslEngine, nothing to do in that case on close. Seen in BZ65131. --- java/org/apache/tomcat/util/net/SecureNio2Channel.java | 9 - java/org/apache/tomcat/util/net/SecureNioChannel.java | 5 + webapps/docs/changelog.xml | 4 3 files changed, 17 insertions(+), 1 deletion(-) diff --git a/java/org/apache/tomcat/util/net/SecureNio2Channel.java b/java/org/apache/tomcat/util/net/SecureNio2Channel.java index 92fe2f0..49768e4 100644 --- a/java/org/apache/tomcat/util/net/SecureNio2Channel.java +++ b/java/org/apache/tomcat/util/net/SecureNio2Channel.java @@ -596,8 +596,15 @@ public class SecureNio2Channel extends Nio2Channel { */ @Override public void close() throws IOException { -if (closing) return; +if (closing) { +return; +} closing = true; +if (sslEngine == null) { +netOutBuffer.clear(); +closed = true; +return; +} sslEngine.closeOutbound(); long timeout = endpoint.getConnectionTimeout(); diff --git a/java/org/apache/tomcat/util/net/SecureNioChannel.java b/java/org/apache/tomcat/util/net/SecureNioChannel.java index f235ade..a38ec47 100644 --- a/java/org/apache/tomcat/util/net/SecureNioChannel.java +++ b/java/org/apache/tomcat/util/net/SecureNioChannel.java @@ -540,6 +540,11 @@ public class SecureNioChannel extends NioChannel { return; } closing = true; +if (sslEngine == null) { +netOutBuffer.clear(); +closed = true; +return; +} sslEngine.closeOutbound(); if (!flush(netOutBuffer)) { diff --git a/webapps/docs/changelog.xml b/webapps/docs/changelog.xml index c8a0e2c..f313899 100644 --- a/webapps/docs/changelog.xml +++ b/webapps/docs/changelog.xml @@ -118,6 +118,10 @@ 65118: Fix a potential NullPointerException when pruning closed HTTP/2 streams from the connection. (markt) + +Avoid NullPointerException when a secure channel is closed before the +SSL engine was initialized. (remm) + - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat-jakartaee-migration] branch master updated (1a8f20a -> e963951)
This is an automated email from the ASF dual-hosted git repository. markt pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/tomcat-jakartaee-migration.git. from 1a8f20a Update chnagelog new c09f8b4 Replace `-verbose` with `-logLevel=` to provide more control new d8373f2 Add Javadoc new 4a0b09d Reduce object creation during conversion new 9acbf5e Simplify error handling new 0a09f52 Track whether source was modified or not. new 066f71d Fix #13 refactor logging new e963951 Fix #3 Add support for excluding files from conversion The 7 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.md | 3 + .../apache/tomcat/jakartaee/ClassConverter.java| 26 ++- .../org/apache/tomcat/jakartaee/Converter.java | 13 +- .../org/apache/tomcat/jakartaee/GlobMatcher.java | 243 + .../apache/tomcat/jakartaee/ManifestConverter.java | 34 ++- .../org/apache/tomcat/jakartaee/Migration.java | 136 .../org/apache/tomcat/jakartaee/MigrationCLI.java | 108 - .../org/apache/tomcat/jakartaee/MigrationTask.java | 24 +- .../tomcat/jakartaee/PassThroughConverter.java | 10 +- .../org/apache/tomcat/jakartaee/TextConverter.java | 18 +- .../tomcat/jakartaee/LocalStrings.properties | 35 ++- .../org/apache/tomcat/jakartaee/MigrationTest.java | 3 +- .../tomcat/jakartaee/PassThroughConverterTest.java | 6 +- .../apache/tomcat/jakartaee/TextConverterTest.java | 4 +- 14 files changed, 536 insertions(+), 127 deletions(-) create mode 100644 src/main/java/org/apache/tomcat/jakartaee/GlobMatcher.java - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat-jakartaee-migration] 07/07: Fix #3 Add support for excluding files from conversion
This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/tomcat-jakartaee-migration.git commit e963951d3f3a7ac0785815dfe44bd008425ccd43 Author: Mark Thomas AuthorDate: Tue Feb 9 17:00:03 2021 + Fix #3 Add support for excluding files from conversion --- CHANGES.md | 3 +- .../org/apache/tomcat/jakartaee/GlobMatcher.java | 243 + .../org/apache/tomcat/jakartaee/Migration.java | 65 +- .../org/apache/tomcat/jakartaee/MigrationCLI.java | 7 +- .../org/apache/tomcat/jakartaee/MigrationTask.java | 17 ++ .../tomcat/jakartaee/LocalStrings.properties | 12 +- 6 files changed, 338 insertions(+), 9 deletions(-) diff --git a/CHANGES.md b/CHANGES.md index 9b23746..4c6b15d 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -22,7 +22,8 @@ - Ensure that all the Manifest attributes are processed during the migration process. (markt) - Include `.properties` and `.json` files in the conversion process. (markt) - Replace `-verbose` with `-logLevel=` to provide more control over logging level. (markt) -- Fix [#13](https://github.com/apache/tomcat-jakartaee-migration/issues/13). Refactor mapping of log messages to log levels. (markt) +- Fix [#13](https://github.com/apache/tomcat-jakartaee-migration/issues/13). Refactor mapping of log messages to log levels. (markt) +- Fix [#3](https://github.com/apache/tomcat-jakartaee-migration/issues/3). Add support for excluding files from conversion. (markt) ## 0.1.0 diff --git a/src/main/java/org/apache/tomcat/jakartaee/GlobMatcher.java b/src/main/java/org/apache/tomcat/jakartaee/GlobMatcher.java new file mode 100644 index 000..33dc338 --- /dev/null +++ b/src/main/java/org/apache/tomcat/jakartaee/GlobMatcher.java @@ -0,0 +1,243 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + +package org.apache.tomcat.jakartaee; + +import java.util.Set; + +/** + * This is a utility class to match file globs. + * The class has been derived from + * org.apache.tools.ant.types.selectors.SelectorUtils. + * + * All methods are static. + */ +public final class GlobMatcher { + +/** + * Tests whether or not a given file name matches any file name pattern in + * the given set. The match is performed case-sensitively. + * + * @see #match(String, String, boolean) + * + * @param patternSet The pattern set to match against. Must not be + *null. + * @param fileName The file name to match, as a String. Must not be + *null. It must be just a file name, without + *a path. + * @param caseSensitive Whether or not matching should be performed + *case sensitively. + * + * @return true if any pattern in the set matches against the + * file name, or false otherwise. + */ +public static boolean matchName(Set patternSet, String fileName, boolean caseSensitive) { +char[] fileNameArray = fileName.toCharArray(); +for (String pattern: patternSet) { +if (match(pattern, fileNameArray, caseSensitive)) { +return true; +} +} +return false; +} + + +/** + * Tests whether or not a string matches against a pattern. + * The pattern may contain two special characters: + * '*' means zero or more characters + * '?' means one and only one character + * + * @param pattern The pattern to match against. + *Must not be null. + * @param str The string which must be matched against the + *pattern. Must not be null. + * @param caseSensitive Whether or not matching should be performed + *case sensitively. + * + * + * @return true if the string matches against the pattern, + * or false otherwise. + */ +public static boolean match(String pattern, String str, +boolean caseSensitive) { + +return match(pattern, str.toCharArray(), caseSensitive); +} + + +/** + * Tests whether or not a
[tomcat-jakartaee-migration] 01/07: Replace `-verbose` with `-logLevel=` to provide more control
This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/tomcat-jakartaee-migration.git commit c09f8b4a1620643c40018cb54730cc06176c35c2 Author: Mark Thomas AuthorDate: Tue Feb 9 12:00:36 2021 + Replace `-verbose` with `-logLevel=` to provide more control --- CHANGES.md | 1 + .../org/apache/tomcat/jakartaee/MigrationCLI.java | 103 +++-- .../tomcat/jakartaee/LocalStrings.properties | 16 +++- 3 files changed, 70 insertions(+), 50 deletions(-) diff --git a/CHANGES.md b/CHANGES.md index af74bb7..99d84e1 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -21,6 +21,7 @@ - Include the Maven Wrapper source files in the source distribution. (markt) - Ensure that all the Manifest attributes are processed during the migration process. (markt) - Include `.properties` and `.json` files in the conversion process. (markt) +- Replace `-verbose` with `-logLevel=` to provide more control over logging level. (markt) ## 0.1.0 diff --git a/src/main/java/org/apache/tomcat/jakartaee/MigrationCLI.java b/src/main/java/org/apache/tomcat/jakartaee/MigrationCLI.java index 5c3bf4f..64af9b9 100644 --- a/src/main/java/org/apache/tomcat/jakartaee/MigrationCLI.java +++ b/src/main/java/org/apache/tomcat/jakartaee/MigrationCLI.java @@ -20,82 +20,87 @@ import java.io.File; import java.io.IOException; import java.util.ArrayList; import java.util.Arrays; +import java.util.Iterator; import java.util.List; +import java.util.Locale; import java.util.logging.Level; import java.util.logging.Logger; public class MigrationCLI { -private static final Logger logger = Logger.getLogger(MigrationCLI.class.getCanonicalName()); private static final StringManager sm = StringManager.getManager(MigrationCLI.class); +private static final String LOGLEVEL_ARG = "-logLevel="; private static final String PROFILE_ARG = "-profile="; - +// Will be removed for 1.0.0 +@Deprecated +private static final String VERBOSE_ARG = "-verbose"; private static final String ZIPINMEMORY_ARG = "-zipInMemory"; -public static void main(String[] args) { -System.setProperty("java.util.logging.SimpleFormatter.format", "%5$s%n %6$s%n"); - -List arguments = new ArrayList<>(Arrays.asList(args)); -if (arguments.contains("-verbose")) { - Logger.getGlobal().getParent().getHandlers()[0].setLevel(Level.FINE); -Logger.getGlobal().getParent().setLevel(Level.FINE); -arguments.remove("-verbose"); -} - -boolean zipInMemory = false; -if (arguments.contains(ZIPINMEMORY_ARG)) { -zipInMemory = true; -arguments.remove(ZIPINMEMORY_ARG); -} +public static void main(String[] args) throws IOException { +// Defaults +System.setProperty("java.util.logging.SimpleFormatter.format", "%5$s%n"); Migration migration = new Migration(); -migration.setZipInMemory(zipInMemory); -boolean valid = false; -String source = null; -String dest = null; -if (arguments.size() == 3) { -if (arguments.get(0).startsWith(PROFILE_ARG)) { -source = arguments.get(1); -dest = arguments.get(2); -valid = true; +// Process argumnets +List arguments = new ArrayList<>(Arrays.asList(args)); + +// Process the custom log level if present +// Use an iterator so we can remove the log level argument if found +Iterator iter = arguments.iterator(); +while (iter.hasNext()) { +String argument = iter.next(); +if (argument.startsWith(LOGLEVEL_ARG)) { +iter.remove(); +String logLevelName = argument.substring(LOGLEVEL_ARG.length()); +Level level = null; try { - migration.setEESpecProfile(EESpecProfile.valueOf(arguments.get(0).substring(PROFILE_ARG.length(; +level = Level.parse(logLevelName.toUpperCase(Locale.ENGLISH)); +} catch (IllegalArgumentException iae) { +invalidArguments(); +} +// Configure the explicit level + Logger.getGlobal().getParent().getHandlers()[0].setLevel(level); +Logger.getGlobal().getParent().setLevel(level); +} else if (argument.startsWith(PROFILE_ARG)) { +iter.remove(); +String profileName = argument.substring(PROFILE_ARG.length()); +try { +EESpecProfile profile = EESpecProfile.valueOf(profileName.toUpperCase(Locale.ENGLISH)); +migration.setEESpecProfile(profile); } catch (IllegalArgumentException e) { // Invalid profile value -
[tomcat-jakartaee-migration] 02/07: Add Javadoc
This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/tomcat-jakartaee-migration.git commit d8373f2e132434e4e8b1aec9fed30f599e6d0eb8 Author: Mark Thomas AuthorDate: Tue Feb 9 14:24:54 2021 + Add Javadoc --- src/main/java/org/apache/tomcat/jakartaee/Converter.java | 10 ++ 1 file changed, 10 insertions(+) diff --git a/src/main/java/org/apache/tomcat/jakartaee/Converter.java b/src/main/java/org/apache/tomcat/jakartaee/Converter.java index 92ef1dd..5a62b5c 100644 --- a/src/main/java/org/apache/tomcat/jakartaee/Converter.java +++ b/src/main/java/org/apache/tomcat/jakartaee/Converter.java @@ -24,5 +24,15 @@ public interface Converter { boolean accepts(String filename); +/** + * Copies the source to the destination, converting it if necessary, + * according to the requirements of the given profile. + * + * @param src The source data to convert + * @param dest The destination to write the converted data + * @param profile The profile that defines the conversion required + * + * @throws IOException If the conversion fails + */ void convert(InputStream src, OutputStream dest, EESpecProfile profile) throws IOException; } - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat-jakartaee-migration] 05/07: Track whether source was modified or not.
This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/tomcat-jakartaee-migration.git commit 0a09f522ac65a4ae587e0c69e273fddd77ebdfdb Author: Mark Thomas AuthorDate: Tue Feb 9 14:35:08 2021 + Track whether source was modified or not. --- .../apache/tomcat/jakartaee/ManifestConverter.java | 32 -- 1 file changed, 24 insertions(+), 8 deletions(-) diff --git a/src/main/java/org/apache/tomcat/jakartaee/ManifestConverter.java b/src/main/java/org/apache/tomcat/jakartaee/ManifestConverter.java index 3a89ae2..e177217 100644 --- a/src/main/java/org/apache/tomcat/jakartaee/ManifestConverter.java +++ b/src/main/java/org/apache/tomcat/jakartaee/ManifestConverter.java @@ -52,10 +52,16 @@ public class ManifestConverter implements Converter { Manifest srcManifest = new Manifest(src); Manifest destManifest = new Manifest(srcManifest); -removeSignatures(destManifest); -updateValues(destManifest, profile); +boolean result = false; -destManifest.write(dest); +result = result | removeSignatures(destManifest); +result = result | updateValues(destManifest, profile); + +if (result) { +destManifest.write(dest); +} else { +srcManifest.write(dest); +} } @@ -90,23 +96,33 @@ public class ManifestConverter implements Converter { } -private void updateValues(Manifest manifest, EESpecProfile profile) { -updateValues(manifest.getMainAttributes(), profile); +private boolean updateValues(Manifest manifest, EESpecProfile profile) { +boolean result = false; +result = result | updateValues(manifest.getMainAttributes(), profile); for (Attributes attributes : manifest.getEntries().values()) { -updateValues(attributes, profile); +result = result | updateValues(attributes, profile); } +return result; } -private void updateValues(Attributes attributes, EESpecProfile profile) { +private boolean updateValues(Attributes attributes, EESpecProfile profile) { +boolean result = false; // Update version info if (attributes.containsKey(Attributes.Name.IMPLEMENTATION_VERSION)) { String newValue = attributes.get(Attributes.Name.IMPLEMENTATION_VERSION) + "-" + Info.getVersion(); attributes.put(Attributes.Name.IMPLEMENTATION_VERSION, newValue); +result = true; } // Update package names in values for (Entry entry : attributes.entrySet()) { -entry.setValue(profile.convert((String) entry.getValue())); +String newValue = profile.convert((String) entry.getValue()); +// Object comparison is deliberate +if (newValue != entry.getValue()) { +entry.setValue(newValue); +result = true; +} } +return result; } } - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat-jakartaee-migration] 06/07: Fix #13 refactor logging
This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/tomcat-jakartaee-migration.git commit 066f71df6bcd647b26af96079235861a035032cb Author: Mark Thomas AuthorDate: Tue Feb 9 15:18:48 2021 + Fix #13 refactor logging INFO - Lists archives and dropped signature files FINE - INFO plus modified files and dropped manifets entries FINEST - Every file --- CHANGES.md | 1 + .../apache/tomcat/jakartaee/ClassConverter.java| 22 +++-- .../org/apache/tomcat/jakartaee/Converter.java | 3 +- .../apache/tomcat/jakartaee/ManifestConverter.java | 2 +- .../org/apache/tomcat/jakartaee/Migration.java | 55 +- .../tomcat/jakartaee/PassThroughConverter.java | 10 +++- .../org/apache/tomcat/jakartaee/TextConverter.java | 18 ++- .../tomcat/jakartaee/LocalStrings.properties | 13 +++-- .../org/apache/tomcat/jakartaee/MigrationTest.java | 3 +- .../tomcat/jakartaee/PassThroughConverterTest.java | 6 ++- .../apache/tomcat/jakartaee/TextConverterTest.java | 4 +- 11 files changed, 88 insertions(+), 49 deletions(-) diff --git a/CHANGES.md b/CHANGES.md index 99d84e1..9b23746 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -22,6 +22,7 @@ - Ensure that all the Manifest attributes are processed during the migration process. (markt) - Include `.properties` and `.json` files in the conversion process. (markt) - Replace `-verbose` with `-logLevel=` to provide more control over logging level. (markt) +- Fix [#13](https://github.com/apache/tomcat-jakartaee-migration/issues/13). Refactor mapping of log messages to log levels. (markt) ## 0.1.0 diff --git a/src/main/java/org/apache/tomcat/jakartaee/ClassConverter.java b/src/main/java/org/apache/tomcat/jakartaee/ClassConverter.java index cc06bde..173ea8e 100644 --- a/src/main/java/org/apache/tomcat/jakartaee/ClassConverter.java +++ b/src/main/java/org/apache/tomcat/jakartaee/ClassConverter.java @@ -19,6 +19,8 @@ package org.apache.tomcat.jakartaee; import java.io.IOException; import java.io.InputStream; import java.io.OutputStream; +import java.util.logging.Level; +import java.util.logging.Logger; import org.apache.bcel.classfile.ClassParser; import org.apache.bcel.classfile.Constant; @@ -27,6 +29,9 @@ import org.apache.bcel.classfile.JavaClass; public class ClassConverter implements Converter { +private static final Logger logger = Logger.getLogger(ClassConverter.class.getCanonicalName()); +private static final StringManager sm = StringManager.getManager(ClassConverter.class); + @Override public boolean accepts(String filename) { String extension = Util.getExtension(filename); @@ -35,26 +40,37 @@ public class ClassConverter implements Converter { @Override -public void convert(InputStream src, OutputStream dest, EESpecProfile profile) throws IOException { +public void convert(String path, InputStream src, OutputStream dest, EESpecProfile profile) throws IOException { ClassParser parser = new ClassParser(src, "unknown"); JavaClass javaClass = parser.parse(); +boolean converted = false; + // Loop through constant pool Constant[] constantPool = javaClass.getConstantPool().getConstantPool(); for (short i = 0; i < constantPool.length; i++) { if (constantPool[i] instanceof ConstantUtf8) { ConstantUtf8 c = (ConstantUtf8) constantPool[i]; String str = c.getBytes(); -String converted = profile.convert(str); +String newString = profile.convert(str); // Object comparison is deliberate -if (converted != str) { +if (newString != str) { c = new ConstantUtf8(profile.convert(str)); constantPool[i] = c; +converted = true; } } } +if (logger.isLoggable(Level.FINE)) { +if (converted) { +logger.log(Level.FINE, sm.getString("classConverter.converted", path)); +} else if (logger.isLoggable(Level.FINEST)) { +logger.log(Level.FINEST, sm.getString("classConverter.noConversion", path)); +} +} + javaClass.dump(dest); } } diff --git a/src/main/java/org/apache/tomcat/jakartaee/Converter.java b/src/main/java/org/apache/tomcat/jakartaee/Converter.java index 5a62b5c..e05c232 100644 --- a/src/main/java/org/apache/tomcat/jakartaee/Converter.java +++ b/src/main/java/org/apache/tomcat/jakartaee/Converter.java @@ -28,11 +28,12 @@ public interface Converter { * Copies the source to the destination, converting it if necessary, * according to the requirements of the given profile. * + * @param path The path to the data being conver
[tomcat-jakartaee-migration] 04/07: Simplify error handling
This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/tomcat-jakartaee-migration.git commit 9acbf5e8089114a8ea3a5118ca4b64beeb5d5b14 Author: Mark Thomas AuthorDate: Tue Feb 9 14:34:49 2021 + Simplify error handling --- .../java/org/apache/tomcat/jakartaee/Migration.java| 18 +++--- .../org/apache/tomcat/jakartaee/MigrationTask.java | 7 +-- .../apache/tomcat/jakartaee/LocalStrings.properties| 2 +- 3 files changed, 9 insertions(+), 18 deletions(-) diff --git a/src/main/java/org/apache/tomcat/jakartaee/Migration.java b/src/main/java/org/apache/tomcat/jakartaee/Migration.java index c6a6371..e970a08 100644 --- a/src/main/java/org/apache/tomcat/jakartaee/Migration.java +++ b/src/main/java/org/apache/tomcat/jakartaee/Migration.java @@ -103,32 +103,28 @@ public class Migration { } -public boolean execute() throws IOException { +public void execute() throws IOException { logger.log(Level.INFO, sm.getString("migration.execute", source.getAbsolutePath(), destination.getAbsolutePath(), profile.toString())); -boolean result = true; + long t1 = System.nanoTime(); if (source.isDirectory()) { if ((destination.exists() && destination.isDirectory()) || destination.mkdirs()) { -result = result && migrateDirectory(source, destination); +migrateDirectory(source, destination); } else { -logger.log(Level.WARNING, sm.getString("migration.mkdirError", destination.getAbsolutePath())); -result = false; +throw new IOException(sm.getString("migration.mkdirError", destination.getAbsolutePath())); } } else { // Single file File parentDestination = destination.getAbsoluteFile().getParentFile(); if (parentDestination.exists() || parentDestination.mkdirs()) { -result = result && migrateFile(source, destination); +migrateFile(source, destination); } else { -logger.log(Level.WARNING, sm.getString("migration.mkdirError", parentDestination.getAbsolutePath())); -result = false; +throw new IOException(sm.getString("migration.mkdirError", parentDestination.getAbsolutePath())); } } logger.log(Level.INFO, sm.getString("migration.done", -Long.valueOf(TimeUnit.MILLISECONDS.convert(System.nanoTime() - t1, TimeUnit.NANOSECONDS)), -Boolean.valueOf(result))); -return result; +Long.valueOf(TimeUnit.MILLISECONDS.convert(System.nanoTime() - t1, TimeUnit.NANOSECONDS; } diff --git a/src/main/java/org/apache/tomcat/jakartaee/MigrationTask.java b/src/main/java/org/apache/tomcat/jakartaee/MigrationTask.java index 6bdda1d..c19599e 100644 --- a/src/main/java/org/apache/tomcat/jakartaee/MigrationTask.java +++ b/src/main/java/org/apache/tomcat/jakartaee/MigrationTask.java @@ -74,15 +74,10 @@ public class MigrationTask extends Task { migration.setEESpecProfile(profile); migration.setZipInMemory(zipInMemory); -boolean success = false; try { -success = migration.execute(); +migration.execute(); } catch (IOException e) { throw new BuildException(e, getLocation()); } - -if (!success) { -throw new BuildException("Migration failed", getLocation()); -} } } diff --git a/src/main/resources/org/apache/tomcat/jakartaee/LocalStrings.properties b/src/main/resources/org/apache/tomcat/jakartaee/LocalStrings.properties index d3274e4..d8150c0 100644 --- a/src/main/resources/org/apache/tomcat/jakartaee/LocalStrings.properties +++ b/src/main/resources/org/apache/tomcat/jakartaee/LocalStrings.properties @@ -16,7 +16,7 @@ migration.archive=Migrating archive [{0}] migration.archiveFailed=Failed to migrate archive [{0}]. Using the "-zipInMemory" option may help. migration.cannotReadSource=Cannot read source location [{0}] -migration.done=Migration completed successfully [{1}] in [{0}] milliseconds +migration.done=Migration completed successfully in [{0}] milliseconds migration.entry=Migrating Jar entry [{0}] migration.error=Error performing migration migration.execute=Performing migration from source [{0}] to destination [{1}] with Jakarta EE specification profile [{2}] - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[GitHub] [tomcat-jakartaee-migration] markt-asf closed issue #13: List modified files
markt-asf closed issue #13: URL: https://github.com/apache/tomcat-jakartaee-migration/issues/13 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[tomcat-jakartaee-migration] 03/07: Reduce object creation during conversion
This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/tomcat-jakartaee-migration.git commit 4a0b09d3e43dfcb35c5a7488a222b1c7ea941669 Author: Mark Thomas AuthorDate: Tue Feb 9 14:25:17 2021 + Reduce object creation during conversion --- src/main/java/org/apache/tomcat/jakartaee/ClassConverter.java | 8 ++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/src/main/java/org/apache/tomcat/jakartaee/ClassConverter.java b/src/main/java/org/apache/tomcat/jakartaee/ClassConverter.java index 9d398d7..cc06bde 100644 --- a/src/main/java/org/apache/tomcat/jakartaee/ClassConverter.java +++ b/src/main/java/org/apache/tomcat/jakartaee/ClassConverter.java @@ -46,8 +46,12 @@ public class ClassConverter implements Converter { if (constantPool[i] instanceof ConstantUtf8) { ConstantUtf8 c = (ConstantUtf8) constantPool[i]; String str = c.getBytes(); -c = new ConstantUtf8(profile.convert(str)); -constantPool[i] = c; +String converted = profile.convert(str); +// Object comparison is deliberate +if (converted != str) { +c = new ConstantUtf8(profile.convert(str)); +constantPool[i] = c; +} } } - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[GitHub] [tomcat-jakartaee-migration] markt-asf closed issue #3: Removing signature from JAR file with JCE providers leads to SecurityException
markt-asf closed issue #3: URL: https://github.com/apache/tomcat-jakartaee-migration/issues/3 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Releasing Tomcat migration tool for Jakarta EE 0.2.0
All, With the various fixes and changes, I think we need another release. I also want to add a basic documentation page to the web site as well as linking to the download page. Please speak up and/or open an issue if there is something you think needs fixing before I tag. Thanks, Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Integrating migration tool into Tomcat 10
Hi all, I've been looking at the options to integrate the Java EE to Jakarta EE migration functionality into Tomcat 10. There are essentially two ways to do this: deployment time and runtime. The simplest solution to implement is probably a separate webapps-legacy directory (or some other name) where WARs and DIRs added to that directory get converted to Jakarta EE with the new version being placed in the webapps directory. There are complexities (such as handling an updates of the legacy WAR) but they should be manageable. A more complex version of the deploy time solution has the legacy web application placed in webapps, identified as a legacy webapp and then replaced with the new version (several ways to do this). The hard part here is the identification of the webapp as a Java EE app. The only reliable way to do this is class scanning and that is slow. The runtime approach converts classes and static resources as they are loaded. The classes are relatively easy to handle. A ClassFileTransformer can be added to the WebappClassLoader to do this. The static files are more of a problem. The good news is that the WebResources refactoring means static file access all goes through the same code but the filtering required essentially means we'd need to load static files into memory - regardless of size, convert them, and then update the cache with the converted version. That is likely to have a performance impact. Because of the performance impacts of handling static resources, the runtime approach also needs a way to identify applications that need conversion. I don't see a reliable, performant way to do that. Which, I think, leaves us with the simple deployment time approach and something (filename, special directory, something else) to mark an app as needing conversion. A final point, which probably should have been first, is is there a demand for this feature? Early indications from the users list and $work is that there is (going to be) a demand for this feature. Thoughts? Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Integrating migration tool into Tomcat 10
On Tue, Feb 9, 2021 at 10:12 PM Mark Thomas wrote: > Hi all, > > I've been looking at the options to integrate the Java EE to Jakarta EE > migration functionality into Tomcat 10. > > There are essentially two ways to do this: deployment time and runtime. > > The simplest solution to implement is probably a separate webapps-legacy > directory (or some other name) where WARs and DIRs added to that > directory get converted to Jakarta EE with the new version being placed > in the webapps directory. There are complexities (such as handling an > updates of the legacy WAR) but they should be manageable. > > A more complex version of the deploy time solution has the legacy web > application placed in webapps, identified as a legacy webapp and then > replaced with the new version (several ways to do this). The hard part > here is the identification of the webapp as a Java EE app. The only > reliable way to do this is class scanning and that is slow. > > The runtime approach converts classes and static resources as they are > loaded. The classes are relatively easy to handle. A > ClassFileTransformer can be added to the WebappClassLoader to do this. > The static files are more of a problem. The good news is that the > WebResources refactoring means static file access all goes through the > same code but the filtering required essentially means we'd need to load > static files into memory - regardless of size, convert them, and then > update the cache with the converted version. That is likely to have a > performance impact. > > Because of the performance impacts of handling static resources, the > runtime approach also needs a way to identify applications that need > conversion. I don't see a reliable, performant way to do that. Which, I > think, leaves us with the simple deployment time approach and something > (filename, special directory, something else) to mark an app as needing > conversion. > > A final point, which probably should have been first, is is there a > demand for this feature? Early indications from the users list and $work > is that there is (going to be) a demand for this feature. > > Thoughts? > I kind of proposed the "simplest" option [the one using a separate appBase for the EE 8 and earlier webapps] a couple times. A slightly more complex deploy time option could look more polished and maybe preferable, like that "filename" idea maybe ? Or a marker file ? I'm not sure. There's a demand for the feature for sure, but probably only up to the point people realize it's not actually that helpful. This adds a step to deployment which may fail. The offline tool however is more efficient and safer. Also the tool now has quite a few "advanced" options [the ones you just added] for migrating trickier webapps, so how is an automatic migration with defaults going to work out ? This looks like asking people to fill up the BZ with stuff. So assuming the feature goes in, maybe a hybrid solution could work better than a pure runtime or deploy time solution. The classloader classes and resources could rely on a runtime conversion [a bit costly but probably safer], while the static resources could be converted at deployment time [it's hard to give decent guarantees on resource use if done at runtime]. Rémy > > Mark > > - > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > For additional commands, e-mail: dev-h...@tomcat.apache.org > >
Re: Integrating migration tool into Tomcat 10
On Tue, Feb 9, 2021 at 14:34 Rémy Maucherat wrote: > On Tue, Feb 9, 2021 at 10:12 PM Mark Thomas wrote: > > > Hi all, > > > > I've been looking at the options to integrate the Java EE to Jakarta EE > > migration functionality into Tomcat 10. > > > > There are essentially two ways to do this: deployment time and runtime. > > > > The simplest solution to implement is probably a separate webapps-legacy > > directory (or some other name) where WARs and DIRs added to that > > directory get converted to Jakarta EE with the new version being placed > > in the webapps directory. There are complexities (such as handling an > > updates of the legacy WAR) but they should be manageable. > > > > A more complex version of the deploy time solution has the legacy web > > application placed in webapps, identified as a legacy webapp and then > > replaced with the new version (several ways to do this). The hard part > > here is the identification of the webapp as a Java EE app. The only > > reliable way to do this is class scanning and that is slow. > > > > The runtime approach converts classes and static resources as they are > > loaded. The classes are relatively easy to handle. A > > ClassFileTransformer can be added to the WebappClassLoader to do this. > > The static files are more of a problem. The good news is that the > > WebResources refactoring means static file access all goes through the > > same code but the filtering required essentially means we'd need to load > > static files into memory - regardless of size, convert them, and then > > update the cache with the converted version. That is likely to have a > > performance impact. > > > > Because of the performance impacts of handling static resources, the > > runtime approach also needs a way to identify applications that need > > conversion. I don't see a reliable, performant way to do that. Which, I > > think, leaves us with the simple deployment time approach and something > > (filename, special directory, something else) to mark an app as needing > > conversion. > > > > A final point, which probably should have been first, is is there a > > demand for this feature? Early indications from the users list and $work > > is that there is (going to be) a demand for this feature. > > > > Thoughts? > > > > I kind of proposed the "simplest" option [the one using a separate appBase > for the EE 8 and earlier webapps] a couple times. A slightly more complex > deploy time option could look more polished and maybe preferable, like that > "filename" idea maybe ? Or a marker file ? I'm not sure. > > There's a demand for the feature for sure, but probably only up to the > point people realize it's not actually that helpful. This adds a step to > deployment which may fail. The offline tool however is more efficient and > safer. Also the tool now has quite a few "advanced" options [the ones you > just added] for migrating trickier webapps, so how is an automatic > migration with defaults going to work out ? This looks like asking people > to fill up the BZ with stuff. > > So assuming the feature goes in, maybe a hybrid solution could work better > than a pure runtime or deploy time solution. The classloader classes and > resources could rely on a runtime conversion [a bit costly but probably > safer], while the static resources could be converted at deployment time > [it's hard to give decent guarantees on resource use if done at runtime]. > > Rémy We’ve spent two decades optimizing the run time within the constraints that the Servlet spec has given to us. This was driven by the market, and accentuated when containerization became a thing. I would suggest that those optimization not be lost and that the extra time spent at startup, a highly repetitive operation these days, would be something the user would have to opt in to. Personally I like a complete offline model, yet there is immense value for organizations that have stale projects to be upgraded easily. (albeit, one can argue they can stay on v9 for a very long time too) > > > > > > Mark > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > > For additional commands, e-mail: dev-h...@tomcat.apache.org > > > > >
Re: Integrating migration tool into Tomcat 10
Le 2021-02-09 22:12, Mark Thomas a écrit : Thoughts? I think this feature is really desirable. For the Debian packaging this would mean a faster transition to Tomcat 10. There are two issues: 1. How to identify a legacy application? 2. How to convert the application once identified? For the identification, either: a. separate the Java EE and Jakarta EE webapps in different directories. It's simple but it's an additional decision point for the user and there will be mistakes. b. keep the webapps in the same directory and scan the content. This degrades the performances for those having switched to Jakarta EE, so this should be optional, and probably disabled by default. c. keep the webapps in the same directory, load them as usual and trigger the conversion mechanism only if a class loading error related to the javax namespace occurs. Regarding the conversion, we can: a. Convert at runtime with class loading tricks, which is quite elegant but the price is paid every time the application is started. b. Convert at deployment time before loading the application, this means more file juggling but only the first start is impacted. The convenience of the automatic migration comes obviously at a price but I think it's important we ensure it doesn't impact those ready to jump to the new Jakarta EE world. For this reason I'm leaning toward the solution 1c + 2b (convert on errors, and convert the webapp before reloading). The start sequence would look like this: 1. foo.war is copied to the webapps directory by the user 2. The foo application is loaded 3. A NoClassDefFoundError regarding the javax.servlet.Servlet class is thrown, the migration mechanism is triggered 4. The migration tool is executed on foo.war. The original file is renamed foo.war.legacy and the migrated file remains as foo.war 5. The application is reloaded And on subsequent starts, the migrated application is loaded directly, with no performance impact. Emmanuel Bourg - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org