Le 17 mars 2019 à 16:22, Romain Manni-Bucau <rmannibu...@gmail.com> a écrit :
> Le dim. 17 mars 2019 à 14:51, Gaël Lalire <gael.lal...@gaellalire.fr> a > écrit : > >> >> Le 17 mars 2019 à 13:21, Romain Manni-Bucau <rmannibu...@gmail.com> a >> écrit : >> >>> Le dim. 17 mars 2019 à 12:56, Gaël Lalire <gael.lal...@gaellalire.fr> a >>> écrit : >>> >>>> Hello Romain, >>>> >>>> I already explained why I do not want to give file or jar:file URL, even >>>> if I have it. >>>> Maven resolver is giving me File, I create a VestigeJar from it >>>> >> https://gaellalire.fr/gitlab/vestige/vestige.spi.resolver/blob/master/src/main/java/fr/gaellalire/vestige/spi/resolver/VestigeJar.java >>>> I also create a mvn: URL accessible through getCodeBase however it is >>>> only for policy permission you should not use openStream on it >> (although I >>>> had to make it possible to get BouncyCastle working). >>>> >>> >>> I get that but it also can conflict with other resolver using mvn:, open >>> new security issues and break some libs - speaking having dropped a lib >>> doing exactly that at work for all these reasons. >> >> mvn: was implemented two weeks before, it is only for ProtectionDomain. >> And ProtectionDomain is used by almost no library. >> VestigeClassloader send vrt: URL. These ones start with / and does not >> break libraries relying on context URL. >> It will break those expecting jar:file URL though but Java9 with its jrt: >> URL will also break them, so I don't care. >> > > Paxurl uses it for a long tile. Also jrt is different cause excluded from > most scanner - it is only the jvm - whereas your jars are not excluded and > scanning will fail. Check out spring, weld, openwebbeans etc... > I mean implemented in Vestige. I use a sub format of Paxurl (no repository, no version range). Also I could use vmvn: instead, as I said it's only for code base URL. Scanning in openwebbeans will not work because it is expecting an URLClassLoader or using java.class.path property. Java9 is not providing URLClassLoader ... these scanners must evolve. I see 2 types of scanning : - hinted one -> for example with beans.xml. These ones can be written to be working only using clean Java (context URL will do the job). - blind one -> searching for annotations in all jars. These ones have to hack the system to find all jars (there is no listAllJar in classloader), they should provide an API so its container can give it the jar list, and by jar list I think of something like (InputStream, JarName, JarSize) rather than a JarFile or URL. > > >>> >>> >>>> I think that SecureClassLoader is not secure enough because it only >> checks >>>> classes not resources. >>>> In a world using XML to create, configure and link instances (Spring), >> it >>>> is terrible to let resources unchecked. >>>> >>>> That's also why VestigeClassLoader#getResource is returning vrt: URL and >>>> not jar:file: URL. >>>> >>> >>> Classloaders generally check in their own calls, not in the resource >>> itself. Can' t you validate the resource before actually reasing it? >> Sounds >>> saner and closer to security manager common configs. >> >> Yes but it is not secure even with read lock. >> The read lock is on the file not the path. You can remove and even replace >> the file while the read lock is still here. >> So my check succeeds, I give a jar:file URL then the file is replaced, the >> jar:file is open and get the new file (hacked one). >> > > Sounds like you want to implement a jvm filesystem and not a new url > handler explained this way otherwise you still have this issue with other > concurrent accesses. > I have not yet implemented it. Concurrent accesses can be solved in multiple ways : - open a new RandomAccessFile each time and check the signature each time (probably the slowest way) - create a cache system - share the RandomAccessFile and synchronize read calls between each InputStream vrt format is vrt:/[ClassLoaderNumber]/[JarOfClassLoaderNumber]!/[PathOfFile] for example vrt:/56/3!/META-INF/MANIFEST.MF I don't see it as a file system. > >>> >>> >>>> The easiest way to implements War Maven artifacts deployment is to use >>>> directly Maven resolver API and give file or jar:file URL to Tomcat. >>>> >>> >>> Or war: since tomcat supports it. >> >> Of course >> >>> >>> >>> >>>> I could have done that, but guess what, I'm willing to give TomEE EAR >>>> Maven artifacts deployment after Tomcat. >>>> In the TomEE my company uses there is a horrible CLASSPATH because they >>>> wanted to avoid RMI call between EAR. >>>> That is where Vestige is useful, it allows to choose which jar(s) should >>>> be shared without any global impact. >>>> >>> >>> Hmm, you use a tomee? Did you check jars.txt? Can be worth pinging tomee >>> about it cause all is there to do it afaik. >> >> Thanks for the pointer, I will check it because I really don't like global >> CLASSPATH. >> For my business case it will almost certainly work, however I'm pretty >> sure it has not the power of Vestige. >> I don't know if you chose to create one classloader grouping all jars in >> jar.txt in which case you will have issues when a library is deployed in 2 >> different versions. >> Or if you chose to create one classloader per entry in jar.txt in which >> case the jar will be missing its dependencies unless you expect them to be >> in MANIFEST.MF. >> >> In Maven repositories Class-Path is generally not set in MANIFEST.MF. >> > > > Jars.txt is just a way to add jars in the classloader it is present in - > webapp or ear lib one - without putting the files physically there. It is > close to old configurable tolcat classloader and new webresource > abstraction which can do it as well. So because two EAR don't share their classloader you will have ClassCastException (even if it is the same file). I will have to keep CLASSPATH variable. > > >>> >>> >>>> You have 3 scopes and 2 modes in Vestige. >>>> Mode CLASSPATH is creating one classloader with all jars inside, not >> easy >>>> to share. >>>> Mode FIXED_DEPENDENCIES is creating one classloader per jar, but implies >>>> some good practice (use of context classloader for example). >>>> >>>> After setting FIXED_DEPENDENCIES you have to use the PLATFORM scope to >> get >>>> it shared, this scope implies some other good practice (static field >>>> immutable for example). >>>> >>>> About memory issue, I don't get your point. >>>> I will not keep all jars content in memory I will use shared locks >> (fcntl) >>>> to keep the content of RandomAccessFile the way it was when I checked >> it. >>>> VestigeJar#open will create an InputStream reading from this locked >>>> RandomAccessFile. >>>> >>> >>> Oki but then you are in jar:file mode ;) >> >> In a way, but I give only the InputStream not the URL so the >> JarURLConnection of JarScannerCallback can't be fetched. >> > > > You can make it work replacing the whole scanner, we do it in tomee and > openwebbeans, mainly to change the scanner impl to xbean, but i wouldnt > chabge urls themselves cause it makes the runtile less predictable until it > is for a single stack - dependencies - you will not change. > Yes I could replace the whole scanner, but I will have to do this for each new Tomcat version. If I do it, I would prefer to have it committed in Tomcat sources. I didn't get the second part, obviously in TomEE you will not convert URL to another format, you already have a jar:file URL. If I didn't care about security, I would have keep jar:file URL in classloader#getResource. Using file URL in VestigeJarFile#getCodeBase does not change anything about security but your policy file becomes $HOME/.m2/repository dependent. > >>> >>> >>>> Regards, >>>> Gaël Lalire >>>> >>>> Le 17 mars 2019 à 10:17, Romain Manni-Bucau <rmannibu...@gmail.com> a >>>> écrit : >>>> >>>> Hi Gaël, >>>> >>>> In Tomee we plugged before to enrich the classloader and then tomcat >> -and >>>> other libs - works normally using jar urls. >>>> >>>> Cant you use a listener to do that and convert m2 urls to plain jar >> files - >>>> at the end it is local files i guess otherwise you generally consume too >>>> much memory to be prod friendly? >>>> >>>> >>>> Le dim. 17 mars 2019 à 09:46, Gaël Lalire <gael.lal...@gaellalire.fr> a >>>> écrit : >>>> >>>> Hello Tomcat developers, >>>> >>>> I made a software to enable update of Java applications named Vestige. >>>> To achieve that, Vestige use Maven, downloading Maven artifacts and >>>> creating classloaders linked with jar inside m2 repository. >>>> >>>> I made it to update my IBM notes connector (POP access provider). >>>> >>>> The fact it is downloading Maven artifacts makes the assembly >>>> (jar-with-dependencies of maven-assembly-plugin) of the connector not >>>> mandatory. >>>> >>>> In a business project I saw that war artifacts were filling the >>>> repository, so they had to regularly remove older version from it. >>>> I thought it would be great if we could remove the WEB-INF/lib from the >>>> published war and still be able to deploy it with Tomcat. >>>> >>>> I did that, the WebResource API helps me a lot. >>>> However I had to disable JarScanner API (tld & fragments) because it's >>>> using JarURLConnection and my API is not providing jar:file: nor file: >> URL. >>>> My API won't provide them because I want to be able to check a pgp >>>> signature before any use of an artifact in m2 repository. >>>> If I check the signature and send a jar:file: or file: URL it won't be >>>> secure because there is no way to prevent the modification of the file >>>> after the check. >>>> To be secure I will probably lock the file for reading, then check the >>>> signature, and give locked InputStream. >>>> >>>> I would like you to change the JarScanner API/Impl so it won't rely on >>>> JarURLConnection anymore (maybe WebResource ?). >>>> Also I have to replace some Tomcat classes ( >>>> >>>> >> https://gaellalire.fr/gitlab/vestige_app/tomcat_vestige/commit/67dea6054c9da30047ebba3e9a376fa44b544f13 >>>> ) >>>> that is not future proof. >>>> Could you provide some extension(s) so I could do the same thing without >>>> replacing any Tomcat class ? >>>> >>>> Hoping that you get interested enough to help me improve the Maven >>>> artifact deployment support, I send you my best regards. >>>> >>>> PS: >>>> You can test the vwar, an xml which describes the war to deploy >>>> (essentially repository URL, groupId, artifactId, version), deployment >> by : >>>> - download (https://gaellalire.fr/vestige/) & install & run Vestige >>>> - go to http://localhost:8480/ >>>> - click on install >>>> - write "tomcat" in repository application name >>>> - write "8.0.32" in repository application version >>>> - write "tc" in local application name >>>> - click install button >>>> - click play button >>>> - go to http://localhost:8080/mywar/hello (servlet test) and >>>> http://localhost:8080/mywar/hi.jsp?max=5 (jsp test) >>>> >>>> The vwar will be at $VESTIGE_BASE/app/tc/webapps/mywar.vwar >>>> Where $VESTIGE_BASE is : >>>> - $HOME/Vestige on Mac OS X >>>> - $HOME/vestige on Linux >>>> - %userprofile%\Vestige on Windows >>>> - the place you unzip the file if you chose to install the standalone >>>> version (a ZIP file) >>>> >>>> You can also see it at >>>> >>>> >> https://gaellalire.fr/gitlab/vestige_app/tomcat_vestige/blob/master/installer/src/main/resources/mywar.vwar >>>> >>>> tomcat_vestige sources at >>>> https://gaellalire.fr/gitlab/vestige_app/tomcat_vestige >>>> tomcat_vestige descriptor at >>>> https://gaellalire.fr/vestige/repository/tomcat/tomcat-8.0.32.xml >>>> mywar sources at https://gaellalire.fr/gitlab/vestige_app/mywar (its >> pom >>>> https://gaellalire.fr/gitlab/vestige_app/mywar/blob/master/pom.xml >>>> excludes lib folder) >>>> >>>> >>>> >>>> >> >>
signature.asc
Description: Message signed with OpenPGP using GPGMail
