Re: [Crypto] requesting help testing native binaries

I will proceed with an RC today! :-) Gary On Fri, Aug 28, 2020 at 10:42 AM Gary Gregory wrote: > Thanks Geoffrey, updating our tally: > > Here is what community testing we have so far for the Crypto.main() smoke > test: > > - darwin64-x86_64-cc; OpenSSL 1.1.1g; Gary Gregory, Alex Remily > - deb

Re: [Crypto] requesting help testing native binaries

Thanks Geoffrey, updating our tally: Here is what community testing we have so far for the Crypto.main() smoke test: - darwin64-x86_64-cc; OpenSSL 1.1.1g; Gary Gregory, Alex Remily - debian-amd64; OpenSSL 1.0.1f; Gary Gregory - debian-amd64; OpenSSL 1.1.1g; Bruno P. Kinoshita - Linux x86_64; Open

Re: [Crypto] requesting help testing native binaries

Hi all, For the simple smoke test, on AArch64 for AmazonLinux2 (OpenSSL 1.0.2k) and Ubuntu 20.04 (OpenSSL 1.1.1f) everything loads ok with the current Jar that Gary posted. -Geoff AL2 output: java -cp commons-crypto-1.1.0-20200824.190246-21.jar org.apache.commons.crypto.Crypto Apache Commons Cry

Re: [Crypto] requesting help testing native binaries

For a library with as many vulnerabilities as OpenSSL, I’m surprised macOS keeps such an ancient version! It’s not like they ship a trimmed down and audited version of LibreSSL, either. On Thu, Aug 27, 2020 at 20:19 Gary Gregory wrote: > The issue for me is that it was a PITA to override macos'

Re: [Crypto] requesting help testing native binaries

The issue for me is that it was a PITA to override macos' baked in (ancient) LibreSSL. Gary On Thu, Aug 27, 2020, 20:03 Alex Remily wrote: > Interesting. If I understand correctly, you did get it to run > successfully to completion, but only after placing a compatible > libcrypto in the direct

Re: [Crypto] requesting help testing native binaries

Interesting. If I understand correctly, you did get it to run successfully to completion, but only after placing a compatible libcrypto in the directory of execution, probably the first place dlopen looks for it. Would you agree then that the error was caused by loading an incompatible libcrypto?

Re: [Crypto] requesting help testing native binaries

Thanks Alex, updating tally: Here is what community testing we have so far for the Crypto.main() smoke test: - darwin64-x86_64-cc; OpenSSL 1.1.1g; Gary Gregory, Alex Remily - debian-amd64; OpenSSL 1.0.1f; Gary Gregory - debian-amd64; OpenSSL 1.1.1g; Bruno P. Kinoshita - Linux x86_64; OpenSSL 1.1.

Re: [Crypto] requesting help testing native binaries

Gary, You can add the Windows 64 build. Unit tests passed as expected and main function output is: Apache Commons Crypto 1.1.0-SNAPSHOT Native code loaded OK: 1.1.0-SNAPSHOT Native name: Apache Commons Crypto Native built: Aug 16 2020 OpenSSL library loaded OK, version: 0x1010104f OpenSSL librar

Re: [Crypto] requesting help testing native binaries

Here is what community testing we have so far for the Crypto.main() smoke test: - darwin64-x86_64-cc; OpenSSL 1.1.1g; Gary Gregory, Alex Remily - debian-amd64; OpenSSL 1.0.1f; Gary Gregory - debian-amd64; OpenSSL 1.1.1g; Bruno P. Kinoshita - Linux x86_64; OpenSSL 1.1.1; Alex Remily Gary On Sat,

Re: [Crypto] requesting help testing native binaries

On Mon, Aug 24, 2020 at 7:28 PM Alex Remily wrote: > Gary, > > Can you check that your libcrypto.dylib is symlinked to the libcrypto > for OpenSSL 1.1.1.g? Mine wasn't, and I was getting different output > from the main function than from the unit test output. I'm not > confident that this is t

Re: [Crypto] requesting help testing native binaries

On Wed, Aug 26, 2020 at 11:00 PM Gary Gregory wrote: > Part 2, trying to get macos to behave: > > echo $PATH > /opt/apache-maven-3.6.3/bin:/usr/local/opt/openssl@1.1 > /bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin > > *export LD_LIBRARY_PATH=/usr/local/Cellar/openssl@1.1/1.1.1g/lib/* > *java

Re: [Crypto] requesting help testing native binaries

Part 2, trying to get macos to behave: echo $PATH /opt/apache-maven-3.6.3/bin:/usr/local/opt/openssl@1.1 /bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin *export LD_LIBRARY_PATH=/usr/local/Cellar/openssl@1.1/1.1.1g/lib/* *java -cp target/classes org.apache.commons.crypto.Crypto* Apache Commons C

Re: [Crypto] requesting help testing native binaries

Alex, All: Thank you for your review Alex. I agree that LibreSSL could be out of scope here. But, FTR, here is what I found: LibreSSL 2.8.3 does not seem to implement ENGINE_load_rdrand. https://www.libressl.org/ points to https://github.com/libressl-portable/openbsd and looking for ENGINE_load_

Re: [Crypto] requesting help testing native binaries

Gary, I tested the most recent jar against Mac x86_64 and Linux x86_64 OpenSSL version 1.1.1. The unit tests passed and the main function ran to completion with expected output. I'll try to get to the Windows x86_64 test tomorrow and the OpenSSL 1.0 tests on the same three architectures by the e

Re: [Crypto] requesting help testing native binaries

>From my end, that latest jar from 8/24 is working on AArch64 with a simple test on OpenSSL 1.1.x and OpenSSL 1.0.x based distros. I'm now trying it out on Spark to see if it links up with OpenSSL instead of the JCE. -Geoff On Tue, Aug 25, 2020 at 9:47 AM Alex Remily wrote: > Bruno, > > Are yo

Re: [Crypto] requesting help testing native binaries

Bruno, Are you certain that OpenSSL isn't installed on your Windows box? It's supposed to fail before it looks for "OpenSSL_version" if there's no OpenSSL installed. Alex On Tue, Aug 25, 2020 at 2:37 AM Bruno P. Kinoshita wrote: > > Hi Gary, > > I did a mvn clean test install from master, and i

Re: [Crypto] requesting help testing native binaries

Hi Gary, I did a mvn clean test install from master, and it worked OK on my Linux box. Then grabbed the snapshot from https://repository.apache.org/content/repositories/snapshots/org/apache/commons/commons-crypto/1.1.0-SNAPSHOT/ (the latest), and ran the command to test it successfully: $ java

Re: [Crypto] requesting help testing native binaries

Gary, Can you check that your libcrypto.dylib is symlinked to the libcrypto for OpenSSL 1.1.1.g? Mine wasn't, and I was getting different output from the main function than from the unit test output. I'm not confident that this is the root of the problem, but it may at least eliminate a possibil

Re: [Crypto] requesting help testing native binaries

Hi. seems all machines that github actions can provide (means [ windows-2019, ubuntu-20.04, ubuntu-18.04, ubuntu-16.04, macos-10.15 ]) can invoke your snapshot correctly... Gary Gregory 于2020年8月25日周二 上午3:40写道： > Here is a jar since I don't have a public place for binaries. > > Gary > > commons-

Re: [Crypto] requesting help testing native binaries

Public place for binaries: https://dist.apache.org/repos/dist/dev/commons/ On Mon, 24 Aug 2020 at 14:40, Gary Gregory wrote: > > Here is a jar since I don't have a public place for binaries. > > Gary > > commons-crypto-1.1.0-20200824.190246-21.jar >

Re: [Crypto] requesting help testing native binaries

Here is a jar since I don't have a public place for binaries. Gary commons-crypto-1.1.0-20200824.190246-21.jar On Mon, Aug 24, 2020 at 12:10 PM Geoffrey Blake wrote: > Hi Gary, > > Which snapshots have Lin

Re: [Crypto] requesting help testing native binaries

The snapshot repo keeps getting overwritten by CI builds, let me find a more stable place... Gary On Mon, Aug 24, 2020 at 12:10 PM Geoffrey Blake wrote: > Hi Gary, > > Which snapshots have Linux binaries in them? As far as I can tell, the > recent ones uploaded are Mac only, and the one from

Re: [Crypto] requesting help testing native binaries

On Mon, Aug 24, 2020 at 12:10 PM Geoffrey Blake wrote: > Hi Gary, > > Which snapshots have Linux binaries in them? As far as I can tell, the > recent ones uploaded are Mac only, and the one from 8/18 had that weird > /native/native directory issue. For what it's worth, Mac binaries are > worki

Re: [Crypto] requesting help testing native binaries

Hi Gary, Which snapshots have Linux binaries in them? As far as I can tell, the recent ones uploaded are Mac only, and the one from 8/18 had that weird /native/native directory issue. For what it's worth, Mac binaries are working on my laptop, though I'm using LibreSSL which is what comes insta

Re: [Crypto] requesting help testing native binaries

FWIW I have the same basic setup. The original LibreSSL is still installed and I used homebrew to install the various versions of OpenSSL, which I put first on my PATH. I don't routinely call the main function when I test, I usually just run the unit tests, so it's quite possible this is an error

Re: [Crypto] requesting help testing native binaries

Let me see if I can reproduce the error. I'm just speculating from memory at this point. On Sun, Aug 23, 2020 at 9:50 PM Gary Gregory wrote: > > I do have LibreSSL but I used homebrew to install OpenSSL 1.1.1g which I > put first on the PATH. Maybe something is off in my setup... > > Gary > > On

Re: [Crypto] requesting help testing native binaries

I do have LibreSSL but I used homebrew to install OpenSSL 1.1.1g which I put first on the PATH. Maybe something is off in my setup... Gary On Sun, Aug 23, 2020, 21:46 Alex Remily wrote: > Gary, > > I'll have a look. I did the 1.1 support stuff and I'm familiar with > that class and that error,

Re: [Crypto] requesting help testing native binaries

Gary, I'll have a look. I did the 1.1 support stuff and I'm familiar with that class and that error, although I don't recall seeing that specific error in that class. The JNI libraries check the OpenSSL version at runtime, but maybe a compile time dependency got through. Out of curiosity, I ass

Re: [Crypto] requesting help testing native binaries

Side note: Your English is good, no need to apologize :-) Gary On Sun, Aug 23, 2020 at 1:56 PM Xeno Amess wrote: > @Gary Gregory > Hi. > I done it at https://github.com/XenoAmess/commons-crypto-smoke-test > Please have a look. > > > I don't quite see how but I am obviously not comprehending yo

Re: [Crypto] requesting help testing native binaries

@Gary Gregory Hi. I done it at https://github.com/XenoAmess/commons-crypto-smoke-test Please have a look. > I don't quite see how but I am obviously not comprehending your plan. Then I will explain it again :) Step 1, I wrote a maven-plugin, whose usage is to copy tests from another repo when b

Re: [Crypto] requesting help testing native binaries

I wondering if anyone can confirm the following issue and/or help explain it, on MacOS 10.15.6 with OpenSSL 1.1.1g, running: mvn package then: java -Xdiag -Xcheck:jni -cp target/classes -Dcommons.crypto.lib.tempdir=target/ org.apache.commons.crypto.Crypto WARNING in native method: JNI call made

Re: [Crypto] requesting help testing native binaries

On Sun, Aug 23, 2020 at 10:46 AM Xeno Amess wrote: > Got it. > I will explain my plan first. > First, I will write a maven-plugin today. > The maven plugin can accept two parameters, one be repo git url, the other > be test codes relatice dir. > When the plugin invoke (at phase prepare-test-sourc

Re: [Crypto] requesting help testing native binaries

Got it. I will explain my plan first. First, I will write a maven-plugin today. The maven plugin can accept two parameters, one be repo git url, the other be test codes relatice dir. When the plugin invoke (at phase prepare-test-source), it will use jgit to clone the repo you set, and copy its test

Re: [Crypto] requesting help testing native binaries

The Apache snapshot repository is here: https://repository.apache.org/snapshots/ You just need a repository entry in your POM pointing to it in order for Maven to resolve versions postfixed with "-SNAPSHOT". Gary On Sun, Aug 23, 2020 at 10:31 AM Xeno Amess wrote: > Hi Gary. > Where is the snaps

Re: [Crypto] requesting help testing native binaries

Hi Gary. Where is the snapshot jar? Is it at oss snapshot center? If so, then I will have a very, very tricky way to test it, and that test is reusable. Gary Gregory 于2020年8月23日周日 下午9:53写道： > Building the binaries can be tricky, be warned. > > The trickiest to validate are aarch64, arm, and art

Re: [Crypto] requesting help testing native binaries

Building the binaries can be tricky, be warned. The trickiest to validate are aarch64, arm, and artmhf. All help is welcome of course. A smoke test would be to make sure that the native code from the SNAPSHOT jar from our SNAPSHOT repo can be loaded, for example, by calling from the command line

Re: [Crypto] requesting help testing native binaries

On Sun, Aug 23, 2020 at 8:31 AM Alex Remily wrote: > Gary, > > I can test the 64 bit Mac, Windows and Linx builds this week. Did you > want to include this test fix in the snapshot: > > https://issues.apache.org/jira/browse/CRYPTO-149 > > There's a pending PR with the fix. It's only one line of

Re: [Crypto] requesting help testing native binaries

Gary, I can test the 64 bit Mac, Windows and Linx builds this week. Did you want to include this test fix in the snapshot: https://issues.apache.org/jira/browse/CRYPTO-149 There's a pending PR with the fix. It's only one line of code so it should merge easily. In the past I've had some succes

Re: [Crypto] requesting help testing native binaries

on the snapshot's release. sorry for the typo Xeno Amess 于2020年8月23日周日 下午2:44写道： > > what is the easiest way to test them? Run some code using the code of > the snapshots, or perhaps run the build+tests from a tag/commit? > How about fork its tests to a new repo, and import the original codes a

Re: [Crypto] requesting help testing native binaries

> what is the easiest way to test them? Run some code using the code of the snapshots, or perhaps run the build+tests from a tag/commit? How about fork its tests to a new repo, and import the original codes as a maven dependency, run the tests on the snapshot's source? Bruno P. Kinoshita 于2020年8

Re: [Crypto] requesting help testing native binaries

Hi Gary, I might be able to test on Win64 and Linux64 this week. You mentioned snapshots… what is the easiest way to test them? Run some code using the code of the snapshots, or perhaps run the build+tests from a tag/commit? Cheers Bruno On Sunday, 23 August 2020, 11:49:21 am NZST, Gary

[Crypto] requesting help testing native binaries

Hi all, I intent on creating a release candidate for Commons Crypto soon. I pushed a snapshot today which contains native binaries for Windows 32 and 64, Linux 32 and 64, Mac 64, and ARM and ARM HF. Please help testing these on whatever platforms you may have access to. Gary