Re: Running all tests via Crave.io

2023-11-22 Thread Mikhail Khludnev
Thank you, all. Build passed! On Thu, Nov 23, 2023 at 12:50 AM David Smiley wrote: > Mikhail, > > This is tracked here: https://issues.apache.org/jira/browse/SOLR-17081 > (thanks Christine) and I've updated it with further details. > I've been in contact with UV. The build image has been update

Re: Running all tests via Crave.io

2023-11-22 Thread David Smiley
Mikhail, This is tracked here: https://issues.apache.org/jira/browse/SOLR-17081 (thanks Christine) and I've updated it with further details. I've been in contact with UV. The build image has been updated with fonts and recent builds have succeeded proof: https://github.com/apache/solr/actions/wor

Re: Running all tests via Crave.io

2023-11-22 Thread Mikhail Khludnev
Hello Yuvraaj, Recently Crave became prone to specific bug. https://github.com/apache/solr/actions/runs/6909103380/job/18849339883?pr=2046 java.lang.RuntimeException: Fontconfig head is null, check your fonts or fonts configuration at java.desktop/sun.awt.FontConfiguration.getVersion(FontConfigur

Re: Running all tests via Crave.io

2023-03-10 Thread Shawn Heisey
On 3/10/23 06:49, Kevin Risden wrote: So in the case of https://github.com/apache/solr/pull/571 that would be concretely: git fetch origin pull/571/head:pr-571 git checkout pr-571 This would work if https://github.com/apache/solr is cloned. There should be no need to try to find a merge base an

Re: Running all tests via Crave.io

2023-03-10 Thread Yuvraaj Kelkar
Disclaimer: technical details that aren't strictly about Solr coming up ahead: Crave builds and tests the Solr code on a remote server. The first step to doing that is to "clone the exact sources on the remote system". A brute-force way to do that would be rsync the sources from the Github runne

Re: Running all tests via Crave.io

2023-03-10 Thread Kevin Risden
I guess I'm not quite sure why Crave is trying to find the merge base... The PR is actually a branch in git - it has the full state of the repo that should be tested. This is accomplished basically by the following [1] git fetch origin pull/ID/head:BRANCH_NAME git checkout BRANCH_NAME So in the c

Re: Running all tests via Crave.io

2023-03-09 Thread Yuvraaj Kelkar
Upvote for Mikhail's suggestion: That's the root cause of the problem: The PR has drifted too much for Crave to get a correct git merge base. Evidence : The PR is at most 8 files. Crave seems to thing it is 42 files and thousands of changes: Run ./crave run --clean 42 files changed, 1032 insertio

Re: Running all tests via Crave.io

2023-03-09 Thread Mikhail Khludnev
Hello Kevin, It happened to me too. I fixed it with rebasing pr branch. $git pull --rebase origin main $git push -f myfork PR-branch On Thu, Mar 9, 2023 at 5:50 PM Kevin Risden wrote: > I've noticed failures like this on PRs quite a few times when github > actions for `./gradlew check` pass no i

Re: Running all tests via Crave.io

2023-03-09 Thread Kevin Risden
I've noticed failures like this on PRs quite a few times when github actions for `./gradlew check` pass no issue. https://github.com/apache/solr/actions/runs/4375627158/jobs/7656583100 Setting up workspace (this could take some time)... > Pulling container image... > Finished pulling container im

Re: Running all tests via Crave.io

2023-02-09 Thread Jan Høydahl
Looks like crave test (in github actions) is still somewhat unstable, sometimes failing to allocate resoources? Also, question regarding how to inspect crave failed tests. How to access the full log, it just says it is at file:///tmp/... but is there a way for crave to generate a URL where one

Re: Running all tests via Crave.io

2023-01-29 Thread Yuvraaj Kelkar
Updated default command to ./gradlew localSettings && ./gradlew --max-workers=`nproc` -Ptests.jvms=96 test Thanks, -Uv On Jan 29 2023, at 5:00 am, Ishan Chattopadhyaya wrote: > Thanks to Crave! Seems like there's Dual XEON 24 core processors > available on the system. > > I pushed the paralleli

Re: Running all tests via Crave.io

2023-01-29 Thread Ishan Chattopadhyaya
Thanks to Crave! Seems like there's Dual XEON 24 core processors available on the system. I pushed the parallelism to 96, and times reduced from 8.5 min to ~5.5 min. ~/crave run -- './gradlew localSettings && ./gradlew --max-workers=`nproc` -Ptests.jvms=96 test' The slowest tests (exceeding

Re: Running all tests via Crave.io

2023-01-28 Thread Eric Pugh
Fantastic! > On Jan 27, 2023, at 10:36 PM, Yuvraaj Kelkar wrote: > > Hurray! > > On Fri, Jan 27, 2023, 19:27 David Smiley wrote: > >> The PR is merged. Henceforth, all java based tests will run on PRs. >> Also "crave run" now has the defaults so you can run that at your terminal >> withou

Re: Running all tests via Crave.io

2023-01-27 Thread Yuvraaj Kelkar
Hurray! On Fri, Jan 27, 2023, 19:27 David Smiley wrote: > The PR is merged. Henceforth, all java based tests will run on PRs. > Also "crave run" now has the defaults so you can run that at your terminal > without adding all the extra args :-) > > I did an experiment[1] wondering how long it wou

Re: Running all tests via Crave.io

2023-01-27 Thread David Smiley
The PR is merged. Henceforth, all java based tests will run on PRs. Also "crave run" now has the defaults so you can run that at your terminal without adding all the extra args :-) I did an experiment[1] wondering how long it would take a typical GitHub Action machine to run all these Solr tests.

Re: Running all tests via Crave.io

2023-01-26 Thread Mike Drob
@Ishan - not necessarily, a committer has to explicitly approve running the GHA checks for first time contributors. So a random person with a drive-by PR won't be automatically handed the keys to our infra. I believe we can set that to needing approval for each run by a non-committer, or maybe that

Re: Running all tests via Crave.io

2023-01-26 Thread Houston Putman
Non-committers’ PRs do not have Github Actions run automatically. A committer has to press a button to allow the tests to run. So unless a committer is malicious, there shouldnt be a problem there. On Thu, Jan 26, 2023 at 10:18 AM Ishan Chattopadhyaya < ichattopadhy...@gmail.com> wrote: > Though,

Re: Running all tests via Crave.io

2023-01-26 Thread Ishan Chattopadhyaya
Though, if a malicious user creates a PR that executes harmful code, that PR will also get executed (via "gradlew test"), right? On Thu, 26 Jan, 2023, 8:36 pm Yuvraaj Kelkar, wrote: > Hm. I see your point. The first solution I thought of is a bit blunt, but > it will work: We disable command lin

Re: Running all tests via Crave.io

2023-01-26 Thread Yuvraaj Kelkar
Hm. I see your point. The first solution I thought of is a bit blunt, but it will work: We disable command line arguments when using ZeroConf. Only preset commands are allowed. Also, to allow the Solr GHA to run whatever commands needed not just for now, but also for any future changes, we set u

Re: Running all tests via Crave.io

2023-01-26 Thread Mike Drob
Having massive infrastructure to run PRs is pretty cool. I'm worried about letting arbitrary people run code on these machines though - a single 'crave run -- mine_bitcoin.exe' would ruin the system for everybody, or it's not hard to imagine a slightly more indirect case where an attacker adds a t

Re: RE%3A Re%3A Running all tests via Crave.io&In-Reply-To=

2023-01-26 Thread Yuvraaj Kelkar
Yes, the stdout and stderr are streamed back to the initiator - the Github runner - which then records it as part of the Actions' logs. On Jan 26 2023, at 5:11 am, Eric Pugh wrote: > Running the tests on every PR raised will make reviewing patches MUCH > simpler. Today, on my laptop, it’s 40 mi

Re: RE%3A Re%3A Running all tests via Crave.io&In-Reply-To=

2023-01-26 Thread Jan Høydahl
I like this. Perhaps it will also put more focus on fixing bad tests, since some PRs are bound to fail the full test suite due to test flakiness, but it will be more evident. Which is a good thing. Jan > 26. jan. 2023 kl. 12:11 skrev Eric Pugh : > > Running the tests on every PR raised will

Re: RE%3A Re%3A Running all tests via Crave.io&In-Reply-To=

2023-01-26 Thread Eric Pugh
Running the tests on every PR raised will make reviewing patches MUCH simpler. Today, on my laptop, it’s 40 minutes to 1 hour to run the full process, and I can’t use my machine during that time..Being able to look at a PR and see that the tests run will make me more likely to review ;-).

RE%3A Re%3A Running all tests via Crave.io&In-Reply-To=

2023-01-25 Thread Yuvraaj Kelkar
Apologies: I just signed up for this email list right now. I was signed up for the old Lucene list but didn't realize that the split would impact the mailing list as well. Explains a lot, now that I think about it. I'll answer all the questions here, that I see from the thread: 1: (Noble) Full t

Re: Running all tests via Crave.io

2023-01-25 Thread Ishan Chattopadhyaya
This is very cool. Thanks for working on this, David. Can multiple developers execute their tests at the same time? On Thu, 26 Jan, 2023, 5:07 am Noble Paul, wrote: > This is interesting. > > So, if the PR is merged , we will have the full test running on crave.io > for every PR raised? > > On T

Re: Running all tests via Crave.io

2023-01-25 Thread Noble Paul
This is interesting. So, if the PR is merged , we will have the full test running on crave.io for every PR raised? On Thu, Jan 26, 2023 at 9:22 AM David Smiley wrote: > We haven't been running all our tests in GitHub Actions (i.e. PR > validation) because it was too time consuming to do so. I

Re: Running all tests via Crave.io

2023-01-25 Thread Gus Heck
This looks awesome. Is Crave really prepared for anyone in the world to run this build on their hardware whenever they want? or does it require membership/paid? If there is a limit is it per person or per project? I don't mind a command of that length so much. That's what bash history is for (or r

Running all tests via Crave.io

2023-01-25 Thread David Smiley
We haven't been running all our tests in GitHub Actions (i.e. PR validation) because it was too time consuming to do so. I don't recall how slow it was when someone last tried; it's probably better now but still slow. To make up for this, there is a GHA only for SolrJ if a PR touches SolrJ. Ther