Re: [DISCUSS] Bring cassandra-harry in tree as a submodule

2023-06-01 Thread David Capwell
Hmmm, did you try without —remote? We 100% rely on git hooks, and this is what we do $ grep -r 'git submodule' .build/ .build//sh/bump-accord.sh: git submodule status modules/accord .build//sh/change-submodule.sh: git submodule set-url "${path}" "${url}" .build//sh/change-submodule.sh: git s

Re: [DISCUSS] Bring cassandra-harry in tree as a submodule

2023-06-01 Thread Josh McKenzie
> Josh, do you see any reports on what isn’t working? I think most people > don’t touch 1% of what git can do… so it might be that 10% is broken but that > no one in our domain actually touches that path? Was changing .gitmodule in harry to point to a branch and git just straight up went out to

Re: [DISCUSS] Bring cassandra-harry in tree as a submodule

2023-06-01 Thread David Capwell
To be clear, we only use the relative syntax during development and not long lived feature branches like cep-15-accord; we use https address there. So when you create a PR you switch to relative paths (if-and-only-if you change the submodule), then on merge you switch back to https pointing to

Re: [DISCUSS] Bring cassandra-harry in tree as a submodule

2023-06-01 Thread David Capwell
Most edge cases we have seen in Accord are working with feature branches from other authors where we use relative paths to make sure the git@ vs https:// doesn’t become a problem for CI (submodule points to https:// to work in CI, but if you do that during feature development it gets annoying to

Re: [DISCUSS] Bring cassandra-harry in tree as a submodule

2023-05-31 Thread Josh McKenzie
Bumping into worktree + submodule pain on some harry related work; it looks like "git worktree" and submodules are not currently fully implemented: https://git-scm.com/docs/git-worktree#_bugs BUGS Multiple checkout in general is still experimental, and the support for submodules is incomplete.

Re: [DISCUSS] Bring cassandra-harry in tree as a submodule

2023-05-28 Thread Alex Petrov
Regarding approachability, one of the things I thought is worth adding is a DSL. I feel like there's enough functionality in Harry and there's enough information for anyone who needs to write even an involved test out there, but adoption doesn't usually start with complex use-cases, so it could

Re: [DISCUSS] Bring cassandra-harry in tree as a submodule

2023-05-25 Thread Abe Ratnofsky
I'm seeing a few distinct topics here: 1. Harry's adoption and approachability I agree that approachability is one of Harry's main improvement areas right now. If our goal is to produce a fuzz testing framework for the Cassandra project, then adoption by contributors and usage for new feature d

Re: [DISCUSS] Bring cassandra-harry in tree as a submodule

2023-05-25 Thread Josh McKenzie
> I would really like us to split out utilities into a common project +1 to the sentiment. Would also advocate strongly for it being more tightly integrated with the base project than what we've been doing with our ecosystem (i.e. completely separate projects, not submodules), mostly from a disc

Re: [DISCUSS] Bring cassandra-harry in tree as a submodule

2023-05-25 Thread Alex Petrov
This was not a talk, but rather an interactive workshop, unfortunately will not work in a recorded way, but I am trying to work out ways to preserve this. On Thu, May 25, 2023, at 10:26 AM, Claude Warren, Jr via dev wrote: > Since the talk was not accepted for Cassandra Summit, would it be possib

Re: [DISCUSS] Bring cassandra-harry in tree as a submodule

2023-05-25 Thread Benedict
I would really like us to split out utilities into a common project, personally. It would be nice to work with a shared palette, including for dtest-api, accord, Harry etc.I think it would help clean up the codebase a bit too, as we have some (minimal) tight coupling with utilities and the C* proce

Re: [DISCUSS] Bring cassandra-harry in tree as a submodule

2023-05-25 Thread Claude Warren, Jr via dev
Since the talk was not accepted for Cassandra Summit, would it be possible to record it as a simple youtube video and publish it so that the detailed information about how to use Harry is not lost? On Thu, May 25, 2023 at 7:36 AM Alex Petrov wrote: > While we are at it, we may also want to pull

Re: [DISCUSS] Bring cassandra-harry in tree as a submodule

2023-05-24 Thread Alex Petrov
While we are at it, we may also want to pull the in-jvm dtest API as a submodule, and actually move some tests that are common between the branches there. On Thu, May 25, 2023, at 6:03 AM, Caleb Rackliffe wrote: > Isn’t the other reason Accord works well as a submodule that it has no > dependen

Re: [DISCUSS] Bring cassandra-harry in tree as a submodule

2023-05-24 Thread Caleb Rackliffe
Isn’t the other reason Accord works well as a submodule that it has no dependencies on C* proper? Harry does at the moment, right? (Not that we couldn’t address that…just trying to think this through…)On May 24, 2023, at 6:54 PM, Benedict wrote:In this case Harry is a testing module - it’s not so

Re: [DISCUSS] Bring cassandra-harry in tree as a submodule

2023-05-24 Thread Benedict
In this case Harry is a testing module - it’s not something we will develop in tandem with C* releases, and we will want improvements to be applied across all branches.So it seems a natural fit for submodules to me.On 24 May 2023, at 21:09, Caleb Rackliffe wrote:> Submodules do have their own ove

Re: [DISCUSS] Bring cassandra-harry in tree as a submodule

2023-05-24 Thread Caleb Rackliffe
> Submodules do have their own overhead and edge cases, so I am mostly in favor of using for cases where the code must live outside of tree (such as jvm-dtest that lives out of tree as all branches need the same interfaces) Agreed. Basically where I've ended up on this topic. > We could go over s

Re: [DISCUSS] Bring cassandra-harry in tree as a submodule

2023-05-24 Thread Alex Petrov
> I'm about to need to harry test for the paging across tombstone work for > https://issues.apache.org/jira/browse/CASSANDRA-18424 (that's where my own > overlapping fuzzing came in). In the process, I'll see if I can't distill > something really simple along the lines of how React approaches it

Re: [DISCUSS] Bring cassandra-harry in tree as a submodule

2023-05-24 Thread Mick Semb Wever
> > So looking at accord trunk, we needed 12 votes for a release, and each > vote is a min of 3 days, so 36 days of overhead vs 5 hours of work? > That's apples and oranges (wait time vs effort time). I was most interested in (and supportive of) your qualified opinion :-) > One thing that ca

Re: [DISCUSS] Bring cassandra-harry in tree as a submodule

2023-05-24 Thread David Capwell
> The time spent on getting that running has been a fair few hours, where we > could have cut many manual module releases in that time. We spent a few hours getting submodules working, and we no longer need to release for every single commit… $ git log b9025e59395f47535e4ed1fec20b1186cdb07db8.

Re: [DISCUSS] Bring cassandra-harry in tree as a submodule

2023-05-24 Thread Josh McKenzie
> I have submitted a proposal to Cassandra Summit for a 4-hour Harry workshop, I'm about to need to harry test for the paging across tombstone work for https://issues.apache.org/jira/browse/CASSANDRA-18424 (that's where my own overlapping fuzzing came in). In the process, I'll see if I can't dist

Re: [DISCUSS] Bring cassandra-harry in tree as a submodule

2023-05-24 Thread Alex Petrov
> I wonder if a mini-onboarding session would be good as a community session - > go over Harry, how to run it, how to add a test? Would that be the right > venue? I just would like to see how we can not only plug it in to regular CI > but get everyone that wants to add a test be able to know h

Re: [DISCUSS] Bring cassandra-harry in tree as a submodule

2023-05-24 Thread Brandon Williams
On Wed, May 24, 2023 at 7:45 AM Josh McKenzie wrote: > What prompted this thread was harry being external to the core codebase and > the lack of adoption and usage of it having led to atrophy of certain aspects > of it, which then led to redundant implementation of some fuzz testing and > lost

Re: [DISCUSS] Bring cassandra-harry in tree as a submodule

2023-05-24 Thread Alex Petrov
I think pulling Harry into the tree will make adoption easier for the folks. I have been a bit swamped with Transactional Metadata work, but I wanted to make some of the things we were using for testing TCM available outside of TCM branch. This includes a bunch of helper methods to perform opera

Re: [DISCUSS] Bring cassandra-harry in tree as a submodule

2023-05-24 Thread Josh McKenzie
> importantly it’s a million times better than the dtest-api process - which > stymies development due to the friction. This is my major concern. What prompted this thread was harry being external to the core codebase and the lack of adoption and usage of it having led to atrophy of certain aspe

Re: [DISCUSS] Bring cassandra-harry in tree as a submodule

2023-05-24 Thread Benedict
It’s not without hiccups, and I’m sure we have more to learn. But it mostly just works, and importantly it’s a million times better than the dtest-api process - which stymies development due to the friction.On 24 May 2023, at 08:39, Mick Semb Wever wrote:WRT git submodules and CASSANDRA-18204, ar

Re: [DISCUSS] Bring cassandra-harry in tree as a submodule

2023-05-24 Thread Mick Semb Wever
WRT git submodules and CASSANDRA-18204, are we happy with how it is working for accord ? The time spent on getting that running has been a fair few hours, where we could have cut many manual module releases in that time. David and folks working on accord ? On Tue, 23 May 2023 at 20:09, Josh Mc

Re: [DISCUSS] Bring cassandra-harry in tree as a submodule

2023-05-23 Thread Josh McKenzie
I'll hold off on this until Alex Petrov chimes in. @Alex -> got any thoughts here? On Tue, May 16, 2023, at 5:17 PM, Jeremy Hanna wrote: > I think it would be great to onboard Harry more officially into the project. > However it would be nice to perhaps do some sanity checking outside of Apple

Re: [DISCUSS] Bring cassandra-harry in tree as a submodule

2023-05-16 Thread Jeremy Hanna
I think it would be great to onboard Harry more officially into the project. However it would be nice to perhaps do some sanity checking outside of Apple folks to see how approachable it is. That is, can someone take it and just run it with the current readme without any additional context? I

Re: [DISCUSS] Bring cassandra-harry in tree as a submodule

2023-05-16 Thread Abe Ratnofsky
Just to make sure I'm understanding the details, this would mean apache/cassandra-harry maintains its status as a separate repository, apache/cassandra references it as a submodule, and clones and builds Harry locally, rather than pulling a released JAR. We can then reference Harry as a library

[DISCUSS] Bring cassandra-harry in tree as a submodule

2023-05-16 Thread Josh McKenzie
Similar to what we've done with accord in https://issues.apache.org/jira/browse/CASSANDRA-18204, I'd like to discuss bringing cassandra-harry in-tree as a submodule. repo link: https://github.com/apache/cassandra-harry Given the value it's brought to the project's stabilization efforts and the