That's correct. One of the reasons I want to solve this is to get everything ready for the version catalogs and potentially the Admin UI (as it uses version catalogs as well). It's hard to keep track and update versions that are not part of the versions.props, and we have quite a few of them. I believe with version catalogs we will get a better overview.
I have read the dev-docs multiple times before I started working on a few updates. One of the blockers some dependencies have is the upgrade to Jetty 11 /12, which is a complex blocker. Another one was the JDK version, but that got sorted out with JDK 21 in main. Besides that, the overview of related dependencies and breaking changes is missing, which was probably one of the ways to contribute to the updates, even without write permissions. While working on the PRs, I faced various situations which I would like to clarify and document at the end: - Should dependency upgrades of major versions be backported? - What happens if a dependency has multiple licenses? We usually include only one, but which one do we choose? Is there a priority list? I found an ASF policy (https://www.apache.org/legal/resolved.html) but it is not very clear to me. - Is the notice file relevant for all licenses? Because we are missing quite a few changes / additions here (I'd like to address them all together in a separate PR, now that we have quite a few differences with the current versions) - Is it sufficient to pick license and notice file from Github repositories (if they are available and the versions tagged)? I'd like to provide information in the documentation where to look for license and notice files, also for non-github projects. - If an upgrade to a version is only planned for main (next major release), like Jetty, should we still upgrade minor and patches in older versions? - I decided to upgrade some minor versions of dependencies on main, so that they can be backported, and then make a major upgrade that is only applicable for main. This may be redundant, but there is no rule here, so input is appreciated. - Solrbot distinguishes between major and minor/patches and creates separate PRs (from what I saw). However, it looks like it does not work for versions with suffixes. Is solrbot supposed to support semantic versioning? - Some dependencies introduce breaking changes (like JDK upgrades) in minor versions. Solrbot will update the same PR as soon as a newer minor or patch version is available. Should we upgrade these dependencies only in major and pick the latest non-breaking change / compatible version for backporting? error_prone_annotations is one example where it requires JDK 17 since v2.32.0. - It seems that Solrbot is not able to identify versions from plugins and JS libraries included in gradle files. Keeping track of them is hard, and should be better if we are able to switch to version catalogs. I am taking notes for the final documentation update, but I'd like to use this thread for gathering more input on special cases like these. So if you have any answers / opinions on how to handle things, or if it is already somewhere documented, please share it. On Sun, Oct 27, 2024 at 6:16 PM David Smiley <dsmi...@apache.org> wrote: > I also agree with it being fine to just defer to the next release. They > can mostly be done all at once at that time. I read the dev-docs on this; > it's pretty clear. > The backports can be done in one fell swoop with > dev-tools/scripts/cherrypick.sh > > Christos, as I understand it, you want to get through these now prior to > the gradle dependency management refactor. Go for it. Once the refactor > happens, the Renovate bot config will need to be updated and furthermore > the backports won't be clean due to main / 9x divergence. > > On Sun, Oct 27, 2024 at 9:55 AM Jan Høydahl <jan....@cominvent.com> wrote: > > > Hi, > > > > We have a dev-doc for upgrading dependencies: > > > https://github.com/apache/solr/blob/main/dev-docs/dependency-upgrades.adoc > > If something is unclear, please improve that doc. > > > > Most SolrBot PRs are small and simple, and can be merged and backported > in > > 2 minutes. By not adding CHANGES.txt entry we avoid lots of merge > conflicst > > due to that one file, and as you say, all SolrBot commits are added to > > CHANGES during release anyway. If you do more than simply hitting the > > "Merge" button, feel free to add yourself as "Co-Authored-By" in the > merge > > message. > > > > As to why not every single SolrBot PR is merged immediately, this may be > > due to > > - Long time until next solr release. To avoid upgrading the same > > dependency 3-4 times, just let it linger and be auto bumped until a few > > weeks before next release. > > - Some upgrades are complex and need to be coordinated with other > > dependencies > > - commons-io is barred from upgrading until a new version of Policeman > > tools, which adds to the time > > - simply lack of hands > > > > I believe a fair amount of the currently stale SolrBot PRs need extra > work > > in terms of adding license files etc. > > > > Jan > > > > > 26. okt. 2024 kl. 19:09 skrev Gus Heck <gus.h...@gmail.com>: > > > > > > Release time is fine in my opinion. If they are automated, I wasn't > > > recalling it. People running unreleased code can reasonably be expected > > to > > > do their own homework, and watch commits etc. However people deciding > if > > > they will conduct an upgrade to a new release, or how to prepare for it > > > will want to pay attention to some dependencies. Solrbot probably has > > less > > > use for the credit and reputation from contributing to Solr than the > > > developer :). > > > > > > I see a re-try checkbox on the PR's by SolrBot. I've not tried it but > it > > > sounds like that could be used to refresh stale PR's? > > > > > > On Sat, Oct 26, 2024 at 11:46 AM Christos Malliaridis < > > > c.malliari...@gmail.com> wrote: > > > > > >> From what I understood, entries in CHANGES.txt are automatically added > > >> during release processes, so no actions are necessary from the > > developer's > > >> point of view when merging the Solrbot PRs. > > >> > > >> If I get you right Gus, you'd like to see at least these entries > > earlier, > > >> and the changes should contain the developer that addressed and merged > > >> solrbot's PR, instead of solrbot. Is that correct? > > >> > > >> If that's the case, I may agree with you, but there is one "issue" I > > want > > >> to understand before adding one more action to the process. > > >> > > >> We have PRs from Solrbot that are more than 6 months old. This means > > that > > >> the process of updating dependencies is complex, time consuming, > > >> error-prone or no one wants to take responsibility. At least that's > > what my > > >> previous experience with updates in other projects was. So I believe > > that > > >> adding one more step into the process may worsen the developer > > experience > > >> of addressing dependency updates. And since these updates often > address > > >> CVEs and other functionality issues, I see this as a crucial part of > > >> maintenance. > > >> > > >> I am trying to understand the reasons for these specific PRs becoming > > >> stale, so that we can take actions and improve this part. If anyone > has > > any > > >> experiences or information please share them. > > >> > > >> On Sat, Oct 26, 2024 at 4:45 PM Gus Heck <gus.h...@gmail.com> wrote: > > >> > > >>> Changes in our dependencies ought to be tracked somewhere more > > digestible > > >>> than VCS. Users who use our test framework or have repositories > > >> containing > > >>> custom code intended to run inside of solr will care about changes in > > >> deps > > >>> for both solrj, and core, and whatever components. CHANGES.txt isn't > > >> great > > >>> either, but it's better than nothing. (i.e. > > >>> > > >>> > > >> > > > https://solr.apache.org/docs/9_7_0/changes/Changes.html#v9.7.0.dependency_upgrades > > >>> ) > > >>> Ideally an automated dependency upgrade report could exist, but I'm > not > > >>> sure what tool might exist for that. > > >>> > > >>> So I think a Changes.txt entry can be added as a follow on in the > case > > >> of a > > >>> direct merge. Simply saying something like we have in the past, just > > the > > >>> library and the committer, and the PR/JIRA as applicable. Solrbot's > > >>> involvement is irrelevant to users, so something like: SOLR-15609: > > >> Upgrade > > >>> log4j to 2.14.1 (hossman) > > >>> > > >>> On Thu, Oct 24, 2024 at 10:16 AM Christos Malliaridis < > > >>> malliari...@apache.org> wrote: > > >>> > > >>>> I got a few questions before interacting with the Solrbot PRs, > because > > >> I > > >>>> haven't done that before. > > >>>> > > >>>> - When are the CHANGES entries added, and when not? Is it just "if > > >>> Solrbot > > >>>> created the PR, then it will be added automatically during next > > >> release"? > > >>>> - In case no additional actions are necessary, would I simply merge > > the > > >>> PR > > >>>> via GitHub and look into backporting the changes as usual? > > >>>> - In case changes are necesasry, would I simply pick up from where > > >>> Solrbot > > >>>> left off and continue working on the PR? > > >>>> - Should any dependency that requires JDK > 11 be backported to 9x? > > >>>> - And if not, should any older version of the dependency that > supports > > >>> JDK > > >>>> 11 but is newer to the one in use be "backported" instead? Like > minor > > >> and > > >>>> patch versions. > > >>>> > > >>>> I'd be happy to tackle the PRs as soon as possible. > > >>>> > > >>>> > --------------------------------------------------------------------- > > >>>> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org > > >>>> For additional commands, e-mail: dev-h...@solr.apache.org > > >>>> > > >>>> > > >>> > > >>> -- > > >>> http://www.needhamsoftware.com (work) > > >>> https://a.co/d/b2sZLD9 (my fantasy fiction book) > > >>> > > >> > > > > > > > > > -- > > > http://www.needhamsoftware.com (work) > > > https://a.co/d/b2sZLD9 (my fantasy fiction book) > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org > > For additional commands, e-mail: dev-h...@solr.apache.org > > > > >
