I don't think the devs would work on all artifacts(projects) a time. If the naming convention of repo for a plugin would be artifactId, like /maven-clean-plugin, then even easy to figure out which one to clone. The most likely the dev would just clone one repo she/he is interested in at the moment, i.e. repository /maven-clean-plugin, let's say. It's good to avoid any shared files across them, even I don't think devs share some in svn today. The release process would be quite usual, i.e. one repo = one project, and new devs already have these experiences which will be simple for them to adapt faster.
On Sat, Oct 7, 2017 at 11:36 PM, Chas Honton <[email protected]> wrote: > Is it usual to need all dozen repositories or is a single repository (or a > handful) more likely? > > Chas > > > On Oct 7, 2017, at 1:32 PM, Arnaud Héritier <[email protected]> wrote: > > > > AFAIR the main concern was for developers to have to clone several dozen > of > > repositories > > I don't think it is a real issue. We can share a script to do it easily > > > > > >> On Sat, Oct 7, 2017 at 10:26 PM, Chas Honton <[email protected]> wrote: > >> > >> What are the concerns with separate git repository per artifact? Is it > >> the monolithic build? > >> > >> Chas > >> > >>> On Oct 7, 2017, at 1:20 PM, Arnaud Héritier <[email protected]> > wrote: > >>> > >>> +1 > >>> > >>>> On Sat, Oct 7, 2017 at 7:11 PM, Hervé BOUTEMY <[email protected]> > >> wrote: > >>>> > >>>> we can start with naming the future git repos: I suppose adding a > table > >> at > >>>> the > >>>> end of the Git migration Wiki page can give a good view of the result > >>>> (with 78 > >>>> new repos + existing 7 repos + the 6 in migration if nobody objects = > 91 > >>>> repos) > >>>> > >>>> once we're all convinced of the target, we'll see how to work with > infra > >>>> and I suppose they already have the right authors list, given the > >> current > >>>> mirrors already published > >>>> > >>>> Regards, > >>>> > >>>> Hervé > >>>> > >>>> Le samedi 7 octobre 2017, 16:05:45 CEST Tibor Digana a écrit : > >>>>> Herve, > >>>>> > >>>>> I think, first of all, we should write a list with exact names of git > >>>>> repositories for which INFRA will crate them in > >>>>> git1-us-west.apache.org/repos/asf > >>>>> and mirror in github.com/apache > >>>>> > >>>>> And then the authors.txt. > >>>>> > >>>>> In my case I did not migrate tags and branches which is my problem > >> given > >>>> in > >>>>> my example with previous e-mail. > >>>>> > >>>>> On Sat, Oct 7, 2017 at 3:48 PM, Tibor Digana <[email protected] > > > >>>> wrote: > >>>>>> Herve, > >>>>>> I think making the tag as I wanted would not be possible in git. > >>>>>> > >>>>>> Let's make it with all 78 projects. > >>>>>> In 2016 I migrated my svn project "audit" located in sub-folder > behind > >>>>>> trunk (trunk/libs/audit) to git which is exactly what we need to do > in > >>>>>> ASF. > >>>>>> > >>>>>> In our case the commands would be 99% the same. The 1% is the source > >>>> SVN > >>>>>> URL and target GitHub URL. That's all. Then if we provide INFRA with > >>>> bunch > >>>>>> of commands like these then they will only execute them with their > >>>>>> permissions. > >>>>>> I do not think they will give me admin permissions to make it. > >>>>>> > >>>>>> git svn clone --no-metadata --authors-file=E:\*authors.txt* http:// > >>>>>> ***/svn/***/*trunk/libs/audit audit* > >>>>>> git config --global user.name "*** ***" > >>>>>> git config --global user.email "***@***.***" > >>>>>> git remote add origin https://***@github.com/apache/ > >>>> *maven-clean-plugin* > >>>>>> .git > >>>>>> git push -u origin --all > >>>>>> > >>>>>> I observed the *authors.txt* from command > >>>>>> *svn log -q* > >>>>>> Each person from between first two pipes, e.g.: > >>>>>> r1234567 | tibordigana | 2015-10-19 02:20:00 > >>>>>> > >>>>>> Every developer added e-mail and full name, i.e.: > >>>>>> tibordigana = Tibor Digana <[email protected]> > >>>>>> etc. > >>>>>> > >>>>>> This list of users which should be created in prior to contact > INFRA. > >>>>>> Not all users are active after years but it would be nice to include > >>>> them > >>>>>> too. > >>>>>> This list of users is one thing which cannot be automated! > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> On Sat, Oct 7, 2017 at 2:48 PM, Tibor Digana < > [email protected]> > >>>>>> > >>>>>> wrote: > >>>>>>> In my company I have such situation in SVN. Making a tag from > >>>> subfolder. > >>>>>>> Git always makes the tag from the root, right? > >>>>>>> Let's google something... > >>>>>>> > >>>>>>> On Sat, Oct 7, 2017 at 2:47 PM, Tibor Digana < > >>>> [email protected] > >>>>>>> > >>>>>>>> wrote: > >>>>>>>> In my company I have such situation. > >>>>>>>> Making a tag from subfolder. > >>>>>>>> Git always makes the tag from the root, right? > >>>>>>>> Let's google something... > >>>>>>>> > >>>>>>>> On Sat, Oct 7, 2017 at 2:05 PM, Hervé BOUTEMY < > >> [email protected] > >>>>> > >>>>>>>> > >>>>>>>> wrote: > >>>>>>>>> thanks for your interest and feedback > >>>>>>>>> > >>>>>>>>> Le samedi 7 octobre 2017, 12:00:32 CEST Tibor Digana a écrit : > >>>>>>>>>> 78 is too much. > >>>>>>>>> > >>>>>>>>> notice that there would also be a question on the git repos > naming > >>>>>>>>> convention, > >>>>>>>>> to avoid flat 78 names but keep at least 3 meaningful groups > >>>> (plugins, > >>>>>>>>> shared, > >>>>>>>>> resources: I think this is not absolutely necessary for > >> doxia-tools) > >>>>>>>>> > >>>>>>>>>> There is no problem to trigger release over sub-folders and tag > it > >>>>>>>>> > >>>>>>>>> with > >>>>>>>>> > >>>>>>>>>> prefix which is already done in SVN. > >>>>>>>>> > >>>>>>>>> is it supported by maven-release plugin with git? > >>>>>>>>> > >>>>>>>>> notice I did not know that git was able to tag only a subtree: > but > >> I > >>>>>>>>> knew I > >>>>>>>>> don't yet understand every aspect of git magic... :) > >>>>>>>>> > >>>>>>>>>> The CI build can always trigger the root and Jenkinsfile would > >>>> have > >>>>>>>>>> 41 > >>>>>>>>>> stages for plugins, 26 stages for Shared, etc. > >>>>>>>>>> It can be done in Jenkinsfile and so the shell would not throw > >>>>>>>>> > >>>>>>>>> exception > >>>>>>>>> > >>>>>>>>>> but status would be set instead and goes to the next stage. > >>>>>>>>>> I do not know how to detect in GitSCM which sub-folder(s) > received > >>>>>>>>> > >>>>>>>>> but I > >>>>>>>>> > >>>>>>>>>> guess this can be investigated. > >>>>>>>>> > >>>>>>>>> +1 > >>>>>>>>> but I don't know where to look for myself: on that point, I hope > to > >>>>>>>>> have some > >>>>>>>>> help from Jenkinsfile experts > >>>>>>>>> and eventually start with svn, to have something before the git > >>>>>>>>> migration > >>>>>>>>> > >>>>>>>>>> Then it can be a kind of switch-case over committed folders in > >>>>>>>>> > >>>>>>>>> Jenkinsfile. > >>>>>>>>> > >>>>>>>>>> This means that every time another stage/sub-folder is shown in > >>>>>>>>> > >>>>>>>>> Pipeline > >>>>>>>>> > >>>>>>>>>> which will be messy in the UI. :( > >>>>>>>>> > >>>>>>>>> notice that we can perhaps split the repos in less parts than we > >>>> have > >>>>>>>>> components: on plugins, for example, we already have 4 categories > >>>> [1] > >>>>>>>>> which > >>>>>>>>> would avoid 1 repo with 41 plugins, but more something like > >>>> 6+10+10+15 > >>>>>>>>> I suppose we could do the same for shared (reporting folder comes > >>>> to my > >>>>>>>>> mind) > >>>>>>>>> > >>>>>>>>> The key feasibility issue is the capacity to release a > >> sub-component > >>>>>>>>> from git > >>>>>>>>> with maven-release-plugin, IMHO > >>>>>>>>> (taking apart the git purists idea of 1 lifecycle = 1 git repo) > >>>>>>>>> > >>>>>>>>> Regards, > >>>>>>>>> > >>>>>>>>> Hervé > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> [1] http://maven.apache.org/plugins/ > >>>>>>>>> > >>>>>>>>>> On Sat, Oct 7, 2017 at 4:17 AM, Hervé BOUTEMY < > >>>> [email protected]> > >>>>>>>>> > >>>>>>>>> wrote: > >>>>>>>>>>> There are 6 svn locations without any special complexity that > >>>> are > >>>>>>>>> > >>>>>>>>> waiting > >>>>>>>>> > >>>>>>>>>>> for > >>>>>>>>>>> a volunteer for git migration for a few years but nobody does > >>>>>>>>> > >>>>>>>>> anything > >>>>>>>>> > >>>>>>>>>>> about: > >>>>>>>>>>> I started 3 days ago to ask for help about it and got pretty no > >>>>>>>>> > >>>>>>>>> feedback > >>>>>>>>> > >>>>>>>>>>> [1] > >>>>>>>>>>> > >>>>>>>>>>> then there are the 4 complex svn locations (plugins, shared, > >>>>>>>>> > >>>>>>>>> resources, > >>>>>>>>> > >>>>>>>>>>> doxia- > >>>>>>>>>>> tools) that require much more work: I suppose we can't tell git > >>>>>>>>> > >>>>>>>>> migration > >>>>>>>>> > >>>>>>>>>>> is > >>>>>>>>>>> abandoned, but it will require someone to work on it seriously > >>>>>>>>>>> Remember that the key question [2] is: do we transform these 4 > >>>> svn > >>>>>>>>>>> locations > >>>>>>>>>>> into 41 +26 + 6 + 5 = 78 independent git repos? > >>>>>>>>>>> Yes, I told that Jenkins configuration is one key aspect we > >>>> need a > >>>>>>>>>>> strategy > >>>>>>>>>>> about, in parallel with git strategy. > >>>>>>>>> > >>>>>>>>>>> then there will be the remaining cases where we need to decide: > >>>>>>>>> lower > >>>>>>>>> > >>>>>>>>>>> impact, > >>>>>>>>>>> lower priority. > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> Summary: nothing is abandoned, but: > >>>>>>>>>>> - simple cases: no volunteer to do the job > >>>>>>>>>>> - hard cases: is there a volunteer either to define a strategy > >>>> or > >>>>>>>>> > >>>>>>>>> do some > >>>>>>>>> > >>>>>>>>>>> work > >>>>>>>>>>> on it? > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> Regards, > >>>>>>>>>>> > >>>>>>>>>>> Hervé > >>>>>>>>>>> > >>>>>>>>>>> [1] https://lists.apache.org/thread.html/ > >>>>>>>>>>> edf3642a7bdd515f0cad421c25589741819446463614bf0515e56dbe@ > >>>>>>>>>>> %3Cdev.maven.apache.org%3E > >>>>>>>>>>> > >>>>>>>>>>> [2] https://cwiki.apache.org/confluence/display/MAVEN/Git > >>>>>>>>>>> +Migration#GitMigration-Migratinganaggregatortreeintoacollec > >>>>>>>>> > >>>>>>>>> tionofgitrepos > >>>>>>>>> > >>>>>>>>>>> Le vendredi 6 octobre 2017, 20:35:45 CEST Arnaud Héritier a > >>>> écrit : > >>>>>>>>>>>> But did we completely abandoned the idea of moving everything > >>>> to > >>>>>>>>> > >>>>>>>>> git ? > >>>>>>>>> > >>>>>>>>>>>> The CI setup was the main stop for it ? > >>>>>>>>>>>> > >>>>>>>>>>>> On Fri, Oct 6, 2017 at 8:05 PM, Hervé BOUTEMY < > >>>>>>>>> > >>>>>>>>> [email protected]> > >>>>>>>>> > >>>>>>>>>>> wrote: > >>>>>>>>>>>>> I was expecting the usual litany > >>>>>>>>>>>>> > >>>>>>>>>>>>> what I'm not confident currently with pipeline on Maven > >>>> core is > >>>>>>>>> > >>>>>>>>> that > >>>>>>>>> > >>>>>>>>>>>>> we > >>>>>>>>>>>>> have > >>>>>>>>>>>>> for example the "maven-3.x-jenkinsfile/MNG-6242 - build #1 > >>>> - > >>>>>>>>> > >>>>>>>>> null" > >>>>>>>>> > >>>>>>>>>>>>> message, > >>>>>>>>>>>>> with this "null" part that makes me wonder if we are using > >>>> it > >>>>>>>>>>>>> as > >>>>>>>>>>> > >>>>>>>>>>> expected. > >>>>>>>>>>> > >>>>>>>>>>>>> And for large multi-module svn trunks (the ones we don't > >>>>>>>>> > >>>>>>>>> migrate to > >>>>>>>>> > >>>>>>>>>>> git: > >>>>>>>>>>>>> mainly plugins and shared), is there a solution to rebuild > >>>> just > >>>>>>>>>>>>> changed > >>>>>>>>>>>>> modules? > >>>>>>>>>>>>> > >>>>>>>>>>>>> ideally, the rebuild when a SNAPSHOT dependency is published > >>>>>>>>> > >>>>>>>>> would > >>>>>>>>> > >>>>>>>>>>>>> have > >>>>>>>>>>>>> been a > >>>>>>>>>>>>> plus, but this is sufficiently a rare use that I won't be > >>>>>>>>> > >>>>>>>>> extremist > >>>>>>>>> > >>>>>>>>>>>>> Then IIUC, this migration job is one additional TODO for > >>>> me... > >>>>>>>>>>>>> > >>>>>>>>>>>>> Regards, > >>>>>>>>>>>>> > >>>>>>>>>>>>> Hervé > >>>>>>>>>>>>> > >>>>>>>>>>>>> Le vendredi 6 octobre 2017, 17:46:53 CEST Arnaud Héritier a > >>>>>>>>> > >>>>>>>>> écrit : > >>>>>>>>>>>>>> I agree that we should use pipeline nowdays > >>>>>>>>>>>>>> perhaps a shared lib if we want to standardize some stuffs > >>>>>>>>> > >>>>>>>>> and a set > >>>>>>>>> > >>>>>>>>>>> of > >>>>>>>>>>> > >>>>>>>>>>>>>> multi-branches jobs (or org folder but it requires GitHub > >>>> :( > >>>>>>>>>>>>>> ) > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> On Fri, Oct 6, 2017 at 9:16 AM, Stephen Connolly < > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> [email protected]> wrote: > >>>>>>>>>>>>>>> On Fri 6 Oct 2017 at 06:32, Hervé BOUTEMY < > >>>>>>>>> > >>>>>>>>> [email protected]> > >>>>>>>>> > >>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>> I just fixed a few failing jobs [1] to have again a > >>>>>>>>>>>>>>>> usable > >>>>>>>>>>> > >>>>>>>>>>> Jenkins. > >>>>>>>>>>> > >>>>>>>>>>>>>>>> Now I'm facing some issues, I suppose caused by newer > >>>>>>>>> > >>>>>>>>> Jenkins > >>>>>>>>> > >>>>>>>>>>>>> versions: > >>>>>>>>>>>>>>>> - Maven 3.0.5 causes NoSuchMethodError: > >>>>>>>>>>>>>>>> o.c.plexus.util.xml.pull. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> MXParser > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> [2] > >>>>>>>>>>>>>>>> - I had to switch to JDK 8 for maven-plugin-tools job, > >>>>>>>>> > >>>>>>>>> since JDK > >>>>>>>>> > >>>>>>>>>>>>> causes > >>>>>>>>>>>>> > >>>>>>>>>>>>>>>> failures (looks like Jenkins uses a hack to inject > >>>> JDK 7 > >>>>>>>>> > >>>>>>>>> as a > >>>>>>>>> > >>>>>>>>>>> tool > >>>>>>>>>>> > >>>>>>>>>>>>> while > >>>>>>>>>>>>> > >>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>> build JVM is Java 8) > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Should we drop Maven 3.0.5 builds and JDK 7? > >>>>>>>>>>>>>>>> Notice I didn't check which is the minimum Maven > >>>> version > >>>>>>>>>>> > >>>>>>>>>>> required... > >>>>>>>>>>> > >>>>>>>>>>>>>>>> Or perhaps simply don't use the Jenkins Maven plugin > >>>> with > >>>>>>>>> > >>>>>>>>> this > >>>>>>>>> > >>>>>>>>>>> Maven > >>>>>>>>>>> > >>>>>>>>>>>>>>> 3.0.5 > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> or > >>>>>>>>>>>>>>>> JDK 7 configuration: default build as Jenkins Maven > >>>>>>>>> > >>>>>>>>> plugin with > >>>>>>>>> > >>>>>>>>>>> JDK > >>>>>>>>>>> > >>>>>>>>>>>>> 8 + > >>>>>>>>>>>>> > >>>>>>>>>>>>>>>> latest > >>>>>>>>>>>>>>>> Maven, and other configurations as scripted jobs? > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> http://javaadventure.blogspot. > >>>> ie/2013/11/jenkins-maven-job-> >>>> > > > > > > > > >>>>>>>>>>>>> > >>>>>>>>>>>>> type-considered-evil.html > >>>>>>>>>>>>> > >>>>>>>>>>>>>>> <http://javaadventure.blogspot.ie/2013/11/jenkins-> > > >>>>>>>>>>>>> > >>>>>>>>>>>>> maven-job-type-considered-evil.html?m=1> > >>>>>>>>>>>>> > >>>>>>>>>>>>>>> We should stop using the evil job type as it’s minimum > >>>> Java > >>>>>>>>>>> > >>>>>>>>>>> version is > >>>>>>>>>>> > >>>>>>>>>>>>>>> dictated by Jenkins’ Java minimum. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> We need to define our common strategy and have a > >>>>>>>>> > >>>>>>>>> consistent > >>>>>>>>> > >>>>>>>>>>>>>>>> configuration > >>>>>>>>>>>>>>>> for > >>>>>>>>>>>>>>>> every job understood by everybody > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> I recommend Jenkinsfile and the `withMaven` wrapper > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Regards, > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Hervé > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> [1] https://builds.apache.org/view/M-R/view/Maven/ > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> [2] > >>>>>>>>>>>>>>>> https://builds.apache.org/ > >>>> view/M-R/view/Maven/job/maven-> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> plugin-tools-jdk-1.7/162/console > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> ------------------------------ > >>>>>>>>> > >>>>>>>>> ------------------------------ > >>>>>>>>> > >>>>>>>>>>>>> --------- > >>>>>>>>>>>>> > >>>>>>>>>>>>>>>> To unsubscribe, e-mail: [email protected]. > >>>> org > >>>>>>>>> > >>>>>>>>>>>>>>>> For additional commands, e-mail: > >>>>>>>>> [email protected] > >>>>>>>>> > >>>>>>>>>>>>>>>> -- > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Sent from my phone > >>>>>>>>>>>>> > >>>>>>>>>>>>> ------------------------------ > >>>> ------------------------------ > >>>>>>>>> > >>>>>>>>> --------- > >>>>>>>>> > >>>>>>>>>>>>> To unsubscribe, e-mail: [email protected] > >>>>>>>>>>>>> For additional commands, e-mail: [email protected] > >>>>>>>>>>> > >>>>>>>>>>> ------------------------------------------------------------ > >>>>>>>>> > >>>>>>>>> --------- > >>>>>>>>> > >>>>>>>>>>> To unsubscribe, e-mail: [email protected] > >>>>>>>>>>> For additional commands, e-mail: [email protected] > >>>>>>>>> > >>>>>>>>> ------------------------------------------------------------ > >>>> --------- > >>>>>>>>> To unsubscribe, e-mail: [email protected] > >>>>>>>>> For additional commands, e-mail: [email protected] > >>>>>>>> > >>>>>>>> -- > >>>>>>>> Cheers > >>>>>>>> Tibor > >>>> > >>>> > >>>> > >>>> --------------------------------------------------------------------- > >>>> To unsubscribe, e-mail: [email protected] > >>>> For additional commands, e-mail: [email protected] > >>>> > >>>> > >>> > >>> > >>> -- > >>> ----- > >>> Arnaud Héritier > >>> http://aheritier.net > >>> Mail/GTalk: aheritier AT gmail DOT com > >>> Twitter/Skype : aheritier > >> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: [email protected] > >> For additional commands, e-mail: [email protected] > >> > >> > > > > > > -- > > ----- > > Arnaud Héritier > > http://aheritier.net > > Mail/GTalk: aheritier AT gmail DOT com > > Twitter/Skype : aheritier > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
