On 9/3/13 8:25 AM, Ehsan Akhgari wrote: > Thanks for the update on this, John! I have a number of questions about > this: > > 1. On the issue of the hg tags, can you please be more specific about the > problem that you're encountering? In my experience, git deals with updated > tags the same way as it does with any non-fast-forward merge, and I've > never experienced any problems with that.
I'm explicitly not converting most tags, and only whitelisting certain ones to avoid the issue caused by moving tags. To quote https://www.kernel.org/pub/software/scm/git/docs/git-tag.html#_on_re_tagging on moving git tags: ``Does this seem a bit complicated? It should be. There is no way that it would be correct to just "fix" it automatically. People need to know that their tags might have been changed.'' We move tags regularly on hg repos; this is standard operating procedure for a release build 2, or if a build 1 has an automation hiccup. While we *could* convert the tags automatically, then either never move them or move them behind users' backs, users would then never get the updated tags unless they explicitly delete and re-fetch the tag by name... something people wouldn't typically do without prompting. In my opinion, tags pointing at the wrong revision are worse than no tags. Also, I need to limit the tags pushed to gecko.git, as there is a hard fastforward-only rule there, and notifying partners to delete and recreate tags seems like a non-starter. So I built in tag-limiting whitelists for safety. However, there appears to be an issue with the way I'm limiting tags. Rather than delay things further, I decided to publish as-is and see if anyone really cares about the tags, or if they would be fine using the tip of each relbranch instead. Since you bring up the point of tags, can you give examples of how you use tags, or how you have seen others use tags? > 2. How frequently are these branches updated? The current job is running on a 5 minute cron job. We can, of course, change that if needed. When I add a new repo or a slew of new branches to convert, the job can take longer to complete, but it typically finishes in about 6 minutes. > 3. What are the plans for adding the rest of the branches that we currently > have in https://github.com/mozilla/mozilla-central, an what is the process > for developers to request their project branches to be added to the > conversion jobs? Right now the process is pretty light-weight (they > usually just ping me on IRC or send me an email). It would be nice if we > kept that property. There are two concerns here: Project branches are often reset, and also individual developers care about different subsets of branches. Providing a core set of branches that everyone uses, and which we can safely support at scale, seems a good set to include by default. For users of other branches, one approach we're looking at is to provide supported documentation that shows developers how to add any-branch-of-their-choice to their local repo. We're still figuring out what this default set should be, and as you have clearly expressed opinions on this in the past, we'd of course be interested in your current opinions here. > 4. About the interaction between gecko.git and beagle, when you say that > they cannot be identical, are you talking about the location where they're > hosted or the SHA1s for the same commits? I think we need to guarantee the > latter being identical going forward, but the former doesn't matter as much. It's a requirement to keep the SHA1s the same, and we are confident we can do that. The location and naming for legacy b2g branches will be different, as well as a hard fastforward-only rule on gecko.git. > 5. Is the git-mapfile generated from the conversion scrips available > somewhere? (Mine is currently available here: > http://git.mozilla.org/?p=users/eakhg...@mozilla.com/mozilla-history-tools.git;a=history;f=updates/git-mapfile;h=5c8ecf4bf1d7339ad1dff1ea4ab62a712da2fb6c;hb=HEAD > ) I'm uploading this per run, but it's currently not going anywhere publicly accessible. We'll change the location of the upload to a public location before signing off on this repo. > 6. Do you have any plans for how we're going to approach the problem of > migrating the current git mirror with beagle? Specifically, I think we > need to: > a) Figure out how github deals with a repository being deleted/recreated, > would that break people's forks, stars, etc.? > b) Are there any alternative solutions in replacing all of the > branches/tags under mozilla/mozilla-central with the new repo? > c) What is the developer work flow going to look like in rebasing the local > branches once the move has been completed? This is a one time hit, but obviously, we want the least churn and disruption possible. We're still putting that plan together, so all good questions to add to the planning criteria. First, lets get feedback that the test-beagle repos looks good and can be signed off. aki (originals to end) > Cheers, > > -- > Ehsan > <http://ehsanakhgari.org/> > > > On Fri, Aug 30, 2013 at 9:29 PM, John O'Duinn <jodu...@mozilla.com> wrote: > >> (cross-posting to make sure this is not missed; please respond in >> dev-planning or bug#847727) >> >> >> >> tl,dr: we're moving replication of gecko+gaia repos to more robust >> systems. This email gives quick status, and summary of rollout plan. >> >> >> >> >> In the rush to B2G 1.0 release, we had to quickly stand up replication >> systems to mirror changes from github to hg.m.o and to git.m.o. These >> locally-patched systems replicate 368 repos, 24x7, and have carried us >> a long way, but they are fragile to network/system hiccups, and tricky >> to configure for new branches. Now that v1.0 has shipped, and we're >> into more predictable development cadence, its time to switch to more >> robust systems. >> >> Over the last few months, we've been standing up new systems alongside >> our current production systems. On the new systems, we've been testing >> newer version of hg-git conversion by running our biggest, most >> comprehensive testdata we have - the entire cvs-history and hg-history >> of mozilla-central. When we hit problems that break the hg-git >> conversion tools, we upstreamed fixes to the hg-git tools. Being able >> to process all the edge-cases that Mozilla developers have thrown at >> cvs and hg since the start of Mozilla gives us confidence that these >> scripts are robust enough to use in production going forward. >> >> The curious can see the generated, fully converted repo, at >> https://github.com/escapewindow/test-beagle. It's roughly equivalent to >> https://github.com/mozilla/mozilla-central, but with SHAs that match >> http://git.mozilla.org/?p=releases/gecko.git;a=summary . >> >> The "test-beagle" repo currently contains: >> * mozilla-central with full CVS history >> * mozilla-b2g18 >> * mozilla-b2g18_v1_1_0_hd >> * mozilla-b2g18_v1_0_1 >> * mozilla-b2g18_v1_0_0 >> * mozilla-aurora >> * mozilla-beta, with relbranches >> * mozilla-release, with relbranches >> * mozilla-esr17 >> * mozilla-inbound >> * b2g-inbound >> * fx-team >> >> If you see anything that looks wrong in this repo, please do let us know. >> >> Note: >> 1) Do *not* use this for any production work or real development >> work... not just yet. This repo is, as the name suggests, a test repo >> that we reserve the right to reset at any point as/when we find problems. >> 2) The branches to be included in this repo, and the final name and >> location of the "test-beagle" repo, are still in flux. We're >> considering having separate conversions of other repos that can be >> pulled into local clones if desired. For more background, read the blog >> post below for details. >> 3) Note that these can not be identical repos due to partner >> requirements for gecko.git; see the blog post below). >> 4) All the release branches generated per release are present, but >> currently no release tags. Moving tags is problematic in git; see the >> blog post below for details. >> >> >> Our rollout plan is to: >> * enable "test-beagle" repo in mozilla account on github (using real >> name of course!) >> * replace/reset these repos on github >> (https://github.com/mozilla/releases-mozilla-central, >> https://github.com/mozilla/releases-mozilla-aurora, >> https://github.com/mozilla/releases-mozilla-release) since they don't >> have CVS history and have a different set of SHAs. If they're unneeded >> and we can nuke 'em, even better. >> * switch over http://git.mozilla.org/?p=releases/gecko.git;a=summary >> to be on these new scripts. As the SHAs are the same, this should be >> invisible to Mozilla devs and partners using it, and should buy us >> better stability. >> >> There's plenty of other work after this deploy is completed (l10n, >> bitbucket, etc), but this is a big leap forward and exciting to see. >> >> >> Hope all that makes sense - for more information, please see: >> * the blog post http://escapewindow.dreamwidth.org/238476.html >> * the bug https://bugzilla.mozilla.org/show_bug.cgi?id=847727 >> >> >> >> Thanks >> Aki, Hal and John. >> >> _______________________________________________ >> dev-platform mailing list >> dev-platform@lists.mozilla.org >> https://lists.mozilla.org/listinfo/dev-platform >> >> _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
