Re: [DISCUSSION] Growing the Git community

2019-10-04 Thread Jakub Narebski
"brian m. carlson" writes: > I think GitGitGadget is a useful tool which I haven't really had the > time to learn how to use. I appreciate that many people prefer a > patch-based workflow, and that using a patch-based workflow and a > mailing list provides the project independence and avoids fav

Re: [DISCUSSION] Growing the Git community

2019-10-04 Thread Jakub Narebski
Derrick Stolee writes: > On 9/25/2019 9:36 AM, Pierre Tardy wrote: [...] >> And why restrict on DVCS? >> Isn't it admitted that the distributed version control is nowadays >> much better in term of software productivity? >> Is there some use cases that "traditional" centralized VCS are better >> o

Re: [DISCUSSION] Growing the Git community

2019-10-04 Thread Jakub Narebski
Elijah Newren writes: > On Thu, Sep 19, 2019 at 11:37 AM Derrick Stolee wrote: [...] >> II. Approach >> >> The action items below match the problems listed above. >> >> 1. Improve the documentation for contributing to Git. >> >> In preparation for this email, I talked to someone familiar with is

Re: [DISCUSSION] Growing the Git community

2019-10-01 Thread Jakub Narebski
Hello, Johannes Schindelin writes: > On Fri, 20 Sep 2019, Mike Hommey wrote: >> 6. Newcomers don't really have any idea /what/ they could contribute. >> They either have to come with their own itch to scratch, or read the >> code to figure out if there's something to fix. > > I think this is ver

[RFC/PATCH] commit-graph: generation v5 (backward compatible date ceiling)

2019-09-18 Thread Jakub Narebski
Derrick Stolee writes: > On 6/25/2019 3:51 AM, Jakub Narebski wrote: >> Jakub Narebski writes: >>> Derrick Stolee writes: [...] >>> O.K., so the "generation number v2 (legacy)" would be incremental and >>> backward-compatibile in use (though not i

Re: [PATCH v3 0/3] [RFC] Create 'core.featureAdoptionRate' setting to update config defaults

2019-07-11 Thread Jakub Narebski
Derrick Stolee writes: > On 7/1/2019 10:29 AM, Derrick Stolee via GitGitGadget wrote: >> >> Here is a second run at this RFC, which aims to create a "meta" config >> setting that automatically turns on other settings according to a user's >> willingness to trade new Git behavior or new feature ris

Re: [Question] Diff text filters and git add

2019-07-11 Thread Jakub Narebski
"Randall S. Becker" writes: > On July 9, 2019 5:51 PM, Peff wrote: [...] >> No, textconv only applies when generating a diff to output, and will never >> impact what's stored in Git. >> >> It sounds like you might want a clean filter instead, to sanitize the file >> contents as they come into Git

Re: [RFC PATCH v2 2/3] trace2: add a schema validator for trace2 events

2019-07-11 Thread Jakub Narebski
Josh Steadmon writes: > trace_schema_validator can be used to verify that trace2 event output > conforms to the expectations set by the API documentation and codified > in event_schema.json (or strict_schema.json). This allows us to build a > regression test to verify that trace2 output does not

Re: [RFC PATCH v2 1/3] trace2: Add a JSON schema for trace2 events

2019-07-10 Thread Jakub Narebski
Josh Steadmon writes: > Define a JSON schema[1] that can be used to validate trace2 event > objects. This can be used to add regression tests to verify that the > event output format does not change unexpectedly. > > Two versions of the schema are provided: Actually, four versions of the schema

Re: Virtual Git Contributor Summit

2019-07-05 Thread Jakub Narebski
Johannes Schindelin writes: > Team, > > I kept talking about this idea of a purely online Git Contributor Summit, > and it is finally time for action. > > The idea: just like the Git Contributor Summits we have on the day before > GitMerge, but instead of traveling to the same place, we'll use an

Re: Surprising use of memory and time when repacking mozilla's gecko repository

2019-07-05 Thread Jakub Narebski
Mike Hommey writes: > On Fri, Jul 05, 2019 at 01:14:13AM -0400, Jeff King wrote: >> On Thu, Jul 04, 2019 at 10:13:20PM +0900, Mike Hommey wrote: [...] >> I think I explained all of the memory-usage questions in my earlier >> response, but just for reference: if you have access to it, valgrind's >

Re: [PATCH v3 1/3] repo-settings: create core.featureAdoptionRate setting

2019-07-04 Thread Jakub Narebski
Duy Nguyen writes: > On Mon, Jul 1, 2019 at 10:32 PM Derrick Stolee via GitGitGadget > wrote: >> @@ -601,3 +602,22 @@ core.abbrev:: >> in your repository, which hopefully is enough for >> abbreviated object names to stay unique for some time. >> The minimum length is 4. >>

Re: [PATCH/RFC] get_oid: new extended SHA-1 syntax to control resolution process

2019-06-30 Thread Jakub Narebski
Bikeshed painting ahead. Nguyễn Thái Ngọc Duy writes: [...] > The problem is we try every possible way to resolve a rev. Let's have > some annotation to express that we only want to resolve a rev in a > certain way: > > - @{hash} only accepts a full hash or a short hash. If it's a > short hash,

Re: standalone library/tool to query commit-graph?

2019-06-25 Thread Jakub Narebski
Ævar Arnfjörð Bjarmason writes: [...] > To clarify (and I should have said) I meant it'll include only packed > commits in the mode Karl Ostmo invoked it in, as Derrick points out. > > But yeah, you can of course give it arbitrary starting points, but > needing to deal with those sorts of caveats

Re: Revision walking, commit dates, slop

2019-06-25 Thread Jakub Narebski
Jakub Narebski writes: > Derrick Stolee writes: >> On 5/20/2019 7:02 AM, Jakub Narebski wrote: >>> >>> Are there any blockers that prevent the switch to this >>> "generation number v2"? >>> >>> - Is it a problem with insufficient

Re: [RFC PATCH 3/3] trace2: add a schema validator for trace2 events

2019-06-21 Thread Jakub Narebski
Josh Steadmon writes: > On 2019.06.12 15:28, Ævar Arnfjörð Bjarmason wrote: >> On Wed, Jun 12 2019, Josh Steadmon wrote: >> >>> trace_schema_validator can be used to verify that trace2 event output >>> conforms to the expectations set by the API documentation and codified >>> in event_schema.jso

Re: [PATCH 2/3] hash-object doc: elaborate on -w and --literally promises

2019-05-24 Thread Jakub Narebski
Jeff King writes: > On Mon, May 20, 2019 at 11:53:11PM +0200, Ævar Arnfjörð Bjarmason wrote: > >> Clarify the hash-object docs to explicitly note that the --literally >> option guarantees that a loose object will be written, but that a >> normal -w ("write") invocation doesn't. > > I had to double

Re: Revision walking, commit dates, slop

2019-05-23 Thread Jakub Narebski
Derrick Stolee writes: > On 5/22/2019 2:29 PM, Jakub Narebski wrote: >> Derrick Stolee writes: >>> On 5/20/2019 7:27 PM, Jakub Narebski wrote: >> >> Restating it yet again: >> >>A. corrected_date(C) = max(committer_date(C), >>

Re: standalone library/tool to query commit-graph?

2019-05-23 Thread Jakub Narebski
Derrick Stolee writes: > On 5/22/2019 2:49 PM, Karl Ostmo wrote: >> After producing the file ".git/objects/info/commit-graph" with the >> command "git commit-graph write", is there a way to answer queries >> like "git merge-base --is-ancestor" without having a .git directory? >> E.g. is there a l

Re: RFC: Separate commit identification from Merkle hashing

2019-05-23 Thread Jakub Narebski
e...@thyrsus.com (Eric S. Raymond) writes: > I have been thinking hard about the problems raised during my > request for unique timestamps. I think I've found a better way > to bust the box I was trying to break out of. I am therefore > withdrawing that proposal and replacing it with this one. >

Re: Revision walking, commit dates, slop

2019-05-22 Thread Jakub Narebski
Derrick Stolee writes: > On 5/20/2019 7:27 PM, Jakub Narebski wrote: >> Jakub Narebski writes: >>> Derrick Stolee writes: >>>> On 5/20/2019 7:02 AM, Jakub Narebski wrote: >>>>> >>>>> Are there any blockers

Re: Finer timestamps and serialization in git

2019-05-20 Thread Jakub Narebski
"Eric S. Raymond" writes: > Jakub Narebski : >>> What "commits that follow it?" By hypothesis, the incoming commit's >>> timestamp is bumped (if it's bumped) when it's first added to a branch >>> or branches, before there are foll

Re: Revision walking, commit dates, slop

2019-05-20 Thread Jakub Narebski
Jakub Narebski writes: > Derrick Stolee writes: >> On 5/20/2019 7:02 AM, Jakub Narebski wrote: >>> >>> Are there any blockers that prevent the switch to this >>> "generation number v2"? [...] >> Using the generation nu

Re: Merging (joining/stiching/rewriting) history of "unrelated" git repositories

2019-05-20 Thread Jakub Narebski
Elijah Newren writes: > On Wed, May 15, 2019 at 8:30 AM Ævar Arnfjörð Bjarmason > wrote: >> On Wed, May 15 2019, Piotr Krukowiecki wrote: >>> >>> I'm migrating two repositories from svn. I already did svn->git >>> migration (git-svn clone) and now have two git repositories. >>> >>> I would like t

Re: Revision walking, commit dates, slop

2019-05-20 Thread Jakub Narebski
Derrick Stolee writes: > On 5/20/2019 7:02 AM, Jakub Narebski wrote: >> >> Are there any blockers that prevent the switch to this >> "generation number v2"? >> >> - Is it a problem with insufficient data to choose the correct numbering >>

Re: Revision walking, commit dates, slop

2019-05-20 Thread Jakub Narebski
Derrick Stolee writes: > On 5/18/2019 12:17 AM, Mike Hommey wrote: >> On Sat, May 18, 2019 at 12:58:28PM +0900, Mike Hommey wrote: >>> On Sat, May 18, 2019 at 03:50:05AM +0200, SZEDER Gábor wrote: All the above is without commit-graph, I presume? If so, then you should give it a tr

Re: Finer timestamps and serialization in git

2019-05-20 Thread Jakub Narebski
"Eric S. Raymond" writes: > Jakub Narebski : >> As far as I understand it this would slow down receiving new commits >> tremendously. Currently great care is taken to not have to parse the >> commit object during fetch or push if it is not necessary (thanks to

Re: Finer timestamps and serialization in git

2019-05-19 Thread Jakub Narebski
Ævar Arnfjörð Bjarmason writes: > On Thu, May 16 2019, Eric S. Raymond wrote: >> Derrick Stolee : >>> On 5/15/2019 3:16 PM, Eric S. Raymond wrote: The deeper problem is that I want something from Git that I cannot have with 1-second granularity. That is: a unique timestamp on each c

Re: Revision walking, commit dates, slop

2019-05-19 Thread Jakub Narebski
SZEDER Gábor writes: > On Sat, May 18, 2019 at 01:17:06PM +0900, Mike Hommey wrote: >> On Sat, May 18, 2019 at 12:58:28PM +0900, Mike Hommey wrote: >>> On Sat, May 18, 2019 at 03:50:05AM +0200, SZEDER Gábor wrote: On Sat, May 18, 2019 at 09:54:12AM +0900, Mike Hommey wrote: [...] > There

Re: if YOU use a Windows GUI for Git, i would appreciate knowing which one and why

2019-05-01 Thread Jakub Narebski
"_g e r r y _ _l o w r y _" writes: > > > QUESTION: if YOU use a Windows GUI for Git, i would appreciate knowing which > one and why > > i have been asked to look at GUI versions of Git for Windows. > > https://git-scm.com/download/gui/windows currently

Re: Feature request: Allow to update commit ID in messages when rebasing

2019-04-19 Thread Jakub Narebski
Hello, Giuseppe Crinò writes: > On Thu, Apr 18, 2019 at 7:32 PM Jakub Narebski wrote: >> Well, what about limiting changes and rewriting only to the commits >> being rewritten by [interactive] rebase? I mean that we would rewrite >> "revert 01a9fe8" only if:

Re: Feature request: Allow to update commit ID in messages when rebasing

2019-04-18 Thread Jakub Narebski
Junio C Hamano writes: > Ævar Arnfjörð Bjarmason writes: >> On Wed, Apr 17 2019, Giuseppe Crinò wrote: >> >>> The feature I'm asking is to add an extra-step during rebasing, >>> checking whether there's a reference to a commit that's not going to >>> be included in history and asks the user wheth

Re: "commit --author=..." does not work if global email and name is not set

2019-04-09 Thread Jakub Narebski
Piotr Krukowiecki writes: > On Mon, Apr 8, 2019 at 1:06 PM Jakub Narebski wrote: >> Piotr Krukowiecki writes: >>>> On Sat, Apr 6, 2019 at 8:25 PM Jakub Narebski wrote: >>>>> >>>>> Better though is to focus on what you want, namely to preven

Re: "commit --author=..." does not work if global email and name is not set

2019-04-08 Thread Jakub Narebski
Piotr Krukowiecki writes: >> On Sat, Apr 6, 2019 at 8:25 PM Jakub Narebski wrote: >>> >>> Better though is to focus on what you want, namely to prevent accidental >>> commits without specified author, instead of how you want to achieve it, >>> i.e.

Re: "commit --author=..." does not work if global email and name is not set

2019-04-06 Thread Jakub Narebski
Piotr Krukowiecki writes: > Hi, > > I have a repo downloaded on machines which do automatic tests. > Sometimes I want to make a quick fix there and push back to origin. > Reading git-commit docs I had impression that I can use "--author=me" > and it will work. But it requires setting global user

Re: Contributor Summit Topics and Logistics

2019-02-02 Thread Jakub Narebski
Jeff King writes: > On Tue, Jan 22, 2019 at 02:50:27AM -0500, Jeff King wrote: > >> If you're not coming, you can probably stop reading this message now. >> The rest is all logistics. > > Here are a few additional last-minute logistics: > >> For people who want to try to join remotely, I don't th

Re: [RFC] Generation Number v2

2018-11-03 Thread Jakub Narebski
Jakub Narebski writes: > Jakub Narebski writes: >> Stefan Beller writes: > [...] >>> How would this impact creation of a commit? >>> >>> The current generation numbers can be lazily updated or not >>> updated at all. In my understanding of the ma

Re: [RFC] Generation Number v2

2018-11-03 Thread Jakub Narebski
Derrick Stolee writes: > On 11/1/2018 8:27 AM, Jakub Narebski wrote: >> Derrick Stolee writes: >> >>> Please also let me know about any additional tests that I could >>> run. Now that I've got a lot of test scripts built up, I can re-run >>> t

Re: [RFC] Generation Number v2

2018-11-02 Thread Jakub Narebski
Derrick Stolee writes: > On 10/31/2018 8:54 AM, Ævar Arnfjörð Bjarmason wrote: >> On Tue, Oct 30 2018, Junio C Hamano wrote: >>> Derrick Stolee writes: In contrast, maximum generation numbers and corrected commit dates both performed quite well. They are frequently the top two

Re: [RFC] Generation Number v2

2018-11-02 Thread Jakub Narebski
Derrick Stolee writes: > On 10/29/2018 11:59 PM, Junio C Hamano wrote: >> Derrick Stolee writes: [...] >>> * **Compatible?** In our test implementation, we use a previously unused >>>byte of data in the commit-graph format to indicate which reachability >>>index version we are using. Exis

Re: [RFC] Generation Number v2

2018-11-02 Thread Jakub Narebski
Jakub Narebski writes: > Stefan Beller writes: [...] >> How would this impact creation of a commit? >> >> The current generation numbers can be lazily updated or not >> updated at all. In my understanding of the maximum generation >> numbers, a new commit wo

Re: [RFC] Generation Number v2

2018-11-01 Thread Jakub Narebski
Derrick Stolee writes: > Here is a re-formatted version of the tables I introduced earlier. > The tables were too wide for public-inbox to render correctly (when > paired with my email client). Hopefully this bulleted-list format > works better. Thanks, Stefan, for pointing out the rendering > pr

Re: [RFC] Generation Number v2

2018-11-01 Thread Jakub Narebski
Stefan Beller writes: >> Based on the performance results alone, we should remove minimum >> generation numbers, (epoch, date) pairs, and FELINE index from >> consideration. There are enough examples of these indexes performing >> poorly. >> >> In contrast, maximum generation numbers and correcte

Re: [RFC] Generation Number v2

2018-11-01 Thread Jakub Narebski
[I have noticed that in some places I wrote A..B instead of B..A. Sorry about that] Derrick Stolee writes: > We've discussed in several places how to improve upon generation > numbers. This RFC is a report based on my investigation into a > few new options, and how they compare for Git's purpo

Re: [PATCH v3 10/12] Add a base implementation of SHA-256 support

2018-10-27 Thread Jakub Narebski
"brian m. carlson" writes: > SHA-1 is weak and we need to transition to a new hash function. For > some time, we have referred to this new function as NewHash. Recently, > we decided to pick SHA-256 as NewHash. Even if we have decided to not repeat the reasoning behind the need to switch away

Re: [PATCH] Poison gettext with the Ook language

2018-10-27 Thread Jakub Narebski
SZEDER Gábor writes: > On Mon, Oct 22, 2018 at 05:36:33PM +0200, Nguyễn Thái Ngọc Duy wrote: >> >> The current gettext() function just replaces all strings with >> '# GETTEXT POISON #' including format strings and hides the things >> that we should be allowed to grep (like branch names, or some ot

Re: [PATCH v4 4/7] revision.c: begin refactoring --topo-order logic

2018-10-26 Thread Jakub Narebski
Junio C Hamano writes: > Derrick Stolee writes: > >>     time git log --topo-order -10 master >/dev/null >> >>     time git log --topo-order -10 maint..master >/dev/null >> >> I get 0.39s for the first call and 0.01s for the second. (Note: I >> specified "-10" to ensure we are only writing 10 co

Re: [PATCH v4 6/7] revision.c: generation-based topo-order algorithm

2018-10-26 Thread Jakub Narebski
Derrick Stolee writes: > On 10/22/2018 9:37 AM, Jakub Narebski wrote: >> "Derrick Stolee via GitGitGadget" writes: [...] >> Sidenote: the new algorithm looks a bit like Unix pipeline, where each >> step of pipeline does not output much more than next step

Re: [PATCH v4 7/7] t6012: make rev-list tests more interesting

2018-10-23 Thread Jakub Narebski
"Derrick Stolee via GitGitGadget" writes: > From: Derrick Stolee > > As we are working to rewrite some of the revision-walk machinery, > there could easily be some interesting interactions between the > options that force topological constraints (--topo-order, > --date-order, and --author-date-o

Re: [PATCH v4 6/7] revision.c: generation-based topo-order algorithm

2018-10-22 Thread Jakub Narebski
"Derrick Stolee via GitGitGadget" writes: > From: Derrick Stolee > > The current --topo-order algorithm requires walking all > reachable commits up front, topo-sorting them, all before > outputting the first value. This patch introduces a new > algorithm which uses stored generation numbers to >

Re: [PATCH v4 5/7] commit/revisions: bookkeeping before refactoring

2018-10-21 Thread Jakub Narebski
"Derrick Stolee via GitGitGadget" writes: > From: Derrick Stolee > > There are a few things that need to move around a little before > making a big refactoring in the topo-order logic: > > 1. We need access to record_author_date() and >compare_commits_by_author_date() in revision.c. These ar

Re: [PATCH v4 4/7] revision.c: begin refactoring --topo-order logic

2018-10-21 Thread Jakub Narebski
"Derrick Stolee via GitGitGadget" writes: > From: Derrick Stolee > > When running 'git rev-list --topo-order' and its kin, the topo_order > setting in struct rev_info implies the limited setting. This means > that the following things happen during prepare_revision_walk(): > > * revs->limited im

Re: [PATCH v4 0/7] Use generation numbers for --topo-order

2018-10-21 Thread Jakub Narebski
"Derrick Stolee via GitGitGadget" writes: > This patch series performs a decently-sized refactoring of the revision-walk > machinery. Well, "refactoring" is probably the wrong word, as I don't > actually remove the old code. Instead, when we see certain options in the > 'rev_info' struct, we redi

Re: [PATCH v4 3/7] test-reach: add rev-list tests

2018-10-21 Thread Jakub Narebski
"Derrick Stolee via GitGitGadget" writes: > From: Derrick Stolee > > The rev-list command is critical to Git's functionality. Ensure it > works in the three commit-graph environments constructed in > t6600-test-reach.sh. Here are a few important types of rev-list > operations: > > * Basic: git r

Re: [PATCH v3 7/7] revision.c: refactor basic topo-order logic

2018-10-06 Thread Jakub Narebski
Derrick Stolee writes: > On 9/21/2018 1:39 PM, Derrick Stolee via GitGitGadget wrote: > Hello, Git contributors. > > I understand that this commit message and patch are pretty > daunting. There is a lot to read and digest. I would like to see if > anyone is willing to put the work in to review t

Re: [PATCH 1/1] commit: don't use generation numbers if not needed

2018-10-06 Thread Jakub Narebski
"Derrick Stolee via GitGitGadget" writes: > From: Derrick Stolee > > In 3afc679b "commit: use generations in paint_down_to_common()", > the queue in paint_down_to_common() was changed to use a priority > order based on generation number before commit date. This served > two purposes: > > 1. Whe

Re: [PATCH 0/1] v2.19.0-rc1 Performance Regression in 'git merge-base'

2018-10-06 Thread Jakub Narebski
"Derrick Stolee via GitGitGadget" writes: > As I was testing the release candidate, I stumbled across a regression in > 'git merge-base' as a result of the switch to generation numbers. The commit > message in [PATCH 1/1] describes the topology involved, but you can test it > yourself by comparin

Re: Git Evolve

2018-10-04 Thread Jakub Narebski
Junio C Hamano writes: > Stefan Xenos writes: > >> What is the evolve command? >> ... >> - Systems like gerrit would no longer need to rely on "change-id" tags >> in commit comments to associate commits with the change that they >> edit, since git itself would have that information. >> ... >> Is

Re: What's cooking in git.git (Aug 2018, #01; Thu, 2)

2018-08-07 Thread Jakub Narebski
Junio C Hamano writes: > * ds/commit-graph-with-grafts (2018-07-19) 8 commits > (merged to 'next' on 2018-08-02 at 0ee624e329) > + commit-graph: close_commit_graph before shallow walk > + commit-graph: not compatible with uninitialized repo > + commit-graph: not compatible with grafts > + c

Re: [PATCH] DO-NOT-MERGE: write and read commit-graph always

2018-07-31 Thread Jakub Narebski
Stefan Beller writes: >> I wonder though if all those changes to the testsuite shouldn't be >> merged. > > I think Stolee doesn't want this to be merged after rereading > subject and the commit message. Yes, I understand that, and for the most part I agree with it. This commit main purpose is t

Re: [PATCH 8/8] commit-graph: close_commit_graph before shallow walk

2018-07-30 Thread Jakub Narebski
"Derrick Stolee via GitGitGadget" writes: > From: Derrick Stolee > > Make close_commit_graph() work for arbitrary repositories. The first line (above) does not match the rest of the commit message. > Call close_commit_graph() when about to start a rev-list walk that > includes shallow commits.

Re: [PATCH] DO-NOT-MERGE: write and read commit-graph always

2018-07-30 Thread Jakub Narebski
Derrick Stolee writes: > This commit is not intended to be merged, but is only to create a test > environment to see what works with the commit-graph feature and what > does not. The tests that fail are primarily related to corrupting the > object store to remove a commit from visibility and test

Re: [PATCH 7/8] commit-graph: not compatible with uninitialized repo

2018-07-29 Thread Jakub Narebski
"Derrick Stolee via GitGitGadget" writes: [I hope that the rest of replies would make it all right through GitGitGadget] > From: Derrick Stolee > > Signed-off-by: Derrick Stolee > --- > commit-graph.c | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/commit-graph.c b/commit-graph.c

Re: [PATCH 6/8] commit-graph: not compatible with grafts

2018-07-29 Thread Jakub Narebski
"Derrick Stolee via GitGitGadget" writes: > From: Derrick Stolee > > Augment commit_graph_compatible(r) to return false when the given > repository r has commit grafts or is a shallow clone. Test that in these > situations we ignore existing commit-graph files and we do not write new > commit-gr

Re: [PATCH 4/8] test-repository: properly init repo

2018-07-29 Thread Jakub Narebski
"Derrick Stolee via GitGitGadget" writes: > From: Derrick Stolee > > Signed-off-by: Derrick Stolee > --- > t/helper/test-repository.c | 10 -- > 1 file changed, 8 insertions(+), 2 deletions(-) > > diff --git a/t/helper/test-repository.c b/t/helper/test-repository.c > index 2762ca656..6

Re: [PATCH 5/8] commit-graph: not compatible with replace objects

2018-07-29 Thread Jakub Narebski
"Derrick Stolee via GitGitGadget" writes: > From: Derrick Stolee > > Create new method commit_graph_compatible(r) to check if a given > repository r is compatible with the commit-graph feature. Fill the > method with a check to see if replace-objects exist. All right, looks sensible. Did you s

Re: [PATCH 3/8] commit-graph: update design document

2018-07-29 Thread Jakub Narebski
"Derrick Stolee via GitGitGadget" writes: > From: Derrick Stolee > > As it exists right now, the commit-graph feature may provide > inconsistent results when combined with commit grafts, replace objects, > and shallow clones. Update the design document to discuss why these > interactions are dif

Re: [PATCH 0/8] Clarify commit-graph and grafts/replace/shallow incompatibilities

2018-07-29 Thread Jakub Narebski
"Derrick Stolee via GitGitGadget" writes: Sidenote: the GitGitGadget doesn't fold/wrap lines properly in the cover-letter. Not that it is much of a problem. > One unresolved issue with the commit-graph feature is that it can > cause issues when combined with replace objects, commit grafts, or >

Re: [RFC PATCH 3/6] commit-graph: enable replace-object and grafts

2018-06-09 Thread Jakub Narebski
Derrick Stolee writes: First, this commit seems to try to do two things at once, and in its current form it should be split into adding destroy_commit_graph() and into grafts / replacements check. Those are separate and unconnected features, and should be in separate patches, in my opinion. > C

Re: [RFC PATCH 0/6] Fix commit-graph/graft/replace/shallow combo

2018-06-08 Thread Jakub Narebski
Derrick Stolee writes: > The commit-graph file stores a condensed version of the commit history. > This helps speed up several operations involving commit walks. This > feature does not work well if those commits "change" using features like > commit grafts, replace objects, or shallow clones. I

Re: [PATCH v3 11/20] commit-graph: verify root tree OIDs

2018-06-02 Thread Jakub Narebski
Derrick Stolee writes: > On 5/30/2018 6:24 PM, Jakub Narebski wrote: [...] >> NOTE: we will be checking Commit Data chunk; I think it would be good >> idea to verify that size of Commit Data chunk matches (N * (H + 16) bytes) >> that format gives us, so that we don't a

Re: [PATCH v3 07/20] commit-graph: verify catches corrupt signature

2018-06-02 Thread Jakub Narebski
Derrick Stolee writes: > On 5/28/2018 10:05 AM, Jakub Narebski wrote: >> Derrick Stolee writes: [...] >>> diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh >>> index 6ca451dfd2..bd64481c7a 100755 >>> --- a/t/t5318-commit-graph.sh >>>

Re: [PATCH v3 06/20] commit-graph: add 'verify' subcommand

2018-06-02 Thread Jakub Narebski
Derrick Stolee writes: > On 5/27/2018 6:55 PM, Jakub Narebski wrote: >> Derrick Stolee writes: [...] >>> +static int verify_commit_graph_error; >>> + >>> +static void graph_report(const char *fmt, ...) >>> +{ >>>

Re: [PATCH v3 02/20] commit-graph: fix GRAPH_MIN_SIZE

2018-06-02 Thread Jakub Narebski
"brian m. carlson" writes: > On Sat, May 26, 2018 at 08:46:09PM +0200, Jakub Narebski wrote: >> One issue: in the future when Git moves to NewHash, it could encounter >> then both commit-graph files using SHA-1 and using NewHash. What about >> GRPH_OID_LEN the

Re: [RFC PATCH 4/6] commit-graph: avoid writing when repo is shallow

2018-06-02 Thread Jakub Narebski
Derrick Stolee writes: > On 5/31/2018 10:30 PM, Junio C Hamano wrote: >> Derrick Stolee writes: >> >>> Shallow clones do not interact well with the commit-graph feature for >>> several reasons. Instead of doing the hard thing to fix those >>> interactions, instead prevent reading or writing a com

Re: [PATCH v3 20/20] commit-graph: update design document

2018-06-02 Thread Jakub Narebski
Derrick Stolee writes: > The commit-graph feature is now integrated with 'fsck' and 'gc', > so remove those items from the "Future Work" section of the > commit-graph design document. It is always nice to have such commit as a summary what was done in the series, and to have up to date roadmap.

Re: [PATCH v3 19/20] gc: automatically write commit-graph files

2018-06-02 Thread Jakub Narebski
Derrick Stolee writes: > The commit-graph file is a very helpful feature for speeding up git > operations. In order to make it more useful, write the commit-graph file > by default during standard garbage collection operations. I think you meant here "make it possible to write the commit-graph f

Re: [PATCH v3 18/20] commit-graph: add '--reachable' option

2018-06-02 Thread Jakub Narebski
Derrick Stolee writes: > When writing commit-graph files, it can be convenient to ask for all > reachable commits (starting at the ref set) in the resulting file. This > is particularly helpful when writing to stdin is complicated, such as a > future integration with 'git gc' which will call > wr

Re: [PATCH v3 17/20] fsck: verify commit-graph

2018-06-02 Thread Jakub Narebski
Derrick Stolee writes: > If core.commitGraph is true, verify the contents of the commit-graph > during 'git fsck' using the 'git commit-graph verify' subcommand. Run > this check on all alternates, as well. All right, so we have one config variable to control the use of serialized commit-graph f

Re: [PATCH v3 16/20] commit-graph: verify contents match checksum

2018-06-02 Thread Jakub Narebski
Derrick Stolee writes: > The commit-graph file ends with a SHA1 hash of the previous contents. If > a commit-graph file has errors but the checksum hash is correct, then we > know that the problem is a bug in Git and not simply file corruption > after-the-fact. > > Compute the checksum right away

Re: [PATCH v3 15/20] commit-graph: test for corrupted octopus edge

2018-06-02 Thread Jakub Narebski
Derrick Stolee writes: > The commit-graph file has an extra chunk to store the parent int-ids for > parents beyond the first parent for octopus merges. Our test repo has a > single octopus merge that we can manipulate to demonstrate the 'verify' > subcommand detects incorrect values in that chunk

Re: [PATCH v3 14/20] commit-graph: verify commit date

2018-06-02 Thread Jakub Narebski
Derrick Stolee writes: Nice and simple. The only possible question may be the ordering of patches in the series, namely whether this change should be before or after test checking generation numbers. > Signed-off-by: Derrick Stolee > --- > commit-graph.c | 6 ++ > t/t5318-commit-

Re: [PATCH v3 13/20] commit-graph: verify generation number

2018-06-02 Thread Jakub Narebski
Derrick Stolee writes: > While iterating through the commit parents, perform the generation > number calculation and compare against the value stored in the > commit-graph. All right, that's good. What about commit-graph files that have GENERATION_NUMBER_ZERO for all its commits (because we ver

Re: [PATCH v3 12/20] commit-graph: verify parent list

2018-06-01 Thread Jakub Narebski
Derrick Stolee writes: > The commit-graph file stores parents in a two-column portion of the > commit data chunk. If there is only one parent, then the second column > stores 0x to indicate no second parent. All right, it is certainly nice to have this information, but isn't it something

Re: [PATCH v3 11/20] commit-graph: verify root tree OIDs

2018-05-30 Thread Jakub Narebski
Derrick Stolee writes: > The 'verify' subcommand must compare the commit content parsed from the > commit-graph and compare it against the content in the object database. You have "compare" twice in the above sentence. > Use lookup_commit() and parse_commit_in_graph_one() to parse the commits >

Re: [PATCH v3 10/20] commit-graph: verify objects exist

2018-05-30 Thread Jakub Narebski
Derrick Stolee writes: > In the 'verify' subcommand, load commits directly from the object > database to ensure they exist. Parse by skipping the commit-graph. All right, before we check that the commit data matches, we need to check that all the commits in cache (in the serialized commit graph)

Re: [PATCH v3 09/20] commit-graph: verify corrupt OID fanout and lookup

2018-05-30 Thread Jakub Narebski
Derrick Stolee writes: > In the commit-graph file, the OID fanout chunk provides an index into > the OID lookup. The 'verify' subcommand should find incorrect values > in the fanout. > > Similarly, the 'verify' subcommand should find out-of-order values in > the OID lookup. O.K., the OID Lookup

Re: [PATCH v3 08/20] commit-graph: verify required chunks are present

2018-05-28 Thread Jakub Narebski
Derrick Stolee writes: > The commit-graph file requires the following three chunks: > > * OID Fanout > * OID Lookup > * Commit Data > > If any of these are missing, then the 'verify' subcommand should > report a failure. This includes the chunk IDs malformed or the > chunk count is truncated. Mi

Re: [PATCH v3 07/20] commit-graph: verify catches corrupt signature

2018-05-28 Thread Jakub Narebski
Derrick Stolee writes: > This is the first of several commits that add a test to check that > 'git commit-graph verify' catches corruption in the commit-graph > file. The first test checks that the command catches an error in > the file signature. This is a check that exists in the existing > com

Re: [PATCH v3 06/20] commit-graph: add 'verify' subcommand

2018-05-27 Thread Jakub Narebski
Derrick Stolee writes: > If the commit-graph file becomes corrupt, we need a way to verify > that its contents match the object database. In the manner of > 'git fsck' we will implement a 'git commit-graph verify' subcommand > to report all issues with the file. > > Add the 'verify' subcommand to

Re: [PATCH v3 05/20] commit-graph: load a root tree from specific graph

2018-05-27 Thread Jakub Narebski
Derrick Stolee writes: > When lazy-loading a tree for a commit, it will be important to select > the tree from a specific struct commit_graph. Create a new method that > specifies the commit-graph file and use that in > get_commit_tree_in_graph(). Is this for the same reason why parse_commit_in_

Re: [PATCH v3 04/20] commit: force commit to parse from object database

2018-05-27 Thread Jakub Narebski
Derrick Stolee writes: > In anticipation of verifying commit-graph file contents against the > object database, create parse_commit_internal() to allow side-stepping > the commit-graph file and parse directly from the object database. > > Due to the use of generation numbers, this method should n

Re: [PATCH v3 03/20] commit-graph: parse commit from chosen graph

2018-05-27 Thread Jakub Narebski
Derrick Stolee writes: > Before verifying a commit-graph file against the object database, we > need to parse all commits from the given commit-graph file. Create > parse_commit_in_graph_one() to target a given struct commit_graph. If I understand it properly the problem is that when verifying a

Re: [PATCH v3 02/20] commit-graph: fix GRAPH_MIN_SIZE

2018-05-26 Thread Jakub Narebski
Derrick Stolee writes: > The GRAPH_MIN_SIZE macro should be the smallest size of a parsable > commit-graph file. However, the minimum number of chunks was wrong. > It is possible to write a commit-graph file with zero commits, and > that violates this macro's value. > > Rewrite the macro, and use

Re: RFC: New reference iteration paradigm

2018-05-26 Thread Jakub Narebski
Jeff King writes: > On Thu, Mar 31, 2016 at 11:01:44AM -0700, Junio C Hamano wrote: >> Michael Haggerty writes: >> >>> the backend now has to implement >>> struct ref_iterator *ref_iterator_begin_fn(const char *submodule, const char *prefix, >

Re: commit-graph: change in "best" merge-base when ambiguous

2018-05-24 Thread Jakub Narebski
Derrick Stolee writes: > On 5/22/2018 1:39 AM, Michael Haggerty wrote: >> On 05/21/2018 08:10 PM, Derrick Stolee wrote: >>> [...] >>> In the Discussion section of the `git merge-base` docs [1], we have the >>> following: >>> >>>     When the history involves criss-cross merges, there can be more

Re: [PATCH v2 03/12] commit-graph: test that 'verify' finds corruption

2018-05-21 Thread Jakub Narebski
Derrick Stolee writes: > Add test cases to t5318-commit-graph.sh that corrupt the commit-graph > file and check that the 'git commit-graph verify' command fails. These > tests verify the header and chunk information is checked carefully. > > Helped-by: Martin Ågren > Signed-off-by: Derrick Stole

Re: [PATCH v2 02/12] commit-graph: verify file header information

2018-05-20 Thread Jakub Narebski
Derrick Stolee writes: > During a run of 'git commit-graph verify', list the issues with the > header information in the commit-graph file. Some of this information > is inferred from the loaded 'struct commit_graph'. Some header > information is checked as part of load_commit_graph_one(). > > Si

Re: [PATCH v2 01/12] commit-graph: add 'verify' subcommand

2018-05-20 Thread Jakub Narebski
Derrick Stolee writes: > If the commit-graph file becomes corrupt, we need a way to verify > that its contents match the object database. In the manner of > 'git fsck' we will implement a 'git commit-graph verify' subcommand > to report all issues with the file. > > Add the 'verify' subcommand to

Re: [RFC] Other chunks for commit-graph, part 1 - Bloom filters, topo order, etc.

2018-05-14 Thread Jakub Narebski
Derrick Stolee writes: > On 5/12/2018 10:00 AM, Jakub Narebski wrote: >> Derrick Stolee writes: >>> On 5/4/2018 3:40 PM, Jakub Narebski wrote: >>>> >>>> With early parts of commit-graph feature (ds/commit-graph and >>>> ds/lazy-load-tree

  1   2   >