Bug#935875: ITP: ocaml-mmap -- file mapping functionality in OCaml

[Stéphane Glondu]

Package: wnpp
Severity: wishlist
Owner: Stéphane Glondu 

* Package name: ocaml-mmap
  Version : 1.1.0
  Upstream Author : Jérémie Dimino and Anton Bachin
* URL : https://github.com/mirage/mmap
* License : LGPL 2.1 with linking exception
  Programming Lang: OCaml
  Description : file mapping functionality in OCaml

This project provides a Mmap.map_file functions for mapping files in
memory. This function is the same as the Unix.map_file funciton added
in OCaml >= 4.06.

This package is a new dependency of lwt.

It will be maintained in ocaml-team.

Bug#935876: ITP: octave-kernel -- Jupyter kernel for Octave (Python 3)

[Joseph Nahmias]

Package: wnpp
Severity: wishlist
Owner: Joseph Nahmias 

* Package name: octave-kernel
  Version : 0.31.1
  Upstream Author : Steven Silvester 
* URL : https://github.com/calysto/octave_kernel
* License : BSD
  Programming Lang: Python
  Description : Jupyter kernel for Octave (Python 3)

This package integrates the use of the Octave language within
the Jupyter Notebook by providing a kernel that communicates
using the standard API with the octave-cli.  It also handles
plotting and displays graphs within the notebook as expected.

Re: Consensus Call: Git Packaging Round 1 [and 1 more messages]

[Ian Jackson]

Andrej Shadura writes ("Re: Consensus Call: Git Packaging Round 1"):
> I noticed some people [citation needed] think it is not important to
> preserve pristine upstream tarballs with the move to Git, and it's
> okay to regenerate them from a Git branch without trying to preserve
> checksums of the tarballs upstream has somehow generated.

I am one of these people.  I have always been sceptical of the need to
preserve upstream pristine tarballs.  I haven't been vocal about this
because no-one is forcing anyone to publish pristine tarballs.  So in
any situation where the maintainer doesn't want to pay the costs of
preserving pristine upstream tarballs, the maintainer can simply not
do so.

That overall stance has a lot of social value for the project, because
it means we can all cooperate without having to have this debate.  We
can save our energy for doing something more useful.

For myself I try to publish pristine upstream tarballs when it's
reasonably convenient, because I feel that this is a thing that some
people value, even though I myself think the value is very limited.
That's part of playing nice in a community like Debian.

If we are going to mandate something - or even, if we are going to
change our current stance (which seems to be that this is a "nice to
have"), then a discussion of the upsides and downsides - particularly,
with a practical focus - is necessary.

> I just had an impression this is not being considered and sort of
> assumed a consensus that we won't keep them in Git when we move to
> git-debpush workflows. Let's discuss it instead of have people
> assuming things potentially incorrect :)

Adoption of git-debpush will be optional, obviously.  I think reasons
to actively encourage it over `dgit push-source' [1] are rather
limited, at least right now.

Also, in principle git-debpush and the tag2upload service could
support eg pristine-tar, although that would involve quite some
development work.  If someone wants to lead that, I would be happy to
review patches, do supporting work, etc.

So certainly, speaking for myself, I don't intend to discourage or
impede anyone from publishing pristine tarballs, even though as I say
I think their value is very limited.

Sam Hartman writes ("Re: Consensus Call: Git Packaging Round 1"):
> That's on my list for discussion in round 3.

I have carefully avoided any discussion of the merits in my message,
and am happy to wait.

Ian.

[1] Both `dgit push-source' and `git debpush' publish your actual git
history on the dgit git server in a form useful for users, so I have
argued, and will continue to argue, that there are good reasons to
encourage maintainers to do one of these things.

-- 
Ian JacksonThese opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Re: Consensus Call: Git Packaging Round 1 [and 1 more messages]

[Colin Watson]

On Tue, Aug 27, 2019 at 11:40:22AM +0100, Ian Jackson wrote:
> Andrej Shadura writes ("Re: Consensus Call: Git Packaging Round 1"):
> > I noticed some people [citation needed] think it is not important to
> > preserve pristine upstream tarballs with the move to Git, and it's
> > okay to regenerate them from a Git branch without trying to preserve
> > checksums of the tarballs upstream has somehow generated.
> 
> I am one of these people.  I have always been sceptical of the need to
> preserve upstream pristine tarballs.

I just wanted to leave a note to the effect that I have some cases where
I think this remains useful.  In deference to Sam's organisation of the
discussion I'll refrain from getting into them just now.

> I haven't been vocal about this because no-one is forcing anyone to
> publish pristine tarballs.  So in any situation where the maintainer
> doesn't want to pay the costs of preserving pristine upstream
> tarballs, the maintainer can simply not do so.
> 
> That overall stance has a lot of social value for the project, because
> it means we can all cooperate without having to have this debate.  We
> can save our energy for doing something more useful.

I definitely agree with this position.

-- 
Colin Watson   [cjwat...@debian.org]

Consensus?

[Bernd Zeimetz]

On 2019-08-26 23:41, Sam Hartman wrote:

I don't think you're part of our consensus.


Yes, that might be very true. But what you describe by
"our consensus"
is the opinion of a few people who actually read this mailinglist
regularily and consider to reply. I really hope that you do not
consider this "consensus" as the opinion of the Debian project.
It is the opinion of 10-20 random people, nothing more.

And therefore I see absolutely no reason why the developer
reference should be changed based on this.

The proper way to handle this would be to amend
https://dep-team.pages.debian.net/deps/dep14/
and drive it forward.

Please remember that it should be easier and more fun to contribute
to Debian. Keeping packaging in the stone age jsut because some
people still live there is not what you should strive for.


--
 Bernd ZeimetzDebian GNU/Linux Developer
 http://bzed.dehttp://www.debian.org
 GPG Fingerprint: ECA1 E3F2 8E11 2432 D485  DD95 EB36 171A 6FF9 435F

Re: tag2upload service architecture and risk assessment - draft v2

[Ian Jackson]

Ian Jackson writes ("tag2upload service architecture and risk assessment - 
draft v2"):
> Thanks for all the comments on the draft service architecture I posted
> in late July. [1]  I have made a v2, incorporating the various helpful
> suggestions, and the information from the thread.

It has been a week.  I'm not sure whether that means people haven't
looked at the linked documents (I guess there might be some tl;dr
going on).

It would be good to get a review of the risk analysis in particular,
hence this ping.  I'd appreciate public follow-ups.

Anyway, I'll leave it at least another week.  If anyone wants more
time than that to properly review my proposal, please let me know when
you expect to be able to respond.

If no-one has any comments, then I intend to use this set of documents
as basis for asking for a formal go-ahead before I do a lot of
implementation work (privsep code etc.)

Thanks,
Ian.

> Some respondents raised archive integrity concerns.  It seemed best to
> address those more formally, and in a more structured way, than as a
> mailing list subthreads.  Accordingly, v2 of my proposal has a formal
> risk assessment, in a format loosely borrowed from health and safety
> management.
> 
> I think I have captured in the risk assessment all the risks mentioned
> in the thread, but I may well have missed something.  Please let me
> know if you think there is anything which is not covered.  Also please
> let me know if any of my analysis seems wrong.
> 
> Please find an introduction, and detailed documentation, here:
>   https://people.debian.org/~iwj/tag2upload/2019-08-20/
> 
> Thanks,
> Ian.
> 
> [1] This message and the subsequent thread:
>   https://lists.debian.org/debian-devel/2019/07/msg00501.html

-- 
Ian JacksonThese opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Re: tag2upload service architecture and risk assessment - draft v2

[Bastian Blank]

On Tue, Aug 20, 2019 at 06:32:30PM +0100, Ian Jackson wrote:
> Thanks for all the comments on the draft service architecture I posted
> in late July. [1]  I have made a v2, incorporating the various helpful
> suggestions, and the information from the thread.

No, you just did a medium break.  Mail is not web, don't do that.  You
need to at least list the differences.

> Some respondents raised archive integrity concerns.  It seemed best to
> address those more formally, and in a more structured way, than as a
> mailing list subthreads.  Accordingly, v2 of my proposal has a formal
> risk assessment, in a format loosely borrowed from health and safety
> management.

Please describe the design changes you added to address our concerns.
The risk assessment still lists things we described as no-go.

Sorry, but I don't see how we can go forward, while you seem to be
either unable to understand what we want to tell you or just decided to
ignore it.  I don't think it makes sense to continue this discussion
without a mediator.

Regards,
Bastian

-- 
Those who hate and fight must stop themselves -- otherwise it is not stopped.
-- Spock, "Day of the Dove", stardate unknown

Re: Consensus?

[Sam Hartman]

> "Bernd" == Bernd Zeimetz  writes:

Bernd> On 2019-08-26 23:41, Sam Hartman wrote:
>> I don't think you're part of our consensus.

Bernd> Yes, that might be very true. But what you describe by "our
Bernd> consensus" is the opinion of a few people who actually read
Bernd> this mailinglist regularily and consider to reply. I really
Bernd> hope that you do not consider this "consensus" as the opinion
Bernd> of the Debian project.  It is the opinion of 10-20 random
Bernd> people, nothing more.

I am not in agreement with the above.

First, this work was discussed in the DPL campaign, in every bits from
the DPL mail I've published, in my keynote, in several sessions at
DebConf, and heavily in the DebConf hallway track.
I think that  most active members of the project are aware the
discussion is going on.
More over, debian-devel is the list where we have such conversations.
Also, I've been saying it was happening on debian-devel.

I know people have joined or started following debian-devel again to
follow this conversation.

So, given that we've made an appropriate effort to inform people, this
is the discussion amongst the people who care about the issue enough to
join a discussion.

Also, one of the assumptions about consensus discussions is that many
more people are reading than contributing.  But if those people disagree
strongly enough they'll speak up.

So, we have:

* The active support of say 10 people.

* We have fairly confidence that members of debian-devel (the list) have
  researched these issues enough to have an informed opinion.

* We'll eventually get to a point where few people disagree with my
  summary enough to comment on it.

* No one disagrees enough to start a GR process.  If we really get this
  wrong here it only takes 6 to make a very loud statement that we need
  to consider more deeply.

And assuming that none of the things that invalidates a consensus
process happens like driving people away or bullying, then yeah, for
this discussion, I think we do have a project level consensus.

--Sam

Re: Bypassing the 2/3/4GB virtual memory space on 32-bit ports

[Sam Hartman]

> "Luke" == Luke Kenneth Casson Leighton  writes:

>On Friday, August 23, 2019, Karsten Merker  wrote:
>   and decide for themselves who is displaying "violent
> hatred" on mailing lists and come to their own judgement about
> your allegations:

Luke>You've now violated the Debian Conduct twice in under an
Luke> hour.  https://www.debian.org/code_of_conduct

I've been debating whether to respond to this and ultimate decided that
I will.
On one level, it has died out and it would be good to let an unfortunate
thread rest.

On another level, people have expressed concerns about the CoC being
weaponized to shutdown legitimate discussion.
Luke is trying to do that here.
Discussions about his behavior belong in private mail.
However, I want to reassure the community that this is not what the CoC
is about.
I regret that Ted and Karsten were not treated with the respect that we
have chosen as our standard.

Sam Hartman
Debian Project Leader


signature.asc
Description: PGP signature

Re: Consensus?

[Wookey]

On 2019-08-27 09:48 -0400, Sam Hartman wrote:
> 
> Also, one of the assumptions about consensus discussions is that many
> more people are reading than contributing.  But if those people disagree
> strongly enough they'll speak up.

Exactly - I'm one of these people.  I'm happy with what Sam said and
the discussion that followed, so felt no need to say anything directly
in the discussion yet.  I don't agree with Bernard.

Sam's process to find consensus where it exists, and points of issue
where it doesn't, seems sensible to me. I am interested in the
results, particularly from the POV of someone who has primarily
'patches, quilt and tarballs' workflows, but can use git if
necessary.

Wookey
-- 
Principal hats:  Linaro, Debian, Wookware, ARM
http://wookware.org/


signature.asc
Description: PGP signature

Re: tag2upload service architecture and risk assessment - draft v2

[Sam Hartman]

> "Bastian" == Bastian Blank  writes:


Bastian> Please describe the design changes you added to address our
Bastian> concerns.  The risk assessment still lists things we
Bastian> described as no-go.

Bastian> Sorry, but I don't see how we can go forward, while you
Bastian> seem to be either unable to understand what we want to tell
Bastian> you or just decided to ignore it.  I don't think it makes
Bastian> sense to continue this discussion without a mediator.

I do think it would be valuable to confirm whether we're at an impasse.
It sounds like Ian may think that  resolving your concerns would be a
no-go and that you think that his design is a no-go.
Confirming whether that's true would actually be valuable.  I think the
ball is probably in Ian's court on that issue.

I also think it would be very interesting to get Joerg's personal
opinion on designs in this space because it sounds like he's thought
about it for a while.

I'd really like to find ways to get more experience with tag2upload-like
things.
I've proposed my ideas for running experiments  with the technology in
limited fassions to Ian.
Sadly none of my ideas were all that helpful so far.

Also, if Ansgar is going to implement his solution it would be valuable
to get experience with that.  One of the factors we as a project will
consider is whether other implementations emerge or whether Ian is the
only one who will choose to implement in this space.

Ultimately, if we are at an impasse, and we've reached the point where
it's time to make a decision, I do think there is a way forward.  We can
delegate a specific decision (probably something like deciding an
initial policy for archive integrity) to a group of developers.  We'd
want to include members of ftpmaster, people who want something like
tag2upload, and some respected neutral parties to balance things out.
Yes, such a delegation would have power that overlapped with ftpmaster.
That's clearly constitutionally fine, but I also think in a limited
circumstance like a specific decision it is both appropriate and a
reasonable way to break a logjam.

Right now, I think it's too early to do something like that.


signature.asc
Description: PGP signature

Re: tag2upload service architecture and risk assessment - draft v2

[Holger Levsen]

Dear Bastian,

On Tue, Aug 27, 2019 at 02:41:28PM +0200, Bastian Blank wrote:
> No, you just did a medium break.  Mail is not web, don't do that.  You
> need to at least list the differences.
[...] 
> Sorry, but I don't see how we can go forward, while you seem to be
> either unable to understand what we want to tell you or just decided to
> ignore it.  I don't think it makes sense to continue this discussion
> without a mediator.

the hostility in your mail makes me very sad and want to go away. I believe 
this is your intention :-(


-- 
cheers,
Holger

---
   holger@(debian|reproducible-builds|layer-acht).org
   PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C


signature.asc
Description: PGP signature

Re: Consensus Call: Git Packaging Round 1

[Ian Jackson]

Bernd Zeimetz :
> On 2019-08-26 23:41, Sam Hartman wrote:
> > I don't think you're part of our consensus.
> 
> Yes, that might be very true. But what you describe by
> "our consensus"

I'm not a fan of this phrasing by Sam.  But he makes a very good
point: you have not answered any of the substantive reasons why people
might have problems with Gitlab MRs.

Taken together with the rest of your mail, it seems to me a bit like
you're complaining that people aren't listening (to you, or to some
silent majority) but you yourself don't seem to have listened very
well to the rest of the thread ?

> is the opinion of a few people who actually read this mailinglist
> regularily and consider to reply. I really hope that you do not
> consider this "consensus" as the opinion of the Debian project.
> It is the opinion of 10-20 random people, nothing more.

I'm not sure how else you would measure the rough consensus.
You'd prefer a GR ?

> The proper way to handle this would be to amend
> https://dep-team.pages.debian.net/deps/dep14/
> and drive it forward.

The DEP process provides a framework for documenting proposed changes,
etc., but I'm not sure how a DEP proponent would judge consensus other
than with traditional methods such as a consensus call here on -devel.

> And therefore I see absolutely no reason why the developer
> reference should be changed based on this.

I'm confused.  In your earlier message you were proposing that "3. If
a package is maintained on salsa, maintainers have to process merge
requests".  That would be a change to the DR, presumably.

Now you are saying that no change should be made to the DR because
this mailing list is not a good place to judge rough consensus ?

I think Sam's proposed change would be to document in the DR that a
maintainer should handle change requests (including code
contributions) sent to the BTS.  That would surely just be documenting
our existing norm.  It seems to me that if you want to change a norm,
it is up to you to argue for it.

> Please remember that it should be easier and more fun to contribute
> to Debian. Keeping packaging in the stone age jsut because some
> people still live there is not what you should strive for.

Please avoid pejorative language like "stone age".

And please avoid describing workflows as obsolete when there are
people making cogent arguments why those established workflows are (at
least in some respects) better than newer ones.

Ian.

-- 
Ian JacksonThese opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Bug#935908: ITP: dmsh -- simple 2D mesh generator inspired by distmesh

[Drew Parsons]

Package: wnpp
Severity: wishlist
Owner: Drew Parsons 

* Package name: python-dmsh
  Version : 0.1.3
  Upstream Author : Nico Schlömer 
* URL : https://github.com/nschloe/dmsh
* License : MIT
  Programming Lang: Python
  Description : simple mesh generator inspired by distmesh

 dmsh: "The worst mesh generator you'll ever use."
 
 Inspired by distmesh, dmsh is slow, requires a lot of memory, and
 isn't terribly robust either.

 On the plus side, it's got a usable interface, is pure Python (and
 hence easily installable on any system), and if it works, it produces
 pretty high-quality meshes.

 Combined with optimesh, dmsh produces the highest-quality 2D meshes
 in the west.

 Example capabilities:
 * Primitives
   - circle, rectangle, polygon
   - halfspace
 * Combinations
   - difference
   - nonconstant edge length
   - union
   - intersection
 * Transformations
   - rotation, translation, scaling
 * Local refinement


A simple-to-use tool for creating 2D meshes. Complements mshr
(which is not actively developed)

To be packaged under the Debian Science team alongside other related
packages by the same author: meshio (mesh file conversion), pygalmesh
(3D meshes)

Some debate about source package name: dmsh? python-dmsh? python3-dmsh?
A quick poll on irc indicates some preference for python-dmsh. Further
debate welcome.

Re: Consensus Call: Git Packaging Round 1

[Alf Gaida]

On Tue, 27 Aug 2019 16:08:59 +0100
Ian Jackson  wrote:

> I think Sam's proposed change would be to document in the DR that a
> maintainer should handle change requests (including code
> contributions) sent to the BTS.  That would surely just be documenting
> our existing norm.  It seems to me that if you want to change a norm,
> it is up to you to argue for it.
> 
> > Please remember that it should be easier and more fun to contribute
> > to Debian. Keeping packaging in the stone age jsut because some
> > people still live there is not what you should strive for.
> 
> Please avoid pejorative language like "stone age".

Nicer would be "lowest common nominator" but "stone age" describe the
process of sending patches via BTS very well. Upps, sorry, not only the
process, but the BTS also. 

We have git, we have salsa, it seems to me that using merge requests is
common sense and should not need any mention. Otherwise - if one is
happy to send patches via BTS - why not, as long one don't bother me
with. And this it a two way thing - i will not bother other people with
patches via BTS (nicer for: if not in salsa, gitlab or github, no
contribution, no patch)

Cheers Alf


-- 
Alf Gaida
BDBF C688 EFAD BA89 5A9F  464B CD28 0A0B 4D72 827C

Re: Consensus Call: Git Packaging Round 1

[Jeremy Stanley]

On 2019-08-27 17:52:02 +0200 (+0200), Alf Gaida wrote:
> On Tue, 27 Aug 2019 16:08:59 +0100 Ian Jackson wrote:
[...]
> > Please avoid pejorative language like "stone age".
> 
> Nicer would be "lowest common nominator" but "stone age" describe
> the process of sending patches via BTS very well. Upps, sorry, not
> only the process, but the BTS also.
> 
> We have git, we have salsa, it seems to me that using merge
> requests is common sense and should not need any mention.
> Otherwise - if one is happy to send patches via BTS - why not, as
> long one don't bother me with. And this it a two way thing - i
> will not bother other people with patches via BTS (nicer for: if
> not in salsa, gitlab or github, no contribution, no patch)

Please don't describe systems as "stone age" simply because you
don't see the value in them. For example, I appreciate the BTS
because it is truly free[*] software and I value software freedom.
Just because some people are happy to compromise these ideals and
use open-core[**] products they find convenient, that doesn't mean
that software freedom is a "stone age" concept from which Debian
should relieve itself.

[*] https://bugs.debian.org/debbugs-source/mainline/COPYING
[**] https://gitlab.com/gitlab-org/gitlab-ee/blob/master/ee/LICENSE
-- 
Jeremy Stanley


signature.asc
Description: PGP signature

Re: tag2upload service architecture and risk assessment - draft v2

[Ian Jackson]

Sam Hartman writes ("Re: tag2upload service architecture and risk assessment - 
draft v2"):
> I do think it would be valuable to confirm whether we're at an impasse.
> It sounds like Ian may think that resolving your concerns would be a
> no-go

I'm definitely trying to have a constructive discussion about the best
design, what the risks are of various approaches, etc.

Indeed I have been trying to resolve people's concerns.  The concerns
were almost all about archive integrity, and I have tried to analyse
the actual risks, and where appropriate propose control measures for
those risks.

My intent was not to ignore Bastian's requirements but instead tease
out the risks that they were aimed at, and make constructive
evaluations of those.  It thought it was reasonably clear what risks
Bastian was concerned with, but I had hoped Bastian would help me out
where I had misunderstood.

Unfortunately it seems that Bastian doesn't have "concerns" as you put
it.  He seems to have hard design demands.  He appears to be offended
that instead of accepting what he sees as non-negotiable requirements,
I have attempted to explain why I disagree with them.


>From my reading of the thread, I think there are two disputed design
demands, which are related.

The most basic demand is that the archive should be able to verify the
whole contents of the .dsc, given data signed by the user.

The risk assessment explains why I don't think this is an appropriate
requirement, but I will go through it again:

The mapping from git tag to .dsc is nontrivial.  git tag to .dsc
construction (or verification) is complex and offers a large attack
surface to the incoming source code.  It ought not to be done near a
powerful key such as the dak master archive signing key.

Furthermore, there is nearly no benefit in redoing this mapping.  In
my design proposal, the conversion occurs on a DSA-managed machine
using fully controlled software, with a copious audit trail.  This is
far superior to the current situation, where .dscs are produced by
uncontrolled git-buildpackage runes run on uploader's own systems (not
even in a clean environment, usually).  So my proposed design is much
better than the status quo.

In principle it would be possible to satisfy this demand.  The
tag2upload service could ship the git data to the archive, bundled
with the .changes (as a git bundle perhaps).  The archive would then
re-run the source package generation (perhaps using the reproduction
script that my proposal already includes), and compare the results.

So this is negotiable, although this seems to me like a silly
direction to be going.  And it would likely involve a long delay to
deployment if the extra dak rebuild machinery were to be regarded as a
blocker.


The second demand (or the second aspect) is that all of this
verification, by the archive, should be doable without relying on the
git SHA-1 object system.

Again, I have analysed the SHA-1 risk in my risk assessment.  So here
too I have explained why I don't think this is an appropriate
requirement to place on the tag2upload service.  We have already
accepted the SHA-1 risk; the mitigations we have are (including the
code in git which deals with at least the known collision attack) are
tolerably sufficient.

The tag2upload proposal doesn't make it worse; in some ways it is
slightly better because git object data comes from a central source
(salsa) rather than the maintainer's machine.

I think this demand cannot be met by anything I would call a "tag to
upload" system.  The tag data would have to contain some kind of file
manifest.  The tag would not be constructable by normal tooling, but
only by a special program.  It would not be simply a git tag.


> and that you think that his design is a no-go.
> Confirming whether that's true would actually be valuable.
> I think the ball is probably in Ian's court on that issue.

I was hoping for constructive engagement with the substance of the
arguments I am making.  I found it difficult to see where to go from
Bastian's most recent message, in which he seemed to me to say he had
been laying down non-negotiable demands and was offended that I
disagreed.


> I also think it would be very interesting to get Joerg's personal
> opinion on designs in this space because it sounds like he's thought
> about it for a while.

I would definitely welcome wider engagement with the substance of the
risks, the design tradeoffs, etc.


>  One of the factors we as a project will consider is whether other
> implementations emerge or whether Ian is the only one who will
> choose to implement in this space.

I would like to point out that tag2upload is by no means all my own
work.

The original design approach (from Vaumarcus) for a Debian git
transition was a joint effort (mostly with Joey), and there are other
contributors to src:dgit (Sean has been invaluable and of course wrote
git-debpush).

Also much of the heavy lifting in tag2upload is not done by src:dgit
itself.  In p

Re: Consensus Call: Git Packaging Round 1

[Adam Borowski]

On Tue, Aug 27, 2019 at 05:52:02PM +0200, Alf Gaida wrote:
> On Tue, 27 Aug 2019 16:08:59 +0100
> Ian Jackson  wrote:
> > Please avoid pejorative language like "stone age".
> 
> Nicer would be "lowest common nominator" but "stone age" describe the
> process of sending patches via BTS very well. Upps, sorry, not only the
> process, but the BTS also. 

New stuff is always better.  Go Electron!

-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁
⢿⡄⠘⠷⠚⠋  The root of a real enemy is an imaginary friend.
⠈⠳⣄

Re: tag2upload service architecture and risk assessment - draft v2

[Ian Jackson]

Ian Jackson writes ("Re: tag2upload service architecture and risk assessment - 
draft v2"):
> [stuff]

Argh.  A bunch of people helped me refine this but I sent an early
draft by mistake.  I guess it's too late to hope people will read only
the better version, but here it is anyway.

If you haven't read the first one yet, please prefer this one.

Sorry,
Ian.


Sam Hartman writes ("Re: tag2upload service architecture and risk assessment - 
draft v2"):
> I do think it would be valuable to confirm whether we're at an impasse.
> It sounds like Ian may think that resolving your concerns would be a
> no-go

I'm definitely trying to have a constructive discussion about the best
design, what the risks are of various approaches, etc.

Indeed I have been trying to resolve people's concerns.  The concerns
were almost all about archive integrity, so I have tried to analyse
the actual risks, and where appropriate propose control measures for
those risks.

My intent was not to ignore Bastian's requirements but instead tease
out the risks that they were aimed at, and make constructive
evaluations of those.  It seemed reasonably clear what risks Bastian
was concerned with, but I had hoped Bastian would help me out where I
had misunderstood..

Unfortunately it seems that rather than a set of concerns, as you put
it, Bastian has some hard design demands.  I didn't mean to cause
offence; I was trying to focus on the exact nature of the problems
these requirements are intended to solve.


>From my reading of the thread, it seems that there are two disputed
design demands, which are related.

The most basic demand is that the archive should be able to verify the
whole contents of the .dsc, given data signed by the user.

The risk assessment explains why I don't think this is an appropriate
requirement, but I will go through it again:

The mapping from git tag to .dsc is nontrivial.  git tag to .dsc
construction (or verification) is complex and offers a large attack
surface to the incoming source code.  It ought not to be done near a
powerful key such as the dak master archive signing key.

Furthermore, there is nearly no benefit in redoing this mapping.  In
my design proposal [1], the conversion occurs on a DSA-managed machine
using fully controlled software, with a copious audit trail.

This contrasts starkly with currently leading approaches for git-based
packaging: currently .dscs are produced from git data by uncontrolled
git-buildpackage runes run on uploader's own systems (not even in a
clean environment, usually).  So my proposal is far superior to the
status quo.

In principle it would be possible to satisfy this demand.  The
tag2upload service could ship the git data to the archive, bundled
with the .changes (as a git bundle perhaps).  The archive would then
re-run the source package generation (perhaps using the reproduction
script that my proposal already includes), and compare the results.

So this is negotiable, although this seems to me like a silly
direction to be going.  And it would likely involve a long delay to
deployment if the extra dak rebuild machinery were to be regarded as a
blocker.


The second demand (or the second aspect) is that all of this
verification, by the archive, should be doable without relying on the
git SHA-1 object system.

Again, I have analysed the SHA-1 risk in my risk assessment.  So here
too I have explained why I don't think this is an appropriate
requirement to place on the tag2upload service.  We have already
accepted the SHA-1 risk; the mitigations we have (including the code
in git which deals with at least the known collision attack) are
tolerably sufficient.

The tag2upload proposal doesn't make it worse; in some ways it is
slightly better because git object data comes from a central source
(salsa) rather than the maintainer's machine.

I think this demand cannot be met by anything I would call a "tag to
upload" system.  The tag data would have to contain some kind of file
manifest.  The tag would not be constructable by normal tooling, but
only by a special program.  It would not be simply a git tag.


> and that you think that his design is a no-go.
> Confirming whether that's true would actually be valuable.
> I think the ball is probably in Ian's court on that issue.

I was hoping for constructive engagement with the substance of the
arguments I am making.  I find it difficult to see where to go from
Bastian's most recent message, in which he seemed to me to say he had
been laying down non-negotiable demands and was offended that I
disagreed.


> I also think it would be very interesting to get Joerg's personal
> opinion on designs in this space because it sounds like he's thought
> about it for a while.

I would definitely welcome wider engagement with the substance of the
risks, the design tradeoffs, etc.


>  One of the factors we as a project will consider is whether other
> implementations emerge or whether Ian is the only one who will
> choose to implement in this 

Re: Consensus Call: Git Packaging Round 1

[Alf Gaida]

Am Dienstag, den 27.08.2019, 19:18 +0200 schrieb Adam Borowski:
> New stuff is always better.  Go Electron!
> 
Like it or not - the idea of pull requests (Github) or merge requests (Gitlab)
isn't exactly new. It might surprise you that people outside of debian are used
to use it a lot. Anyways, it's the method i like most if available. If not
available the good old patch via mail is ok too.

There are things i really like about PRs or MRs - they can be reviewed,
commented, changed without problems and fast. Just a "new" workflow. I can talk
only for myself, but i really like it - it speeds up development if used right.

Cheers Alf


signature.asc
Description: This is a digitally signed message part

Re: tag2upload service architecture and risk assessment - draft v2

[Sam Hartman]

> "Ian" == Ian Jackson  writes:

Ian> From my reading of the thread, it seems that there are two
Ian> disputed design demands, which are related.

Ian> The most basic demand is that the archive should be able to
Ian> verify the whole contents of the .dsc, given data signed by the
Ian> user.

Ian> The risk assessment explains why I don't think this is an
Ian> appropriate requirement, but I will go through it again:

Ian> The mapping from git tag to .dsc is nontrivial.  git tag to
Ian> .dsc construction (or verification) is complex and offers a
Ian> large attack surface to the incoming source code.  It ought not
Ian> to be done near a powerful key such as the dak master archive
Ian> signing key.

It's nontrivial in your design.  It sounds like in Ansger's approach
where effectively all the data you need for the dsc is included in the
tag, it would be a lot more trivial.  Obviously that design fails to
give you some things you value.  But it seems to me at least that if you
are willing to do significantly more work on the tagger's computer you
can make tag to dsc verification a lot simpler.

Also, I think the question of whether third-parties should be able to
verify that a dsc was properly generated without going back to a git tag
is interesting.

Ian> The second demand (or the second aspect) is that all of this
Ian> verification, by the archive, should be doable without relying
Ian> on the git SHA-1 object system.


If we get down to a point where this is a major issue, I'm happy to step
in as someone who has a lot of protocol security design experience and
reach out to other Internet security experts for their review.  I could
explain why I think relying on Git's integrity mechanisms for verifying
the authenticity of objects stored in Git (which is exactly what you are
doing) is a reasonable thing to do from a security standpoint.  My
answer will look a lot like Russ's answer.

I understand why SHA-1 looks worrying, and I was initially worried.  I'd
need to do a bit more review before I'd feel comfortable doing a write
up on this, but I think Russ's analysis is looking fairly good from
where I sit now.

Re: Consensus Call: Git Packaging Round 1

[Sam Hartman]

> "Alf" == Alf Gaida  writes:


Alf> There are things i really like about PRs or MRs - they can be
Alf> reviewed, commented, changed without problems and fast.

And as Sean pointed out, it's hard to understand the history of the
changes and comments after they hppened.  What happens when I'm trying
to review a MR three years later and the MR was rebased 4 times during
the lifetime of the MR prior to merge.

You may not care about that, but others do.

That's why there are interesting trade offs to balance.

--Sam

Re: Consensus Call: Git Packaging Round 1

[Alf Gaida]

On Tue, 27 Aug 2019 14:52:01 -0400
Sam Hartman  wrote:

> And as Sean pointed out, it's hard to understand the history of the
> changes and comments after they hppened.  What happens when I'm trying
> to review a MR three years later and the MR was rebased 4 times during
> the lifetime of the MR prior to merge.
> 
> You may not care about that, but others do.
> 
> That's why there are interesting trade offs to balance.
> 
> --Sam

To be honest, i find it hard even to read my own code after one, three,
ten or god beware 30 years - have done so several times. Ok - i had no
problem with the 30 y.o. things and pull requests - there was no SCM
involved. 

-- 
Alf Gaida
BDBF C688 EFAD BA89 5A9F  464B CD28 0A0B 4D72 827C

Re: Consensus Call: Git Packaging Round 1

[Antonio Terceiro]

On Tue, Aug 27, 2019 at 02:52:01PM -0400, Sam Hartman wrote:
> > "Alf" == Alf Gaida  writes:
> 
> 
> Alf> There are things i really like about PRs or MRs - they can be
> Alf> reviewed, commented, changed without problems and fast.
> 
> And as Sean pointed out, it's hard to understand the history of the
> changes and comments after they hppened.  What happens when I'm trying
> to review a MR three years later and the MR was rebased 4 times during
> the lifetime of the MR prior to merge.

FWIW, nowadays gitlab keeps track of every push, including rebases, to a
single merge request. It even adds a "compare to previous version",
where you can see the diff between the latest, maybe rebased, version of
the branch, and the previous one.

It _used_ to be the case that rebasing and force-pushing to the branch
referenced by a merge request would make you lose the history, but that
hasn't been true for a while.


signature.asc
Description: PGP signature

Re: Consensus Call: Git Packaging Round 1

[Alf Gaida]

On Tue, 27 Aug 2019 15:35:37 -0400
Milan Kupcevic  wrote:

> I fully agree with the initial best practices proposal stating that
> merge requests in salsa have to be attended to or otherwise this
> feature has to be disabled as per package maintainer preference.

I only stated that the Debian BTS feels like stone age - but i see no
replacement for when i look into things like Gitlab or Gitlab
"bugtracking" - i suggest we should only discuss possible git workflows
here - gbp is already invented, switched all my packages to. The only
thing i don't use right now is the changelog functionality, this will
come later. And with right written commits it should work well with the
Debian BTS - so best of both worlds.

Cheers

-- 
Alf Gaida
BDBF C688 EFAD BA89 5A9F  464B CD28 0A0B 4D72 827C

Re: Consensus Call: Git Packaging Round 1

[Milan Kupcevic]

On 8/27/19 1:37 PM, Alf Gaida wrote:
> Am Dienstag, den 27.08.2019, 19:18 +0200 schrieb Adam Borowski:
>> New stuff is always better.  Go Electron!
>>
> Like it or not - the idea of pull requests (Github) or merge requests (Gitlab)
> isn't exactly new. It might surprise you that people outside of debian are 
> used
> to use it a lot. Anyways, it's the method i like most if available. If not
> available the good old patch via mail is ok too.



Exactly. The tracking, branching and merging code is around literally
since the IT stone age[1]. There were various tools and services built
around SCCS, RCS, CVS, SVN, Git and others and yet they were and still
are lacking features we really need for proper bug tracking in packages.
Salsa is a nice maintenance flow supplement but can not replace Debian
BTS. Sorry, that is how it is.

I fully agree with the initial best practices proposal stating that
merge requests in salsa have to be attended to or otherwise this feature
has to be disabled as per package maintainer preference.

Milan

[1] http://sccs.sourceforge.net/PWB.html



signature.asc
Description: OpenPGP digital signature

Re: tag2upload service architecture and risk assessment - draft v2

[Scott Kitterman]

On Tuesday, August 27, 2019 1:19:14 PM EDT Ian Jackson wrote:
> Ian Jackson writes ("Re: tag2upload service architecture and risk assessment 
- draft v2"):
> > [stuff]
> 
> Argh.  A bunch of people helped me refine this but I sent an early
> draft by mistake.  I guess it's too late to hope people will read only
> the better version, but here it is anyway.
> 
> If you haven't read the first one yet, please prefer this one.
> 
> Sorry,
> Ian.
> 
> Sam Hartman writes ("Re: tag2upload service architecture and risk assessment 
- draft v2"):
> > I do think it would be valuable to confirm whether we're at an impasse.
> > It sounds like Ian may think that resolving your concerns would be a
> > no-go
> 
> I'm definitely trying to have a constructive discussion about the best
> design, what the risks are of various approaches, etc.
> 
> Indeed I have been trying to resolve people's concerns.  The concerns
> were almost all about archive integrity, so I have tried to analyse
> the actual risks, and where appropriate propose control measures for
> those risks.
> 
> My intent was not to ignore Bastian's requirements but instead tease
> out the risks that they were aimed at, and make constructive
> evaluations of those.  It seemed reasonably clear what risks Bastian
> was concerned with, but I had hoped Bastian would help me out where I
> had misunderstood..
> 
> Unfortunately it seems that rather than a set of concerns, as you put
> it, Bastian has some hard design demands.  I didn't mean to cause
> offence; I was trying to focus on the exact nature of the problems
> these requirements are intended to solve.
> 
> 
> From my reading of the thread, it seems that there are two disputed
> design demands, which are related.
> 
> The most basic demand is that the archive should be able to verify the
> whole contents of the .dsc, given data signed by the user.
> 
> The risk assessment explains why I don't think this is an appropriate
> requirement, but I will go through it again:
> 
> The mapping from git tag to .dsc is nontrivial.  git tag to .dsc
> construction (or verification) is complex and offers a large attack
> surface to the incoming source code.  It ought not to be done near a
> powerful key such as the dak master archive signing key.
> 
> Furthermore, there is nearly no benefit in redoing this mapping.  In
> my design proposal [1], the conversion occurs on a DSA-managed machine
> using fully controlled software, with a copious audit trail.
> 
> This contrasts starkly with currently leading approaches for git-based
> packaging: currently .dscs are produced from git data by uncontrolled
> git-buildpackage runes run on uploader's own systems (not even in a
> clean environment, usually).  So my proposal is far superior to the
> status quo.
> 
> In principle it would be possible to satisfy this demand.  The
> tag2upload service could ship the git data to the archive, bundled
> with the .changes (as a git bundle perhaps).  The archive would then
> re-run the source package generation (perhaps using the reproduction
> script that my proposal already includes), and compare the results.
> 
> So this is negotiable, although this seems to me like a silly
> direction to be going.  And it would likely involve a long delay to
> deployment if the extra dak rebuild machinery were to be regarded as a
> blocker.
> 
> 
> The second demand (or the second aspect) is that all of this
> verification, by the archive, should be doable without relying on the
> git SHA-1 object system.
> 
> Again, I have analysed the SHA-1 risk in my risk assessment.  So here
> too I have explained why I don't think this is an appropriate
> requirement to place on the tag2upload service.  We have already
> accepted the SHA-1 risk; the mitigations we have (including the code
> in git which deals with at least the known collision attack) are
> tolerably sufficient.
> 
> The tag2upload proposal doesn't make it worse; in some ways it is
> slightly better because git object data comes from a central source
> (salsa) rather than the maintainer's machine.
> 
> I think this demand cannot be met by anything I would call a "tag to
> upload" system.  The tag data would have to contain some kind of file
> manifest.  The tag would not be constructable by normal tooling, but
> only by a special program.  It would not be simply a git tag.
> 
> > and that you think that his design is a no-go.
> > Confirming whether that's true would actually be valuable.
> > I think the ball is probably in Ian's court on that issue.
> 
> I was hoping for constructive engagement with the substance of the
> arguments I am making.  I find it difficult to see where to go from
> Bastian's most recent message, in which he seemed to me to say he had
> been laying down non-negotiable demands and was offended that I
> disagreed.
> 
> > I also think it would be very interesting to get Joerg's personal
> > opinion on designs in this space because it sounds like he's thought
> > about it for a while.
>

Re: tag2upload service architecture and risk assessment - draft v2

[Russ Allbery]

Scott Kitterman  writes:

> As an example, I recall concerns about there not being an uploader
> signature on the source anymore, so we would lose the ability to verify
> from the archive who was responsible for the upload.

Does anyone do this?  Does it work today?

I'm dubious that you would be able to successfully verify all of the
archive from *.dsc signatures now.  Maybe you would be able to verify the
pieces that are the most important to you, though?

I think this requirement is a bit incomplete, in that I don't understand
the use case that would lead you to want to do this.  It's more of a
description of an implementation strategy than a use case, which makes it
hard to find other ways of accomplishing the same use case.

-- 
Russ Allbery (r...@debian.org)   

Re: tag2upload service architecture and risk assessment - draft v2

[Scott Kitterman]

On Tuesday, August 27, 2019 8:04:06 PM EDT Russ Allbery wrote:
> Scott Kitterman  writes:
> > As an example, I recall concerns about there not being an uploader
> > signature on the source anymore, so we would lose the ability to verify
> > from the archive who was responsible for the upload.
> 
> Does anyone do this?  Does it work today?
> 
> I'm dubious that you would be able to successfully verify all of the
> archive from *.dsc signatures now.  Maybe you would be able to verify the
> pieces that are the most important to you, though?
> 
> I think this requirement is a bit incomplete, in that I don't understand
> the use case that would lead you to want to do this.  It's more of a
> description of an implementation strategy than a use case, which makes it
> hard to find other ways of accomplishing the same use case.

I sometimes use who-uploads from devscripts when I want to find out who 
actually did an upload.  In theory, it could be re-written to support 
whatever.

I also check that the signature validates when I download a package from the 
archive.  I like the fact that this signature connects to a developer key in 
the keyring.

That said, I'm not the one who suggested losing this would be a problem in the 
previous thread, so I can't say what they were thinking.  I just don't think 
the threat assessment is a serious response to what people were suggesting.  
It would be a mistake to assume silence is concurrence.

I may be wrong, but I think Ian's made up his mind what he wants to do, so 
there's not a lot of point in convincing him otherwise.

Scott K

Re: tag2upload service architecture and risk assessment - draft v2

[Tobias Frost]

On Tue, Aug 27, 2019 at 05:04:06PM -0700, Russ Allbery wrote:
> Scott Kitterman  writes:
> 
> > As an example, I recall concerns about there not being an uploader
> > signature on the source anymore, so we would lose the ability to verify
> > from the archive who was responsible for the upload.
> 
> Does anyone do this?  Does it work today?
> 
> I'm dubious that you would be able to successfully verify all of the
> archive from *.dsc signatures now.  Maybe you would be able to verify the
> pieces that are the most important to you, though?
> 
> I think this requirement is a bit incomplete, in that I don't understand
> the use case that would lead you to want to do this.  It's more of a
> description of an implementation strategy than a use case, which makes it
> hard to find other ways of accomplishing the same use case.

Not sure if I understood this correctly, but the MIA team (via echolon?)
uses the information to tell us if there is an upload from a prossible
MIA person. (IOW the person is still active.)
I also use who-uploads occasionally to see if a sponsor knows about
where-abouts of some possible MIA persons.

> -- 
> Russ Allbery (r...@debian.org)   
>