Re: Security concerns with minified javascript code

[Samuel Thibault]

Russ Allbery, le Tue 01 Sep 2015 18:05:09 -0700, a écrit :
> Healthy language communities have their own metadata systems and
> standardized build systems that allow Debian packaging to be nearly
> automated, *provided* that we use the same unit of distribution as
> upstream.

I understand that using the same unit of distribution helps, but I'd
tend to think that with not too much work you can achieve automated
packaging of collections of upstream packages.

Notably, the whole minification toolchain could be uploaded as just one
package, using on each upload the set of versions that upstream is known
to be using.

Samuel

Re: Security concerns with minified javascript code

[Vincent Bernat]

 ❦  2 septembre 2015 09:32 +0300, Lars Wirzenius  :

> However, I want to raise the point that upstreams do not always make
> sensible decisions, and if they don't, it's good to raise that with
> them. For example, there was recently an ITP bug for
> node-number-is-nan. Upstream source code is at
> https://github.com/sindresorhus/number-is-nan, and the whole package
> boils down to these four lines of code:
>
> 'use strict';
> module.exports = Number.isNaN || function (x) {
> return x !== x;
> };
>
> (https://github.com/sindresorhus/number-is-nan/blob/master/index.js)
>
> If something or someone needs this, we should package it, and it seems
> Grunt needs it. However, it doesn't seem sensible to have a package
> for every one-liner Javascript function, either in Debian or upstream.
> That's going to be a lot of packages, and that alone makes things
> harder to manage for everyone. It'd make sense for the Javascript
> community to produce a more general library to make ES5 look more like
> ES6, which would include a number of such functions.

It exists. For polyfills:

 https://github.com/zloirock/core-js

And for the syntax:

 https://babeljs.io/
-- 
Alas, how love can trifle with itself!
-- William Shakespeare, "The Two Gentlemen of Verona"


signature.asc
Description: PGP signature

Re: Security concerns with minified javascript code

[Vincent Bernat]

 ❦  2 septembre 2015 09:28 +0200, Samuel Thibault  :

>> Healthy language communities have their own metadata systems and
>> standardized build systems that allow Debian packaging to be nearly
>> automated, *provided* that we use the same unit of distribution as
>> upstream.
>
> I understand that using the same unit of distribution helps, but I'd
> tend to think that with not too much work you can achieve automated
> packaging of collections of upstream packages.
>
> Notably, the whole minification toolchain could be uploaded as just one
> package, using on each upload the set of versions that upstream is known
> to be using.

There is no such thing as a "whole minification toolchain".

If you talk about uglifyjs or the like, it is already packaged and
doesn't solve all the problems we have (see my message to Odyx,
).

If you talk about Grunt, Grunt comes with a lot of plugins (and does
almost nothing without those) and each upstream will require different
plugins with different versions (Grunt plugin versions are evolving
fast). See the tree I posted for jQuery 3.x in
. All this dependency chain is maintained
by a variety of upstreams with different release schedules and goals.
-- 
Let the data structure the program.
- The Elements of Programming Style (Kernighan & Plauger)


signature.asc
Description: PGP signature

Re: Security concerns with minified javascript code

[Samuel Thibault]

Vincent Bernat, le Wed 02 Sep 2015 09:47:23 +0200, a écrit :
> If you talk about Grunt,

That's what I'm talking about.

> Grunt comes with a lot of plugins (and does almost nothing without
> those) and each upstream will require different plugins with different
> versions (Grunt plugin versions are evolving fast). See the tree I
> posted for jQuery 3.x in .

That's precisely what I'm talking about.

> All this dependency chain is maintained by a variety of upstreams with
> different release schedules and goals.

Sure, but apparently the set of plugin versions which fit together is
known?  I.e. I guess you didn't write the tree by hand?

Samuel

Re: Security concerns with minified javascript code

[Vincent Bernat]

 ❦  2 septembre 2015 09:54 +0200, Samuel Thibault  :

>> If you talk about Grunt,
>
> That's what I'm talking about.
>
>> Grunt comes with a lot of plugins (and does almost nothing without
>> those) and each upstream will require different plugins with different
>> versions (Grunt plugin versions are evolving fast). See the tree I
>> posted for jQuery 3.x in .
>
> That's precisely what I'm talking about.
>
>> All this dependency chain is maintained by a variety of upstreams with
>> different release schedules and goals.
>
> Sure, but apparently the set of plugin versions which fit together is
> known?  I.e. I guess you didn't write the tree by hand?

Yes, but it's only to compile jQuery. If I take another random project,
I will get a different set of plugins.

Or maybe you propose to just ship the whole "node_modules" directory
(which has all the dependencies) with jQuery sources? This would incur
some work on d/copyright and I don't see like this would be a good
practice. But this would solve some of the problems, yes.
-- 
Why is it that we rejoice at a birth and grieve at a funeral?  It is because we
are not the person involved.
-- Mark Twain, "Pudd'nhead Wilson's Calendar"


signature.asc
Description: PGP signature

Re: Security concerns with minified javascript code

[Samuel Thibault]

Vincent Bernat, le Wed 02 Sep 2015 10:10:55 +0200, a écrit :
>  ❦  2 septembre 2015 09:54 +0200, Samuel Thibault  :
> 
> >> If you talk about Grunt,
> >
> > That's what I'm talking about.
> >
> >> Grunt comes with a lot of plugins (and does almost nothing without
> >> those) and each upstream will require different plugins with different
> >> versions (Grunt plugin versions are evolving fast). See the tree I
> >> posted for jQuery 3.x in .
> >
> > That's precisely what I'm talking about.
> >
> >> All this dependency chain is maintained by a variety of upstreams with
> >> different release schedules and goals.
> >
> > Sure, but apparently the set of plugin versions which fit together is
> > known?  I.e. I guess you didn't write the tree by hand?
> 
> Yes, but it's only to compile jQuery. If I take another random project,
> I will get a different set of plugins.

Ok.

> Or maybe you propose to just ship the whole "node_modules" directory
> (which has all the dependencies) with jQuery sources?

That'd be a lot better than nothing.

> This would incur some work on d/copyright and I don't see like this
> would be a good practice.

Yes, but that work on d/copyright is *needed*: if we don't know for sure
that the compiler itself is really free, then we can't call the result
free and put it in main.

I however agree that it seems poor practice to duplicate these build
modules in every packages. But if the required versions are different,
there is no real other way. If there is a set of modules which are known
to be used widely and with stable versions, then they could be put in a
shared package.

> But this would solve some of the problems, yes.

What problems remain?

Samuel

Bug#797749: ITP: python-jsonpath-rw-ext -- extensions for JSONPath RW

[Thomas Goirand]

Package: wnpp
Severity: wishlist
Owner: Thomas Goirand 

* Package name: python-jsonpath-rw-ext
  Version : 0.1.7
  Upstream Author : Mehdi Abaakouk 
* URL : https://github.com/sileht/python-jsonpath-rw-ext
* License : Apache-2.0
  Programming Lang: Python
  Description : extensions for JSONPath RW

 Jsonpath-rw-ext extends json-path-rw capabilities by adding multiple
 extensions: 'len' that allows to get the length of a list. 'sorted' that
 returns a sorted version of a list, 'arithmetic' that permits to make math
 operation between elements and 'filter' to select only certain elements of
 a list.

Note: this is a new dependency for OpenStack Liberty, to be released next
October.

Re: Security concerns with minified javascript code

[Vincent Bernat]

 ❦  2 septembre 2015 10:18 +0200, Samuel Thibault  :

>> Or maybe you propose to just ship the whole "node_modules" directory
>> (which has all the dependencies) with jQuery sources?
>
> That'd be a lot better than nothing.

OK. Also, node_modules for jQuery is 76M (for 3.x, 70M for 2.x). I still
find using pre-minification jquery.js to be a better alternative due to
the fact is far far simpler. But whatever the consensus we may reach.

>> This would incur some work on d/copyright and I don't see like this
>> would be a good practice.
>
> Yes, but that work on d/copyright is *needed*: if we don't know for sure
> that the compiler itself is really free, then we can't call the result
> free and put it in main.

Yes, but at each release, node_modules will need to be regenerated and
inspected again.

> I however agree that it seems poor practice to duplicate these build
> modules in every packages. But if the required versions are different,
> there is no real other way. If there is a set of modules which are known
> to be used widely and with stable versions, then they could be put in a
> shared package.

I can't say for sure.

>> But this would solve some of the problems, yes.
>
> What problems remain?

Parametrized and custom builds. But we don't have to solve everything at
once.
-- 
Every cloud engenders not a storm.
-- William Shakespeare, "Henry VI"


signature.asc
Description: PGP signature

Re: Security concerns with minified javascript code

[Samuel Thibault]

Vincent Bernat, le Wed 02 Sep 2015 11:20:32 +0200, a écrit :
>  ❦  2 septembre 2015 10:18 +0200, Samuel Thibault  :
> >> Or maybe you propose to just ship the whole "node_modules" directory
> >> (which has all the dependencies) with jQuery sources?
> >
> > That'd be a lot better than nothing.
> 
> OK. Also, node_modules for jQuery is 76M (for 3.x, 70M for 2.x).

Is is arch:all?  If so, I guess 76M is completely fine.

> >> This would incur some work on d/copyright and I don't see like this
> >> would be a good practice.
> >
> > Yes, but that work on d/copyright is *needed*: if we don't know for sure
> > that the compiler itself is really free, then we can't call the result
> > free and put it in main.
> 
> Yes, but at each release, node_modules will need to be regenerated and
> inspected again.

Which maps to the fact that we need to continuously make sure that the
software we're basing on is free.

Samuel

Re: Security concerns with minified javascript code

[Thorsten Glaser]

Vincent Bernat  debian.org> writes:

>  2. Upstream may generate the final pre-minification file with complex
> tools, like an AMD loader or an ES6/ES5 transpiler, along with the
> use of non-packaged build tools like Grunt.

> problem. For the second one, a solution would be to consider the
> pre-minification JS code to be perfectly valid source code
> (indentations, comments, variable names, everything is here).

There is (I just had an epiphany) another possible criterium to apply
for to determine what the preferred form of modification is:

Does upstream accept patches for that form?

(The Perl configure script discussion also comes into play, here.)

bye,
//mirabilos

Re: Security concerns with minified javascript code

[Marvin Renich]

* Thorsten Glaser  [150902 07:50]:
> There is (I just had an epiphany) another possible criterium to apply
> for to determine what the preferred form of modification is:
   ^ for
  [Okay, so I'm being pedantic, but this is a common mistake.]

> Does upstream accept patches for that form?

I thoroughly and whole-heartedly disagree with this criterion.  As I
stated in an earlier message, the purpose of the source requirement in
the DFSG (and GPL, etc.) is not to protect the rights of the persons
distributing software, but those receiving the software.  There is no
requirement that changes to the software be returned to upstream; such a
requirement would violate the dissident and desert island tests¹.

The source requirement is so that the recipient can make changes if
desired, and if the changes are redistributed (not passed back to
upstream), the second-level recipient may also make changes.

Any test of preferred form for modification must be in terms of how the
recipient is able to use it, not how the distributor would like it.

...Marvin

[1] https://people.debian.org/~bap/dfsg-faq.html#testing

Bug#797775: ITP: python-tosca-parser -- parser for TOSCA Simple Profile in YAML

[Thomas Goirand]

Package: wnpp
Severity: wishlist
Owner: Thomas Goirand 

* Package name: python-tosca-parser
  Version : 0.1.0
  Upstream Author : OpenStack Foundation 
* URL : https://github.com/openstack/tosca-parser
* License : Apache-2.0
  Programming Lang: Python
  Description : parser for TOSCA Simple Profile in YAML

 The TOSCA Parser is an OpenStack project. It is developed to parse TOSCA
 Simple Profile in YAML. It reads the TOSCA templates and creates an
 in-memory graph of TOSCA nodes and their relationship.
 .
 TOSCA stands for: Topology and Orchestration Specification for Cloud
 Application.

Re: Security concerns with minified javascript code

[Ben Hutchings]

On Wed, 2015-09-02 at 08:59 -0400, Marvin Renich wrote:
> * Thorsten Glaser  [150902 07:50]:
> > There is (I just had an epiphany) another possible criterium to apply
> > for to determine what the preferred form of modification is:
>^ for
>   [Okay, so I'm being pedantic, but this is a common mistake.]
> 
> > Does upstream accept patches for that form?
> 
> I thoroughly and whole-heartedly disagree with this criterion.  As I
> stated in an earlier message, the purpose of the source requirement in
> the DFSG (and GPL, etc.) is not to protect the rights of the persons
> distributing software, but those receiving the software.  There is no
> requirement that changes to the software be returned to upstream; such a
> requirement would violate the dissident and desert island tests¹.
> 
> The source requirement is so that the recipient can make changes if
> desired, and if the changes are redistributed (not passed back to
> upstream), the second-level recipient may also make changes.
> 
> Any test of preferred form for modification must be in terms of how the
> recipient is able to use it, not how the distributor would like it.

My preferred form is a git repository of code written in C, Python, or
some other language I know.  That doesn't mean that a tarball of
Haskell code is non-free!

The preferred form for modification is generally whatever form an
upstream developer will load into a text editor or other interactive
editing tool.

Still, I think there are some exceptions to this.  I used to maintain
the sfc driver in Linux, which has some C header files generated by
script from Verilog or YAML files that aren't published.  In case I
received patches for these headers (usually spelling fixes) I would
make the corresponding change to the unpublished file as well.  I think
that, given the choice, outside developers would still have preferred
editing the C header files, so I was fairly comfortable with this.

Ben.

> [1] https://people.debian.org/~bap/dfsg-faq.html#testing
> 
-- 
Ben Hutchings
Horngren's Observation:
   Among economists, the real world is often a special case.



signature.asc
Description: This is a digitally signed message part

Re: Security concerns with minified javascript code

[Neil Williams]

On Wed, 2 Sep 2015 08:59:11 -0400
Marvin Renich  wrote:

> * Thorsten Glaser  [150902 07:50]:
> > There is (I just had an epiphany) another possible criterium to
> > apply for to determine what the preferred form of modification is:
>^ for
>   [Okay, so I'm being pedantic, but this is a common mistake.]
> 
> > Does upstream accept patches for that form?
> 
> I thoroughly and whole-heartedly disagree with this criterion.  As I
> stated in an earlier message, the purpose of the source requirement in
> the DFSG (and GPL, etc.) is not to protect the rights of the persons
> distributing software, but those receiving the software.  There is no
> requirement that changes to the software be returned to upstream;
> such a requirement would violate the dissident and desert island
> tests¹.

DFSG tests aside, there is copyleft. So where the recipient does
distribute their changes and the code is under a copyleft licence, it
is not just the next recipient who can benefit but also upstream.

There is little point meeting only DFSG requirements if the changes
have to be manually ported from one upstream release to the next simply
because the changes are made using a form which is not used by
upstream. Even without copyleft, the maintainer still has a clear
interest in getting those changes applied upstream or there's little
point in thinking of the software as free - we could easily be accused
of "hoarding" the changes in a way that works against other
distributions by "cutting out the upstream". That is actively unhelpful. 

> The source requirement is so that the recipient can make changes if
> desired, and if the changes are redistributed (not passed back to
> upstream), the second-level recipient may also make changes.
> 
> Any test of preferred form for modification must be in terms of how
> the recipient is able to use it, not how the distributor would like
> it.

Upstream is another recipient of code distributed under copyleft.
Having changes in a format which upstream can use is absolutely a
sensible and sane criterion for what is regarded as the form of the
code for modification. To do otherwise is to make the maintenance
burden untenable.

Every recipient needs to get the source code and the maintainer changes
in a format which is suitable for modification and that includes
the work of modification required to incorporate those changes into the
next upstream release. To rule out upstream requirements is nonsense.

-- 


Neil Williams
=
http://www.linux.codehelp.co.uk/



pgpw8VDlMGOXt.pgp
Description: OpenPGP digital signature

Strange units in systemd graphical.target

[Jayson Willson]

systemctl list-dependencies:
http://pastebin.com/EkdYjp7X

In my system default.target is symlink to graphical.target.
It does not seem obvious to me, why such services as exim4, tor, gpm, fail2ban 
are both in default.target==graphical.target and in multi-user.target.

Seems like only lightdm is specific for graphical system. 
Could you please clarify this situation? Thank you.

Bug#797798: ITP: bagel -- Computational Chemistry Package

[Michael Banck]

Package: wnpp
Severity: wishlist
Owner: Debichem Team 

* Package name: bagel
  Version : (git snapshot)
  Upstream Author : Shiozaki Group, Northwestern University
* URL : http://www.nubakery.org/
* License : GPL3+
  Programming Lang: C++
  Description : Computational Chemistry Package

 BAGEL (Brilliantly Advanced General Electronic-structure Library) is a
 computational chemistry package aimed at large-scale parallel
 computations.  It specializes on highgly accurate methods and includes
 density-fitting and relativistic effects for most of the methods it
 implements.
 .
 Features include:
  * HF
  * DFT
  * MP2
  * CASSCF
  * NEVPT2
  * CASPT2
  * MRCI
 .
 BAGEL does not include a disk interface, so computations need to fit in
 memory.

Re: Strange units in systemd graphical.target

[Michael Biebl]

Am 02.09.2015 um 17:00 schrieb Jayson Willson:
> systemctl list-dependencies:
> http://pastebin.com/EkdYjp7X
> 
> In my system default.target is symlink to graphical.target.
> It does not seem obvious to me, why such services as exim4, tor, gpm, 
> fail2ban are both in default.target==graphical.target and in 
> multi-user.target.
> 
> Seems like only lightdm is specific for graphical system. 
> Could you please clarify this situation? Thank you.

The services which are listed both in default.target (graphical.target)
and multi-user.target (exim4, tor, gpm, ...) do not provide a native
service file but are generated legacy SysV init scripts, which have

# Default-Start: 2 3 4 5

in their LSB header. Debian traditionally did not differentiate between
a "graphical" runlevel and a "multiuser" runlevel. That's why they show
up in both graphical.target and multi-user.target.

Michael
-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?



signature.asc
Description: OpenPGP digital signature

Re: Strange units in systemd graphical.target

[Jayson Willson]

Thank you, everything is clear now.

Re: Security concerns with minified javascript code

[Marvin Renich]

* Neil Williams  [150902 10:22]:
> Upstream is another recipient of code distributed under copyleft.
> Having changes in a format which upstream can use is absolutely a
> sensible and sane criterion for what is regarded as the form of the
> code for modification. To do otherwise is to make the maintenance
> burden untenable.
> 
> Every recipient needs to get the source code and the maintainer changes
> in a format which is suitable for modification and that includes
> the work of modification required to incorporate those changes into the
> next upstream release. To rule out upstream requirements is nonsense.

The whole point of this discussion is what does Debian require of
upstream for upstream to get its software distributed in Debian main.
It is presumed that upstream already has what it considers "source"; in
the case of this thread, that is minified JS.

My point is that if what upstream considers to be "source" is not
acceptable to Debian, and the Debian packager has to grab real source
from other places and use a build process that is different from what
upstream uses in order to make the Debian package satisfy the DFSG, then
upstream's wishes are not relevant to whether the Debian package
conforms to the DFSG.

Furthermore, if the Debian packager does not like upstream's arrangement
of source, even if it would satisfy the DFSG, and wishes to rearrange
it, whether or not the packager's arrangement satisfies the DFSG's
meaning of source should be judged on its own merit, not on whether
upstream is willing to accept patches based on the Debian packager's
arrangement.

I am _not_ saying this this is necessarily a good decision.  The
distinction is between the DFSG, which is one part of the Debian Social
Contract, and the whole DSC.  DSC point 2 requires that the Debian
maintainer give back to upstream.  But that has nothing to do with what
satisfies the DSFG definition of source.

My argument is not that Debian should not use a form that upstream
likes, but that the definition of "source" for purposes of the DFSG is
independent of upstream's definition of source.  If both source forms A
and B satisfy the DFSG, and upstream uses form A, that does not make
form B fail to satisfy the DFSG, even for Debian packages of upstream's
software.

...Marvin

Bug#797803: ITP: libcsp -- The Cubesat Space Protocol Library

[Agustin Henze]

Package: wnpp
Severity: wishlist
X-Debbugs-CC: debian-devel@lists.debian.org


   Package name: libcsp
Version: 1.4
Upstream Author: 2012 GomSpace ApS (http://www.gomspace.com)
 2012 AAUSAT3 Project (http://aausat3.space.aau.dk)
URL: https://github.com/GomSpace/libcsp
License: LGPL-2+
Description: The Cubesat Space Protocol Library
 Cubesat Space Protocol (CSP) is a small protocol stack written in C. CSP is
 designed to ease communication between distributed embedded systems in smaller
 networks, such as Cubesats. The design follows the TCP/IP model and includes a
 transport protocol, a routing protocol and several MAC-layer interfaces. The
 core of libcsp includes a router, a socket buffer pool and a connection
 oriented socket API.

-- 
TiN

Re: Security concerns with minified javascript code

[Marvin Renich]

* Ben Hutchings  [150902 10:12]:
> My preferred form is a git repository of code written in C, Python, or
> some other language I know.  That doesn't mean that a tarball of
> Haskell code is non-free!

I can't tell whether you are agreeing or disagreeing with me!

> The preferred form for modification is generally whatever form an
> upstream developer will load into a text editor or other interactive
> editing tool.

No, "A preferred form" is what upstream uses.  The DFSG does not use the
term "THE preferred form", and I believe that was wise.  There can be
multiple "preferred forms" for some software, and all are, in my
opinion, acceptable by the DFSG.  The real question is whether it is
reasonable to expect someone who wishes to modify the software to
consider the form "source".

Again, this is specifically about the DFSG, not the whole Debian Social
Contract.

This whole thread is about how Debian can conform to some points of the
DSC (e.g. points 4 and 2) by providing packages containing minified JS,
and still conform to the DFSG, and whether that means some packages that
are currently in main need to be moved to contrib or non-free.  The
point of contention is not "should we package this software for the
benefit of our users" but whether the packages are allowed in main,
which is strictly based on whether or not they conform to the DFSG.

...Marvin

Re: Security concerns with minified javascript code

[Neil Williams]

On Wed, 2 Sep 2015 13:14:31 -0400
Marvin Renich  wrote:

> * Neil Williams  [150902 10:22]:
> > Upstream is another recipient of code distributed under copyleft.
> > Having changes in a format which upstream can use is absolutely a
> > sensible and sane criterion for what is regarded as the form of the
> > code for modification. To do otherwise is to make the maintenance
> > burden untenable.
> > 
> > Every recipient needs to get the source code and the maintainer
> > changes in a format which is suitable for modification and that
> > includes the work of modification required to incorporate those
> > changes into the next upstream release. To rule out upstream
> > requirements is nonsense.
> 
> The whole point of this discussion is what does Debian require of
> upstream for upstream to get its software distributed in Debian main.
> It is presumed that upstream already has what it considers "source";
> in the case of this thread, that is minified JS.

Actually, not. Source, for upstream of JQuery at least, is a set of
directives to include files into a non-minified jquery.js which is then
minified as part of the build in Debian. At that point, we're all good.
It's when other packages have to have other bits of jquery which aren't
packaged or are out of date in Debian and end up pulling those in
as .min.js - there needs to be a way of still using those whilst
allowing for fixes to be applied and used. That is what we lack - the
ability to rebuild embedded .min.js from *any* source for modification.
The problem is that the further that tool diverges from the variety of
tools used by the variety of JS upstreams, the more likely it is that
Debian will introduce further bugs into the .min.js which the JS
upstream will not be able to reproduce.

There are three players:

0: Debian - needs to fix problems in JS wherever we find them.

1: The JS upstream and their non-packaged minifier - may be unaware of
the problem and have their own "source for modification" format. There
may be a maintainer in Debian, likely there is not or the package is
too far out of date. (JS upstreams where the code is up to date must be
dealt with by symlinks into the packages using that JS.)

2: The embedding upstream who use the .min.js but need a way of
patching actual source JS and rebuilding a new .min.js which is,
hopefully, as close to the old one + that one fix as possible without
introducing further bugs.

If there's no embedding upstream, the JS doesn't matter to Debian - we
aren't likely to find bugs in unused libraries of any language, except
FTBFS and someone will probably file an RM RoQA.

If the embedding upstream is writing JS of their own, then they need a
minifier in Debian main too.

None of those players are using or can use the minified JS as source or
for any form of modification other than rebuilding. The current players
are using *different* formats for modification and all lack
packaged or consistent tools to get from one to another.

It is a complete mess - with the embedding upstreams lost in the middle
with no idea of what they can do to avoid getting in the way.

> My point is that if what upstream considers to be "source" is not
> acceptable to Debian, and the Debian packager has to grab real source
> from other places and use a build process that is different from what
> upstream uses in order to make the Debian package satisfy the DFSG,
> then upstream's wishes are not relevant to whether the Debian package
> conforms to the DFSG.

There are different upstreams here. The upstream of the package
embedding the minified JS, yes, that situation needs a fix. The
upstream of the original JS that went into the minified JS has a
different source for modification again - and that one is likely to be
one of the ones which needs a whole gamut of packages to run the
minifier that actually produces the embedded .min.js.

No JS or embedding upstream is going to work with the .min.js as
modifiable source. That's clearly nonsense. What is happening is that
upstreams who need to embed JS are not finding the packages available
and therefore embed and then reference that file in that upstream
as .min.js.

The solution is to have a way of rebuilding that .min.js in Debian so
that fixes can be applied to the available JS for that file - and yet
the upstream for that JS file is likely to have a different build
process from that, so we end up with three possible source formats for
modification - JS upstream, embedded unminified and the one nobody can
actually modify but the one everyone wants to rebuild: minified.

> Furthermore, if the Debian packager does not like upstream's
> arrangement of source, even if it would satisfy the DFSG, and wishes
> to rearrange it, whether or not the packager's arrangement satisfies
> the DFSG's meaning of source should be judged on its own merit, not
> on whether upstream is willing to accept patches based on the Debian
> packager's arrangement.

It's up to the maintainer to handle upstreaming 

Re: Security concerns with minified javascript code

[Neil Williams]

On Wed, 2 Sep 2015 13:33:57 -0400
Marvin Renich  wrote:

> * Ben Hutchings  [150902 10:12]:
> > My preferred form is a git repository of code written in C, Python,
> > or some other language I know.  That doesn't mean that a tarball of
> > Haskell code is non-free!

> No, "A preferred form" is what upstream uses.  The DFSG does not use
> the term "THE preferred form", and I believe that was wise.  There
> can be multiple "preferred forms" for some software, and all are, in
> my opinion, acceptable by the DFSG.  The real question is whether it
> is reasonable to expect someone who wishes to modify the software to
> consider the form "source".

Minified isn't source for modification, that much is as far as we've
got on consensus in this thread. However, lintian has had checks on
minified without unminified JS available for some time and embedding
upstreams can follow that, so there is source for modification which
the embedding upstream would be able to use but, likely as not, the
original JS upstream would not. Equally, fixing the unminified source
likely also means patching the embedding upstream templates to use
the updated unminified JS as the .min.js cannot be built in the same
way as the original JS upstream. There are other minifiers available
but those could introduce subtle new bugs.

Debian could mandate that embedding upstreams do not include .min.js at
all and I could live with that. It doesn't fix the problem that Debian
lacks a sane way of maintainers keeping javascript packages up to date
with javascript upstreams in ways that embedding upstreams can actually
use.

The result of that is making software in Debian worse by having more
unminified embedded copies at different versions across lots of
packages, exponentially more security issues and more JS packages out of
date relative to the JS upstream. That would not be good.

Embedding software is a bad idea - Debian needs a way of keeping
javascript packages up to date with JS upstreams so that other
upstreams don't need to embed (minified or non-minified) JS in the first
place and everyone has a source for modification in a single place, not
spread across a dozen large packages that simply add in some JS to make
one small (but important) bit of code work.

> This whole thread is about how Debian can conform to some points of
> the DSC (e.g. points 4 and 2) by providing packages containing
> minified JS, and still conform to the DFSG, and whether that means
> some packages that are currently in main need to be moved to contrib
> or non-free.  The point of contention is not "should we package this
> software for the benefit of our users" but whether the packages are
> allowed in main, which is strictly based on whether or not they
> conform to the DFSG.

It's not about contrib or main, that is roughly equivalent to thinking
that every DFSG problem looks like a nail because all you have available
is a large hammer instead of solving the problem inside main.

It is about getting the tools to convert the source for modification
(unminified JS) which is included in packages in main which embed JS
into a minified JS that doesn't make the security problems worse by
introducing it's own bugs by not building the .min.js in the same way
as the JS upstream. It's also about implementing a method which allows
JS maintainers to keep up to date with JS upstreams so that embedding
upstreams don't have to do any of this in the first place.

-- 


Neil Williams
=
http://www.linux.codehelp.co.uk/



pgprkWwZuZgRK.pgp
Description: OpenPGP digital signature

Bug#797818: ITP: r-cran-coin -- GNU R package providing conditional inference procedures for the general independence problem

[Jonathon]

Package: wnpp
Severity: wishlist
Owner: Jonathon 

* Package name: r-cran-coin
  Version : 1.1-0
  Upstream Author : Torsten Hothorn 
* URL : http://coin.r-forge.r-project.org/
* License : GPL
  Programming Lang: R, C
  Description : GNU R package providing conditional inference procedures 
for the general independence problem

r-cran-coin is a GNU R package providing conditional inference procedures for 
the general independence problem including two-sample, K-sample (non-parametric 
ANOVA), correlation, censored, ordered and multivariate problems.

Bug#797819: ITP: r-cran-afex -- GNU R package providing convenience functions for analyzing factorial experiments using ANOVA or mixed models

[Jonathon]

Package: wnpp
Severity: wishlist
Owner: Jonathon 

* Package name: r-cran-afex
  Version : 0.14-2
  Upstream Author : Henrik Singmann 
* URL : https://github.com/singmann/afex
* License : GPL
  Programming Lang: R
  Description : GNU R package providing convenience functions for analyzing 
factorial experiments using ANOVA or mixed models

r-cran-afex is a GNU R package providing convenience functions for analyzing 
factorial experiments using ANOVA or mixed models. aov_ez(), aov_car(), and 
aov_4() allow specification of between, within (i.e., repeated-measures), or 
mixed between-within (i.e., split-plot) ANOVAs for data in long format (i.e., 
one observation per row), potentially aggregating multiple observations per 
individual and cell of the design. mixed() fits mixed models using lme4::lmer() 
and computes p-values for all fixed effects using either Kenward-Roger 
approximation for degrees of freedom (LMM only), parametric bootstrap (LMMs and 
GLMMs), or likelihood ratio tests (LMMs and GLMMs). afex uses type 3 sums of 
squares as default (imitating commercial statistical software).

Bug#797820: ITP: r-cran-bayesfactor -- GNU R package providing a suite of functions for computing various Bayes factors for simple designs

[Jonathon]

Package: wnpp
Severity: wishlist
Owner: Jonathon 

* Package name: r-cran-bayesfactor
  Version : 0.9.11-1
  Upstream Author : Richard D. Morey 
* URL : http://bayesfactorpcl.r-forge.r-project.org/
* License : GPL
  Programming Lang: R, C
  Description : GNU R package providing a suite of functions for computing 
various Bayes factors for simple designs

r-cran-bayesfactor is a GNU R package providing a suite of functions for 
computing various Bayes factors for simple designs, including contingency 
tables, one- and two-sample designs, one-way designs, general ANOVA designs, 
and linear regression.

Re: Security concerns with minified javascript code

[Jeroen Dekkers]

At Tue, 1 Sep 2015 18:56:45 +0200,
Raphael Hertzog wrote:
> For me, the javascripts bits in wordpress/publican are not part of the
> product, they are external libraries whose preferred form of use is
> by embedding a copy of the library... that sucks but it's the way it is.
> 
> I do not see significant value in extending my packaging to rebuild
> the minified files from source as part of the wordpress/publican source
> package. On the opposite, it has a significant cost:
> - I have to add the sources when upstream does not ship them
>   (which is not a problem for many upstreams since the BSD-ish
>   licences do not require you to provide the sources)
> - ensure the sources are in sync with the minified copy
>   (even when friendly upsreams provide the required sources
>   on our request, they sometimes updates only one the minified file
>   and forget about the sources in some other directory)
> - if the minifier is not the same as upstream, it will create
>   a divergence with upstream and can always be a source of
>   suspicion when we report issues to upstream...

I do see at least one very significant advantage of rebuilding: it's a
lot easier to check that no malicious code is inserted. And if I
understand you correctly you're shipping minified files modified by
upstream for which there might not even be complete up-to-date source
available at all, so how do you know that neither upstream nor someone
who compromised the server used by upstream inserted any backdoor in
the minified file?

What you're saying is like it's fine to have a precompiled static
auxiliary C library that get's linked into the big main program
because making sure that you've got the correct corresponding source
for that small library is hard, the whole world just uses the static
library anyway and in case we do have the correct source then we
probably don't use the exact same compiler as upstream so the object
files would diverge from upstream. Doesn't that sound a bit ridiculous
if we're talking about C? So why would it be okay if we're talking
about javascript?


Kind regards,

Jeroen Dekkers

Re: Security concerns with minified javascript code

[Bas Wijnen]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Wed, Sep 02, 2015 at 07:33:10PM +0100, Neil Williams wrote:
> On Wed, 2 Sep 2015 13:33:57 -0400
> Marvin Renich  wrote:
> 
> > * Ben Hutchings  [150902 10:12]:
> > > My preferred form is a git repository of code written in C, Python,
> > > or some other language I know.  That doesn't mean that a tarball of
> > > Haskell code is non-free!
> 
> > No, "A preferred form" is what upstream uses.  The DFSG does not use
> > the term "THE preferred form", and I believe that was wise.

The DFSG doesn't define source at all.  There seems to be consensus (you're the
only one who doesn't seem to agree) that the definition from the GPL is a good
one, and that does say "the".

> > There can be multiple "preferred forms" for some software, and all are, in
> > my opinion, acceptable by the DFSG.  The real question is whether it is
> > reasonable to expect someone who wishes to modify the software to consider
> > the form "source".

I disagree partly.  It is possible to copy a generated file and use that as
source.  IMO that isn't the case until there have actually been made
modifications to that file, though.  If an upstream (which doesn't need to be
the original upstream) actually uses a file to make modifications, an argument
can be made that this format is source.

At the same time, we should try to convince upstreams that do such a thing to
stop it; it causes code duplication and a (security) support nightmare.

"Someone might think they can make modifications to this file" is much too
broad; for some modifications a hex editor is good enough.  And in some cases
that is totally reasonable, such as an executable for which you don't have
source.  That doesn't make binary exectutables source.

A requirement that there is a (serious, not set up to circumvent this rule)
upstream that actually uses this format as their source is still too broad, but
it's a lot better than the overly broad definition you propose.  With your
definition, literally everything is source.

Here's a rule to limit the selection a bit: a file is certainly not source if
it was originally generated from a different file, and has not been modified.

> Minified isn't source for modification, that much is as far as we've
> got on consensus in this thread. However, lintian has had checks on
> minified without unminified JS available for some time and embedding
> upstreams can follow that, so there is source for modification which
> the embedding upstream would be able to use but, likely as not, the
> original JS upstream would not.

If the embedding upstream wants to use updates from the original upstream, they
will not want to make changes to the generated code.  By the sound of it they
haven't figured that out yet, but if they port their changes to new releases of
the original upstream, then those releases are definitely not source,
regardless of whether the embedding upstream is using them for modification.

> Debian could mandate that embedding upstreams do not include .min.js at
> all and I could live with that. It doesn't fix the problem that Debian
> lacks a sane way of maintainers keeping javascript packages up to date
> with javascript upstreams in ways that embedding upstreams can actually
> use.

Why?  If we rebuild everything, the embedding upstreams can copy our build
procedure, or they can take the minified file we generated.

> The result of that is making software in Debian worse by having more
> unminified embedded copies at different versions across lots of
> packages, exponentially more security issues and more JS packages out of
> date relative to the JS upstream. That would not be good.

I thought the minified files were going to be in their own package so there can
be symlinks?  Any other solution involves code duplication which causes those
problems.

> Embedding software is a bad idea - Debian needs a way of keeping
> javascript packages up to date with JS upstreams so that other
> upstreams don't need to embed (minified or non-minified) JS in the first
> place and everyone has a source for modification in a single place, not
> spread across a dozen large packages that simply add in some JS to make
> one small (but important) bit of code work.

Upstreams will always continue embedding though, because even if Debian fixes
it, not everyone will.  And they want their software to run on those other
platforms as well.  So what we want, is to make sure that Debian doesn't need
the embedded copy.  It can still be there, we just shouldn't use it.

> It's not about contrib or main, that is roughly equivalent to thinking
> that every DFSG problem looks like a nail because all you have available
> is a large hammer instead of solving the problem inside main.

But in the end, it is.  If a maintainer refuses to make their package conform
to the DFSG, it cannot be in main.  We all agree that fixing the problem is
better, but people are now claiming that literally every file is source, and
previous

Re: Security concerns with minified javascript code

[Russ Allbery]

The below is very much a tangent from the minified Javascript case, and
not applicable to that case.

Bas Wijnen  writes:

> Here's a rule to limit the selection a bit: a file is certainly not
> source if it was originally generated from a different file, and has not
> been modified.

This makes files for which the source has been irrevocably lost
non-DFSG-free.  I don't think that's a good feature, nor is that the
standard that ftpmaster has used in the past for the archive.

Admittedly, that's something of an edge case, but I've uploaded PostScript
files with that property in one package in the past because they were
still the best available documentation for part of a software package (and
called this out in debian/copyright, and had the package approved by
ftpmaster).  An extensive search had been done for the original source
(which was originally in an internal IBM documentation generation system),
and everyone including IBM was pretty sure that the source was gone
forever and will never be found.

I think reading "preferred form of modification" from the perspective of
upstream is a useful standard because it handles some edge cases like
that, and because it feels ethically consistent with free software
principles.  The goal is that everyone with a copy of the software should
be on equal footing.  The person distributing the software should have no
special access to sources that those receiving the software don't get.

If *no one* has access to anything better than a binary file, then
possession of that binary file puts you on an equal footing with everyone
else in the world, which I think is all that we can reasonably ask.

-- 
Russ Allbery (r...@debian.org)   

Bug#797837: ITP: node-hashish -- maniuplate hash data structures in Node.js

[Bastien Roucariès]

Package: wnpp
Severity: wishlist
Owner: "Bastien Roucariès" 

* Package name: node-hashish
  Version : 0.4.0
  Upstream Author : James Halliday 
* URL : https://github.com/substack/node-hashish/
* License : X11
  Programming Lang: js
  Description : maniuplate hash data structures in Node.js

 Hashish is a Node.js library for manipulating hash data structures. It is
 distilled from the finest that ruby, perl, and haskell have to offer by way of
 hash/map interfaces.

 Needed for packaging browserify needed to package a reverse dep of grunt.

Re: Security concerns with minified javascript code

[Dmitry Smirnov]

On Tuesday 01 September 2015 17:46:30 Josh Triplett wrote:
> Nikolaus Rath wrote:
> > I don't think 28 kB vs 73 kB is a difference that people will notice
> > over the network in *most* situations. Even at just 100 kB/s that's
> > 0.28 vs 0.73 seconds, and only when the page is first loaded.
> 
> That's absolutely a critical difference, even on a faster connection.
> Multiple studies have demonstrated that page load time matters for user
> retention.

Correlation does not imply causation.


> Amazon did a study that showed every ~100ms of page load
> delay lost them 1% in sales.

It could be that small percentage of Amazon users are impulsive trigger-happy 
buyers. :)
However that conclusion is probably wrong due to number of reasons:

 * 1% loss of sales is probably caused not by minification but by other 
means. Remember that minification affects only first page load. I suppose 
most buyers do not just purchase something from the very first page they've 
opened.

 * It is easy to claim loss of sales (and blame it on page load speed) 
without checking how percentage of returns is affected. Impulsive buyers 
never change their minds, right? ;)

 * Most people tend to think harder about more expensive purchases. Therefore 
"loss of revenue" is likely to be much smaller than "loss of sales".


> Google found that half a second slower
> load time for results pages drove off 20% of users.

Who says that loss of this audience matters? Have you ever closed a web site 
just because it is loaded few seconds slower than the other and not due to 
its content?


> Google also prioritizes faster sites in search results.

I doubt that search engine optimisation is important in this context. Google 
have no DFSG concerns and they may prioritise based on other things like SPDY 
support etc. Are you sure Google prioritises smaller web pages over heavy 
ones? (because this is what you're saying as minified content reduces size of 
the web site and not necessary speed of its loading). Besides Google search 
engine is probably just ignores JavaScripts and CSS hence minification should 
not make any difference (unless it is HTML minification).


> So yes, minifiers matter.

IMHO there is more harm than good. The only case for minification that I can 
think of is to increase web server capacity a little to cope with flow of new 
users following some sort of AD campaign. A poor substitute for capacity 
planning or a case when network link is congested.

Minification makes multiple assumptions such as that web app is perfect and 
nobody would ever need to open JS console and report errors. Or that nobody 
would like to learn about web site features from non-minified CSS and JS. Let 
alone debugging some of us do not like proprietary javascripts running in our 
browsers -- minification kills opportunity for security peer review etc.

Finally, one may think that maintenance cost of minified JavaScripts in 
Debian outweighs all the "benefits" by huge margin.

Again, here is my summary why minification is unnecessary:

https://wiki.debian.org/onlyjob/no-minification

I recognise importance of user experience and I know that UI responsiveness 
is important for perception. Actually this is one of my concerns about 
minification: although it negligibly improves speed of downloading of web 
pages for a first time, in theory it may negatively affect JavaScript run-
time performance (e.g. speed of initialisation or UI lag)...

-- 
Regards,
 Dmitry Smirnov.

---

Odious ideas are not entitled to hide from criticism behind the human
shield of their believers' feelings.
-- Richard Stallman


signature.asc
Description: This is a digitally signed message part.

Re: Security concerns with minified javascript code

[Vincent Bernat]

 ❦  3 septembre 2015 12:23 +1000, Dmitry Smirnov  :

>> Amazon did a study that showed every ~100ms of page load
>> delay lost them 1% in sales.
>
> It could be that small percentage of Amazon users are impulsive trigger-happy 
> buyers. :)
> However that conclusion is probably wrong due to number of reasons:
[...]

Please, publish your own study. This number is well known and supported
by an entity which is likely to have a population large enough to be
significant.

>> So yes, minifiers matter.
>
> IMHO there is more harm than good. The only case for minification that I can 
> think of is to increase web server capacity a little to cope with flow of new 
> users following some sort of AD campaign. A poor substitute for capacity 
> planning or a case when network link is congested.
>
> Minification makes multiple assumptions such as that web app is perfect and 
> nobody would ever need to open JS console and report errors. Or that nobody 
> would like to learn about web site features from non-minified CSS and JS. Let 
> alone debugging some of us do not like proprietary javascripts running in our 
> browsers -- minification kills opportunity for security peer review etc.
>
> Finally, one may think that maintenance cost of minified JavaScripts in 
> Debian outweighs all the "benefits" by huge margin.

Without minification, we'll just ship packages that people won't
use. Why would I run a crippled installation of Wordpress that will
drive of part of my users to another competitor?

We don't turn C into an interpreted language because it would be easier
to inspect the resulting binaries.
-- 
Don't stop with your first draft.
- The Elements of Programming Style (Kernighan & Plauger)


signature.asc
Description: PGP signature