Re: Not running tests because tests miss source code is not useful

[Pirate Praveen]

On 7 October 2021 3:02:55 am IST, Thomas Goirand  wrote:
>On 10/6/21 6:53 PM, Pirate Praveen wrote:
>> [adding -devel]
>> 
>> On ബു, ഒക്ടോ 6 2021 at 12:16:07 വൈകു +0200 +0200, Jonas Smedegaard
>>  wrote:
>>> Quoting Yadd (2021-10-06 11:43:40)
  On Lu, 04 oct 21, 16:40:48, Bastien Roucari�s wrote:
  > Source: src:node-lodash
  > Version: 4.17.21+dfsg+~cs8.31.173-1
  > Severity: serious
  > Justification: do not compile from source
  >
  > Dear Maintainer,
  >
  > The vendor directory should be emptied
  >
  > The debug version is compiled without source (lintian warn) and
 moreover the
  > rest of file are already packaged
  >
  > grep -R vendor * gives only a few hit that could be cured by
 symlinking
  >
  > Bastien
  Hi,

  this files are used for test only, maybe severity could be decreased.
>>>
>>> I find the severity accurate: Relying on non-source code is a severe
>>> violation of Debian Policy, not matter the purpose of relying on it.
>> 
>> I think we should change the policy here. Running tests helps improve
>> the quality of the software we ship. Many times the vendored code is
>> used to ensure the code does not break in a specific situation. I don't
>> think reducing test coverage in such situations is really helpful.
>
>Right, running tests helps improve the quality of software we ship.
>Which is why you probably need to test using what's shipped in Debian
>rather than using a vendored source-less code.

We are not shipping the source less code. This is used only during tests. I 
don't think we are not gaining anything by removing tests here. Just making it 
harder for the package maintainer to run tests.

>If we rely on non-free code for tests, that's really bad too, and that
>must be avoided just like we're avoiding source-less code everywhere
>else in Debian. The policy shall not change, please.
>

The code is not non-free here, just a specific version of a Free Software code 
built outside Debian.

I think tools required for tests should be considered separately from tools 
required to compile. I think it should be treated similar to test data.

What you are proposing would require the package maintainer to adapt these 
tests to versions available (many times with different API versions) in Debian 
and the easier choice is disabling tests.

I think blindly applying a rule without thinking of any consequences is bad 
too. Just because it is bad in one situation does not mean it will be bad in 
every situation. We should evaluate pros and cons of each situation before 
making a decision. Blind faith is more suitable for religions and not for a 
project like ours.

I think a nocheck build profile which excludes these files from build is 
sufficient to ensure we are not using these to create binary package. This way 
we guarantee only packages in main is used to generate the binary, but still 
allows to run tests optionally making it easy to find problems, especially 
during transitions. Currently when tests are missing transitions are harder 
because we can't find breakages easily since tests are disabled.

The current policy is not making Debian better.
-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: Q. What is the best practice about +dfsg and +ds extension?

[Kentaro Hayashi]

Hi,


It seems that it is reasonable to do so.
(Use +dfsg-1 first, then switch to +dfsgN-1)

Thanks!


2021年10月5日(火) 13:57 Russ Allbery :
>
> Kentaro Hayashi  writes:
>
> > What do you think about it?
>
> > 1. We should use +dfsg-1 style
> > 2. We should use +dfsgN-1 style
> > 3. We should use +dfsg.N-1 style
> > 4. Other
>
> I would start with +dfsg-1 because it's fairly rare to have to iterate on
> the repackaging.  You can then switch to +dfsgN-1 with the second and
> subsequent repackagings if needed.  (Although if I knew in advance I would
> probably need to iterate, I'd start with +dfsgN-1.)
>
> There's an argument for consistency to always use +dfsgN-1, I guess, but I
> don't think it matters enough to bother.
>
> I would not use +dfsg.N-1.  It's not consistent with the other places
> where we add suffixes, such as backporting and stable updates.
>
> --
> Russ Allbery (r...@debian.org)  
>


-- 
Kentaro Hayashi 

Re: [Pkg-javascript-devel] Bug#995722: Not running tests because tests miss source code is not useful

[Jonas Smedegaard]

Quoting Yadd (2021-10-07 07:06:42)
> Le 06/10/2021 à 23:32, Thomas Goirand a écrit :
>> On 10/6/21 6:53 PM, Pirate Praveen wrote:
>>> On ബു, ഒക്ടോ 6 2021 at 12:16:07 വൈകു +0200 +0200, Jonas Smedegaard
>>>  wrote:
 Quoting Yadd (2021-10-06 11:43:40)
>  On Lu, 04 oct 21, 16:40:48, Bastien Roucari�s wrote:
>> Source: src:node-lodash
>> Version: 4.17.21+dfsg+~cs8.31.173-1
>> Severity: serious
>> Justification: do not compile from source
>>
>> Dear Maintainer,
>>
>> The vendor directory should be emptied
>>
>> The debug version is compiled without source (lintian warn) and 
>> moreover the rest of file are already packaged
>>
>> grep -R vendor * gives only a few hit that could be cured by 
>> symlinking

>  this files are used for test only, maybe severity could be 
> decreased.

 I find the severity accurate: Relying on non-source code is a 
 severe violation of Debian Policy, not matter the purpose of 
 relying on it.
>>>
>>> I think we should change the policy here. Running tests helps 
>>> improve the quality of the software we ship. Many times the vendored 
>>> code is used to ensure the code does not break in a specific 
>>> situation. I don't think reducing test coverage in such situations 
>>> is really helpful.
>> 
>> Right, running tests helps improve the quality of software we ship. 
>> Which is why you probably need to test using what's shipped in Debian 
>> rather than using a vendored source-less code.
>> 
>> If we rely on non-free code for tests, that's really bad too, and 
>> that must be avoided just like we're avoiding source-less code 
>> everywhere else in Debian. The policy shall not change, please.
> 
> We are not talking about really-non-free code, but minified JavaScript 
> code released under a free license.
> 
> If we want to be strict here, there will be some excluded package: for 
> example most of the softwares listed here will be excluded: 
> https://lintian.debian.org/tags/embedded-javascript-library
> 
> Is it what you want ?

We all want to do most possible with Free software, and call that 
"main".

Some of us additionally want to extend that with possibilities beyond 
Free software, and call that "contrib" and "non-free".

We all want to be strict about using only Free software, but we do not 
necessarily want to throw away minified code.

We often throw away upstream-generated minified code because it is an 
easy way to ensure that we are strictly using only Free software, but 
alternatives exist: One alternative is to somehow ensure that the 
minified code is Free software - i.e. that all source for that code 
exist in Debian and if source changes then we are able to generate that 
minified code purely from the Debian-included sources.


 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

signature.asc
Description: signature

Re: Not running tests because tests miss source code is not useful

[Marvin Renich]

* Pirate Praveen  [211007 05:41]:
> On 7 October 2021 3:02:55 am IST, Thomas Goirand  wrote:
> >On 10/6/21 6:53 PM, Pirate Praveen wrote:
> >> On ബു, ഒക്ടോ 6 2021 at 12:16:07 വൈകു +0200 +0200, Jonas Smedegaard > 
> >>  wrote:
> >>> Quoting Yadd (2021-10-06 11:43:40)
>   On Lu, 04 oct 21, 16:40:48, Bastien Roucari�s wrote:
>   > Source: src:node-lodash
>   > Version: 4.17.21+dfsg+~cs8.31.173-1
>   > Severity: serious
>   > Justification: do not compile from source
>   >
>   > Dear Maintainer,
>   >
>   > The vendor directory should be emptied
>   >
>   > The debug version is compiled without source (lintian warn)
>   > and moreover the rest of file are already packaged
>   >
>   > grep -R vendor * gives only a few hit that could be cured by
>   > symlinking
>   >
>   > Bastien
>   Hi,
> 
>   this files are used for test only, maybe severity could be decreased.
> >>>
> >>> I find the severity accurate: Relying on non-source code is a severe
> >>> violation of Debian Policy, not matter the purpose of relying on it.
> >> 
> >> I think we should change the policy here. Running tests helps improve
> >> the quality of the software we ship. Many times the vendored code is
> >> used to ensure the code does not break in a specific situation. I don't
> >> think reducing test coverage in such situations is really helpful.
> >
> >Right, running tests helps improve the quality of software we ship.
> >Which is why you probably need to test using what's shipped in Debian
> >rather than using a vendored source-less code.
> 
> We are not shipping the source less code. This is used only during
> tests. I don't think we are not gaining anything by removing tests
> here. Just making it harder for the package maintainer to run tests.

If the original report is correct (I haven't looked at the package),
then Debian is, indeed, shipping the source-less code.  Debian ships
both source packages and binary packages.  The source-less code is in
the source package, and is thus shipped by Debian.  The DFSG applies to
both source and binary packages.

> >If we rely on non-free code for tests, that's really bad too, and that
> >must be avoided just like we're avoiding source-less code everywhere
> >else in Debian. The policy shall not change, please.

+1

> The code is not non-free here, just a specific version of a Free
> Software code built outside Debian.

I think you are confusing "Free Software" with "DFSG Free Software".
Debian has more strict standards than some other parts of the Free
Software community.

> I think tools required for tests should be considered separately from
> tools required to compile. I think it should be treated similar to
> test data.

Actually, I agree with this, but not exactly in the way you intended.  I
think there should be a way for a source package to specifically
identify a Build-Dep as a test (probably with a separate field, maybe
Test-Depends), and to identify tests that must run for the build to be
considered successful, and tests that are optional (perhaps
Test-Suggests).  This would allow Non-Free tests (and test data) to be
packaged separately and included in non-free where they belong, with the
requirement that a source package can Test-Suggests a non-free package,
but it cannot Test-Depends such a package.  This would also allow tests
that require external networking resources, or tests that consumed an
inordinate amount of CPU, or..., to be Test-Suggested, without
interfering with a normal binary build.

The buildds could be enhanced to build the binary packages without any
of the test dependencies present, then install the desired set of test
dependencies and run them.  This would ensure that the binary packages
were built without the test packages even present (a huge benefit, in my
opinion), but that tests were still performed.

I'm not sure how many existing tests require the built programs to still
have the build environment present (i.e. vestiges of the build process
are used by the test), but it would be nice if the binary package could
be sent to another machine where the binary package (without the source
package) was installed with the desired test packages, so the tests
could be run on the properly installed binary package.

This would, of course, require a moderate amount of implementation in
the buildds and core tools such as dpkg, and I am not in a position to
help with this, so the people who can do so would have to like my idea
enough to spend their own time to implement it.

> I think blindly applying a rule without thinking of any consequences is bad 
> too.

I don't think anyone is blindly applying the DFSG to the source package
here.  It is important to separate "DFSG Free" (main) from "Free but not
DFSG Free" (non-free) in all aspects of the source package, including
tests.  The problem is that there is currently no mechanism available to
have the build process identify and select either free

Bug#995883: ITP: python-npx -- extensions for NumPy

[Drew Parsons]

Package: wnpp
Severity: wishlist
Owner: Drew Parsons 
X-Debbugs-Cc: debian-devel@lists.debian.org, debian-pyt...@lists.debian.org, 
debian-scie...@lists.debian.org

* Package name: python-npx
  Version : 0.0.20
  Upstream Author : Nico Schlömer 
* URL : https://github.com/nschloe/npx
* License : BSD-3-clause
  Programming Lang: Python
  Description : extensions for NumPy

NumPy is a large library used everywhere in scientific computing.
That's why breaking backwards-compatibility comes at a significant
cost and is almost always avoided, even if the API of some methods is
arguably lacking. This package provides drop-in wrappers "fixing" those.

Provides alternative algorithms for dot, solve, sum_at/add_at,
unique_rows, isin_rows, mean.

Required by new releases of python-meshplex.

To be maintained under the Debian Python Team alongside numpy.

Re: Not running tests because tests miss source code is not useful

[Richard Laager]

On 10/7/21 4:40 AM, Pirate Praveen wrote:

What you are proposing would require the package maintainer to adapt these 
tests to versions available (many times with different API versions) in Debian 
and the easier choice is disabling tests.


I haven't looked into the specifics of this situation, but in general, 
tests should be run against the same versions of dependencies that the 
actual code will use, for what should be obvious reasons. If Debian has 
the dependencies with different API versions, then it's all the more 
important that the tests run against the Debian versions of the 
dependencies.


Running tests against vendored dependencies one isn't going to use at 
run-time is of limited usefulness. Presumably upstream is already 
running those tests against the vendored dependencies, so the odds of 
Debian finding breakage in the _source_ at packaging time is low.


--
Richard



OpenPGP_signature
Description: OpenPGP digital signature

Re: Not running tests because tests miss source code is not useful

[Russ Allbery]

Richard Laager  writes:

> I haven't looked into the specifics of this situation, but in general,
> tests should be run against the same versions of dependencies that the
> actual code will use, for what should be obvious reasons. If Debian has
> the dependencies with different API versions, then it's all the more
> important that the tests run against the Debian versions of the
> dependencies.

I believe the dependencies in question are test dependencies.  In other
words, they're dependencies required to drive the test suite machinery,
not dependencies of the code under test.

-- 
Russ Allbery (r...@debian.org)  

Re: Not running tests because tests miss source code is not useful

[Richard Laager]

On 10/7/21 11:35 AM, Russ Allbery wrote:

Richard Laager  writes:


I haven't looked into the specifics of this situation, but in general,
tests should be run against the same versions of dependencies that the
actual code will use, for what should be obvious reasons. If Debian has
the dependencies with different API versions, then it's all the more
important that the tests run against the Debian versions of the
dependencies.


I believe the dependencies in question are test dependencies.  In other
words, they're dependencies required to drive the test suite machinery,
not dependencies of the code under test.


Thanks for the clarification of the specifics!

In that case, I personally don't see a big problem with them being 
vendored as opposed to using system copies.


But AFAIK, they do still need to be DFSG-free to be in main if they are 
in the source tarball. And I personally would not consider minified JS 
(if that is indeed at issue here) to be "source code".


--
Richard



OpenPGP_signature
Description: OpenPGP digital signature

Re: Not running tests because tests miss source code is not useful

[Jonas Smedegaard]

Quoting Richard Laager (2021-10-07 19:49:56)
> On 10/7/21 11:35 AM, Russ Allbery wrote:
> > Richard Laager  writes:
> > 
> >> I haven't looked into the specifics of this situation, but in 
> >> general, tests should be run against the same versions of 
> >> dependencies that the actual code will use, for what should be 
> >> obvious reasons. If Debian has the dependencies with different API 
> >> versions, then it's all the more important that the tests run 
> >> against the Debian versions of the dependencies.
> > 
> > I believe the dependencies in question are test dependencies.  In 
> > other words, they're dependencies required to drive the test suite 
> > machinery, not dependencies of the code under test.
> 
> Thanks for the clarification of the specifics!
> 
> In that case, I personally don't see a big problem with them being 
> vendored as opposed to using system copies.
> 
> But AFAIK, they do still need to be DFSG-free to be in main if they 
> are in the source tarball. And I personally would not consider 
> minified JS (if that is indeed at issue here) to be "source code".

Right: It is ok to use upstream-provided pre-minified code, as long as 
that code is DFSG-free, which requires the source of that code must 
exist in Debian.

...and because that is often complicated to ensure (not because it 
violates DFSG in itself), it is easier to avoid upstream-provided 
pre-minified code.


 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

signature.asc
Description: signature

Re: Not running tests because tests miss source code is not useful

[Russ Allbery]

Jonas Smedegaard  writes:

> Right: It is ok to use upstream-provided pre-minified code, as long as
> that code is DFSG-free, which requires the source of that code must
> exist in Debian.

> ...and because that is often complicated to ensure (not because it
> violates DFSG in itself), it is easier to avoid upstream-provided
> pre-minified code.

Test suites are often a licensing mess.  Another common case that's not in
play here but that I've seen before is that long-standing projects that
have been used commercially often have test snippets with unclear
licensing that check for regressions that were previously seen in
proprietary environments.

Debian historically has erred on the side of maintaining clear source
availability and licensing status for everything in Debian (which includes
everything in any source package) at the cost of not availing ourselves of
test suites that would otherwise be useful.  That's unfortunately probably
the easy path here as well, until someone has time to find non-minified
versions of the test dependencies and either package them or include them
in the package.  It's frustrating to remove the tests, but the DFSG source
requirements as currently applied do not distinguish between code shipped
only in source packages and code also shipped in binary packages.

I can see an argument that we should not worry about minified files in
main that are (a) only in the source package and not in any binary
package, and (b) only used to run tests, not to build the binary packages.
(I'm not saying I agree or disagree, just that I can see the argument.)
But given the apparent consensus on this in past discussions, I suspect
that changing that rule would be GR material rather than debian-devel
thread material.  Making that sort of change without a GR to be sure the
project is behind it feels like the kind of thing that's likely to spawn
endless arguments that will sap everyone's will to live.

-- 
Russ Allbery (r...@debian.org)  

Bug#995884: ITP: zycore-c -- Zyan Core Library for C

[Andrea Pappacoda]

Package: wnpp
Severity: wishlist
Owner: Andrea Pappacoda 
X-Debbugs-Cc: debian-devel@lists.debian.org

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

* Package name: zycore-c
  Version : 1.0.0
  Upstream Author : zyantific 
* URL : https://github.com/zyantific/zycore-c
* License : Expat
  Programming Lang: C
  Description : Zyan Core Library for C

Internal library providing platform independent types, macros and a fallback
for environments without LibC.

This is a dependency of Dynarmic, a dependency of the yuzu emulator -
https://bugs.debian.org/947399


-BEGIN PGP SIGNATURE-

iIoEARYIADIWIQRm3vFSgpkMIZnvqAGooSioqxzuSQUCYV8edxQcYW5kcmVhQHBh
cHBhY29kYS5pdAAKCRCooSioqxzuSSHIAQDu4MylvOAgPfIl+nOHbZbs1C8C5rbM
8PeLOtZLuB7v6AEAm4BECL5aYeJ64g5PjJsu62GdzhgHIfD6e51emQGbsgM=
=Mk4T
-END PGP SIGNATURE-

Work-needing packages report for Oct 8, 2021

[wnpp]

The following is a listing of packages for which help has been requested
through the WNPP (Work-Needing and Prospective Packages) system in the
last week.

Total number of orphaned packages: 1279 (new: 40)
Total number of packages offered up for adoption: 203 (new: 0)
Total number of packages requested help for: 61 (new: 0)

Please refer to https://www.debian.org/devel/wnpp/ for more information.



The following packages have been orphaned:

   gdigi (#995878), orphaned today
 Description: utility to control DigiTech effect pedals
 Installations reported by Popcon: 66
 Bug Report URL: https://bugs.debian.org/995878

   gkrellm-gkrellmpc (#995836), orphaned yesterday
 Description: GKrellM plugin for controlling MPD
 Installations reported by Popcon: 38
 Bug Report URL: https://bugs.debian.org/995836

   lua-apr (#995499), orphaned 5 days ago
 Description: Apache Portable Runtime library for the Lua language
 Reverse Depends: lua-apr-dev
 Installations reported by Popcon: 22
 Bug Report URL: https://bugs.debian.org/995499

   lua-bit32 (#995500), orphaned 5 days ago
 Description: Backport of the Lua 5.2 bit32 library to Lua 5.1
 Reverse Depends: lua-bit32-dev lua-http lua-posix
 Installations reported by Popcon: 265
 Bug Report URL: https://bugs.debian.org/995500

   lua-bitop (#995502), orphaned 5 days ago
 Description: fast bit manipulation library for the Lua language
 Reverse Depends: liblua5.1-bitop-dev liblua5.1-bitop0
   libquvi-scripts-0.9 lua-bitop-dev minetest-mod-pycraft neovim
   redis-tools
 Installations reported by Popcon: 62196
 Bug Report URL: https://bugs.debian.org/995502

   lua-cgi (#995503), orphaned 5 days ago
 Description: CGI library for the Lua language
 Reverse Depends: lua-wsapi
 Installations reported by Popcon: 58
 Bug Report URL: https://bugs.debian.org/995503

   lua-copas (#995501), orphaned 5 days ago
 Description: Copas is a dispatcher of concurrent TCP/IP requests
 Reverse Depends: xavante
 Installations reported by Popcon: 30
 Bug Report URL: https://bugs.debian.org/995501

   lua-cosmo (#995505), orphaned 5 days ago
 Description: Template library for the Lua language
 Reverse Depends: deets lua-orbit
 Installations reported by Popcon: 42
 Bug Report URL: https://bugs.debian.org/995505

   lua-coxpcall (#995504), orphaned 5 days ago
 Description: Protected function calls across coroutines for Lua
 Reverse Depends: lua-copas lua-nvim lua-wsapi
 Installations reported by Popcon: 126
 Bug Report URL: https://bugs.debian.org/995504

   lua-curl (#995507), orphaned 5 days ago
 Description: libcURL bindings for the Lua language
 Reverse Depends: lua-curl-dev
 Installations reported by Popcon: 113
 Bug Report URL: https://bugs.debian.org/995507

   lua-doc (#995509), orphaned 5 days ago
 Description: Documentation generation library for the Lua language
 Reverse Depends: luadoc
 Installations reported by Popcon: 75
 Bug Report URL: https://bugs.debian.org/995509

   lua-expat (#995506), orphaned 5 days ago
 Description: libexpat bindings for the Lua language
 Reverse Depends: libcegui-mk2-dev libquvi-scripts-0.9 lua-cgi
   lua-expat-dev lua-soap lua-svn lua-xmlrpc prosody
 Installations reported by Popcon: 57926
 Bug Report URL: https://bugs.debian.org/995506

   lua-filesystem (#995508), orphaned 5 days ago
 Description: luafilesystem library for the Lua language
 Reverse Depends: corsix-th genometools lmod lua-busted lua-cgi
   lua-check lua-doc lua-filesystem-dev lua-ldoc lua-lxc (7 more
   omitted)
 Installations reported by Popcon: 3264
 Bug Report URL: https://bugs.debian.org/995508

   lua-leg (#995510), orphaned 5 days ago
 Description: Lua 5.1 grammar, with parsing and manipulation
   facilities
 Reverse Depends: shake
 Installations reported by Popcon: 53
 Bug Report URL: https://bugs.debian.org/995510

   lua-logging (#995512), orphaned 5 days ago
 Description: Logging library for the Lua language
 Reverse Depends: lua-doc
 Installations reported by Popcon: 100
 Bug Report URL: https://bugs.debian.org/995512

   lua-lpeg (#995514), orphaned 5 days ago
 Description: LPeg library for the Lua language
 Reverse Depends: corsix-th genometools lua-cosmo lua-json lua-leg
   lua-lpeg-dev lua-lpeg-patterns nmap wordgrinder-ncurses
   wordgrinder-x11
 Installations reported by Popcon: 64298
 Bug Report URL: https://bugs.debian.org/995514

   lua-lpty (#995511), orphaned 5 days ago
 Description: PTY library for the Lua language
 Reverse Depends: lua-lpty-dev
 Installations reported by Popcon: 28
 Bug Report URL: https://bugs.debian.org/995511

   lua-markdown (#995513), orphaned 5 days ago
 Description: Pure Lua