Bug#831995: ITP: parsnp -- rapid core genome multi-alignment

[Andreas Tille]

Package: wnpp
Severity: wishlist
Owner: Andreas Tille 

* Package name: parsnp
  Version : 1.2
  Upstream Author : Todd J Treangen 
* URL : https://github.com/marbl/parsnp
* License : BSD
  Programming Lang: C, Python
  Description : rapid core genome multi-alignment
 Parsnp was designed to align the core genome of hundreds to thousands of
 bacterial genomes within a few minutes to few hours. Input can be both
 draft assemblies and finished genomes, and output includes variant (SNP)
 calls, core genome phylogeny and multi-alignments. Parsnp leverages
 contextual information provided by multi-alignments surrounding SNP
 sites for filtration/cleaning, in addition to existing tools for
 recombination detection/filtration and phylogenetic reconstruction.


Remark: This package is maintained by the Debian Med team at
https://anonscm.debian.org/git/debian-med/parsnp.git

Bug#832005: ITP: r-cran-tibble -- GNU R Simple Data Frames

[Andreas Tille]

Package: wnpp
Severity: wishlist
Owner: Andreas Tille 

* Package name: r-cran-tibble
  Version : 1.1
  Upstream Author : Hadley Wickham, Romain Francois, Kirill Müller, RStudio
* URL : https://cran.r-project.org/web/packages/tibble
* License : MIT/X
  Programming Lang: GNU R
  Description : GNU R Simple Data Frames
 This GNU R package provides a 'tbl_df' class that offers better checking
 and printing capabilities than traditional data frames.


Remark: This package is a new dependency of r-cran-dplyr and will be maintained
 by the Debian Med team at
svn://anonscm.debian.org/debian-med/trunk/packages/R/r-cran-tibble/trunk/

Re: New default -fdebug-prefix-map build flag for dpkg

[Mattia Rizzolo]

On Mon, Jul 18, 2016 at 08:57:01AM +, Mattia Rizzolo wrote:
> Hi *,
> 
> As part of the Reproducible Builds effort [0], we would like to enable
> a new default build flag from the reproducible/fixdebugpath feature
> area in order to prevent issues with build paths.

[...]

> clang 3.8 supports the build flag and does not save the path in
> DW_AT_producer (although the source path gets included in the .strtab
> section if the source path is passed absolutely).  We asked [3] the
> clang maintainers whether they would be willing to backport the
> -fdebug-prefix-path, but in the worst case there are only 3 clang
> reverse build-deps FTBFS due to this [4].

[...]

> We enabled the reproducible/fixdebugpath feature in the Reproducible
> Builds CI one month ago and whilst we have about 3k packages yet to
> build we are already confident that there won't be any major regression
> related to this other than those 3 packages.

We have now completed the rebuild, and the packages FTBFSing with this
new build flag are these:
 - afl #828178
 - libblocksruntime #832015
 - libclc # 832014
 - ncl #832011
 - clustalx #832013
 - clonalframe #832012

The first 3 are due to clan 3.7 not supporting the build flag, so it's
either clang being fixed (#819185) , or packages workarounding it.

> Thus, following the dpkg team's policy [5] about adding a new default
> build flag, I'm seeking a wider discussion to see whether somebody has
> any concern we haven't already taken care of.

I think the failing packages are nothing particularly interesting, and
it's very easy to fix, so if no concerns arise in the next very few
days, I'm going to open a bug to dpkg to have the build flag turned on
by default.

-- 
regards,
Mattia Rizzolo

GPG Key: 66AE 2B4A FCCF 3F52 DA18  4D18 4B04 3FCD B944 4540  .''`.
more about me:  https://mapreri.org : :'  :
Launchpad user: https://launchpad.net/~mapreri  `. `'`
Debian QA page: https://qa.debian.org/developer.php?login=mattia  `-


signature.asc
Description: PGP signature

TMPDIR - Do we also need a drive backed TPMDIR ?

[Ritesh Raj Sarraf]

Hi,

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=831998

This bug is an example of the problem. I am bringing it here to know views of
fellow developers.


I am not sure if there's an fd.o defined standard for environment variables, but
a quick web search pointed to the IEEE and Open Group link.
http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html

None of the standards talk about where TMPDIR should be pointing to. On
GNU/Linux, I recollect it pointing to /tmp and always wondered what environment
variable would point to /var/tmp

On the Debian front, I can't recollect what our official stand is. I could find
a conversation at the following link. It is fairly recent from 2012.
https://lists.debian.org/debian-devel/2012/06/msg00311.html

My /tmp is on tmpfs. I just don't recollect if it was the default or I enabled
it explicitly. Currently, it is managed by systemd, but I do recollect
initscripts providing similar feature.

● tmp.mount - Temporary Directory
   Loaded: loaded (/usr/share/systemd/tmp.mount; enabled; vendor preset:
enabled)
   Active: active (mounted) since Wed 2016-07-20 21:50:05 IST; 19h ago
Where: /tmp
 What: tmpfs
 Docs: man:hier(7)
   http://www.freedesktop.org/wiki/Software/systemd/APIFileSystems
   CGroup: /system.slice/tmp.mount

Jul 20 21:50:05 learner systemd[1]: tmp.mount: Directory /tmp to mount over is
not empty, mount
Jul 20 21:50:05 learner systemd[1]: Mounting Temporary Directory...
Jul 20 21:50:05 learner systemd[1]: Mounted Temporary Directory.



Now, back to the actual problem. For many applications, we rely on the TMPDIR
environments. Tools like Python's modules help use these variables and not worry
about the underneath platform.

Under Linux, with /tmp more commonly on tmpfs, how are developers dealing with
it? tmpfs is limited and multi gigabyte operations may end up filling it all (as
is the case in the debdelta bug report above).

The whole idea of tmpfs for /tmp was performance. Email attachments could be
temporarily opened under tmpfs


But what do we do about larger operations where temporary data may have the
possibility of being huge. I think most video/audio transcoding workflows may be
hit by this.

Should there be another TPMDIR? Like DTMPDIR, pointing to the underneath disk,
where size is limited by the capacity available to partition/disk ?

This could allow developers to choose one over the other based on their needs.
It could also help standardize it as a fallback.

-- 
Given the large number of mailing lists I follow, I request you to CC
me in replies for quicker response

signature.asc
Description: This is a digitally signed message part

Re: TMPDIR - Do we also need a drive backed TPMDIR ?

[Ritesh Raj Sarraf]

On Thu, 2016-07-21 at 17:19 +0530, Ritesh Raj Sarraf wrote:
> 
> I am not sure if there's an fd.o defined standard for environment variables,
> but
> a quick web search pointed to the IEEE and Open Group link.
> http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html
> 
> None of the standards talk about where TMPDIR should be pointing to. On
> GNU/Linux, I recollect it pointing to /tmp and always wondered what
> environment
> variable would point to /var/tmp

Some more search pointed the following link that talks about /var/tmp

http://www.pathname.com/fhs/pub/fhs-2.3.html#VARTMPTEMPORARYFILESPRESERVEDBETWEE

To quote:

*
/var/tmp : Temporary files preserved between system reboots

Purpose

The /var/tmp directory is made available for programs that require temporary
files or directories that are preserved between system reboots. Therefore, data
stored in /var/tmp is more persistent than data in /tmp.

Files and directories located in /var/tmp must not be deleted when the system is
booted. Although data stored in /var/tmp is typically deleted in a 
site-specific 
manner, it is recommended that deletions occur at a less frequent interval than
/tmp.
*

So this could be the answer to DTMPDIR (Disk TMPDIR).
The only missing bit is the Environment Variable. Does anyone know of a standard
way to referencing this ?


-- 
Given the large number of mailing lists I follow, I request you to CC
me in replies for quicker response

signature.asc
Description: This is a digitally signed message part

Re: TMPDIR - Do we also need a drive backed TPMDIR ?

[Neil Williams]

On Thu, 21 Jul 2016 17:19:27 +0530
Ritesh Raj Sarraf  wrote:

> Hi,
> 
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=831998
> 
> This bug is an example of the problem. I am bringing it here to know
> views of fellow developers.
> 
> 
> I am not sure if there's an fd.o defined standard for environment
> variables, but a quick web search pointed to the IEEE and Open Group
> link.
> http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html
> 
> None of the standards talk about where TMPDIR should be pointing to.
> On GNU/Linux, I recollect it pointing to /tmp and always wondered
> what environment variable would point to /var/tmp
> 
> On the Debian front, I can't recollect what our official stand is. I
> could find a conversation at the following link. It is fairly recent
> from 2012. https://lists.debian.org/debian-devel/2012/06/msg00311.html
> 
> My /tmp is on tmpfs. I just don't recollect if it was the default or
> I enabled it explicitly. Currently, it is managed by systemd, but I
> do recollect initscripts providing similar feature.
> 
> ● tmp.mount - Temporary Directory
>    Loaded: loaded (/usr/share/systemd/tmp.mount; enabled; vendor
> preset: enabled)
>    Active: active (mounted) since Wed 2016-07-20 21:50:05 IST; 19h ago
> Where: /tmp
>  What: tmpfs
>  Docs: man:hier(7)
>    http://www.freedesktop.org/wiki/Software/systemd/APIFileSystems
>    CGroup: /system.slice/tmp.mount
> 
> Jul 20 21:50:05 learner systemd[1]: tmp.mount: Directory /tmp to
> mount over is not empty, mount
> Jul 20 21:50:05 learner systemd[1]: Mounting Temporary Directory...
> Jul 20 21:50:05 learner systemd[1]: Mounted Temporary Directory.
> 
> 
> 
> Now, back to the actual problem. For many applications, we rely on
> the TMPDIR environments. Tools like Python's modules help use these
> variables and not worry about the underneath platform.
> 
> Under Linux, with /tmp more commonly on tmpfs, how are developers
> dealing with it? tmpfs is limited and multi gigabyte operations may
> end up filling it all (as is the case in the debdelta bug report
> above).

As a drive backend, why doesn't swap work for this? There's no mention of swap 
in the original bug report.

> The whole idea of tmpfs for /tmp was performance. Email attachments
> could be temporarily opened under tmpfs
> 
> 
> But what do we do about larger operations where temporary data may
> have the possibility of being huge. I think most video/audio
> transcoding workflows may be hit by this.
> 
> Should there be another TPMDIR? Like DTMPDIR, pointing to the
> underneath disk, where size is limited by the capacity available to
> partition/disk ?
> 
> This could allow developers to choose one over the other based on
> their needs. It could also help standardize it as a fallback.
> 
> -- 
> Given the large number of mailing lists I follow, I request you to CC
> me in replies for quicker response


-- 


Neil Williams
=
http://www.linux.codehelp.co.uk/



pgpnv8xVMxVQ2.pgp
Description: OpenPGP digital signature

Re: TMPDIR - Do we also need a drive backed TPMDIR ?

[Ian Jackson]

Ritesh Raj Sarraf writes ("Re: TMPDIR - Do we also need a drive backed TPMDIR 
?"):
> Some more search pointed the following link that talks about /var/tmp
...
> So this could be the answer to DTMPDIR (Disk TMPDIR).

You are using `survives reboots' as a proxy for `on disk'; and using
`on disk' as a proxy for `has enough space for large amounts of data'.
I don't think this is a good approach.

It's true that /tmp has traditionally been smaller than /var/tmp,
partly as an accident of partition and filesystem layout.

As a practical matter, there are big performance gains to be had from
not requiring across-reboot (and, particularly, across-crash)
persistence.

Perhaps the right answer is instead that we should simply configure
more swap by default ?  (IIRC tmpfs data can be swapped.)

Ian.

Re: TMPDIR - Do we also need a drive backed TPMDIR ?

[Ritesh Raj Sarraf]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

On Thu, 2016-07-21 at 13:15 +0100, Neil Williams wrote:
> > Now, back to the actual problem. For many applications, we rely on
> > the TMPDIR environments. Tools like Python's modules help use these
> > variables and not worry about the underneath platform.
> > 
> > Under Linux, with /tmp more commonly on tmpfs, how are developers
> > dealing with it? tmpfs is limited and multi gigabyte operations may
> > end up filling it all (as is the case in the debdelta bug report
> > above).
> 
> As a drive backend, why doesn't swap work for this? There's no mention of swap
> in the original bug report.

Swap will come into effect when the kernel needs more memory.

As I understand it, TMPDIR on tmpfs is capped, based on the amount of real RAM
the machine has. And any application's view of TMPDIR is based on that capped
size.

- From the tmpfs(5) manpage, which seems to be Debian specific, it mentions that
50% of RAM will be used, by default.

On the machine where the (debdelta) bug is seen, I have enough RAM.

rrs@chutzpah:~/Community/Packaging$ free -m
  totalusedfree  shared  buff/cache   available
Mem:   738714034727  3712565677
Swap:  8579   08579


And the documented defaults for tmpfs reflect below.

rrs@chutzpah:~/Community/Packaging$ df -h /tmp/
Filesystem  Size  Used Avail Use% Mounted on
tmpfs   3.7G 0  3.7G   0% /tmp


- -- 
Ritesh Raj Sarraf | http://people.debian.org/~rrs
Debian - The Universal Operating System
-BEGIN PGP SIGNATURE-

iQIcBAEBCgAGBQJXkMTnAAoJEKY6WKPy4XVpkd0P/35T8/2Gct3cJieKrQtZY+ac
Uw4sAhv/OeUHAQ1A19SHClIZvRwCznhwEayg8+AfjRkYJjyhsAH/j9OTALWN18z1
SUzxlOpWiQd/oIu6k7scd6xLsKd/9TbuVoeWfrNu9mSciNacS7kK4oaQKGLF5+Jg
+3DG3L/lwrDDzO/kvhlQwnNkVoJGNOTz8ss5MydxS5dIjagz+vMqQFgel9vBPrfE
hVjcWZZFUPXTXPuwjNGRX/NthDfbHq6Ix9fbBTSHzNKt4aftKBRfGVc5c2y/WWKl
S3bW2zui+yo42g+5Qe+eKIxc+KMDs/IAEgLhe7F044WC7h3CP1M0sGEO5Rk7lzje
/AX8qeH3A5sS+54PS1v3hv+Q4AZna3/S5GkoFS9D7iYWPrkAzQdVPoVUYlbJn07t
nTdBnVdwwh5Y0lNCreRkQ7WVmLHnz5BA+370kgKM3hRVZotzxba/DGD+ODU8s7Iy
aL/UtWoWXo2orrVWJAtI7Lp8TV0iINmCMM9DUckF8b5+CtUEuqkQJ1zr5kR6J1WM
TG3b15NIiKmBwVKLNyASXQ61am7/vfz1O9V57MVdjkqjLhe9RL/jSBcCswmZ6cZ1
KeYwvEfrW+kvBCie9utkdfvIqfYzOd+qgcfPmh8JEaxSSV7yHe77LyfeoVvkvokC
LU/Pz2EaILrGGU33LOnt
=7EXe
-END PGP SIGNATURE-

Re: TMPDIR - Do we also need a drive backed TPMDIR ?

[Dmitry Bogatov]

> > Should there be another TPMDIR? Like DTMPDIR, pointing to the
> > underneath disk, where size is limited by the capacity available to
> > partition/disk ?

Why? How would you define line between 'TMPDIR' and 'DTMPDIR'?

For example, I, as happy owner of 8GB RAM, is perfectly fine with 3GB
in /tmp, because I actually use less then 1Gb. On other hand, would I
start 50 instances of Firefox, Gimp and other stuff, I would object
putting 3GB in my /tmp, since it would make system swap.

In summary, only I, as user, can decide whether is fine to put several
gigabytes in tmpfs, and it depends not only on RAM size. And I have such
control by 'export TMPDIR=$HOME/.tmp'.

So regarding problem about debdelta, I think the right solution would
be to put note in manpage, like 'debdelta need several gigabytes in
TMPDIR for proper operation. Adjust TMPDIR accordingly.' and to *not*
complicate programs.

-- 
Accept: text/plain, text/x-diff
Accept-Language: eo,en,ru
X-Web-Site: sinsekvu.github.io

Re: TMPDIR - Do we also need a drive backed TPMDIR ?

[Vincent Lefevre]

On 2016-07-21 18:19:43 +0530, Ritesh Raj Sarraf wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA512
> 
> On Thu, 2016-07-21 at 13:15 +0100, Neil Williams wrote:
> > > Now, back to the actual problem. For many applications, we rely on
> > > the TMPDIR environments. Tools like Python's modules help use these
> > > variables and not worry about the underneath platform.
> > > 
> > > Under Linux, with /tmp more commonly on tmpfs, how are developers
> > > dealing with it? tmpfs is limited and multi gigabyte operations may
> > > end up filling it all (as is the case in the debdelta bug report
> > > above).
> > 
> > As a drive backend, why doesn't swap work for this? There's no
> > mention of swap in the original bug report.
> 
> Swap will come into effect when the kernel needs more memory.

Anyway, even if it had worked, swap should not be a solution for the
limited /tmp size. The reason is that enabling swap can make the whole
system freeze for several dozens of minutes when a program misbehaves.
A misbehaving program should crash, not freeze the whole system.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: TMPDIR - Do we also need a drive backed TPMDIR ?

[Philip Hands]

Ritesh Raj Sarraf  writes:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA512
>
> On Thu, 2016-07-21 at 13:15 +0100, Neil Williams wrote:
>> > Now, back to the actual problem. For many applications, we rely on
>> > the TMPDIR environments. Tools like Python's modules help use these
>> > variables and not worry about the underneath platform.
>> > 
>> > Under Linux, with /tmp more commonly on tmpfs, how are developers
>> > dealing with it? tmpfs is limited and multi gigabyte operations may
>> > end up filling it all (as is the case in the debdelta bug report
>> > above).
>> 
>> As a drive backend, why doesn't swap work for this? There's no mention of 
>> swap
>> in the original bug report.
>
> Swap will come into effect when the kernel needs more memory.
>
> As I understand it, TMPDIR on tmpfs is capped, based on the amount of real RAM
> the machine has. And any application's view of TMPDIR is based on that capped
> size.

It's configurable.

See TMPFS_SIZE in /etc/defaults/tmpfs

And it'll cheerfully use swap, so just make sure you have enough swap
for your use case, and set the limit to suit your needs.

Alternatively, you can always create a new temporary location where you
have enough space (i.e. ~/.tmp/ ) and then either set TMPDIR to that in
the relevant user's shell profile/rc or define an alias such as:

  alias debdiff="TMPDIR=~/.debdiff-scratch-space debdiff" 

Cheers, Phil.
-- 
|)|  Philip Hands  [+44 (0)20 8530 9560]  HANDS.COM Ltd.
|-|  http://www.hands.com/http://ftp.uk.debian.org/
|(|  Hugo-Klemm-Strasse 34,   21075 Hamburg,GERMANY


signature.asc
Description: PGP signature

Re: TMPDIR - Do we also need a drive backed TPMDIR ?

[Vincent Lefevre]

On 2016-07-21 15:24:10 +0200, Philip Hands wrote:
> It's configurable.
> 
> See TMPFS_SIZE in /etc/defaults/tmpfs

If you mean /etc/default/tmpfs, it doesn't work with systemd
(as documented), and there's the global problem with swap.

> And it'll cheerfully use swap, so just make sure you have enough swap
> for your use case, and set the limit to suit your needs.
> 
> Alternatively, you can always create a new temporary location where you
> have enough space (i.e. ~/.tmp/ ) and then either set TMPDIR to that in
> the relevant user's shell profile/rc or define an alias such as:
> 
>   alias debdiff="TMPDIR=~/.debdiff-scratch-space debdiff" 

That's just a workaround. If some tool uses TMPDIR, it should work
in the first place.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: TMPDIR - Do we also need a drive backed TPMDIR ?

[Ritesh Raj Sarraf]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

On Thu, 2016-07-21 at 13:38 +0100, Ian Jackson wrote:
> You are using `survives reboots' as a proxy for `on disk'; and using
> `on disk' as a proxy for `has enough space for large amounts of data'.

Yes. :-)

> I don't think this is a good approach.
> 
> It's true that /tmp has traditionally been smaller than /var/tmp,
> partly as an accident of partition and filesystem layout.
> 
> As a practical matter, there are big performance gains to be had from
> not requiring across-reboot (and, particularly, across-crash)
> persistence.
> 
> Perhaps the right answer is instead that we should simply configure
> more swap by default ?  (IIRC tmpfs data can be swapped.)

Yes. But that's not the default in our setups. Perhaps next step is to file a
wishlist bug report.


- -- 
Ritesh Raj Sarraf | http://people.debian.org/~rrs
Debian - The Universal Operating System
-BEGIN PGP SIGNATURE-

iQIcBAEBCgAGBQJXkOYrAAoJEKY6WKPy4XVpaiEQAK+3+s4hV/GGvpsZrFo0ePii
PDQ6uFLEGEjfqgE3Y6Syw6gcrt/gmuC9DSVZ1kG0hXbPU9/9fhtQFZnM8tcu/Pfo
SVIODMxaZ9omDBYYl3XHjevM4nkJTPniRs2Mo6frsxE4n6bD1sjb+zMhIT4NFnzM
FzLXi8D4xyiTNhqcESrLz2+GyaOC6gEoHc8xZgaC4+REif66AlCoCzWwRuZzki0S
l5kMP5hjcZm0rTylqAHB1oAyw8Xd4snFYZm2U5qA+THSgI/mDQYCl4a8IlMj+5cn
qB+u43nGjYjeoDsXDkkVVNfBLsm/XFSE48D8OWDfqSZOsGZirJuLk7SYtcgdSXPO
0Iktf25ezLbnhzind1slONgy80Qge5kMDnOJYVT0Cssa4Yfr2ufuA/8pDignxY+L
eJnKqzGMHDnf/Q1I6gt6yj1wNa36gPMYe46EZFoCktQxW78Zpw0bUEtZYJ4xUQjd
Tq/n5ThTtEHV4x6ji8f3UoeSf8ycyBAVz3J5CXqKGoak6+kocCYIogVvILsOFuCf
V14FEuPnr69koT2eGW6R6R02dAU/wrKUcKhljTEabd0/g5Qx/IdhtkEzGLolvWRK
XhZ37jCuQNtKGlcKSwKdRsUjlGnfxHisoUgUhy57gL57DQeRcnH/3hOoQbybtXU9
siLMDcSuAGRqw1UNDD7W
=wbOb
-END PGP SIGNATURE-

Bug#832033: ITP: python-cotyledon -- framework for defining long-running services

[Thomas Goirand]

Package: wnpp
Severity: wishlist
Owner: Thomas Goirand 

* Package name: python-cotyledon
  Version : 1.2.6
  Upstream Author : Mehdi Abaakouk 
* URL : https://github.com/sileht/cotyledon
* License : Apache-2.0
  Programming Lang: Python
  Description : framework for defining long-running services

 Cotyledon provides a framework for defining long-running services. It provides
 handling of Unix signals, spawning of workers, supervision of children
 processes, daemon reloading, sd-notify, rate limiting for worker spawning, and
 more.

Note: This is a new dependency for OpenStack Ceilometer and Aodh.

Re: TMPDIR - Do we also need a drive backed TPMDIR ?

[Ritesh Raj Sarraf]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Hi Dmitry,

On Thu, 2016-07-21 at 16:06 +0300, Dmitry Bogatov wrote:
> 
> > > Should there be another TPMDIR? Like DTMPDIR, pointing to the
> > > underneath disk, where size is limited by the capacity available to
> > > partition/disk ?
> 
> Why? How would you define line between 'TMPDIR' and 'DTMPDIR'?
> 

I am exploring if we can/should have another one. I still haven't found any
environment variable to reference /var/tmp/

> For example, I, as happy owner of 8GB RAM, is perfectly fine with 3GB
> in /tmp, because I actually use less then 1Gb. On other hand, would I
> start 50 instances of Firefox, Gimp and other stuff, I would object
> putting 3GB in my /tmp, since it would make system swap.
> 

What about machines where RAM is much lesser ?

> In summary, only I, as user, can decide whether is fine to put several
> gigabytes in tmpfs, and it depends not only on RAM size. And I have such
> control by 'export TMPDIR=$HOME/.tmp'.
> 
> So regarding problem about debdelta, I think the right solution would
> be to put note in manpage, like 'debdelta need several gigabytes in
> TMPDIR for proper operation. Adjust TMPDIR accordingly.' and to *not*
> complicate programs.

I don't think this is a debdelta problem. If the defaults is to have TMPDIR as a
tmpfs, and if the defaults are to bind 50% of RAM for TMPDIR, then there can be
multiple apps breaking. It just depends on how much RAM your machine has.

I think a more sensible approach would be to allocate 50% of Virtual Memory for
TMPDIR, as the defaults.


- -- 
Ritesh Raj Sarraf | http://people.debian.org/~rrs
Debian - The Universal Operating System
-BEGIN PGP SIGNATURE-

iQIcBAEBCgAGBQJXkOgvAAoJEKY6WKPy4XVpBwoP/0HA1rOwhdrzngR8S04DijSi
mFCiMhHuXF6DlK5eb9w7oloVoQ2yJ74puxi4f0jZLDYyvX5fY6rMUy02lL9btUuY
FZsHl4GfQawjPLuolMbcTsYiPbJbe0hyu+zuUx8Rs+SYefQPbkcVABhgK0aOIQcC
RZguye/6Iq1QW8uRD/0kPBCLM3k8av0Pp5X3OsF9tdpu0ZGxNrmBrt22aC8Wgpik
wML1IQwPbgNsO7Eq7Eh7hVBCyMu3JU2P2iOWWRKKk5jLt9TBvEZHIMGKhTJC0Bh3
A54eNeaOZEPjGQDanDH5eImI2NK8YNBAxy1F9TEXbHbesKzNHoAEbE8QXQEqpnGu
tCVrZ4QvQEbexrj3OxO9Ze2dTMIsASfhw9nqFBHEHHqaDE5i7MKU6/NmYFztVFJ+
6IyW92zeHEOQbNZjfP9wKiWGv3qrc6aNWyYG2orna8s5oYfrXTC/8OX+8C/3ZBZj
UOrxvSnjulDldpxiVPpdLZLuK9bf3VxuR49RDn2Zx2p2UugzF2hW71VO/yhqsUhM
E37rrUYEJazPuOwz4Fk85yhYFEBVltazGas19Mz5CZoDjt5l9AnXTWdYXpN6FnF5
dT0UBS3l5qCgM5P7x5zL66hB4GB+BwKlB/y5lzu4c+xrGIPZeEJdUVwSjoAg3LK4
I+u7zLHQjXVrqQYEV0WD
=6Qyg
-END PGP SIGNATURE-

Re: TMPDIR - Do we also need a drive backed TPMDIR ? [and 1 more messages]

[Ian Jackson]

Philip Hands writes ("Re: TMPDIR - Do we also need a drive backed TPMDIR ?"):
> It's configurable.
> See TMPFS_SIZE in /etc/defaults/tmpfs

A sensible question to ask is whether the default is right.

I recently installed stretch on a new laptop with a large SSD and 16G
of RAM.  I set it up with 27G of swap (because I hate running out of
swap).  I have a bunch of per-user /run directories at 1.6G each.

I don't seem to be using tmpfs for /tmp by default.  My /tmp seems to
be on /.  I wonder if that's just because I'm not using systemd, or
whether it's because I did the partitioning by hand.

I'm tempted to change to RAMTMP=yes in /etc/default/tmpfs.  The
default size of that tmpfs (20%VM = 8.6G) would arguably be rather
small compared to the amount of underlying storage available.

> And it'll cheerfully use swap, so just make sure you have enough swap
> for your use case, and set the limit to suit your needs.

I wonder if the performance will be acceptable if one starts to use
tmpfs for large objects which require swapping.  On spinning rust the
default IO scheduler (cfq) is very poor for interactive workloads.


Vincent Lefevre writes ("Re: TMPDIR - Do we also need a drive backed TPMDIR ?"):
> On 2016-07-21 18:19:43 +0530, Ritesh Raj Sarraf wrote:
> > Swap will come into effect when the kernel needs more memory.
> 
> Anyway, even if it had worked, swap should not be a solution for the
> limited /tmp size. The reason is that enabling swap can make the whole
> system freeze for several dozens of minutes when a program misbehaves.
> A misbehaving program should crash, not freeze the whole system.

Uhhh.  You run your systems with no swap at all ?

That's your prerogative, of course.  But it's far from a default (or
recommended) configuration.  I think that if you configure your system
without swap, it is up to you do whatever else is necessary to make it
work.  That might involve making /tmp not be on a tmpfs (if it is the
default for some reason).

Ian.

Re: TMPDIR - Do we also need a drive backed TPMDIR ?

[Ritesh Raj Sarraf]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

On Thu, 2016-07-21 at 15:24 +0200, Philip Hands wrote:
> It's configurable.
> 
> See TMPFS_SIZE in /etc/defaults/tmpfs
> 
> And it'll cheerfully use swap, so just make sure you have enough swap
> for your use case, and set the limit to suit your needs.
> 

Should this be the default ? Maybe I'll just file a bug report and see where it
goes.


> Alternatively, you can always create a new temporary location where you
> have enough space (i.e. ~/.tmp/ ) and then either set TMPDIR to that in
> the relevant user's shell profile/rc or define an alias such as:
> 
>   alias debdiff="TMPDIR=~/.debdiff-scratch-space debdiff"

- -- 
Ritesh Raj Sarraf | http://people.debian.org/~rrs
Debian - The Universal Operating System
-BEGIN PGP SIGNATURE-

iQIcBAEBCgAGBQJXkOk/AAoJEKY6WKPy4XVppfMQAKDiF84P7L211yEld58z/Mma
T8aCKoDSe6ZwbDU3Ex7y+n8rtjG0FMEVDNZUmKW/MBR9dPyyULnO2VOyitBq1n2/
wFcs4BFcyAxVquJbVFSFwGbM4LZwStd8kZlR5PoAcpTQtOA1R2b15p1ceuCYqDus
N47D9RkDWLK5ZJL5yu+4CpsMX5sYynLceLQv9WxfAdOE+3DIQ5HEsO7FeOL//TPx
WLIZIrAbfZQl0OQDpzRHvkIfefvqvVH09+GhrR5cUQWmepi7/dbNZDsIYEC5lazJ
vjKDGTs5UJDGjhcsSME4yLRABqzsA65spEe5jNUWMbIOU1BhFOS+3NCjvpGwyQ6W
x/AyErKhpWXO2Xlwp/ib9eiP5x72RkdZJE4/Pp5/ucS6nbEMAHYsJtNrGNTLaKgY
4LRmU6yUeITZlyfYlZWvf8rIt3JQ4VuQEmpKcxfeJrnq8L0Gn2ZJnS4/OxU/0Mxh
cf9wyDEK9uEaQJyO3qHEzx01fRSPtGAwMpyOKUqeN9QFCMZsQxz3aSVuB7jCmE16
iZ6YSIu89gElFlcOgEUkY5+bXiLoUd6v1Z2v8VUwdrPDDItrtX3nnx/P/nvympcO
fECqLFQyaVgX2lCVd8qRz2P0MKcJzU6+ZlroWzNf65kAvP940mHp1fUOKbFP4Fi8
nUdPfiGddXHk9Ri3xPIn
=u0Hb
-END PGP SIGNATURE-

Re: TMPDIR - Do we also need a drive backed TPMDIR ?

[Ritesh Raj Sarraf]

On Thu, 2016-07-21 at 15:42 +0200, Vincent Lefevre wrote:
> On 2016-07-21 15:24:10 +0200, Philip Hands wrote:
> > It's configurable.
> > 
> > See TMPFS_SIZE in /etc/defaults/tmpfs
> 
> If you mean /etc/default/tmpfs, it doesn't work with systemd
> (as documented), and there's the global problem with swap.

And I was thinking of asking the same to the systemd maintainers. So thanks for
mentioning this. Though I wish systemd honored values from these files.


-- 
Given the large number of mailing lists I follow, I request you to CC
me in replies for quicker response

signature.asc
Description: This is a digitally signed message part

Re: Running without swap messages]

[Jonathan McDowell]

On Thu, Jul 21, 2016 at 04:24:36PM +0100, Ian Jackson wrote:
> Uhhh.  You run your systems with no swap at all ?
> 
> That's your prerogative, of course.  But it's far from a default (or
> recommended) configuration.  I think that if you configure your system
> without swap, it is up to you do whatever else is necessary to make it
> work.  That might involve making /tmp not be on a tmpfs (if it is the
> default for some reason).

I run my laptop without swap and with /tmp on tmpfs; I have 16G RAM. I
have not noticed problems. I do this because I'm using a single SSD and
I'd rather not end up hammering the SSD if something is doing insane
things with memory. I'd assumed this was a reasonable configuration
these days - am I misinformed?

J.

-- 
101 things you can't have too much of : 3 - Sleep.

Re: TMPDIR - Do we also need a drive backed TPMDIR ?

[Ritesh Raj Sarraf]

On Thu, 2016-07-21 at 21:22 +0530, Ritesh Raj Sarraf wrote:
> On Thu, 2016-07-21 at 15:42 +0200, Vincent Lefevre wrote:
> > On 2016-07-21 15:24:10 +0200, Philip Hands wrote:
> > > It's configurable.
> > > 
> > > See TMPFS_SIZE in /etc/defaults/tmpfs
> > 
> > If you mean /etc/default/tmpfs, it doesn't work with systemd
> > (as documented), and there's the global problem with swap.
> 
> And I was thinking of asking the same to the systemd maintainers. So thanks
> for
> mentioning this. Though I wish systemd honored values from these files.

So, as I understand it, systemd inherits the size value from, either /etc/fstab
or the kernel defaults (which is 50% of RAM).

And given that /tmp in Debian doesn't have a default listing in /etc/fstab, the
defaults would be picked up from the kernel defaults. Which would still be
capable of making many applications break, on a machine with lesser amount of
RAM.

BTW, is there an equivalent of, whats documented in /etc/default/tmpfs about:
TMPFS_SIZE=80%VM, in systemd ? I looked into tmpfs mount manpage section and it
does not talk about VM.

@Vincent: I agree with the pain about swap that Linux brings in. The assumption
everyones making is that faster block devices (SSDs mostly) will mitigate the
problem. And to a large extent it seems to have done that.
But I still have read [1] users report the bug existent, even on Android/Linux,
where I assume they are more commonly using Flash storage. 
Which confirms the arguments that slow rotational media is just part of the
problem (the bug provoker, as I like to put), the real problem lies in how I/O
scheduling works in Linux.

[1] https://bugzilla.kernel.org/show_bug.cgi?id=12309#c639



-- 
Given the large number of mailing lists I follow, I request you to CC
me in replies for quicker response

signature.asc
Description: This is a digitally signed message part

Re: Running without swap messages]

[Ritesh Raj Sarraf]

On Thu, 2016-07-21 at 16:58 +0100, Jonathan McDowell wrote:
> 
> I run my laptop without swap and with /tmp on tmpfs; I have 16G RAM. I
> have not noticed problems. I do this because I'm using a single SSD and
> I'd rather not end up hammering the SSD if something is doing insane
> things with memory. I'd assumed this was a reasonable configuration
> these days - am I misinformed?

For the hardware part, this would be my dream configuration. And I guess the
same for many more in the Indian sub-continent.

-- 
Given the large number of mailing lists I follow, I request you to CC
me in replies for quicker response

signature.asc
Description: This is a digitally signed message part

Bug#832042: ITP: ignition-msgs -- Standard set of message definitions, used by Ignition Transport, and other applications

[Jose Luis Rivero]

Package: wnpp
Severity: wishlist
Owner: Jose Luis Rivero 

* Package name: ignition-msgs
  Version : 0.1.0
  Upstream Author : Open Source Robotics Foundation
* URL : http://ignitionrobotics.org/libraries/common
* License : Apache
* Programming Lang: C++
  Description : Standard set of message definitions, used by Ignition 
Transport, and other applications

Ignition msgs is a component in the Ignition framework, a set of
librariesdesigned to rapidly develop robot applications. The library
defines common protobuf messages used by the robotics community.

Future versions of ignition-transport (starting with 2.x series) 
will use this library so we will need it as dependecy.

Re: TMPDIR - Do we also need a drive backed TPMDIR ?

[Simon McVittie]

On Thu, 21 Jul 2016 at 20:41:38 +0530, Ritesh Raj Sarraf wrote:
> On Thu, 2016-07-21 at 13:38 +0100, Ian Jackson wrote:
> > Perhaps the right answer is instead that we should simply configure
> > more swap by default ?  (IIRC tmpfs data can be swapped.)
> 
> Yes. But that's not the default in our setups.

To be fair, neither is a tmpfs /tmp, and the Debian systemd maintainers
have gone to some length to keep /tmp on-disk by default (#783509).
If a sysadmin does one non-default thing, it seems reasonable to require
them to do another non-default thing to make it work optimally.

S

Re: TMPDIR - Do we also need a drive backed TPMDIR ? [and 1 more messages]

[Simon McVittie]

On Thu, 21 Jul 2016 at 16:24:36 +0100, Ian Jackson wrote:
> I don't seem to be using tmpfs for /tmp by default.  My /tmp seems to
> be on /.  I wonder if that's just because I'm not using systemd, or
> whether it's because I did the partitioning by hand.

tl;dr: the Debian default is the same as you have, even under systemd.

Partitioning is only relevant if you allocate a /tmp partition and
configure it in fstab, in which case systemd (and hopefully all other
inits) will respect that.

Debian's systemd defaults to leaving /tmp part of the root filesystem,
but with a one-time migration to tmpfs /tmp if previously configured
by enabling tmp.mount or setting RAMTMP=yes. This is a Debian-specific
change, and could be reverted if there's ever consensus that it should be.

Upstream systemd uses tmpfs /tmp by default. This can be prevented
by masking tmp.mount, or reconfigured by mentioning /tmp (as
tmpfs, a disk partition or even a bind mount) in fstab, as part of the
general rule that /etc takes precedence over /[usr/]lib.

S

Re: TMPDIR - Do we also need a drive backed TPMDIR ? [and 1 more messages]

[Russ Allbery]

Ian Jackson  writes:

> Uhhh.  You run your systems with no swap at all ?

> That's your prerogative, of course.  But it's far from a default (or
> recommended) configuration.  I think that if you configure your system
> without swap, it is up to you do whatever else is necessary to make it
> work.  That might involve making /tmp not be on a tmpfs (if it is the
> default for some reason).

I never use swap on servers any more.  My experience is that swap is so
slow that if the system starts using swap, the services are all
essentially down anyway, and they're down in pretty much the worst case
scenario for clients (extremely slow and unreliable returns with no clean
"server is down" signal).  It's generally better for the system to panic
or start killing random processes than for it to go into swap.

This depends heavily on what type of server you have and what your
situation is, of course.  But swapless configurations are pretty common
these days in large-scale server farms that already have to tolerate any
one server going down.  It's a lot easier to tolerate a server just going
away than a server that goes into pathological performance from using
swap.

Desktops and laptops are obviously a different issue with different
tradeoffs.

-- 
Russ Allbery (r...@debian.org)   

Re: TMPDIR - Do we also need a drive backed TPMDIR ?

[Dmitry Bogatov]

[2016-07-21 21:22] Gabor Gombas 
>
> On Thu, Jul 21, 2016 at 04:06:26PM +0300, Dmitry Bogatov wrote:
>
> > For example, I, as happy owner of 8GB RAM, is perfectly fine with 3GB
> > in /tmp, because I actually use less then 1Gb. On other hand, would I
> > start 50 instances of Firefox, Gimp and other stuff, I would object
> > putting 3GB in my /tmp, since it would make system swap.
>
> I don't quite buy your argument. If you want to write 3GB to disk, then
> you need to write 3GB to disk - it's the same amount of data, whether
> you call it "filesystem" or "swap".

Let me explain. If I know that program need for it's operation
temporary 3GB of data, and I and only I can decide whether it is
apporiate to put it in tmpfs. As such, I find any attempt to guess on
the side of program vanity. Program must do it's job, not try to guess
my wishes.

-- 
Accept: text/plain, text/x-diff
Accept-Language: eo,en,ru
X-Web-Site: sinsekvu.github.io

Bug#832054: ITP: libimporter-perl -- alternative but compatible interface to modules that export symbols

[gregor herrmann]

Package: wnpp
Owner: gregor herrmann 
Severity: wishlist
X-Debbugs-CC: debian-devel@lists.debian.org, debian-p...@lists.debian.org

* Package name: libimporter-perl
  Version : 0.014
  Upstream Author : Chad Granum 
* URL : https://metacpan.org/release/Importer
* License : Artistic or GPL-1+
  Programming Lang: Perl
  Description : alternative but compatible interface to modules that export 
symbols

Importer acts as a layer between Exporter and modules which consume exports.
It is feature-compatible with Exporter, plus some much needed extras. You can
use this to import symbols from any exporter that follows Exporters
specification. The exporter modules themselves do not need to use or inherit
from the Exporter module, they just need to set @EXPORT and/or other
variables.

The package will be maintained under the umbrella of the Debian Perl Group.


signature.asc
Description: Digital Signature

Bug#832056: ITP: libcgi-github-webhook-perl -- Easily write CGI-based GitHub webhooks in Perl

[Axel Beckert]

Package: wnpp
Owner: Axel Beckert 
Severity: wishlist

* Package name: libcgi-github-webhook-perl
  Version : 0.06
  Upstream Author : Axel Beckert 
* URL : https://metacpan.org/release/CGI-Github-Webhook
* License : Artistic or GPL-1+
  Programming Lang: Perl
  Description : Easily write CGI-based GitHub webhooks in Perl

CGI::Github::Webhook provides an easy way to write CGI-based GitHub
webhooks in Perl. In most cases three statements already
suffice: Configure a script to trigger, a shared secret and a log
file and you're ready to go.

Re: TMPDIR - Do we also need a drive backed TPMDIR ?

[Gabor Gombas]

On Thu, Jul 21, 2016 at 04:06:26PM +0300, Dmitry Bogatov wrote:

> For example, I, as happy owner of 8GB RAM, is perfectly fine with 3GB
> in /tmp, because I actually use less then 1Gb. On other hand, would I
> start 50 instances of Firefox, Gimp and other stuff, I would object
> putting 3GB in my /tmp, since it would make system swap.

I don't quite buy your argument. If you want to write 3GB to disk, then
you need to write 3GB to disk - it's the same amount of data, whether
you call it "filesystem" or "swap". But if you use a real filesystem to
store that data, then you will also have to deal metadata, which needs
to be protected from crashes so you want to flush caches from time to
time, which is expensive. tmpfs does not have such requirements, so it
should give you better performance. I guess someone should measure it...

Gabor

Re: TMPDIR - Do we also need a drive backed TPMDIR ? [and 1 more messages]

[Ben Hutchings]

On Thu, 2016-07-21 at 11:40 -0700, Russ Allbery wrote:
> Ian Jackson  writes:
> 
> > Uhhh.  You run your systems with no swap at all ?
> 
> > That's your prerogative, of course.  But it's far from a default (or
> > recommended) configuration.  I think that if you configure your system
> > without swap, it is up to you do whatever else is necessary to make it
> > work.  That might involve making /tmp not be on a tmpfs (if it is the
> > default for some reason).
> 
> I never use swap on servers any more.  My experience is that swap is so
> slow that if the system starts using swap, the services are all
> essentially down anyway,

It's true that if the system has to swap a lot of data out then it is
likely to become unresponsive.  However, I think there are good reasons
to enable a small amount of swap space:

- Some long-running applications have, effectively, memory leaks of a
bounded size that can profitably be swapped out
- The exact same problem can occur with memory-mapped files, and by
disabling swap you prevent the kernel from balancing demand for
anonymous and file-backed memory

Linux used to have the problem that allocating multi-page blocks
(commonly needed on architectures with 4K pages) could require
reclaiming single pages in the way.  This is much less of an issue now
that the VM manager implements compaction (since 2.6.35).

In some kernels and configurations, forking a large process requires
reserving memory for another copy of all its anonymous pages, so it's
more likely to fail if there's no swap available.  I don't think this
is an issue with the default overcommit setting in Linux, though.

[...]
> It's a lot easier to tolerate a server just going
> away than a server that goes into pathological performance from using
> swap.

Agreed.

Ben.

> Desktops and laptops are obviously a different issue with different
> tradeoffs.
> 
-- 

Ben Hutchings
Absolutum obsoletum. (If it works, it's out of date.) - Stafford Beer


signature.asc
Description: This is a digitally signed message part

Re: TMPDIR - Do we also need a drive backed TPMDIR ?

[Ben Hutchings]

On Thu, 2016-07-21 at 21:22 +0200, Gabor Gombas wrote:
> On Thu, Jul 21, 2016 at 04:06:26PM +0300, Dmitry Bogatov wrote:
> 
> > For example, I, as happy owner of 8GB RAM, is perfectly fine with 3GB
> > in /tmp, because I actually use less then 1Gb. On other hand, would I
> > start 50 instances of Firefox, Gimp and other stuff, I would object
> > putting 3GB in my /tmp, since it would make system swap.
> 
> I don't quite buy your argument. If you want to write 3GB to disk, then
> you need to write 3GB to disk - it's the same amount of data, whether
> you call it "filesystem" or "swap". But if you use a real filesystem to
> store that data, then you will also have to deal metadata, which needs
> to be protected from crashes so you want to flush caches from time to
> time, which is expensive. tmpfs does not have such requirements, so it
> should give you better performance. I guess someone should measure it...

I'm not sure the performance comparison is quite as simple as this.
Commonly used filesystems try to avoid fragmentation of files, but
AFAIK there's nothing to prevent fragmentation of tmpfs files in swap.
So reading a file back from swap can involve more seeks than doing the
same from a filesystem.

Ben.

-- 

Ben Hutchings
Absolutum obsoletum. (If it works, it's out of date.) - Stafford Beer


signature.asc
Description: This is a digitally signed message part

Work-needing packages report for Jul 22, 2016

[wnpp]

The following is a listing of packages for which help has been requested
through the WNPP (Work-Needing and Prospective Packages) system in the
last week.

Total number of orphaned packages: 800 (new: 43)
Total number of packages offered up for adoption: 171 (new: 2)
Total number of packages requested help for: 48 (new: 1)

Please refer to http://www.debian.org/devel/wnpp/ for more information.



The following packages have been orphaned:

   apoo (#831680), orphaned 3 days ago
 Description: An Assembly course aid
 Installations reported by Popcon: 31

   aptsh (#831493), orphaned 5 days ago
 Description: apt interactive shell
 Installations reported by Popcon: 195

   audiolink (#831544), orphaned 4 days ago
 Description: makes managing and searching for music easier
 Installations reported by Popcon: 25

   balder2d (#831545), orphaned 4 days ago
 Description: A 2D shooter in zero gravity
 Reverse Depends: balder2d
 Installations reported by Popcon: 165

   bootchart (#831555), orphaned 4 days ago
 Description: Boot process performance analyser
 Installations reported by Popcon: 128

   cappuccino (#831558), orphaned 4 days ago
 Description: utility to let your boss think that you're working hard
 Installations reported by Popcon: 133

   cgvg (#831700), orphaned 3 days ago
 Description: command-line source browsing tool
 Installations reported by Popcon: 60

   childsplay (#831701), orphaned 3 days ago
 Description: Suite of educational games for young children
 Reverse Depends: childsplay-alphabet-sounds-bg
   childsplay-alphabet-sounds-ca childsplay-alphabet-sounds-de
   childsplay-alphabet-sounds-el childsplay-alphabet-sounds-en-gb
   childsplay-alphabet-sounds-es childsplay-alphabet-sounds-fr
   childsplay-alphabet-sounds-it childsplay-alphabet-sounds-nb
   childsplay-alphabet-sounds-nl (5 more omitted)
 Installations reported by Popcon: 809

   childsplay-alphabet-sounds-bg (#831702), orphaned 3 days ago
 Description: Bulgarian sound files for childsplay
 Installations reported by Popcon: 76

   childsplay-alphabet-sounds-ca (#831703), orphaned 3 days ago
 Description: Catalan sound files for childsplay
 Installations reported by Popcon: 436

   childsplay-alphabet-sounds-de (#831704), orphaned 3 days ago
 Description: German sound files for childsplay
 Installations reported by Popcon: 44

   childsplay-alphabet-sounds-el (#831705), orphaned 3 days ago
 Description: Greek sound files for childsplay
 Installations reported by Popcon: 17

   childsplay-alphabet-sounds-en-gb (#831706), orphaned 3 days ago
 Description: British sound files for childsplay
 Installations reported by Popcon: 85

   childsplay-alphabet-sounds-es (#831707), orphaned 3 days ago
 Description: Spanish sound files for childsplay
 Installations reported by Popcon: 51

   childsplay-alphabet-sounds-fr (#831708), orphaned 3 days ago
 Description: French sound files for childsplay
 Installations reported by Popcon: 65

   childsplay-alphabet-sounds-it (#831709), orphaned 3 days ago
 Description: Italian sound files for childsplay
 Installations reported by Popcon: 19

   childsplay-alphabet-sounds-nb (#831710), orphaned 3 days ago
 Description: Bokmål sound files for childsplay
 Installations reported by Popcon: 16

   childsplay-alphabet-sounds-nl (#831711), orphaned 3 days ago
 Description: Dutch sound files for childsplay
 Installations reported by Popcon: 13

   childsplay-alphabet-sounds-pt (#831712), orphaned 3 days ago
 Description: Portugese sound files for childsplay
 Installations reported by Popcon: 16

   childsplay-alphabet-sounds-ro (#831713), orphaned 3 days ago
 Description: Romanian sound files for childsplay
 Installations reported by Popcon: 12

   childsplay-alphabet-sounds-ru (#831714), orphaned 3 days ago
 Description: Russian sound files for childsplay
 Installations reported by Popcon: 16

   childsplay-alphabet-sounds-sl (#831715), orphaned 3 days ago
 Description: Slovenian sound files for childsplay
 Installations reported by Popcon: 125

   childsplay-alphabet-sounds-sv (#831716), orphaned 3 days ago
 Description: Swedish sound files for childsplay
 Installations reported by Popcon: 12

   cvs-autoreleasedeb (#831624), orphaned 4 days ago
 Description: Automatically release/upload debian packages from CVS
 Installations reported by Popcon: 38

   cvschangelogbuilder (#831679), orphaned 3 days ago
 Description: perl utility to generate ChangeLogs or html cvs reports
 Installations reported by Popcon: 8

   doc-debian-fr (#831625), orphaned 4 days ago
 Description: Debian Manuals, FAQs and other documents in French
 Installations reported by Popcon: 18601

   filetraq (#831717), orphaned 3 days ago
  

Re: TMPDIR - Do we also need a drive backed TPMDIR ? [and 1 more messages]

[Vincent Lefevre]

> Vincent Lefevre writes ("Re: TMPDIR - Do we also need a drive backed TPMDIR 
> ?"):
> > On 2016-07-21 18:19:43 +0530, Ritesh Raj Sarraf wrote:
> > > Swap will come into effect when the kernel needs more memory.
> > 
> > Anyway, even if it had worked, swap should not be a solution for the
> > limited /tmp size. The reason is that enabling swap can make the whole
> > system freeze for several dozens of minutes when a program misbehaves.
> > A misbehaving program should crash, not freeze the whole system.
> 
> Uhhh.  You run your systems with no swap at all ?

What's the problem if swap is *never* used under normal conditions?
On current machines, I typically use less than 10% the amount of
physical RAM.

On an older laptop, I had 4 GB RAM, and under normal conditions,
I was using only around 1 GB. Swap was used only in two cases:

1. Because Firefox was taking more than 4 GB after some time, but
in this case, the system was more or less frozen. So, the only thing
I could do was to kill Firefox from a terminal, which was taking me
several minutes... I ended up by using a wrapper script that did:

  ulimit -v 4096000
  exec /usr/bin/firefox "$@"

so that Firefox would crash instead of freezing the system. Now, it
seems that Firefox has improved and no longer take so much memory.

2. Due to bugs in my programs, or something similar. Ditto, I would
have preferred a crash than a frozen system.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: TMPDIR - Do we also need a drive backed TPMDIR ? [and 1 more messages]

[Vincent Lefevre]

On 2016-07-21 21:53:11 +0100, Ben Hutchings wrote:
> It's true that if the system has to swap a lot of data out then it is
> likely to become unresponsive.  However, I think there are good reasons
> to enable a small amount of swap space:
> 
> - Some long-running applications have, effectively, memory leaks of a
> bounded size that can profitably be swapped out
> - The exact same problem can occur with memory-mapped files, and by
> disabling swap you prevent the kernel from balancing demand for
> anonymous and file-backed memory

Unfortunately, AFAIK, one cannot tell the kernel to use swap only
in these cases.

I'd prefer to buy "a small amount" of additional RAM that would
replace swap in these cases, and avoid a system freeze if something
goes wrong.

Swap could actually be interesting if the swap/RAM ratio were
important, say 10 or more, because in such a case one couldn't
avoid swap by buying more RAM (too expensive or not possible).
But this means that one should have control of how swap is used.
To give an example, I think that /tmp = 4 GB tmpfs + 80 GB swap
could be a better solution than /tmp being a part of the root
filesystem. Something like that.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Summary of the ARM ports BoF at DC16

[Steve McIntyre]

[ Please note the cross-post and Reply-To ]

Hi folks,

As promised, here's a quick summary of what was discussed at the ARM
ports BoF session in Cape Town.

This year, unfortunately the session did not have video so I can't
link to it. I've taken a copy of the Gobby notes from the session,
alongside my small set of slides for the session (that also didn't get
shown!). [1]

We spoke about the past/present/future state of ARM ports in Debian.

armel
=

armel is the longest-running of our current ARM ports in Debian,
starting in 2007. It's starting to become difficult to support. Debian
is the only common distro still supporting ARMv4t. While there is
still good upstream support in most major packages (e.g. Linux and
gcc), we're seeing a growing number of issues. Some packages
(e.g. NodeJS) don't support pre-ARMv7 at all any more. Another problem
is the lack of direct support for C++11 atomics - it's possible to use
a kernel helper to cover for this lack, but performance will be
terrible.

Possible future options for armel:

 * Partial armel architecture?

   We've talked about this in the past for various architectures, but
   nobody has stepped up to work on it. The vast majority of the
   outstanding armel use cases are going to be in headless servers so
   we could maybe drop the desktop tasks and dependencies and keep
   things like web server / mail server tasks that are much simpler.

   Downside: There's no clear plan for how to create/maintain a
   partial architecture, let alone how to release one. Potentially
   huge amount of work needed.

 * Update to ARMv5t (and maybe go headless)

   We might be able to extend the life of armel by upping the minimum
   spec, and would be able to continue supporting Kirkwood and Orion
   based machines (e.g. NAS boxes) for a while longer. This would kill
   support for v4t devices like Openmoko, but there are precious few
   of these older devices still around.

   Downside: as above if we try to do the partial architecture route;
   if we don't we'll still have to support a full range of software on
   older hardware.

   Will need volunteers to make this work in either form, and while
   some people are prepared to *help* (e.g. bdale and zumbi), nobody
   stepped up in the BoF to lead such an effort. It needs both skill
   and commitment of time.

 * Release armel with stretch, then drop it

   **This is the favoured option.** That would give EOL for armel in
   2020, *potentially* later if there's an LTS release for stretch and
   armel is included (as it is in wheezy). That would depend on
   external labour or funding.

   It's not easy (or popular) to remove an architecture from Debian,
   but we're giving 4 years' notice of end of support.

armhf
=

Much as in the Heidelberg BoF in terms of support and future.

Current port, first released with Wheezy. Due to cross-distro effort,
this setup (ARMv7 EABI using VFPv3D-16) is the default supported
32-bit ARM architecture in all distros now. We've got a couple of
kernel variants that will support just about any new devices shipping,
given updated drivers and device tree support.

Despite the ubiquity of v7 hardware that works, some people are still
making CPUs that *don't* work, e.g. new CPUs this year without any
hardware FPU. Not much we can or will do to support such broken
hardware.

Ignoring known-broken hardware stuff, almost any currently-shipping
ARMv7 device is likely to work with armhf. Yay!

There's been a recent effort to add an EFI shim layer into modern
U-Boot, which may allow for easier consistent boot support but there's
still issues to think about in terms of EFI variables etc.

arm64
=

Most recent ARM port. All looking good now - we've been mostly able to
move on from Juno development platforms to real server hardware
now. We're using some APM Mustang machines and an AMD Seattle box
hosted by ARM and Linaro at the moment, and even real arm64 server
machines are finally coming available. Looking to move more of our ARM
buildds over onto these arm64 servers in the future, as this will
improve reliability and manageability.

We're using a single kernel for arm64, using DTB or ACPI for
configuration. Works well.

Affordable, usable machines are available now, e.g. the Cello or
SoftIron 1000 for ~$600. Linaro's 96boards machines are not using a
standard form factor which is sub-optimal. There's a Marvell arm64
board due soon (H2 2016) in ATX. The SoftIron 1000 doesn't have PCI,
but should be good for development/buildd otherwise. Another cheap
option is the Pine64. It's stuck on a vendor kernel for now, but
that's being steadily worked on. Cheap: quad A53, 2GB RAM, $29.

Other issues:

 * ci.debian.net could do with more arm64 hardware, we'll see what we
   can do to help
 * UEFI Secure Boot for arm64 has an issue - there's nobody providing
   a root CA. HP are doing their own for now; not sure what other
   vendors can/will do. It's complicated (TM).

arm64 boards - http://www.96boa

ARM64 (was: Summary of the ARM ports BoF at DC16)

[Jeffrey Walton]

> arm64
> =
>
> Most recent ARM port. All looking good now - we've been mostly able to
> move on from Juno development platforms to real server hardware
> now. We're using some APM Mustang machines and an AMD Seattle box
> hosted by ARM and Linaro at the moment, and even real arm64 server
> machines are finally coming available. Looking to move more of our ARM
> buildds over onto these arm64 servers in the future, as this will
> improve reliability and manageability.
>
> We're using a single kernel for arm64, using DTB or ACPI for
> configuration. Works well.
>
> Affordable, usable machines are available now, e.g. the Cello or
> SoftIron 1000 for ~$600. Linaro's 96boards machines are not using a
> standard form factor which is sub-optimal. There's a Marvell arm64
> board due soon (H2 2016) in ATX. The SoftIron 1000 doesn't have PCI,
> but should be good for development/buildd otherwise. Another cheap
> option is the Pine64. It's stuck on a vendor kernel for now, but
> that's being steadily worked on. Cheap: quad A53, 2GB RAM, $29.

Hats off for the ARM64 support. The best I can tell, the support is
complete - ARM64 performs just like its i386 and x86_64 older
brothers.

There are three other ARM64 gadgets worth mentioning... We purchased
them after our package maintainer, László Böszörményi, filed some bug
reports against us. We did not like playing catch-up in the QEMU
Chroot's; we wanted to get ahead of things.

* LeMaker HiKey (http://www.amazon.com/dp/B019O3QTSA), $169 USD
* SnapDragon 410c (http://www.amazon.com/dp/B01600X7IU), $75 USD
* Raspberry Pi 3 Model B (http://www.amazon.com/dp/B01CD5VC92), $38 USD

The Mustang board is a nice test platform because its an early ARMv8
board. While its ARMv8, it lacks CRC and Crypto extensions. Robert
Nelson lends us time on his Mustang, and it uncovered two bugs in our
CPU feature detection logic.

Jeff

Bug#832080: ITP: golang-github-jamesclonk-vultr --

[Nicolas Braud-Santoni]

Package: wnpp
Severity: wishlist
Owner: Nicolas Braud-Santoni 
Control: block 813502 by -1

* Package name: golang-github-jamesclonk-vultr
  Version : 1.8-1
  Upstream Author : Fabio Berchtold
* URL : https://github.com/jamesclonk/vultr
* License : Expat
  Programming Lang: Go
  Description : Client for the vultr API

  Provides both a CLI client and a Go library.

  This is a dependency of golang-github-xenolf-lego,
  itself  a dependency of caddy.

Bug#832081: ITP: golang-github-miekg-dns --

[Nicolas Braud-Santoni]

Package: wnpp
Severity: wishlist
Owner: Nicolas Braud-Santoni 
Control: block 813502 by -1

* Package name: golang-github-miekg-dns
  Version : 0.0~git20160614.0.5d001d0-1
  Upstream Author : Miek Gieben 
* URL : https://github.com/miekg/dns
* License : BSD-3
  Programming Lang: Go
  Description : DNS library in Go, forked from the standard lib. code

  This is a dependency of lego, among other software.

Bug#832082: ITP: golang-github-ovh-go-ovh --

[Nicolas Braud-Santoni]

Package: wnpp
Severity: wishlist
Owner: Nicolas Braud-Santoni 
Control: block 813502 by -1

* Package name: golang-github-ovh-go-ovh
  Version : 0.0~git20160604.0.d2b2eae-1
  Upstream Author : OVH SAS
* URL : https://github.com/ovh/go-ovh
* License : BSD-3
  Programming Lang: Go
  Description : Lightweight Go wrapper for the OVH API

  This is a dependency of lego, which is a dep. of caddy.

Bug#832084: ITP: golang-github-jawher-mow.cli --

[Nicolas Braud-Santoni]

Package: wnpp
Severity: wishlist
Owner: Nicolas Braud-Santoni 
Control: block 832080 by -1


* Package name: golang-github-jawher-mow.cli
  Version : 0.0~git20160720.0.0de8a76-1
  Upstream Author : Jawher Moussa
* URL : https://github.com/jawher/mow.cli
* License : Expat
  Programming Lang: Go
  Description : CLI arguments parsing and validation, in Go

  This is a framework for CLI applications written in Go.
  It is a dependency of vultr.

Bug#832083: ITP: golang-github-weppos-dnsimple-go --

[Nicolas Braud-Santoni]

Package: wnpp
Severity: wishlist
Owner: Nicolas Braud-Santoni 
Control: block 813502 by -1

* Package name: golang-github-weppos-dnsimple-go
  Version : 0.0~git20160204.0.65c1ca7-1
  Upstream Author : Scott Barron
* URL : https://github.com/weppos/dnsimple-go
* License : Expat
  Programming Lang: Go
  Description : Go client for the DNSimple API

  This is a dependency of lego, which is a dep of caddy.

Bug#832086: ITP: vrfydmn -- Milter for ensuring email message from matches mail from

[Scott Kitterman]

Package: wnpp
Severity: wishlist
Owner: Scott Kitterman 

* Package name: vrfydmn
  Version : 0.7.1
  Upstream Author : Christian Rößner 
* URL : https://github.com/croessner/vrfydmn
* License : GPL 3+
  Programming Lang: Python
  Description : Milter for ensuring email message from matches mail from

 This milter is used with postfix or sendmail to either reject mail from/body
 from mismatches or to fix up the body from to match mail from.  This is
 intended for applications where local constraints on domains in use are
 required for sending mail.

 This package makes use of the python-milter bindings for the Sendmail
 libmilter.  I intend to maintain this with the Python Applications Packaging
 Team (PAPT).