On Tue, 30 Jan 2018 08:25:28 +0100
Michał Górny wrote:
> W dniu wto, 30.01.2018 o godzinie 14∶21 +1300, użytkownik Kent Fredric
> napisał:
> > On Sat, 27 Jan 2018 12:41:58 +0100
> > Michał Górny wrote:
> >
> > > find -name 'foo.tar.gz'
> >
> > Other than being *worse* than the current "l
W dniu wto, 30.01.2018 o godzinie 14∶21 +1300, użytkownik Kent Fredric
napisał:
> On Sat, 27 Jan 2018 12:41:58 +0100
> Michał Górny wrote:
>
> > find -name 'foo.tar.gz'
>
> Other than being *worse* than the current "ls" situation due to the
> existence of distfiles/git3-src/ and distfiles/git-
> On Tue, 30 Jan 2018, Kent Fredric wrote:
> Please download:
>
> src_url1/filename1.tar.gz
> src_url2/filename2.tar.gz
> src_url3/filename3.tar.gz
>
> And install them to your $DISTDIR with
DISTDIR is not valid in pkg_nofetch(), though.
>
> edistadd ./filename1.tar.gz ./filename2.ta
> On Tue, 30 Jan 2018, Kent Fredric wrote:
>> How about "consecutive non-negative integer keys"?
> "Unsigned" ?
Even better.
pgp7Pa8YgSSFa.pgp
Description: PGP signature
On Tue, Jan 30, 2018 at 02:21:34PM +1300, Kent Fredric wrote:
> On Sat, 27 Jan 2018 12:41:58 +0100
> Michał Górny wrote:
>
> > find -name 'foo.tar.gz'
>
> Other than being *worse* than the current "ls" situation due to the
> existence of distfiles/git3-src/ and distfiles/git-src/
find -name '
On Sun, 28 Jan 2018 14:03:07 +0100
Ulrich Mueller wrote:
> How about "consecutive non-negative integer keys"?
"Unsigned" ?
pgpbMe9DskHZO.pgp
Description: OpenPGP digital signature
On Sat, 27 Jan 2018 13:24:44 -0500
Michael Orlitzky wrote:
> Fetch instructions for app-cat/pkg:
> *
> * Please download file1 from:
> * wherever file1 can be found
> * and move it to $DISTDIR/subdir1
> *
> * Please download file2 from
> * wherever file2 can be found
> * and move it to $D
On Sat, 27 Jan 2018 12:41:58 +0100
Michał Górny wrote:
> find -name 'foo.tar.gz'
Other than being *worse* than the current "ls" situation due to the
existence of distfiles/git3-src/ and distfiles/git-src/
pgpyYGhahM8Dq.pgp
Description: OpenPGP digital signature
W dniu pon, 29.01.2018 o godzinie 20∶00 +, użytkownik Robin H.
Johnson napisał:
> On Mon, Jan 29, 2018 at 08:37:47PM +0100, Michał Górny wrote:
> > Migrating mirrors to the hashed structure
> > -
>
> ...
> > The hard link solution allows us to save space
gt; > the ability of creating arbitrary length hashes instead of truncating
> > the standard-length hash. However, not all implementations of BLAKE2
> > support that and relying on it could reduce portability for no apparent
> > gain.
> >
> >
> > Backwards C
---
> The exact means of preserving backwards compatibility in package manager
> storage are left to the package manager authors. However, it is
> recommended that package managers continue to support the flat layout
> even if it is no longer the default. The package manager may either
On Mon, Jan 29, 2018 at 08:37:47PM +0100, Michał Górny wrote:
> Migrating mirrors to the hashed structure
> -
...
> The hard link solution allows us to save space on the master mirror.
> Additionally, if ``-H`` option is used by the mirrors it avoids
> transf
d security implications
(https://www.gentoo.org/glep/glep-0059.html)
.. [#BUG534528] Bug 534528 - distfiles should be sorted into subdirectories
of DISTDIR
(https://bugs.gentoo.org/534528)
.. [#ML1] [gentoo-dev] [pre-GLEP] Split distfile mirror directory structure
(https://archives.
> I don't really like the idea of having to reshuffle all the mirrors all
> of a sudden.
If keeping invariable directory structure is the goal (even though it
should be possible to shuffle files among the directories without having
to retransfer the whole tree with hard links) in addition to be ab
On Sun, Jan 28, 2018 at 03:01:11PM +0800, Jason Zaman wrote:
> Another thing im wondering is if we can just use the same dir layout as
> the packages themselves. that would fix texlive since it has a whole lot
> of separate packages. eg /usr/portage/distfiles/app-cat/pkg/pkg-1.0.tgz
Texlive is wors
W dniu nie, 28.01.2018 o godzinie 21∶43 +0100, użytkownik Andrew Barchuk
napisał:
> [my apologies for posting the message to a wrong thread before]
>
> Hi everyone,
>
> > three possible solutions for splitting distfiles were listed:
> >
> > a. using initial portion of filename,
> >
> > b. using
> In order to use that for distfiles mirrors, such that clients could know
> where to fetch the files from, you'd need the mirror's http server to
> redirect the request to the appropriate location (since the location
> would not be predictable from the client side).
That's not necessary: the cli
On 01/28/2018 02:00 PM, Andrew Barchuk wrote:
>> To the contrary, that would not remain balanced, because your
>> boundaries are entirely dependent on exactly what is in the tree at
>> the moment you run your script. Now the package manager has to perform
>> directory listing, sort and find the fil
On Sun, Jan 28, 2018 at 4:00 PM, Andrew Barchuk wrote:
> I don't see a reason to organize distfiles in a
> multi-level hierarchy: e.g. if the goal is to keep no more than 1000
> files in a folder than the limit of single level hierarchy is a million
> which is more than enough for foreseeable futu
> To the contrary, that would not remain balanced, because your
> boundaries are entirely dependent on exactly what is in the tree at
> the moment you run your script. Now the package manager has to perform
> directory listing, sort and find the file name that's closest, open
> that directory, find
On Sun, Jan 28, 2018 at 2:43 PM, Andrew Barchuk wrote:
> There's another option to use character ranges for each directory
> computed in a way to have the files distributed evenly. One way to do
> that is to use filename prefix of dynamic length so that each range
> holds the same number of files.
[my apologies for posting the message to a wrong thread before]
Hi everyone,
> three possible solutions for splitting distfiles were listed:
>
> a. using initial portion of filename,
>
> b. using initial portion of file hash,
>
> c. using initial portion of filename hash.
>
> The significant adva
> On Sun, 28 Jan 2018, Michał Górny wrote:
> How about this then:
> | This specification currently defines one section: ``[structure]``.
> | This section defines one or more repository structure definitions
> | using non-negative sequential integer keys. The definition with
> | the ``0`` key
W dniu nie, 28.01.2018 o godzinie 11∶22 +0100, użytkownik Ulrich Mueller
napisał:
> > > > > > On Sun, 28 Jan 2018, Michał Górny wrote:
> > > > This specification currently defines one section: ``[structure]``.
> > > > This section defines one or more repository structure definitions
> > > > using s
> On Sun, 28 Jan 2018, Michał Górny wrote:
>> > This specification currently defines one section: ``[structure]``.
>> > This section defines one or more repository structure definitions
>> > using sequential integer keys. The definition keyed as ``0``
>> > is the most preferred structure. Th
W dniu nie, 28.01.2018 o godzinie 11∶14 +0100, użytkownik Ulrich Mueller
napisał:
> > > > > > On Sat, 27 Jan 2018, Michał Górny wrote:
> > This specification currently defines one section: ``[structure]``.
> > This section defines one or more repository structure definitions
> > using sequential in
> On Sat, 27 Jan 2018, Michał Górny wrote:
> This specification currently defines one section: ``[structure]``.
> This section defines one or more repository structure definitions
> using sequential integer keys. The definition keyed as ``0``
> is the most preferred structure. The package ma
W dniu nie, 28.01.2018 o godzinie 15∶01 +0800, użytkownik Jason Zaman
napisał:
> On Sat, Jan 27, 2018 at 12:24:39AM +0100, Michał Górny wrote:
> > Migrating mirrors to the hashed structure
> > -
> > The hard link solution allows us to save space on the master
On Sat, Jan 27, 2018 at 12:24:39AM +0100, Michał Górny wrote:
> Migrating mirrors to the hashed structure
> -
> The hard link solution allows us to save space on the master mirror.
> Additionally, if ``-H`` option is used by the mirrors it avoids
> transferr
On 01/27/2018 02:47 PM, Michał Górny wrote:
>> *
>> * Please download fileN from
>> * wherever fileN can be found
>> * and move it to $DISTDIR/subdirN
>
> Do you really believe this to be more friendly than a helper that places
> all the files in correct directories?
Well, it's no worse than
On 01/27/2018 02:01 PM, Gordon Pettey wrote:
> On Sat, Jan 27, 2018 at 10:48 AM, Michael Orlitzky wrote:
>> On 01/27/2018 11:42 AM, Gordon Pettey wrote:
>>> Why not use a hash of the file name instead of its contents?...
>>
>> That's the proposal =P
>
> I'm not following, then. What's all this ab
W dniu sob, 27.01.2018 o godzinie 13∶24 -0500, użytkownik Michael
Orlitzky napisał:
> On 01/27/2018 01:14 PM, Michał Górny wrote:
> > >
> > > If we have a tool like edistadd, then I see the problem. But if we were
> > > going to use file-data based hashes, then there would be no need for a
> > > t
On Sat, Jan 27, 2018 at 10:48 AM, Michael Orlitzky wrote:
> On 01/27/2018 11:42 AM, Gordon Pettey wrote:
>> Why not use a hash of the file name instead of its contents? That
>> seems like it would be much simpler, and that's not going to reduce
>> the output space for balance...
>
> That's the pro
On 01/27/2018 01:14 PM, Michał Górny wrote:
>>
>> If we have a tool like edistadd, then I see the problem. But if we were
>> going to use file-data based hashes, then there would be no need for a
>> tool in most cases. As a developer, "repoman manifest" would handle it.
>> As a user, I'm going to s
W dniu sob, 27.01.2018 o godzinie 11∶47 -0500, użytkownik Michael
Orlitzky napisał:
> On 01/27/2018 03:30 AM, Michał Górny wrote:
> > >
> > > What are we worried about in using a temporary directory? Copying across
> > > filesystem boundaries? Except in rare cases, $DISTDIR itself will be
> > > us
On 01/27/2018 11:42 AM, Gordon Pettey wrote:
> Why not use a hash of the file name instead of its contents? That
> seems like it would be much simpler, and that's not going to reduce
> the output space for balance...
That's the proposal =P
On 01/27/2018 03:30 AM, Michał Górny wrote:
>>
>> What are we worried about in using a temporary directory? Copying across
>> filesystem boundaries? Except in rare cases, $DISTDIR itself will be
>> usable a temporary location (on the same filesystem), won't it?
>
> Why add the extra complexity whe
Why not use a hash of the file name instead of its contents? That
seems like it would be much simpler, and that's not going to reduce
the output space for balance...
On Sat, Jan 27, 2018 at 5:41 AM, Michał Górny wrote:
> W dniu sob, 27.01.2018 o godzinie 11∶36 +, użytkownik Roy Bamford
> napi
W dniu sob, 27.01.2018 o godzinie 11∶36 +, użytkownik Roy Bamford
napisał:
> On 2018.01.27 08:30, Michał Górny wrote:
> > W dniu pią, 26.01.2018 o godzinie 20∶48 -0500, użytkownik Michael
> > Orlitzky napisał:
> > > On 01/26/2018 06:24 PM, Michał Górny wrote:
> > > >
> > > > The alternate opti
On 2018.01.27 08:30, Michał Górny wrote:
> W dniu pią, 26.01.2018 o godzinie 20∶48 -0500, użytkownik Michael
> Orlitzky napisał:
> > On 01/26/2018 06:24 PM, Michał Górny wrote:
> > >
> > > The alternate option of using file hash has the advantage of
> having
> > > a more balanced split. Furthermo
W dniu pią, 26.01.2018 o godzinie 20∶48 -0500, użytkownik Michael
Orlitzky napisał:
> On 01/26/2018 06:24 PM, Michał Górny wrote:
> >
> > The alternate option of using file hash has the advantage of having
> > a more balanced split. Furthermore, since hashes are stored
> > in Manifests using them
On Fri, Jan 26, 2018 at 7:48 PM, Michael Orlitzky wrote:
> On 01/26/2018 06:24 PM, Michał Górny wrote:
>>
>> The alternate option of using file hash has the advantage of having
>> a more balanced split. Furthermore, since hashes are stored
>> in Manifests using them is zero-cost. However, this s
On 01/26/2018 06:24 PM, Michał Górny wrote:
>
> The alternate option of using file hash has the advantage of having
> a more balanced split. Furthermore, since hashes are stored
> in Manifests using them is zero-cost. However, this solution has two
> significant disadvantages:
>
> 1. The hash v
Hi, everyone.
Here's a little new something we've been silently debating back
in the day, then forgotten about it, then I've written a GLEP about it.
The number's not official yet.
HTML (with plots): https://dev.gentoo.org/~mgorny/tmp/glep-0075.html
---
GLEP: 75
Title: Split distfile mirror dire
44 matches
Mail list logo
