Re: apt-torrent (WAS: Re: apt PARALLELISM)

2006-01-09 Thread Adam Heath
On Mon, 9 Jan 2006, Arnaud Kyheng wrote: > > Hello all and Happy New Year, > > > Thanks to George, apt-torrent has been mentioned in the Debian Devel > list :o) > > I've just noticed it, and the fun part of this discovery, is that I also > found why my ISP has closed sianka.free.fr: Too much hits

Re: apt-torrent (WAS: Re: apt PARALLELISM)

2006-01-09 Thread Nathanael Nerode
> It'll take me some time to find a new, and more appropriate home for > apt-torrent. The Debian archive ("experimental" distribution) would be a *very* appropriate home. It won't provide a testbed package seeder or place to download .torrent files, but that can be done later (and by any numbe

Re: apt-torrent (WAS: Re: apt PARALLELISM)

2006-01-08 Thread Christian Perrier
> I've just noticed it, and the fun part of this discovery, is that I also > found why my ISP has closed sianka.free.fr: Too much hits since the > latest Debian Weekly News, and the new apt-torrent 0.3.1-1 package ! The solution is simple: get it in the Debian archive..:) -- To UNSUBSCRIBE, e

apt-torrent (WAS: Re: apt PARALLELISM)

2006-01-08 Thread Arnaud Kyheng
Hello all and Happy New Year, Thanks to George, apt-torrent has been mentioned in the Debian Devel list :o) I've just noticed it, and the fun part of this discovery, is that I also found why my ISP has closed sianka.free.fr: Too much hits since the latest Debian Weekly News, and the new apt-tor

Re: apt PARALLELISM

2006-01-06 Thread Michelle Konzack
Am 2005-12-27 16:25:10, schrieb Bernd Eckenfels: > In article <[EMAIL PROTECTED]> you wrote: > >apt-get update && apt-get --download-only upgrade > > It would make more sense to send out the DIFFs to the packages.gz, so you > dont actually need to download the packages file every five minutes

Re: apt PARALLELISM

2005-12-27 Thread Bernd Eckenfels
In article <[EMAIL PROTECTED]> you wrote: > <[EMAIL PROTECTED]> subscribe with an seperated E-Mail and track it. > Check the E-Mail in a delay of 5 minutes. Write a script (we do not > want to download Packages.gz, if there is no Pakage of interest) which > check, whether the new package is instal

Re: apt PARALLELISM

2005-12-27 Thread Michelle Konzack
Am 2005-12-21 16:32:20, schrieb Goswin von Brederlow: > I have 10240kBit downstream and get way less from security.debian.org. > Especialy when there is a security release of X or latex. They are two possibilitys: <[EMAIL PROTECTED]> subscribe with an seperated E-Mail and track it. Check the E-M

Re: apt PARALLELISM

2005-12-23 Thread Goswin von Brederlow
Olaf van der Spek <[EMAIL PROTECTED]> writes: > On 12/21/05, Goswin von Brederlow <[EMAIL PROTECTED]> wrote: >> > Who need PARALELISM and who has a bandwidth of more then 8 MBit? >> >> I have 10240kBit downstream and get way less from security.debian.org. >> Especialy when there is a security rele

Re: apt PARALLELISM

2005-12-21 Thread Henrique de Moraes Holschuh
On Wed, 21 Dec 2005, Olaf van der Spek wrote: > On 12/21/05, Goswin von Brederlow <[EMAIL PROTECTED]> wrote: > > > Who need PARALELISM and who has a bandwidth of more then 8 MBit? > > > > I have 10240kBit downstream and get way less from security.debian.org. > > Especialy when there is a security r

Re: apt PARALLELISM

2005-12-21 Thread Olaf van der Spek
On 12/21/05, Goswin von Brederlow <[EMAIL PROTECTED]> wrote: > > Who need PARALELISM and who has a bandwidth of more then 8 MBit? > > I have 10240kBit downstream and get way less from security.debian.org. > Especialy when there is a security release of X or latex. But parallel downloads won't solv

Re: apt PARALLELISM

2005-12-21 Thread Goswin von Brederlow
Michelle Konzack <[EMAIL PROTECTED]> writes: > Am 2005-12-06 09:53:43, schrieb Ivan Adams: >> Hi again, >> in my case: >> I have slow internet connection. BUT I have friends with the same > ^^^ >> connection >> in my local area network, who have apt-proxy. >> My goal

Re: apt PARALLELISM

2005-12-21 Thread Goswin von Brederlow
Michelle Konzack <[EMAIL PROTECTED]> writes: > Am 2005-12-12 13:23:01, schrieb Goswin von Brederlow: > >> Actualy one thing apt could do: >> >> [EMAIL PROTECTED]:~% host security.debian.org >> security.debian.org A 82.94.249.158 >> security.debian.org A 128.101.80.133 >> secur

Re: apt PARALLELISM

2005-12-20 Thread Michelle Konzack
Am 2005-12-12 17:06:28, schrieb Bas Zoetekouw: > But what would you gain from that? In my experience, the mirrors are > fast enough to saturate anything but the fastest (100Mb) links. FullACK! - OK, I have currently only an E3 in Paris, but if all goes right I will get my own FiberOptic STM-

Re: apt PARALLELISM

2005-12-20 Thread Michelle Konzack
Am 2005-12-12 13:23:01, schrieb Goswin von Brederlow: > Actualy one thing apt could do: > > [EMAIL PROTECTED]:~% host security.debian.org > security.debian.org A 82.94.249.158 > security.debian.org A 128.101.80.133 > security.debian.org A 194.109.137.218 > > Why not

Re: apt PARALLELISM

2005-12-20 Thread Michelle Konzack
Am 2005-12-06 09:53:43, schrieb Ivan Adams: > Hi again, > in my case: > I have slow internet connection. BUT I have friends with the same ^^^ > connection > in my local area network, who have apt-proxy. > My goal is: When I need to install new system (Debian) on new us

Re: apt PARALLELISM

2005-12-20 Thread Michelle Konzack
Am 2005-12-05 16:11:35, schrieb Joe Smith: > This person is requesting parallel downloads from multiple servers. So > basicly during package download, if there are three full and up-to-date > mirrors in sources.list, there should be simulatious downloads of different > packages from all three d

Re: apt PARALLELISM

2005-12-14 Thread Wesley J. Landaker
On Wednesday 14 December 2005 14:41, Olaf van der Spek wrote: > On 13 Dec 2005 15:56:00 +0100, Claus Färber <[EMAIL PROTECTED]> wrote: > > Olaf van der Spek <[EMAIL PROTECTED]> schrieb/wrote: > > > That's not true. Suppose you've only got 3 users. If each user > > > connects to one (different) mir

Re: apt PARALLELISM

2005-12-14 Thread Olaf van der Spek
On 13 Dec 2005 15:56:00 +0100, Claus Färber <[EMAIL PROTECTED]> wrote: > Olaf van der Spek <[EMAIL PROTECTED]> schrieb/wrote: > > That's not true. Suppose you've only got 3 users. If each user > > connects to one (different) mirror, he gets 1/1 of that mirror's > > bandwidth. If each user connects

Re: apt PARALLELISM

2005-12-14 Thread Claus Färber
Olaf van der Spek <[EMAIL PROTECTED]> schrieb/wrote: > That's not true. Suppose you've only got 3 users. If each user > connects to one (different) mirror, he gets 1/1 of that mirror's > bandwidth. If each user connects to each mirror, he only gets 1/3 of > that mirror's bandwidth. They could get

Re: apt PARALLELISM

2005-12-14 Thread Robert Lemmen
On Wed, Dec 14, 2005 at 01:23:17PM +0100, Martijn van Oosterhout wrote: > ISTM the easiest would be for apt to lookup the hostname itself and > treat the single entry as a list of entires, one for each possible > address the hostname can resolve to. If one fails, try the next. apt already does tha

Re: apt PARALLELISM

2005-12-14 Thread Martijn van Oosterhout
2005/12/13, Henrique de Moraes Holschuh <[EMAIL PROTECTED]>: > > Time to devise a way to teach it about that, then. HOW to do it is the big > problem, though. How should one deal with round-robin DNS mirrors which are > supposed to be equal, but are not. What are the failure modes to cater > fo

Re: apt PARALLELISM

2005-12-13 Thread Henning Makholm
Scripsit Olaf van der Spek <[EMAIL PROTECTED]> > On 12/13/05, Henning Makholm <[EMAIL PROTECTED]> wrote: >> Alternatively each user can spread his load over all three servers; >> his download now takes 5 minutes, and each server _still_ sees >> 600*5 = 3000 active connections at any time. Thus _al

Re: apt PARALLELISM

2005-12-13 Thread Olaf van der Spek
On 12/13/05, Henning Makholm <[EMAIL PROTECTED]> wrote: > Assume a situation where mirror bandwidth is the limiting factor, and > imagine a world with 3 mirrors. Say that during a certain time of the > day 600 users each minute start to download updated x.org packages. > Either they can do their d

Re: apt PARALLELISM

2005-12-13 Thread Henning Makholm
Scripsit Henrique de Moraes Holschuh <[EMAIL PROTECTED]> > THis is not something that would bother anyone if it is a single user... but > if you have 10k users doing that, often close enough in time, well, things > should get MUCH worse as far as I can see. If they are doing this at random > time

Re: apt PARALLELISM

2005-12-12 Thread Henrique de Moraes Holschuh
On Mon, 12 Dec 2005, Joey Hess wrote: > Marco d'Itri wrote: > > > Or at least fall back to the other IPs if the first one gives an > > > error? > > I hope that this already happens... > > apt doesn't know anything about round robin dns, and especially with > secure apt, if one mirror gets out of s

Re: apt PARALLELISM

2005-12-12 Thread Joey Hess
Marco d'Itri wrote: > > Or at least fall back to the other IPs if the first one gives an > > error? > I hope that this already happens... apt doesn't know anything about round robin dns, and especially with secure apt, if one mirror gets out of sync things break horribly. This recently happened wi

Re: apt PARALLELISM

2005-12-12 Thread Simon Richter
Hi, Bas Zoetekouw wrote: But what would you gain from that? In my experience, the mirrors are fast enough to saturate anything but the fastest (100Mb) links. I think the idea is a) load-balancing over multiple DSL lines b) checking a bunch of apt-proxy servers whether they can provide the

Re: apt PARALLELISM

2005-12-12 Thread Bas Zoetekouw
Hi Ivan! You wrote: > > As far as I read the proposal, it is about downloading _different_ > files from different mirrors - if you have 25 packages to get for your > 'apt-get update' operation, download 5 packages from each of 5 > different servers, with one connection to each se

Re: apt PARALLELISM

2005-12-12 Thread Linas Zvirblis
Ivan Adams wrote: My goal is using more bandwidth with apt-proxy servers on my friends, who have other internet connection. And I want to download first packet from my internet and second packet at the same time from the apt-proxy with my friend internet connection. Now, I can only download all

Re: apt PARALLELISM

2005-12-12 Thread Ivan Adams
My goal is using more bandwidth with apt-proxy servers on my friends, who have other internet connection.And I want to download first packet from my internet and second packet at the same time from the apt-proxy with my friend internet connection. Now, I can only download all packets from my intern

Re: apt PARALLELISM

2005-12-12 Thread Olaf van der Spek
On 12/12/05, Ivan Adams <[EMAIL PROTECTED]> wrote: > > > > As far as I read the proposal, it is about downloading _different_ > > files from different mirrors - if you have 25 packages to get for your > > 'apt-get update' operation, download 5 packages from each of 5 > > different servers, with one

Re: apt PARALLELISM

2005-12-12 Thread Ivan Adams
As far as I read the proposal, it is about downloading _different_files from different mirrors - if you have 25 packages to get for your 'apt-get update' operation, download 5 packages from each of 5different servers, with one connection to each server active at atime.That is what I mean ...  

Re: apt PARALLELISM

2005-12-12 Thread Henrique de Moraes Holschuh
On Mon, 12 Dec 2005, Henning Makholm wrote: > As far as I read the proposal, it is about downloading _different_ > files from different mirrors - if you have 25 packages to get for your > 'apt-get update' operation, download 5 packages from each of 5 > different servers, with one connection to each

Re: apt PARALLELISM

2005-12-12 Thread Wouter Verhelst
On Mon, Dec 12, 2005 at 12:52:09PM +0100, Goswin von Brederlow wrote: > Wouter Verhelst <[EMAIL PROTECTED]> writes: > > True, but that's not what's being asked here. If multiple URLs could > > serve requests for a single repository---i.e., if you've got both > > deb http://ftp1.CC.debian.org/debia

Re: apt PARALLELISM

2005-12-12 Thread Henning Makholm
Scripsit Henrique de Moraes Holschuh <[EMAIL PROTECTED]> > 1. We care about a large lot of people a lot more than we care for an >individual's downloading speed > 2. Thus we try to keep the mirror load down, and downloading hundreds of >megabytes using multiple connections to multiple sou

Re: apt PARALLELISM

2005-12-12 Thread Henrique de Moraes Holschuh
On Mon, 12 Dec 2005, Marco d'Itri wrote: > On Dec 12, Henrique de Moraes Holschuh <[EMAIL PROTECTED]> wrote: > > > use more than a single server by itself... I didn't think it common > > > practice > > > for large mirror to configure multi-megabyte windows... > > TCP/IP windows? Or user bw shapin

Re: apt PARALLELISM

2005-12-12 Thread Marco d'Itri
On Dec 12, Henrique de Moraes Holschuh <[EMAIL PROTECTED]> wrote: > > use more than a single server by itself... I didn't think it common practice > > for large mirror to configure multi-megabyte windows... > TCP/IP windows? Or user bw shaping? He means the TCP window size, and it *is* common pra

Re: apt PARALLELISM

2005-12-12 Thread Marco d'Itri
On Dec 12, Goswin von Brederlow <[EMAIL PROTECTED]> wrote: > Why not open 3 connections one to each host? Why do? > Or at least fall back to the other IPs if the first one gives an > error? I hope that this already happens... -- ciao, Marco signature.asc Description: Digital signature

Re: apt PARALLELISM

2005-12-12 Thread Henrique de Moraes Holschuh
On Mon, 12 Dec 2005, Martijn van Oosterhout wrote: > 2005/12/12, Henrique de Moraes Holschuh <[EMAIL PROTECTED]>: > > We don't want them to open multiple connections even to MULTIPLE > > servers... > > That's odd though, because apt *does* open connections to multiple servers > all the time. To fe

Re: apt PARALLELISM

2005-12-12 Thread Goswin von Brederlow
Martijn van Oosterhout <[EMAIL PROTECTED]> writes: > 2005/12/12, Henrique de Moraes Holschuh <[EMAIL PROTECTED]>: > > We don't want them to open multiple connections even to > MULTIPLE servers... > > > That's odd though, because apt *does* open connections to multiple servers all >

Re: apt PARALLELISM

2005-12-12 Thread Ivan Adams
Can have option in /etc/apt/apt.conf or apt-get (install || upgrade) with -? some char here for parallelism.And by default can be disabled ...

Re: apt PARALLELISM

2005-12-12 Thread Goswin von Brederlow
Wouter Verhelst <[EMAIL PROTECTED]> writes: > On Sun, Dec 11, 2005 at 02:45:50AM +0100, Marco d'Itri wrote: >> On Dec 11, Charles Fry <[EMAIL PROTECTED]> wrote: >> >> > But if multiple URLs could satisfactorily serve requests for a single >> > repository, only one of them is currently used. >> Wh

Re: apt PARALLELISM

2005-12-12 Thread Martijn van Oosterhout
2005/12/12, Henrique de Moraes Holschuh <[EMAIL PROTECTED]>: We don't want them to open multiple connections even to MULTIPLE servers... That's odd though, because apt *does* open connections to multiple servers all the time. To fetch packages lists, or if a package is only available on one of the

Re: apt PARALLELISM

2005-12-12 Thread Ivan Adams
Thanks ...We don't want them to open multiple connections even to MULTIPLE servers...

Re: apt PARALLELISM

2005-12-11 Thread Henrique de Moraes Holschuh
On Sun, 11 Dec 2005, Marco d'Itri wrote: > On Dec 11, Charles Fry <[EMAIL PROTECTED]> wrote: > > But if multiple URLs could satisfactorily serve requests for a single > > repository, only one of them is currently used. > Which is fine, because we do not want people to open multiple > connections to

Re: apt PARALLELISM

2005-12-11 Thread Marco d'Itri
On Dec 11, Wouter Verhelst <[EMAIL PROTECTED]> wrote: > True, but that's not what's being asked here. If multiple URLs could > serve requests for a single repository---i.e., if you've got both > deb http://ftp1.CC.debian.org/debian unstable main > and > deb http://ftp2.CC.debian.org/debian unstab

Re: apt PARALLELISM

2005-12-11 Thread Wouter Verhelst
On Sun, Dec 11, 2005 at 02:45:50AM +0100, Marco d'Itri wrote: > On Dec 11, Charles Fry <[EMAIL PROTECTED]> wrote: > > > But if multiple URLs could satisfactorily serve requests for a single > > repository, only one of them is currently used. > Which is fine, because we do not want people to open m

Re: apt PARALLELISM

2005-12-10 Thread Marco d'Itri
On Dec 11, Charles Fry <[EMAIL PROTECTED]> wrote: > But if multiple URLs could satisfactorily serve requests for a single > repository, only one of them is currently used. Which is fine, because we do not want people to open multiple connections to the same server. -- ciao, Marco signature.asc

Re: apt PARALLELISM

2005-12-10 Thread Charles Fry
> Seperate repositiories are already parallelized, which is what I think > you are asking for here. But if multiple URLs could satisfactorily serve requests for a single repository, only one of them is currently used. Ideally, the amount of parallelism could match the number of redundant URLs pro

Re: apt PARALLELISM

2005-12-10 Thread Stephen Gran
This one time, at band camp, Cosimo Alfarano said: > On Tue, Dec 06, 2005 at 09:09:39AM -0200, Henrique de Moraes Holschuh wrote: > > I doubt very much so parallel downloads will be added to apt. > > Could be added Release-file based. > Parallelize only unrelated pools, and disabled by default to

Re: apt PARALLELISM

2005-12-10 Thread Cosimo Alfarano
On Tue, Dec 06, 2005 at 09:09:39AM -0200, Henrique de Moraes Holschuh wrote: > I doubt very much so parallel downloads will be added to apt. Could be added Release-file based. Parallelize only unrelated pools, and disabled by default to avoid conflict wiht apt-proxy (if any). ie: Debian and Foo v

Re: apt PARALLELISM

2005-12-06 Thread Henrique de Moraes Holschuh
On Tue, 06 Dec 2005, George Danchev wrote: > apt-torrent seems to approach that too: > http://sianka.free.fr/documentation.html Now, THAT is something nice. BitTorrent won't overload the mirrors ever. -- "One disk to rule them all, One disk to find them. One disk to bring them all and in th

Re: apt PARALLELISM

2005-12-06 Thread George Danchev
On Tuesday 06 December 2005 13:09, Henrique de Moraes Holschuh wrote: > On Tue, 06 Dec 2005, Ivan Adams wrote: > > I have slow internet connection. BUT I have friends with the same > > connection in my local area network, who have apt-proxy. > > My goal is: When I need to install new system (Debian

Re: apt PARALLELISM

2005-12-06 Thread Henrique de Moraes Holschuh
On Tue, 06 Dec 2005, Ivan Adams wrote: > I have slow internet connection. BUT I have friends with the same connection > in my local area network, who have apt-proxy. > My goal is: When I need to install new system (Debian) on new user, or > dist-upgrade on entire system, I need the unstable packets

Re: apt PARALLELISM

2005-12-05 Thread Ivan Adams
Hi again,in my case:I have slow internet connection. BUT I have friends with the same connection in my local area network, who have apt-proxy.My goal is: When I need to install new system (Debian) on new user, or dist-upgrade on entire system, I need the unstable packets from site. In this case I n

Re: apt PARALLELISM

2005-12-05 Thread Henrique de Moraes Holschuh
On Tue, 06 Dec 2005, Romain Beauxis wrote: > Le Mardi 6 Décembre 2005 02:50, Joe Smith a écrit : > > Now it is useless for users where the bottleneck is on their end. > > Well, it can also be usefull in case of a broken mirror can't it? apt already handles that and skips to the next mirror. --

Re: apt PARALLELISM

2005-12-05 Thread Romain Beauxis
Le Mardi 6 Décembre 2005 02:50, Joe Smith a écrit : > Now it is useless for users where the bottleneck is on their end. Well, it can also be usefull in case of a broken mirror can't it? Romain -- Not even the dog That piss against the wall of Babylon, Shall escape his judgement

Re: apt PARALLELISM

2005-12-05 Thread Henrique de Moraes Holschuh
On Mon, 05 Dec 2005, Joe Smith wrote: > "Olaf van der Spek" <[EMAIL PROTECTED]> wrote in message > news:[EMAIL PROTECTED] > >On 12/5/05, Ivan Adams <[EMAIL PROTECTED]> wrote: > >>Example: (/etc/apt/sources.list) > >>deb http://ftp.en.debian.org/debian main stable contrib non-free > >>deb http://ft

Re: apt PARALLELISM

2005-12-05 Thread Joe Smith
"Olaf van der Spek" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] On 12/5/05, Joe Smith <[EMAIL PROTECTED]> wrote: "Olaf van der Spek" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] > On 12/5/05, Ivan Adams <[EMAIL PROTECTED]> wrote: >> Example: (/etc/apt/sources.li

Re: apt PARALLELISM

2005-12-05 Thread Olaf van der Spek
On 12/5/05, Joe Smith <[EMAIL PROTECTED]> wrote: > > "Olaf van der Spek" <[EMAIL PROTECTED]> wrote in message > news:[EMAIL PROTECTED] > > On 12/5/05, Ivan Adams <[EMAIL PROTECTED]> wrote: > >> Example: (/etc/apt/sources.list) > >> deb http://ftp.en.debian.org/debian main stable contrib non-free >

Re: apt PARALLELISM

2005-12-05 Thread Joe Smith
"Olaf van der Spek" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] On 12/5/05, Ivan Adams <[EMAIL PROTECTED]> wrote: Example: (/etc/apt/sources.list) deb http://ftp.en.debian.org/debian main stable contrib non-free deb http://ftp.de.debian.org/debian main stable contrib non-free

Re: apt PARALLELISM

2005-12-05 Thread Olaf van der Spek
On 12/5/05, Ivan Adams <[EMAIL PROTECTED]> wrote: > Example: (/etc/apt/sources.list) > deb http://ftp.en.debian.org/debian main stable contrib non-free > deb http://ftp.de.debian.org/debian main stable contrib non-free > > in this case the stable packages will be ONLY downloaded from first server >

apt PARALLELISM

2005-12-05 Thread Ivan Adams
Hi, I'll be so glad to see next (or one of the next) version of the apt package, to have PARALLELISM.For example if I have more than one row pointing to one stage (stable testing or unstable) in /etc/apt/sources.list, apt to get one packet from one server, and at the same time to get second packet