Rob posted on Mon, 04 Jul 2011 14:36:23 -0400 as excerpted: > On Monday 04 July 2011 11:42, Alan Meyer wrote: >> Of course in theory, theory and practice are the same. But in practice >> ... maybe I'm totally screwed up here. > > Practically speaking, I hit binsearch.info and found an Ubuntu ISO to > test with, currently the first result here:
Thanks for putting the practice to the theory for us. That's the great thing about such discussions on lists/newsgroups. If there's a reasonably sized group involved in the discussion, there's generally at least one person who can explain the theory, and one that's willing to test to see if reality and theory match. Alan and I seem to be the former; you're the first of the latter so far, so... thanks! =:^) Unsplit the url: > http://binsearch.info/?q=ubuntu-10.04.1-desktop- i386.iso&max=100&adv_age=600&server= > I was going to post the result of [10/4/1] connections, but [meatspace] > and the single-connection one is still going. Each time, I created a > new directory, put the nzb file into it, purged Pan's article cache, > adjusted the max connections to the appropriate number and watched > jnettop to verify it was working after issuing the following command > line: > > time pan --no-gui --nzb ubuntu\ iso.nzb -o `pwd` > > I did 10 connections first, so that if there were any server-side > caching it would be disadvantaged. [Here's 10/4, respectively.] > > real 14m1.951s user 1m46.775s sys 0m31.602s > > real 35m31.818s user 4m37.997s sys 1m11.544s > > [60% savings] 10 connections vs. 4. > [T]he single-connection one is still going [after 25 min.] > [Note that] the NZB created by the above query creates 2 copies of > the ISO, [thus the 15 minutes for a 700 MB iso.] > > This was using Giganews, a service which advertises 20 simultaneous > connections for its lowest-end subscription level. For my subscription > level, it advertises 50 simultaneous connections. > > Whether it's my ISP, Giganews, my router or something else throttling > things on a per-connection basis, the only conclusion I can come up with > is that Pan's attempts at GNKSA compliance are hamstringing its binary > performance relative to more modern newsreaders, except for those of us > who are comfortable tweaking XML files by hand. That's the bit that /does/ bother me. There's a few questions remaining with the above tests, too. 1) What's your max allowed pipe speed? Current typical mid-range (US) DSL/Cable download speeds run probably 10-20 Mbps, down. If you're connected via the 100 Mbps + available in some places, tho, that's a bit different, as would be the 3 Mbps or less connections typical a few years ago, and still the only reasonable cost alternative in many locations. 2) What's the encoding? If it's traditional MIME/Base64 or UUE, there's an encoding overhead of 33-40%, making that ~1.4 gig closer to 1.9 gig of actual downloaded encoded data. If it's yEnc, the overhead is a much more manageable ~5%, ~1.5 gig encoded. That doesn't mess strictly relative comparisons like the above, but as the highest speed approaches your pipe max, it makes a big difference. (Actually, after looking at the search, assuming you're talking the Peter Hannay posting to a.b.test, it says in the title, yEnc. Thus answering /that/ question. =:^) 3) This bit of course requires XML file editing as well (cache size would need to be >= 2 gig, up from the 10 MB default that pretty much forces download-and-save at once), but it'd be interesting to see what the download-to-cache time/speed was. If there's a big difference tilted in favor of download-to-cache as I suspect there likely is, one way around the problem while staying within the current GNKSA 4-connection limit would be to up pan's default cache size and make download-to-cache the encouraged and default behavior. Another way around it that would help for some, would be to move the decoding/saving to separate threads, so the download threads could immediately go back to downloading. However, results there would vary, depending on local computer resources (single-core and speed vs. dual vs. quad vs. 6 vs. 8/12/whatever, and local storage speed, high-speed RAID-0 or RAID-10, or high speed SSD, in preference to single spinning disk, and for the latter, current generation is likely to be somewhat better than few years old). Threading the decode separately would only help if you're not CPU-bound or storage-bandwidth-bound already. Obviously, they're not in your case (tho the 10-connection one /might/ be), but I doubt you're on a single-core <2 GHz with a single spinning drive from the era when <100 gig drives was still big, either. 4) Talking about local computer resources, I suppose at least dual-core is a reasonable assumption these days, but if you're getting that dramatic a difference on < quad-core and a single mechanical drive, it's WAY more interesting than if you're getting it on a 6-core+ high-speed SSD or RAID-0/RAID-10 setup. 5) Perhaps I'm old fashioned but I tend to still prefer wired local connections, for many reasons including local connection reliability, speed and test duplicability. If you're on wireless, all sorts of weird stuff could interfere with an individual download, making individual results count rather less. 6) Lastly, to answer the "where's the knee" question, results from 6 and 8 connections, and possibly from say 13,16,20,30 and 50 (since you have it...) if the results keep getting better as you go up, would be useful as well, but obviously that increases the work you'd have to do as well. And I don't know what your giganews per-month limit is. If it's not unlimited, that number of tests could use up a rather larger chunk of that than you wish to dedicate to simple testing, as well. #3 is the most interesting to me, but also requires the most work from you, especially if combined with #6. Most of the others are simply reporting data you probably already have at hand. But the results are certainly interesting (and potentially real-world useful for binary downloaders outside this debate as well) even as they stand now. Thanks VERY much! -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman _______________________________________________ Pan-users mailing list Pan-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/pan-users
