Duncan posted on Mon, 09 Oct 2017 02:58:17 +0000 as excerpted: > Dave posted on Sun, 08 Oct 2017 23:46:08 +0000 as excerpted: > >> About the most complicated thing I do these days >> apart from a bit of bash scripting is edit the Pan source code to >> increase the number of threads to 30 per server whenever a FreeBSD port >> comes along. > > As you may know but it's worth repeating for others that may be reading > in any case, editing source code isn't absolutely essential for that > tweak.
> [T]he non-code workaround is to edit the config file, in this case > servers.xml in the pan config dir (~/.pan2 by default[).] > Meanwhile, I've one more point to make on the server connections topic, > but I'll make it in a followup post, as it could potentially lead to its > own subthread... So here's that followup... Are you sure you actually need 30 connections? In my experience, there are only three (rather narrow) cases in which more than, say, 8-10 connections, and often, any more than 4, is actually useful, and one of those three cases won't count here. IOW, more connections doesn't always mean better throughput. In fact, in practice more connections often means either worse throughput or more CPU and memory usage to get the same thruput, due to all those extra fetch threads trying to use the same already maxed out pipe. The three exceptions are: 1) Users where the server caps the per-connection bandwidth but not the total bandwidth, so the more connections you can use the more bandwidth you get. But providers with per-connection bandwidth caps generally allow only a handful of connections anyway; usually in the single digits and often only 2-4, only very rarely 20+ connections, because they're using a per- connection bandwidth limit along with a total connections limit, as a substitute for what they /really/ intend, a per-user bandwidth limit, because the per-connection limit is easier or less resources intensive to track in their server software. Given that you mentioned a 30-connection limit, this wouldn't seem to apply to you. 20+ connections allowed tends to be much more common where they're encouraging you to use more bandwidth, either to encourage tier upgrades on per-month plans, or to encourage you to eat thru your block plan as fast as possible so you'll need to renew it. 2) Those with poor quality internet connections in general. In this case, the individual connections will be throttled to something well below the nominal pipe bandwidth, both that of the user's internet connection and that allowed by the server, due to TCP's packet-loss throttling mechanisms. If this is you, OUCH, I feel for you! Been there, and it SUCKS! While upping the number of news server connections may help to some degree for news, it doesn't help for general web browsing, media streaming (which can be simply unworkable, period, unless you can download it to cache to watch later... which of course is something you might be using the binary newsgroups for in place of real-time streaming), any sort of interactive use, etc. Due to the degree to which such poor internet situations suck in general, most people eventually either give up or switch to something else, if they possibly can. But 20-30 connections may indeed help somewhat for news, in the mean time. This one's the most legit reason, but as I said, if it applies, I sure feel for you! And the one that won't apply in this case, but can be legit where it does... 3) Multiple different users and/or devices connecting to the same NSP account at once. A good example I know is the guy who had pan downloading big binaries (TV programs, many from overseas) to his media server with some of his connections, a couple others used by pan on his personal computer for text groups, and a few others used by his wife/gf on her machine, keeping up with her embroidery or whatever it was (cross- stitch, some such...) groups. Obviously this makes good use of an NSP's 20+ connection limit, but doesn't require a whole lot of connections per individual pan instance. In fact, the GNKSA-max four connections per server may well be plenty, even for that binary-downloader media machine, if neither of the first two apply as well. Meanwhile, every connection has its own overhead, TCP-connection related, CPU-thread context switches, and multiple write threads accessing storage at the same time thereby heavily fragmenting all files compared to fewer download threads at once. (Of course this latter cost will apply mostly to conventional "spinning-rust", not so much to SSDs, which should handle up to gigabit pipes at least, with ease.) If you can get the same thruput with fewer connections, it'll mean lower CPU usage, less disk thrashing if you're still on spinning rust, and less connection overhead as well. So fewer connections tends to be better, as long as you're not throttling thruput, which you shouldn't be under normal circumstances. In fact, here, at least some years ago, I found I could max my local internet pipe connection with only two connections! Of course my local internet pipe is bigger now (I just upgraded to 100 megabit, there's 300 available but I can't justify it yet, and gigabit available in some areas of town but not yet to me), I'm on ssds so the storage IO shouldn't be a bottleneck, and my middle-aged AMD bulldozer-1 fx6100 6-core moderately overclocked to 4 GHz is better than back then as well, but I'd still be extremely surprised if 2-4 connections couldn't max things out just as well as 8 or 20 or 50, and above say 8 connections, I'd expect CPU- overhead on the 6-core to have an increasing effect. So what I'm saying is if you've not actually checked and can justify double-digit download connections, you may well want to do some checking, because it's actually quite likely 30 connections is doing you more harm than good. OTOH if you're running a 16-core-plus monster with a gigabit internet pipe and storing to ssds, then perhaps all those threads /do/ increase your thruput, as they might if you have a really poor internet connection and are seeing tcp-throttling due to lost packets on individual connections. But other than that, if you haven't checked, I'd suggest doing so. I certainly changed my mind when I saw two connections maxing out my pipe just as well as many more did. (Tho I did see a reasonable boost with two over just one.) -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman _______________________________________________ Pan-users mailing list Pan-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/pan-users
