Volker Wysk posted on Sun, 29 Jul 2018 20:18:57 +0200 as excerpted: > I've figured out, what's going on.
=:^) > I had been trying to use the newsserver of the internet provider > IN-Berlin (in-berlin.de). I've contacted their support, and he kind of > hinted that this is a server issue. They are deleting all but the last > part of multipart messages, on purpose (so it seems). They don't want > users to download binary news. I can kind of understand this, but just > doing so, without any warning, is very bad. Now that we know the problem, I can expand on the brief treatment of this possibility in my earlier reply... It's not that they're /deleting/ all but the last part, it's that they have a size filter on their incoming feed from whoever they're peering with, probably because they get charged by the GiB, and because they are an ISP that sells connections to the net, not specifically an NSP that specifically sells news, so news isn't their priority (the actual net connectivity is, and should be), they don't really care that the size filter breaks most binary news. That may actually be a legal benefit as well, but it's almost certainly more accident than intent. Because text posts tend to be far smaller than binaries, the filter lets text posts thru, as it will very small binaries. But when binary files are large enough to split into parts for posting, unless the size of the file is an exact multiple of the split-size (set by the poster or their client), all the parts but the last will be whatever the split-size is set at, with the last part being whatever odd bit smaller than the split- size remains. Think of a binary file of say 13.3 MiB, with a split size of say 1 MiB. To make the math easy we'll pretend yenc is 100% efficient and the 5% encoding overhead doesn't exist. That'll be a 14-part post, with 13 parts of the 1 MiB split-size each, and a last part of 0.3 MiB. Now say the ISP's size filter is set to half a MiB. All but the last part will not be downloaded to the ISP's server, but the last part will be, because at 0.3 MiB it fits under the 0.5 MiB filter limit. Of course the obvious workaround for a poster concerned about people on such poor quality news providers is to cut their split size down similarly, to say a tenth of a MiB instead of the 1 MiB split-size we used in the example above. And a few posters actually do that. But of course that means nearly ten times the number of parts as well, so instead of 14 parts, we have /almost/ 140, actually 133 if the file is /exactly/ 13.3 MiB or just a bit smaller. And the more parts there are the more chances for one of them to get lost, plus there's actually more overhead as well, because a higher percentage of each part is headers and similar overhead, so there's a definite negative to it. And most posters consider it a losing battle to try to make allowances for poor quality news providers beyond a certain point anyway, whether it be outsmarting size filters or providing ridiculous amounts of redundancy in the form of par files. Instead, users who care enough are assumed to eventually switch to a good quality news provider, where they won't have these sorts of problems. And the fact is, even the bad dedicated news providers tend to be better than all but the extreme best bundled news provider, because unlike the bundled news providers that are actually selling something else, say connections to the net, news is what the dedicated news providers are /selling/, and if they don't do at least a half-decent job at it, they pretty quickly lose their customers and go out of business. So like it or not, the fact is that if you care about news, particularly binary news, you pretty much *must* pay for a dedicated news provider. But there are ways to make it more economical. See below. > I've tried it with another news provider (Disputo), and it appears to > works fine, with the same version of pan. =:^) FWIW, there are two primary sales models for dedicated news providers, the price-per-period model, usually monthly tho some offer discounts for pay-in-advance out to annual or so, and the price-per-GiB, aka the "block- at-a-time" model, generally with the price-per-gig going down substantially as the size of the block goes up. The key is that with proper block plans the credits don't expire, so if you don't download much, your block can last you months or years. The per-period model is the most common, but for the many not downloading at TiB/month scale, it's also the most expensive, often by far. The key is to figure out how much you're likely to actually be downloading, and see which is most economical from there. There are two block-mode news sellers I know of, astraweb.com, which I have and which has both blocks and monthly subscriptions available, and blocknews.net. Astraweb's biggest block (and thus lowest per-gig price) available is 1000 GB (base-10), $50, so $.05/GB (USD from the US, German prices may differ). The credits don't expire so if you don't use it that much. Headers are discounted to 20% so 10 GB of headers will be charged 2 GB of usage. Blocknews seems to be higher priced, at least at the top end, 1024 GB for $75 or 3072 for $215. They only do blocks but refer to usenetnow.net for monthly accounts. So usenetnow has three months for $30.49, say $10/mo (tho it's $12/mo if done monthly). And astraweb has $20/3mo (so ~$7/mo) for 10 Mbit/sec speed, $39/3mo (or $15/mo) uncapped speed. I don't know what the going monthly rate is but we'll take those as representative. Which makes astraweb's $50 1000 GB block ~7.5 months of astraweb's $20/3mo service, a bit under 4 months of their $$39/3mo uncapped service, or ~5 months of usenetnow's uncapped service. Taking the longest of those, 7.5 months, the breakover point would be ~133 GB/mo. If you're downloading more than that, the monthly may make sense. If you're downloading less, or if you sometimes go quite some time with little to no activity and would still be paying, then the block account is very likely a better deal for you. Of course at the other end you have (under) 4 months, which would be a 250 GB breakover. You can do the math for your own figures. Here, since I don't do much main usenet, binary or text, at all (I do a reasonable amount of text, but it's all on mailing lists via gmane's list2news service, as I prefer my mailing lists in news format), when I spent that $50 for 1000 GB from astraweb over two years ago (with over 9/10ths of it still left today), I figured at my current usage that may well last my lifetime, and even if it didn't, I figured it'd last me at least a year and thus be a much better buy than monthly, and the very fact that it was used so fast would mean that I got back into binaries heavily, probably /because/ I had the block account to play with, which would still make it a good buy. OTOH, I know one guy who downloads multiple TV series, for friends and extended family as well as his own household, who often clears a TB/mo on news. Obviously a block account's not a good buy at all for him! Now if you're just now sorting out the problem above, that means you're not yet, at least, downloading news at TB scale, and likely not at the 133 GB/mo scale either, or if you are, it may not last. Which means a block account that you can use when you want, and forget about it for awhile without it expiring so you can come back to it months later, without paying for months not used in the mean time, might actually make a lot of sense for now, even if you ultimately get to downloading enough to consistently hit the breakover point and make a monthly account make sense. Something to consider anyway, and for me, it made it economical to have a dedicated provider. =:^) Meanwhile, if anyone knows of any other block account NSPs, please post 'em! Even if they're more expensive, having more than two alternatives for block providers is something I'd consider a good thing! And since block accounts don't expire, I'd consider getting another account somewhere else too, just to give myself some flexibility. (Note that setting a server's connection count to 0 in pan disables it without losing the config, or setting priority to backup means it isn't used unless all primary servers don't have a particular message. Further note that while the GUI only has primary/backup, those translate to numbers in the config file, and by hand-editing, you can add as many levels of backup as desired. So it's possible to set a configured server to only use if necessary (backup), or to disabled but still configured (0 connections), if desired.) Meanwhile (2), FWIW, while most dedicated news providers allow say 20-50 connections, if you need more than say the four connections that pan's GUI allows to max your allowed bandwidth, something's probably wrong somewhere. Number of connections shouldn't be an issue, unless like one guy I know, you have your main downloader machine with its connections, your just browsing machine with a few more connections, your wife has her own machine with her own groups for a few more, etc. And then it's not to maximize the bandwidth, it's simply to allow multiple machines in your household to connect without having to worry too much about exceeding your allowed number of connections. (And BTW, if you /do/ want to up pan's number of connections, simply edit servers.xml by hand. Pan's GUI allows no more than 4 connections per server as that's what GNKSA requires, and GNKSA compliance is something pan's users and devs tend to be proud of. But GNKSA does /not/ say that pan can't honor a hand-edited config, just that pan can't allow a user to configure more than that. So that's how we handle it, if a user wants more than four connections per server they can hand configure it and pan will honor that.) Hope it's useful! =:^) -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman _______________________________________________ Pan-users mailing list Pan-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/pan-users
