Charles Kerr posted <[EMAIL PROTECTED]>, excerpted below, on Fri, 05 May 2006 02:48:14 -0500:
>> Duncan wrote: >>> Talking about which, how does the caching work in 0.9x (second time >>> I've asked)? I don't see a size preference. >>> >> Here's how I changed the cache size. >> gui/pan.cc line 186 >> >> ArticleCache cache (cache_path,100); // replace 100 with the max size in >> MB. the default is 10. >> >> Now I just have to remember to do this for each new tarball. > > I can put in a preferences line to let you change this, but let me ask a > naive question -- why do you want to? Since Pan won't expire any article > that a task's got a reference to, the queue can grow far larger than the > preferred maximum so long as the task exists. You could, for example, > download a dvd iso without having to increase the cache size past 10M. I believe the issue is one of folks (me) using PAN in ways the author didn't envision. <g> As is common in such instances, one must explain the use in ordered to get the point across, since the author obviously hadn't considered that particular use. <g> There are two issues, the one I originally ran into back when old-PAN had a 1 gig limit, and especially now, a newer/different one. The issue back then was a basic one of how one approaches downloading and sorting binaries. Your example, downloading a single ISO complete, saving it, and moving on, is one way, and may work. Even still, I'm not sure how well it will work if one is grabbing from multiple groups at a time. Will the second ISO start overwriting the cached parts of the first while they are both downloading, the first overwriting the second, so neither one ends up with more than about 5MB saved after downloading 10 /gigabytes/!!? That's essentially what used to happen to me, but for a different reason. Rather than saving everything directly to binary and moving on, then sorting and deleting or final-saving the binaries later, working directly with the filesystem, not the newsclient, I normally prefer to use a newsclient in a dramatically different way. After deleting obvious spam and reposts I already have, I'll select everything remaining and set PAN downloading it all. Then I do the second step within PAN, viewing downloaded messages within PAN (if it's a simple jpeg or whatever, that PAN will display), or saving them out individually and opening them, deciding whether I want to keep them or not, and where I want to save them if so, before moving on. This way, I have not just the filename as attached, but all the extra data in the posts that went with it, the author, the date posted, the title of the post, etc, that I can use to help sort and categorize the resulting binaries. Saving the binaries out directly, rather than caching the posts and processing them within PAN, I'd lose the benefit of all that extra metadata! Naturally, this sort of processing requires a much larger cache, typically a gig at minimum, for an active still-image binary downloader, several gig for an active MP3 downloader, tens of gigs if one is downloading full DVD ISOs. Without that, I was **VERY** dismayed to find after downloading several gigs of content, over a session lasting several hours, that most of it was already deleted before I'd even looked at it, because PAN was practically operating with no cache (well, a few megs, but that's practically no cache, when one's dealing with content in the hundreds of megabytes or tens of gigabytes range!). (Not exactly no topic, but I've found it helpful, and this seems a good place to mention it... For many years, from back in my MSWormOS days, in fact, I've actually run a dedicated news partition. Formerly, it was my news cache, and I setup an 8 gig partition for that purpose when I switched to Linux. After I found PAN couldn't use it, I switched to using that for my intermediate stage of binary processing -- after I'd decided I wanted to save the stuff and had performed the initial sort into subdir categories, but before I saved it to its semi-permanent location on my media partition, for eventual burn-off to ISO or deletion if I decided it wasn't worth keeping around after all, a few months later. I like that dedicated partition management, and as a result thereof, would actually find an "unlimited" cache option quite useful as well, with PAN not managing cache until it ran into filesystem full errors.) Naturally, I soon discovered the cache setting in the old (old) PAN and set it to maximum, which helped, but back then, it was only a gigabyte, which still wasn't enough. That's why I requested that the max be raised to at least 4 gigs and was very happy to see you raise it to 20 gig, must have been about 0.13 or so I'd guess. Of course, with broadband speeds increasing and full DVD ISOs now commonly traded, that 20 gig is only about 4 DVDs worth of content, so even that's not likely to be enough for those serious DVD downloaders. Personally, I'd consider making it a terabyte or similar, which should stave off requests for upping it, for awhile anyway. <g> I should also mention that I almost dumped PAN due to this "bug", before I figured out what was happening. Setting up your newsclient to download several gigs of content before you go to work, returning after work expecting to find it all ready and waiting, only to find less than 1% actually still there, is a STRONG signal the newsclient you are testing is NOT going to workout! =8^( While complaints haven't been common on the list, I'm not the only one who has come across this, as I recall explaining the situation to a couple of others over the years who've bothered to complain about PAN's not retaining stuff they downloaded, on the user list. I'd guess that's the tip of the iceberg in terms of the number that have actually tried PAN and simply moved on to something else, when it appeared not to be able to even reliably keep track of what it was downloading. That's the first problem. The second problem is eventually going to hit people even if they save binaries directly, rather than downloading to cache as I prefer and processing later. With PAN headed for better multi-server support, it's going to be used for binaries more and more often, and we're just heading for a collision with this one if something isn't changed to fix it. Ask yourself what happens to the two months of saved messages configured for a text group, when someone downloads that ISO you mentioned. BAM!!! ALL GONE!!! How many text group users, certain ones of which I know actually keep a permanent archive of messages going back /years/, are going to be happy with a "feature" that autodeletes all those messages -- again, going back /years/ in some cases -- when they download just a /single/ largish (10MB) MP3? I think it should be obvious at this point where that impending crash I mentioned is coming from. =8^( Hopefully, I've explained it well enough to get at least an edit-the-config-file setting, if not a spot in preferences. Maybe, considering the text-message deletion scenario I mentioned, thought should actually be given to something seriously high (in PAN terms) as the default. A gig default would allow folks to downoload a full CD ISO with a reasonable safety margin for saved text posts. A 6 gig default would allow folks to download a 4.7 gig DVD with a reasonable safety margin, even if they never touched their cache settings. As I mentioned above, an "unlimited" option (say, a -1 in the config file) would also be of some use, where PAN wouldn't manage cache until it started running into filesystem error messages. That's perfect for those of us that like dedicated news-cache partitions, among others, tho I'd not advocate making it the default. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman in http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html _______________________________________________ Pan-users mailing list Pan-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/pan-users
