Thufir Hawat posted on Tue, 26 Mar 2013 02:06:39 +0000 as excerpted: > Are all articles equally "cache"-able?
In my experience, yes... with one exception that's server-side, thus out of pan's control (plus another couple that would be local admin issues, not under pan's control). See below... > http://www.zimagez.com/zimage/screenshot-03252013-070130pm.php > > I selected all read articles, cached them ok. The I selected more > articles, the unread articles you see in the screenshot, and selected to > cache them. > > Now, I have a cache set to 100MB but might not have that much RAM free. > In this case, cache means to RAM or hdd? I tried setting 1000MB but > still see this, so brought cache back to 100MB. The cache is definitely filesystem storage. I've been using a multi-gig cache for years, back long before I had multi-gig memory! (For my pan binary instance, I have pan's cache symlinked to a dedicated 12-gig partition, with pan set to use it all. For my text instance, pan is set to ~5 gigs in my home dir, with all servers set unexpiring and an actual usage after several years of under a gig, and several "private" groups from a no-longer existing ISP-server archived. While I have a 16-gig RAM machine now, IIRC when I set this up it was half a gig, and back with pre- C++-rewrite pan 0.14.x, I remember using a 4-gig cache with a quarter gig of RAM, possibly less.) At present, I believe pan keeps the overviews (basically the headers seen in the headers pane, tho that's not all headers) for all not deleted messages in memory at once, thus its scaling issues on extremely large groups, particularly for 32-bit systems where a single app's RAM can't exceed 2 GB with a normal kernel, 4 GB with the "big-mem" kconfig options applied. Additionally, pan reloads and re-threads all those messages at startup, so with my unexpiring text instance, for instance, initial (system level) cold filesystem cache loading takes several minutes -- I start it with my kde/X session and let it churn, and a few minutes later it's ready. (Once the system has warmed up its filesystem cache, restarting pan is still practically instantaneous; it's the first cold- cache load that takes the time. Also note that a file-level backup and restore does make the load time dramatically shorter, as the files are defragged as they're copied back. I imagine load times would be near half an hour here now, almost definitely at least 15 minutes, if I hadn't periodically done that over the years. Of course if I had the files on SSD instead of "spinning rust"...) > In any event, why weren't the read articles unchached so that the unread > articles could be cached? > > This is not an isolated instance, just a good screenshot. Some articles > get cached, others don't. Why/not? In the screenshot, it looks like they're all cached but one. I'd guess that one was server error when pan tried to fetch it. You can click it and see if pan loads it on demand. In my experience it seldom can, as the server has either expired it, or removed it due to message cancel, or simply lost track of it due to a bug, or whatever. It's one of the older but not the oldest message in the screenshot and I don't know your server's expiration policy, but it's worth noting that servers typically don't expire strictly by posted date, more often by received date (which you won't see), and that when they expire, it's bunches at a time, so as expiration time nears, more and more posts will be unavailable, but you'll often see posts older (by posting date) than the unfetchable ones that fetch just fine. Meanwhile, just because pan has a set cache size doesn't mean there's actually that much room in the cache partition to save the posts, and for systems with user quotas or the like, it could be the quota that your hitting, not partition size limits. Finally, filesystem damage can have some curious effects, so if you've not fscked the filesystem in awhile, try that too. But for a single post like that, I'd assign about 98% probability to it being the server's issue -- the server still had the message in its overviews when they were fetched, but when you went to fetch the message itself a bit later, it couldn't find it, and returned a "no such message" error. There's nothing pan can do about that. Meanwhile, your question asking why pan didn't uncache the read articles in ordered to cache the unread articles suggests a misunderstanding of how pan's caching works, as well. Pan was originally designed for either text-message use, or for binaries, direct-download-and-save, basically no pre-caching, at least for binaries. This is quickly evident with the default 10 MB cache size if you try downloading to cache a bunch of binary articles. I remember being caught out by this early in my pan experience, probably early in 2002, as I settled into Linux after switching from MS and MSOE in late 2001 and early 2002. I set pan to cache a bunch of messages and went away to do something else (probably work or sleep), expecting to come back and find them all downloaded and ready to process locally. But when I came back, most of them were missing! After grumbling and perhaps swearing a bit, I set about redownloading, knowing some of them would be expired now and thus unavailable. As I watched, pan downloaded a few, and deleted others that I hadn't read yet right in front of my eyes! Of course it was running into the 10 MB default cache limit, and deleting the oldest, still unread, cached messages. =:^( Once I figured that out, I set a nice big (for the time) 4 GB cache. (I had to edit the config file directly to do it as at the time pan's max cache size setting in the GUI was 1 GB. A request to bump that was one of the first bugs I filed and Charles responded by bumping the GUI-max to a then unheard of 20 GB!) You're running into the opposite effect. Having set pan's cache size to 100 MB while apparently doing only text, pan has plenty of room to cache, and won't be deleting messages, read or unread, from cache for quite some time. (I don't know how your text-group usage compares to mine, but as I said, I have several years of text-group usage cached, and last I checked, my usage was still under a gig, nowhere near the 5 gig cache size limit I set. So if your text group usage is comparable to mine and assuming you don't go doing a bunch of binaries with the same pan instance, thus hogging the cache with binaries, a 100 MB cache may be something like a year's worth of messages.) Of course if that bothers you, you can return the cache to something closer to its default 10 MB, or even reduce it to say 2 MB. But for text groups anyway, most people like to be able to go back and reread the thread context once in awhile, and local-caching few extra MB of read messages doesn't hurt on today's terabyte-scale disks, so... Of course a cache size larger than needed for your expire time doesn't help... but doesn't really hurt, either. But do be aware that the number of headers/overviews in combination with having them all cached does dramatically affect startup time. On my pan binary instance, I have no-expire set as well, and as I said, a 12-gig dedicated partition cache (actually, I think I upped it to 15 gig the last time I upgraded disks...). But I multi-select and delete messages as soon as I'm done with them, in ordered to help keep the load time down. On the text instance, I accumulate, but I start pan with X/KDE and let it churn away, too, so it's ready a few minutes later when I'm finished with checking mail, maybe playing a game of solitaire or whatever, etc. And I normally suspend-to-ram rather than shutting down the system as well, with the system-level cache staying intact, and pan is either still running or a quick restart with hot cache is near instant, so I don't really have to worry about my pan text instance cold- cache-load-time that often. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman _______________________________________________ Pan-users mailing list Pan-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/pan-users
