Jim Henderson posted on Thu, 01 Mar 2012 21:32:44 +0000 as excerpted: > I've got a handful of servers I subscribe to groups from. About once > every day or two, with random groups (which one varies, and all of them > have been hit from one time to another), Pan will decide that instead of > using the headers in its cache, it's going to pull all the headers for > that group. > > I observed this behaviour before I started using multiple machines, but > it wasn't a huge issue - but now I sync the .pan2 and News directories > to other machines using Dropbox, so having Pan decide to download all > headers means that the group's file in the .pan2/articles directory > grows to a fairly large size - which affects how much space is used on > my Dropbox account and how long it takes to sync the data. > > I've been working around the issue by killing the task, deleting the > group's headers, and telling Pan to download 14 days' worth of headers. > But it's somewhat annoying to have to do that, so I thought I'd ask. > > I'm running Pan 0.135 on openSUSE 12.1. I also observed this behaviour > on earlier 0.13x releases on earlier releases of openSUSE (back to at > least 11.2).
I had this marked for later reply... but didn't think it'd be /this/ much later! However, despite the time, hitting view-all (including read), I see no other responses. Maybe this will bring some. I don't believe I've ever seen this sort of behavior (since before pan 0.90) and I'm trying to think what might cause it. The way this normally works, with most news clients including pan, is that a server sequentially numbers the posts in each group, and when updating a group, a reply with three numbers, low water mark (the oldest message still around, lowest number), high water mark (the newest, highest number message), and an ESTIMATE of the number of messages available, will be given. The client then compares its old high water mark with the new one and assuming the low water mark on the server overlaps the old high water mark at the client (no gap), the client requests all message headers (as overviews) between the old high water mark and the new one. If the user hasn't checked in in awhile and the server has expired some messages between the old high water mark and its new low water mark, the client knows to only get the messages between the current low and high water marks. OK, that's the general process. It's also worth knowing that pan tracks the message sequence numbers it has seen in the newsrc files, one for each server, since the newsrc format assumed only a single server. If you look in this file, you'll see the ranges of numbers that pan says it has already seen, so doesn't need to request again, for each group you've ever downloaded headers for. OK, that's where pan keeps track of those numbers. What would make pan try to redownload all headers, instead of just the ones since it last downloaded? Somehow, that tracking is getting out of sync. The most common way it gets out of sync is if you transfer servers and try to use the same sequence numbering, since each server will naturally have its own sequencing for a particular group. This can also happen if a server's numbering gets "reset" for some reason or other, maybe because it lost its own records and got rebuilt. If the new server's numbers are LOWER than the ones the client was tracking, then the client believes it has already seen everything, so won't pull new headers. That's when you force it to pull all headers, or you setup a new server instead of trying to use the old one, or you either delete the newsrc or edit it to remove its tracking for just that group. If the new server's numbers are HIGHER, then obviously, the client will think all the messages are new, and try to download all the headers available, back either to the client's old high water mark, or the server's current low water mark, whichever is fewer messages. You mention that you're syncing between different machines. If you sync newsrcs, but NOT servers.xml, AND you use the default newsrc numbering (that is, you didn't rename the newsrcs in servers.xml and the filename to reflect the server name instead of an arbitrary number, as I did here), it could be that the servers were added in a different order on the one machine. IDR exactly what the default newsrc names are, but it's something like newsrc-1, newsrc-2, etc. Now if newsrc-1 corresponds to server A on machine 1, but server A has newsrc2 on machine 2, then syncing them WITHOUT syncing the servers.xml file that maps one to the other as well, will screw you up, because pan will now believe the per- server per-group sequential message numbering belonging to one server/ group actually belongs to a different one! That's one possibility. Another would be that something's corrupting or deleting the newsrcs on one or more of your machines, such that pan's losing track of which messages it has actually seen, and thus when told to download all new, downloads all of them. Yet another possiblity, probably remote, is that one or more of your servers is having bad enough problems to lose its own message sequencing on a group once in awhile. A variant of that one would be if the server is actually a server farm, probably served round-robin, and one of say five or ten servers is bad, out of sync with the rest. Every time you happen to get the bad server... (This one's actually reasonable and I've known it to happen with Highwinds based servers, in particular. One bad front-end in their round-robin balancing can be a real frustration! What can make it worse is that nobody likes the bad server, so it always looks as if it has less load than the others, and if they're using load stats rather than pure round-robin the odds of getting the bad front-end thus go up dramatically! ) Yet another variant would be "cheap" providers like the old newsfeeds used to do. They had lots of servers in various places, but they weren't coordinated or number-synchronized, so if you used one one day and another a different day, you pretty much HAD to use message-IDs, not message sequence numbering AT ALL. Anyway, maybe you've fixed it by now, maybe not, but those are the triggers I came up with in the few minutes I was thinking about it writing this post. If it's not fixed by now, see if any of those match what you are seeing... -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman _______________________________________________ Pan-users mailing list Pan-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/pan-users
