Thanks for the detailed suggestions, comments below.
Duncan wrote:
Glen Walpert posted on Fri, 08 Apr 2011 10:33:33 -0400 as excerpted:
I have been using Pan 0.132 on Ubuntu 8.04 for a year with no problems
until a few days ago when all posts to sci.electronics.design started
being ignored ("get headers in subscribed groups" lists over 100
messages but they all disappear when the group is selected, no headers
for the last 3 days even when no filters are selected so they are not
just marked as read.)
All of my other subscribed groups have no problems. If it is an issue
with my news server (news.astraweb.com) it does not affect an old copy
of Agent on XP, which still reads new messages on
sci.electronics.design. I tried temporarily removing .pan2/Score with
no effect (not a bad scoring rule) and reproduced the problem on a clean
install of Pan 0.133 under Ubuntu 9.04 with no scoring rules. What else
should I look for?
First, note that pan 0.132 and previous had a security vulnerability that
was fixed in 0.133. pan is obscure enough it's unlikely to be a problem
unless someone's specifically targeting you, but you really want to
upgrade that 0.132 instance so you're no longer exposed.
It's also worth noting that 0.134 is out now, with various mostly bugfix
patches that have been floating around for awhile. You may wish to
consider upgrading to it, as it's possible the issue you are seeing is one
of those bugs (see below for details). But for sure you want at least
0.133, due to that security issue fix!
OK, I will compile and install 0.134 as soon as I can finish some other
work, and let you know the results.
As to your problem, there are three possibilities here. Well, four
actually. I added this top (0) one later, on top, since it's easiest to
deal with and get out of the way.
Zeroeth, easy to test but too vaguely defined to know a fix immediately,
report the results for further investigation. What happens if you use the
get headers... dialog, and select get ALL headers? It's usual that pan
might fill in the occasional missed post here or there, and they may or
may not appear read, but does get all headers get the new ones that get
headers in subscribed groups missed, or not? If it does, report that
behavior and perhaps we can figure out what's happening. If not, well, we
know it doesn't help, now. The reasoning here is that the get all headers
function uses somewhat different methods to request them than the other
options do, and it can at times get stuff that isn't picked up using the
normal fetch new headers functionality. The problem in that case may well
be a peculiarity of the server behavior, or a pan bug, or... but knowing
one way or the other is more data on it than we have now.
Testing with 0.133 now:
Get all headers retrieves a lot of headers, but does not get all the
missing headers, and they are all marked as read! Even stranger, if I
wait a while and get new headers (without deselecting
sci.electronics.design), some new headers are retrieved, but they only
show up in the headers pane when "show only unread messages" is selected!
First, easiest to fix, but probably not /your/ problem (unless you copies
over the pan data sans scorefile from the old install to the clean
install), and simply listed here for completeness, pan's read-message
tracking depends on the server keeping its per-group message sequence
numbers in order. If the server starts its numbering over, as it might do
if it had a storage problem for the group and restored from a backup, or
if you switch servers by just giving pan a new server address for an
existing server config, instead of creating a new server config for the
new server, the new message sequence numbers may appear to pan to be for
old messages it long since expired, instead of the new ones they actually
are.
The fix for that is to either create a fresh server config so pan starts
with a fresh idea of what the server message sequence numbers should be,
OR, with pan not running naturally, to manually text-edit the appropriate
newsrc file for that server (for multi-server people, the servers.xml file
tracks which server uses which newsrc file), deleting pan's tracking of
read-messages for that group, or simply delete that newsrc file entirely,
if losing that info for all groups for that server isn't a problem.
The fresh server config (I even uninstalled pan, deleted the .pan2
directory, and reinstalled with a new manual config) does not affect the
problem.
Second, the problem alluded to above, fixed by an upgrade to 0.134. The
problem here is that in huge groups, typically binary groups with tens of
thousands of individual daily articles (a full ISO binary post may be
split over thousands of individual articles for just that one post!), the
message sequence numbers exceed the maximum number (a bit under 4.3
billion) that can be recorded with a 32-bit number! 32-bit pan especially
(I'm not sure about 64-bit pan, BTW not /just/ pan, it was a problem for a
number of news clients) used to use a 32-bit int for tracking message
sequence numbers.
The patch alluded to above switched pan to using a 64-bit number for the
same thing. That's not going to run out in the foreseeable future! But
that patch didn't make it into 0.133, only 0.134. Thus, the fix in this
case is to upgrade to 0.134. However, a sci.* group doesn't normally
allow binaries (does it?) and as such, is quite unlikely to have surpassed
the 32-bit limit of some 4 billion plus posts, in which case this patch
won't solve the problem you are seeing.
There are less than 200,000 posts to sci.electronics.design on astraweb.
Third... not so good news. There has been a longstanding but quite
obscure bug in pan for quite some time, related in some way to cross-
posts. The bug is obscure enough not to affect most folks, and I'd be
unsure it was even there, but for the fact that I have it myself!
Unfortunately, it's obscure enough and complicated enough nobody has been
able to trace down the problem, and it remains unfixed.
What I know about the problem so far is that it is somehow triggered by
crossposts to multiple groups. I /think/ it only happens when one group
involved in the cross-post gets ahead of another, perhaps you don't visit
or update the behind group for some days, or perhaps you're just adding it
as a new subscription after seeing sufficient cross-post activity to
become interested in the other group (my case here). Somehow, something
about the cross-posts gets pan mixed up, with the result being,
apparently, a case of the first possibility above -- pan somehow records
way high message sequence numbers for the one group, perhaps recording the
numbers for the first group cross-posted to, to the second instead of
using the lower numbers of the second group. As such, it then believes
any new messages it sees are in fact years old and refuses to download
them!
If you are seeing this rather obscure behavior, I'm sad for you, but glad,
in that we now have another data-point to compare against mine, and maybe
some progress can finally be made on this thing!
Two questions should help prove this the case or not:
1) Does the problem group get cross-posts from another group you regularly
visit, or may have visited in the past? (If no such cross-posts are
involved, it's unlikely you're experiencing this same bug.)
Some users of sci.electronics.design do inexplicably routinely crosspost
text only messages to alt.binaries.schematics.electronics, another group
I follow, so this may be the crosspost related bug.
2) If you start a clean pan instance (say by moving the ~/.pan2 directory
elsewhere, when pan's not running of course, so it has to start a new
one), setup the server, and ONLY visit the problem group, WITHOUT visiting
anything else first, so the ONLY data it has is from that problem group,
the group should load just fine. However, start trying to bring the saved
data from the other groups back in and the problem group may go
unresponsive again. The existing posts will remain (until they expire
anyway), but pan will refuse to update the group any further. (This is
what I see here. Again, if with a completely clean pan instance, with the
problem group the FIRST you visit so no history or possibility at all for
the cross-posted messages to interfere since the other group hasn't been
visited yet, if it does NOT load just fine, you're seeing a different
problem.)
I got exactly these results, but strangely the problem re-occurs when I
load other groups not including the cross-post suspect
alt.binaries.schematics.electronics. I am not sure if the problem would
re-occur in a few uses even with no other groups visited yet since I
added new groups as soon as it appeared that sci.electronics.design was
working, I will test that later, after taking care of some other work.
Thanks,
Glen
_______________________________________________
Pan-users mailing list
Pan-users@nongnu.org
http://lists.nongnu.org/mailman/listinfo/pan-users
