Re: [Mailman-Users] A scrubber issue

2006-12-09 Thread Todd Zullinger
I wrote: > Tokio Kikuchi wrote: [...] >> But as to the default charset is 'us-ascii' problem, if we put the >> part together the parts, some language text (like japanese) become >> irreversibly unreadable. It is safe to keep it in a separate file >> if you can't archive the whole message in multip

Re: [Mailman-Users] A scrubber issue

2006-12-09 Thread Todd Zullinger
Tokio Kikuchi wrote: > Folks, > > I'm OK with changing the recomposing part in Scrubber.py: > > if not part or part.is_multipart(): > continue > > to > > if part.is_multipart(): > continue > > It looks like the email package is more robust than it was when the > bug report was issued

Re: [Mailman-Users] A scrubber issue

2006-12-09 Thread Todd Zullinger
Mark Sapiro wrote: > Todd Zullinger wrote: [...] >>Poking in the email package (on python 2.4.4) shows: >> >>def get_content_charset(self, failobj=None): >>"""Return the charset parameter of the Content-Type header. >> >>The returned string is always coerced to lower case. If t

Re: [Mailman-Users] A scrubber issue

2006-12-09 Thread Todd Zullinger
Mark Sapiro wrote: > In another reply, I suggested a simpler change Yeah, I should have just waited a little longer before posting a half-baked diff. :-) > In order to fix the bug we really only need to skip parts with payload > = None so if we want to keep the 'empty' part, the fix should be >

Re: [Mailman-Users] A scrubber issue

2006-12-09 Thread Tokio Kikuchi
Folks, I'm OK with changing the recomposing part in Scrubber.py: if not part or part.is_multipart(): continue to if part.is_multipart(): continue It looks like the email package is more robust than it was when the bug report was issued and the Scrubber code was patched. But as to t

Re: [Mailman-Users] A scrubber issue

2006-12-09 Thread Mark Sapiro
Todd Zullinger wrote: > >Related to the second part of Werner's message being scrubbed with the >message: > >An embedded and charset-unspecified text was scrubbed... > >Poking in the email package (on python 2.4.4) shows: > >def get_content_charset(self, failobj=None): >"""Return th

Re: [Mailman-Users] A scrubber issue

2006-12-09 Thread Mark Sapiro
Todd Zullinger wrote: > >--- Scrubber.py~2006-10-01 16:28:57.0 -0400 >+++ Scrubber.py 2006-12-09 11:41:25.0 -0500 >@@ -334,7 +334,12 @@ > text = [] > for part in msg.walk(): > # TK: bug-id 1099138 and multipart >-if not part or part.is

Re: [Mailman-Users] A scrubber issue

2006-12-09 Thread Mark Sapiro
Tokio Kikuchi wrote: > >This program prints out like this: > >multipart/mixed >None >text/plain >text/plain >Another text with a header > >Note that 'A message without header' is not printed out. > >If I remove 'If p:' and print the payload unconditionally, I get: > >multipart/mixed >None >text/pla

Re: [Mailman-Users] A scrubber issue

2006-12-09 Thread Todd Zullinger
Related to the second part of Werner's message being scrubbed with the message: An embedded and charset-unspecified text was scrubbed... Poking in the email package (on python 2.4.4) shows: def get_content_charset(self, failobj=None): """Return the charset parameter of the Conten

Re: [Mailman-Users] A scrubber issue

2006-12-09 Thread Todd Zullinger
Tokio Kikuchi wrote: > It looks like the problem is something to do with email package > behavior. Here is a test code to reproduce the problem: That's neat. I like how you can do if not p: print p and it'll print out "A message without header". :) Would an (ugly, amateurish) work-around

Re: [Mailman-Users] A scrubber issue

2006-12-08 Thread Tokio Kikuchi
Hi all, Mark Sapiro wrote: > Todd Zullinger wrote: >> Attached is the original message from the list mbox and one that I >> munged up to included a content-type: text/plain; charset=3Dus-ascii. > > > I see the symptom with the original message. I'll look further, but it > seems to have something

Re: [Mailman-Users] A scrubber issue

2006-12-08 Thread Mark Sapiro
Todd Zullinger wrote: > >Attached is the original message from the list mbox and one that I >munged up to included a content-type: text/plain; charset=3Dus-ascii. I see the symptom with the original message. I'll look further, but it seems to have something to do with the fact that the first sub-

Re: [Mailman-Users] A scrubber issue

2006-12-08 Thread Todd Zullinger
Mark Sapiro wrote: > It shouldn't be a content filtering issue. If a part is missing a > Content-Type: header, the message methods get_content_type() and > get_content_maintype() which are used by MimeDel.py (content > filtering) return the default types which are text/plain and text > except for s

Re: [Mailman-Users] A scrubber issue

2006-12-08 Thread Mark Sapiro
Todd Zullinger wrote: > >However, I created a fresh list on my system to see whether this was a >list configuration issue or not and it's reproduceable using a default >list setup. I initially thought it must be some over-agressive >content filtering, but after having it work on a virgin list I do

Re: [Mailman-Users] A scrubber issue

2006-12-08 Thread Todd Zullinger
Hi Mark, Mark Sapiro wrote: > There seems to be another issue in that there is no 'link' to the > scrubbed part, just a relative URL which doesn't work. Yes, there are other issues with the configuration there. Click on the list info link and then follow that to the archives. You get into the t

Re: [Mailman-Users] A scrubber issue

2006-12-08 Thread Mark Sapiro
Todd Zullinger wrote: > >In honor of recent discussions on pipermail, I think I've found >another issue with archiving, though this seems to be in >Mailman.Scrubber. > >In a few recent posts to the GnuPG lists, Werner Koch sent along some >signed patches fixing issues in the gpg code. Unfortunatel

[Mailman-Users] A scrubber issue

2006-12-08 Thread Todd Zullinger
In honor of recent discussions on pipermail, I think I've found another issue with archiving, though this seems to be in Mailman.Scrubber. In a few recent posts to the GnuPG lists, Werner Koch sent along some signed patches fixing issues in the gpg code. Unfortunately, the archives ate his posts[