[issue14452] SysLogHandler sends invalid messages when using unicode

Daniel Pocock added the comment: As a workaround, Python 3.2 users can clobber the global variable codecs like this from their own init code: logging.handlers.codec = None There is a more complete example here: https://github.com/dpocock/python-rfc5424-logging-formatter -- nosy:

[issue14452] SysLogHandler sends invalid messages when using unicode

Changes by Roy Smith : -- nosy: +roysmith ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.or

[issue14452] SysLogHandler sends invalid messages when using unicode

Changes by Jody McIntyre : -- nosy: +scjody ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.

[issue14452] SysLogHandler sends invalid messages when using unicode

Roundup Robot added the comment: New changeset 603301cfb194 by Vinay Sajip in branch 'default': Closes #14452: brought tests in line with removal of BOM insertion code. http://hg.python.org/cpython/rev/603301cfb194 -- status: open -> closed ___ Pytho

[issue14452] SysLogHandler sends invalid messages when using unicode

R. David Murray added the comment: This appears to be failing on the buildbots: http://www.python.org/dev/buildbot/all/builders/x86%20OpenIndiana%203.x/builds/3358/steps/test/logs/stdio http://www.python.org/dev/buildbot/all/builders/x86%20Gentoo%20Non-Debug%203.x/builds/2037/steps/test/logs/st

[issue14452] SysLogHandler sends invalid messages when using unicode

Roundup Robot added the comment: New changeset af46a001d5ec by Vinay Sajip in branch '2.7': Issue #14452: remove BOM insertion code. http://hg.python.org/cpython/rev/af46a001d5ec New changeset 89ab589f6fa7 by Vinay Sajip in branch '3.2': Closes #14452: remove BOM insertion code. http://hg.pytho

[issue14452] SysLogHandler sends invalid messages when using unicode

Vinay Sajip added the comment: > What would the behavior be when run on an older version without support for > that placeholder be? Then it would fail when the format string contained e.g. %(UTF8BOM)s and there was no corresponding attribute in the LogRecord - but that's true of any feature

[issue14452] SysLogHandler sends invalid messages when using unicode

Gregory P. Smith added the comment: Your three step approach makes sense... But it _is_ still technically a new API though in that the UTF8BOM placeholder for LogRecord's is being introduced. What would the behavior be when run on an older version without support for that placeholder be? I'

[issue14452] SysLogHandler sends invalid messages when using unicode

Vinay Sajip added the comment: > But why on earth would one want a BOM in UTF-8-encoded data? It is > byte-order independent! Lord only knows, but the RFC does call for it - msg157572 has an actual excerpt from RFC 5424. -- ___ Python tracker

[issue14452] SysLogHandler sends invalid messages when using unicode

Tim Golden added the comment: It's used by some systems (Windows Notepad does this if you save as UTF8) to indicate that the byte stream *is* UTF8-encoded. It's not so much a BOM as a magic cookie. I can't speak for syslog, I'm afraid TJG -- nosy: +tim.golden

[issue14452] SysLogHandler sends invalid messages when using unicode

Guido van Rossum added the comment: But why on earth would one want a BOM in UTF-8-encoded data? It is byte-order independent! -- nosy: +gvanrossum ___ Python tracker ___

[issue14452] SysLogHandler sends invalid messages when using unicode

Vinay Sajip added the comment: I have a possible suggestion about how to resolve this issue: The SysLogHandler will not do BOM insertion unless the message is Unicode. If it is Unicode, it will add the attribute 'UTF8BOM' to the LogRecord, with the value u'\ufeff'. The record will then be for

[issue14452] SysLogHandler sends invalid messages when using unicode

Changes by Vinay Sajip : -- assignee: -> vinay.sajip ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://ma

[issue14452] SysLogHandler sends invalid messages when using unicode

marko kreen added the comment: Note additional brokenness in BOM addition: * It does add BOM even when msg is ascii, just because it was in unicode() string. * It does not add BOM to UTF8 string if it is already encoded to str(). So highly doubt anybody actually relies on it. And keeping such

[issue14452] SysLogHandler sends invalid messages when using unicode

Vinay Sajip added the comment: Ok, I see what the problem is. I could go for option 1 - leave the BOM out, encode the string as UTF-8 but send it as just a bunch of bytes, i.e. the MSG-ANY variant of the spec. However, this could break any existing code that doesn't use structured data before

[issue14452] SysLogHandler sends invalid messages when using unicode

marko kreen added the comment: The 'msg' in SysLogHandler does not correspond to MSG in RFC. Nor to %(message)s in log record. RFC: SYSLOG-MSG = HEADER SP STRUCTURED-DATA [SP MSG] HEADER = PRI VERSION SP TIMESTAMP SP HOSTNAME SP APP-NAME

[issue14452] SysLogHandler sends invalid messages when using unicode

Vinay Sajip added the comment: > the RFC specifies that the BOM should prefix only unstructured > message part, but current Python implementation puts it in the > middle of structured part, thus confusing RFC-compliant receivers. How do you work that out? The implementation puts the BOM before

[issue14452] SysLogHandler sends invalid messages when using unicode

Changes by R. David Murray : -- versions: -Python 2.6, Python 3.1, Python 3.4 ___ Python tracker ___ ___ Python-bugs-list mailing lis

[issue14452] SysLogHandler sends invalid messages when using unicode

Changes by R. David Murray : -- nosy: +vinay.sajip ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.

[issue14452] SysLogHandler sends invalid messages when using unicode

Changes by marko kreen : -- type: -> behavior ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pyth

[issue14452] SysLogHandler sends invalid messages when using unicode

New submission from marko kreen : SysLogHandler converts message to utf8 and adds BOM, supposedly to conform with RFC5424, but the implementation is broken: the RFC specifies that the BOM should prefix only unstructured message part, but current Python implementation puts it in the middle of stru