Thanks for double-checking this. I realized that there's a problem with my
patch: skipping the encoding step when using Python 3 could cause problems when
we use difflib.SequenceMatcher.
I don't know how to fix this, since the strings in vim.current.buffer appear to
be 'raw'. They are not encoded in any valid way. This happens with both the
python 2/3 vim interfaces. You can test this by creating a buffer with an emoji
in it ('😀') and then running:
:set fileencoding=utf8
:python import vim
:python print(len(vim.current.buffer[0]))
4
That makes us responsible for picking an encoding and using it consistently [*].
However, I don't know how to properly encode an invalid string in Python 3. Is
there some way to 'reinterpret_cast' it to a bytes object?
vedant
[*] I'd rather get rid of the encoding/decoding steps entirely, but I don't
think
this is possible, because that would prevent us from using the JSON module.
> On Dec 10, 2016, at 6:16 AM, Nico Weber <tha...@chromium.org> wrote:
>
> Shouldn't this at least use >=? (But it feels like there is probably a way
> that doesn't have to do UA sniffing, so to speak)
>
> On Dec 9, 2016 8:04 PM, "Vedant Kumar via cfe-commits"
> <cfe-commits@lists.llvm.org> wrote:
> Author: vedantk
> Date: Fri Dec 9 18:54:13 2016
> New Revision: 289308
>
> URL: http://llvm.org/viewvc/llvm-project?rev=289308&view=rev
> Log:
> [clang-format] Another attempt at python 3 compatibility
>
> The entries in vim.current.buffer appear to be decoded strings, which
> means that python3 won't allow invoking 'decode' on them. Keep the old
> behavior when running under python2, but skip the error-inducing decode
> step with python3..
>
> Modified:
> cfe/trunk/tools/clang-format/clang-format.py
>
> Modified: cfe/trunk/tools/clang-format/clang-format.py
> URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-format/clang-format.py?rev=289308&r1=289307&r2=289308&view=diff
> ==============================================================================
> --- cfe/trunk/tools/clang-format/clang-format.py (original)
> +++ cfe/trunk/tools/clang-format/clang-format.py Fri Dec 9 18:54:13 2016
> @@ -29,6 +29,7 @@ from __future__ import print_function
>
> import difflib
> import json
> +import platform
> import subprocess
> import sys
> import vim
> @@ -48,10 +49,15 @@ fallback_style = None
> if vim.eval('exists("g:clang_format_fallback_style")') == "1":
> fallback_style = vim.eval('g:clang_format_fallback_style')
>
> +def get_buffer(encoding):
> + if platform.python_version_tuple()[0] == '3':
> + return vim.current.buffer
> + return [ line.decode(encoding) for line in vim.current.buffer ]
> +
> def main():
> # Get the current text.
> encoding = vim.eval("&encoding")
> - buf = [ line.decode(encoding) for line in vim.current.buffer ]
> + buf = get_buffer(encoding)
> text = '\n'.join(buf)
>
> # Determine range to format.
>
>
> _______________________________________________
> cfe-commits mailing list
> cfe-commits@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
_______________________________________________
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
