[issue24869] shlex lineno inaccurate with certain inputs

Gareth Rees added the comment: I've made a pull request. (Not because I expect it to be merged as-is, but to provide a starting point for discussion.) -- nosy: +petri.lehtinen, vinay.sajip ___ Python tracker _

[issue24869] shlex lineno inaccurate with certain inputs

Changes by Gareth Rees : -- pull_requests: +2849 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.p

[issue24869] shlex lineno inaccurate with certain inputs

Gareth Rees added the comment: Here's a patch that implements my proposal (1) -- under this patch, tokens read from an input stream belong to a subtype of str with startline and endline attributes giving the line numbers of the first and last character of the token. This allows the accurate re

[issue24869] shlex lineno inaccurate with certain inputs

Changes by Christopher Hoadley : -- type: -> behavior versions: +Python 3.6 ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue24869] shlex lineno inaccurate with certain inputs

Gareth Rees added the comment: A third alternative: 3. Add a method whose effect is to consume comments and whitespace, but which does not yield a token. You could then call this method, and then look at shlex.lineno, which will be the line number of the first character of the next token (if

[issue24869] shlex lineno inaccurate with certain inputs

Gareth Rees added the comment: Just to restate the problem: The use case is that when emitting an error message for a token, we want to include the number of the line containing the token (or the number of the line where the token started, if the token spans multiple lines, as it might if it's

[issue24869] shlex lineno inaccurate with certain inputs

Christopher Hoadley added the comment: This problem makes it impossible to use shlex to parse commands where a newline is intended to separate commands. In the attached sample script, I created two input strings with the same tokens in the same order: the only difference is newlines and space

[issue24869] shlex lineno inaccurate with certain inputs

New submission from Robert Escriva: The newlines calculated by the shlex module are inaccurate for certain inputs with comments inline. I've attached a simple script that illustrates the problem. My assumption here is that the lineno is supposed to match a line related to the current token.