[issue37953] Fix ForwardRef equality checks
New submission from Dominic Littlewood <11dlittlew...@gmail.com>:
Apologies for issuing a pull request without an associated issue. I'm kind of
new to this. Nevermind, I'm making one now.
The typing module currently contains a bug where ForwardRefs change their hash
and equality once they are evaluated. Consider the following code:
import typing
ref = typing.ForwardRef('MyClass')
ref_ = typing.ForwardRef('MyClass')
class MyClass:
def __add__(self, other: ref): ...
# We evaluate one forward reference, but not the other.
typing.get_type_hints(MyClass.__add__)
# Equality is violated
print(ref == ref_) # False
# This can cause duplication in Unions.
# The following prints:
# typing.Union[ForwardRef('MyClass'), ForwardRef('MyClass')]
# when it should be: typing.Union[ForwardRef('MyClass')]
wrong = typing.Union[ref, ref_]
print(wrong)
# The union also does not compare equality properly
should_be_equal = typing.Union[ref]
print(should_be_equal == wrong) # False
# In fact this applies to any generic
print(typing.Callable[[ref],None] == typing.Callable[[ref_],None]) # False
--
components: Library (Lib)
messages: 350531
nosy: plokmijnuhby
priority: normal
severity: normal
status: open
title: Fix ForwardRef equality checks
type: behavior
versions: Python 3.9
___
Python tracker
<https://bugs.python.org/issue37953>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37953] Fix ForwardRef equality checks
Change by Dominic Littlewood <11dlittlew...@gmail.com>: -- keywords: +patch pull_requests: +15235 stage: -> patch review pull_request: https://github.com/python/cpython/pull/15400 ___ Python tracker <https://bugs.python.org/issue37953> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38297] Imports at top of module is often not used
New submission from Dominic Littlewood <11dlittlew...@gmail.com>: In PEP 8, it is stated that: "Imports are always put at the top of the file, just after any module comments and docstrings, and before module globals and constants." Note the word "always". This advice makes sense because it cuts down on performance costs. I recently got into an argument about this, and so I created a script to find all the times in the standard library that an import statement was used inside a function or method. I was expecting to get one or two, but it's safe to say that the answer 1576 was a little surprising. (I was on 3.7.4, if anyone wants to check.) It turns out that more than one in five modules (defined as any .py file) contain at least one violation of this rule. In the standard library, there are more violations of the rule than there are while loops. Do we need to either a) fix large parts of the standard library or b) rewrite PEP 8 to say that this might not actually be all that bad? -- components: Library (Lib) messages: 353385 nosy: plokmijnuhby priority: normal severity: normal status: open title: Imports at top of module is often not used type: performance ___ Python tracker <https://bugs.python.org/issue38297> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38297] Imports at top of module is often not used
Dominic Littlewood <11dlittlew...@gmail.com> added the comment:
I've slightly adjusted the script to check for module-level imports not at the
top of the file. If we permit things like this:
if condition:
import module
else:
do_something_sensible()
as long as they are at the top of the file, it seems that only one in ten
modules have a problem. Which is better but not great.
A common pattern which breaks the rules is this:
# Large amounts of code here
if __name__ == '__main__':
import unittest
unittest.main('test/test_thisfile')
although that by no means accounts for all of the problems.
--
___
Python tracker
<https://bugs.python.org/issue38297>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38297] Imports at top of module is often not used
Dominic Littlewood <11dlittlew...@gmail.com> added the comment: With all due respect, I disagree with what you're saying here. "It's a bit misleading to use the stdlib as a guideline" ... when looking at the style guide for the stdlib? I appreciate that PEP 8 is used for other purposes, but its main purpose is clear from its very first line. "There are reasons we do this in certain spots" Yes, there are. And given that there are so many of these different spots where it would give better performance to ignore the rule, PEP 8 should reflect that. -- ___ Python tracker <https://bugs.python.org/issue38297> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38721] modulefinder should use io.open_code() instead of open()
New submission from Dominic Littlewood <11dlittlew...@gmail.com>: modulefinder currently will detect modules imported by a script dynamically by running the script, and to do this it has to open the script as a file. This would also fix what appears to be a bug where modulefinder failed to open files in bytes mode sometimes, causing it to fail for some modules (like functools) if the wrong encoding is used. I say "appears to be", because the structure of the module seems to imply this was intended behaviour, but I can't think why anyone would want this. This is similar to two other issues I'm posting, but they're in different modules, so different bugs. -- components: Library (Lib) messages: 356143 nosy: plokmijnuhby priority: normal severity: normal status: open title: modulefinder should use io.open_code() instead of open() type: security versions: Python 3.9 ___ Python tracker <https://bugs.python.org/issue38721> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38723] Pdb._runscript should use io.open_code() instead of open()
Change by Dominic Littlewood <11dlittlew...@gmail.com>: -- components: +Library (Lib) type: -> security versions: +Python 3.9 ___ Python tracker <https://bugs.python.org/issue38723> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38722] runpy should use io.open_code() instead of open()
New submission from Dominic Littlewood <11dlittlew...@gmail.com>: Fairly obviously, if you're using something called runpy you're probably trying to run some code. To do this it has to open the script as a file. This is similar to two other issues I'm posting, but they're in different modules, so different bugs. -- components: Library (Lib) messages: 356144 nosy: plokmijnuhby priority: normal severity: normal status: open title: runpy should use io.open_code() instead of open() type: security versions: Python 3.9 ___ Python tracker <https://bugs.python.org/issue38722> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38723] Pdb._runscript should use io.open_code() instead of open()
New submission from Dominic Littlewood <11dlittlew...@gmail.com>: Fairly obviously, if you're using something called _runscript you're probably trying to run some code. To do this it has to open the script as a file. This is similar to two other issues I'm posting, but they're in different modules, so different bugs. -- messages: 356145 nosy: plokmijnuhby priority: normal severity: normal status: open title: Pdb._runscript should use io.open_code() instead of open() ___ Python tracker <https://bugs.python.org/issue38723> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38721] modulefinder should use import hooks properly
Dominic Littlewood <11dlittlew...@gmail.com> added the comment: Okay, I've started putting together a proper PR, and I've had some thoughts. There's a useful script at the bottom of the importlib documentation that readers should consult. This can be used to correctly find the spec for a module, and therefore the loader. (AddPackagePath and ReplacePackage are not needed, and should be deprecated.) For modules already on sys.path, the loader can be identified from module.__loader__. If the loader is an InspectLoader, the code can be retrieved and examined to see what is imported. (Remember to check whether None is returned.) If not, we will assume nothing is imported - which is what modulefinder currently does with extension modules. Since run_script takes a file path rather than a module name, it will have to find the appropriate loader for the task. This will be a simple choice between SourceFileLoader and SourcelessFileLoader, depending on what kind of file is being used. It is also possible to use functions in importlib._bootstrap instead. This enhancement would indirectly cause open_code to be used properly, so the bug I originally posted is redundant. -- title: modulefinder should use io.open_code() instead of open() -> modulefinder should use import hooks properly type: security -> enhancement ___ Python tracker <https://bugs.python.org/issue38721> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38721] modulefinder should use import hooks properly
Dominic Littlewood <11dlittlew...@gmail.com> added the comment: Okay, I've encountered a snag. To use meta path finders, if the path is None, it's sometimes okay to substitute self.path, and sometimes not. The only way to make absolutely sure the finder works correctly is to change sys.path. To use namespace packages (and possibly some other types of modules finders might produce), it's also necessary to adjust sys.modules. However, both of these things may cause problems if a hook wants to import something in the standard way so it can run properly. This will require some thought. -- ___ Python tracker <https://bugs.python.org/issue38721> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38721] modulefinder should use import hooks properly
Dominic Littlewood <11dlittlew...@gmail.com> added the comment: I now have a PR which - dare I say it - appears to be working as it should. I'll just write some tests for it and then I'll send it off for review. -- ___ Python tracker <https://bugs.python.org/issue38721> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38721] modulefinder should use import hooks properly
Change by Dominic Littlewood <11dlittlew...@gmail.com>: -- keywords: +patch pull_requests: +16813 stage: -> patch review pull_request: https://github.com/python/cpython/pull/17326 ___ Python tracker <https://bugs.python.org/issue38721> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38721] modulefinder should use import hooks properly
Dominic Littlewood <11dlittlew...@gmail.com> added the comment: You are correct about modulefinder being static. I read "run_script" and thought it was running a script, but it turns out I was being silly. I shall correct the sys.path issue. -- ___ Python tracker <https://bugs.python.org/issue38721> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38721] modulefinder should use import hooks properly
Dominic Littlewood <11dlittlew...@gmail.com> added the comment: In regards to the "unnecessary refactoring", here's a full list of what I've changed. 1. Previously, the module revolved around passing files and file descriptors around. Now, it uses specs. This means all the functions that used file descriptors have to be updated. 2. Due to the previous issue, several functions had to change their call signatures. The functions that called them had to change to react. 3. Packages work differently now. They do not have to have an __init__ submodule any more, so there's no need for a load_package function - it's integrated with load_module. 4. determine_parent was having trouble finding where the relative imports were relative to. I changed it to use _calc___package__, same as the import system. Half of determine_parent was deleted, because I couldn't understand how it worked and eventually worked out that there was literally no way it could ever run, so it didn't matter. 5. Modules with no __path__ attribute are not packages. Modules with a __path__ attribute of None *are* packages. Code had to be changed to reflect that. Looking back, this is possibly a pedantic enough point that these changes could be removed and hopefully nothing will go wrong. 6. fix_sys was used to update sys.path (although I have since changed it so it does not) and sys.modules. Code had to be changed to make use of it. 7. The only other thing I can think of is that I changed which modules were imported in the first few lines of modulefinder, since some had become unnecessary. (In fact types and warnings were already unnecessary, but never mind about that.) -- ___ Python tracker <https://bugs.python.org/issue38721> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
