builtins confusion
For some reason I find my windows 32 bit python 2.7.13 has a package called builtins installed in site-packages. This creates
confusion about how to look at builtins.
The __init__.py looks like this
from __future__ import absolute_import
import sys
__future_module__ = True
if sys.version_info[0] < 3:
from __builtin__ import *
# Overwrite any old definitions with the equivalent future.builtins ones:
from future.builtins import *
else:
raise ImportError('This package should not be accessible on Python 3. '
'Either you are trying to run from the python-future src
folder '
'or your installation of python-future is corrupted.')
this creates complete confusion about how to import __builtin__ versus builtins. I think this is associated in some way with
pyinstaller, but am not sure.
Is this a reasonable way to import either builtins / __builtin__
try:
import __builtin__
except ImportError:
import builtins as __builtin__
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
Re: Asyncio tasks getting cancelled
On 05/11/2018 07:55, Ian Kelly wrote: >> I assume it's kind of a run_forever() with some code before it >> to schedule the coroutine. > > My understanding of asyncio.run() from > https://github.com/python/asyncio/pull/465 is that asyncio.run() is > more or less the equivalent of loop.run_until_complete() with > finalization, not loop.run_forever(). Thus this result makes sense to > me: asyncio.run() runs until main() finishes, then stops. Without the > sleep(2), main() starts creates the foobar() task and then returns > without awaiting it, so the sleep(1) never finishes. asyncio.run() > also finalizes its event loop, so I assume that the loop being > run_forever must be a different event loop since running it doesn't > just raise an exception. Therefore it doesn't resume the foobar() > coroutine since it doesn't know about it. That totally make sense. I agree with this supposition. > If the goal here is for the task created by main() to complete before > the loop exits, then main() should await it, and not just create it > without awaiting it. I said the following: > Usually, when you're already running into a coroutine, you're using > "await other_coroutine()" instead of > "asyncio.create_task(other_coroutine())". Which is not accurate. What Ian said is accurate. One indeed may need to schedule a coroutine in some situation. I just assumed that's wasn't what the OP intended to do given the name of the "main" function. I also said: > But anyway, I highly recommend you to use the "await other_coroutine()" > syntax I talked about earlier. It may even fix the issue (90% chance). This should indeed fix the issue, but this is definitely not what one is looking for if one really want to _schedule_ a coroutine. - Léo -- https://mail.python.org/mailman/listinfo/python-list
Re: builtins confusion
On 2018-11-05 10:47, Robin Becker wrote:
> raise ImportError('This package should not be accessible on Python 3. '
> 'Either you are trying to run from the python-future
> src folder '
> 'or your installation of python-future is corrupted.')
It would appear that this is from the python-future package.
The PyPI page makes the "why?" fairly clear:
https://pypi.org/project/future/
--
https://mail.python.org/mailman/listinfo/python-list
Re: builtins confusion
On 05/11/2018 10:00, Thomas Jollans wrote:
On 2018-11-05 10:47, Robin Becker wrote:
raise ImportError('This package should not be accessible on Python 3. '
'Either you are trying to run from the python-future src
folder '
'or your installation of python-future is corrupted.')
It would appear that this is from the python-future package.
The PyPI page makes the "why?" fairly clear:
https://pypi.org/project/future/
yes I think it comes from pefile which pyinstaller uses. Problem is that this module just hides an import error and may change
other people's attempts at feature recognition.
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list
Re: gmpy2 problem
Sorry Rhodri, my gmail set your email as default recipient when I was
responding, I forgot to change that. Sorry for that
Regards
On Mon, 5 Nov 2018 at 13:06, Rhodri James wrote:
> Jacob, please reply to the list, not to me personally. As I said, I
> have no idea about gmpy2 or pip, I just know how to google.
>
> On 01/11/2018 23:28, jacob m wrote:
> > I tried to install libgmp3-dev, but it didn't solved my problem
> >
> > On Thu, 1 Nov 2018 at 18:44, Rhodri James wrote:
> >
> >> Replying to the list since I have absolutely no idea about gmpy2 or
> pip...
> >>
> >> On 01/11/2018 16:56, jacob m wrote:
> >>> Hi Rodhri, of course I can paste the text :)
> >>> "
> >>> creating build
> >>> creating build/temp.linux-x86_64-3.7
> >>> creating build/temp.linux-x86_64-3.7/src
> >>> gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g
> -fwrapv
> >> -O3
> >>> -Wall -fPIC -DWITHMPFR -DWITHMPC
> -I/home/lib/python3.7/include/python3.7m
> >>> -c src/gmpy2.c -o build/temp.linux-x86_64-3.7/src/gmpy2.o
> >>> In file included from src/gmpy2.c:426:0:
> >>> src/gmpy.h:106:12: fatal error: gmp.h: No such file or directory
> >>># include "gmp.h"
> >>> ^~~
> >>> compilation terminated.
> >>> error: command 'gcc' failed with exit status 1
> >>>
> >>>
> >>> Command "/home/lib/python3.7/bin/python3.7 -u -c "import setuptools,
> >>>
> >>
> tokenize;__file__='/tmp/pip-install-kiqba_j_/gmpy2/setup.py';f=getattr(tokenize,
> >>> 'open', open)(__file__);code=f.read().replace('\r\n',
> >>> '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record
> >>> /tmp/pip-record-2jatj1mi/install-record.txt
> >>> --single-version-externally-managed --compile" failed with error code 1
> >> in
> >>> /tmp/pip-install-kiqba_j_/gmpy2/
> >>> "
> >>
> >> A little googling suggests that you need to install the libgmp3-dev
> >> package:
> >>
> >> $ sudo apt install libgmp3-dev
> >>
> >>> On Thu, 1 Nov 2018 at 16:46, Rhodri James
> wrote:
> >>>
> On 01/11/2018 14:40, jacob m wrote:
> > Hi guys
> > I have some problems with gmpy2 module installation - pip install
> gmpy2
> > doesn't work for me
> > [image: image.png]
> >
> > How to fix it?
>
> Hi Jacob,
>
> I'm afraid the screen-shot that you sent was automatically stripped
> off
> by the mailing list. Could you copy the error text (preferably
> cut-and-paste the whole thing rather than retyping!)? The more
> information you can give us, the better the advice will be from the
> people who actually understand pip :-)
> >>
> >>
> >> --
> >> Rhodri James *-* Kynesim Ltd
> >> --
> >> https://mail.python.org/mailman/listinfo/python-list
> >>
> >
>
>
> --
> Rhodri James *-* Kynesim Ltd
>
--
https://mail.python.org/mailman/listinfo/python-list
Re: gmpy2 problem
Jacob, please reply to the list, not to me personally. As I said, I
have no idea about gmpy2 or pip, I just know how to google.
On 01/11/2018 23:28, jacob m wrote:
I tried to install libgmp3-dev, but it didn't solved my problem
On Thu, 1 Nov 2018 at 18:44, Rhodri James wrote:
Replying to the list since I have absolutely no idea about gmpy2 or pip...
On 01/11/2018 16:56, jacob m wrote:
Hi Rodhri, of course I can paste the text :)
"
creating build
creating build/temp.linux-x86_64-3.7
creating build/temp.linux-x86_64-3.7/src
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv
-O3
-Wall -fPIC -DWITHMPFR -DWITHMPC -I/home/lib/python3.7/include/python3.7m
-c src/gmpy2.c -o build/temp.linux-x86_64-3.7/src/gmpy2.o
In file included from src/gmpy2.c:426:0:
src/gmpy.h:106:12: fatal error: gmp.h: No such file or directory
# include "gmp.h"
^~~
compilation terminated.
error: command 'gcc' failed with exit status 1
Command "/home/lib/python3.7/bin/python3.7 -u -c "import setuptools,
tokenize;__file__='/tmp/pip-install-kiqba_j_/gmpy2/setup.py';f=getattr(tokenize,
'open', open)(__file__);code=f.read().replace('\r\n',
'\n');f.close();exec(compile(code, __file__, 'exec'))" install --record
/tmp/pip-record-2jatj1mi/install-record.txt
--single-version-externally-managed --compile" failed with error code 1
in
/tmp/pip-install-kiqba_j_/gmpy2/
"
A little googling suggests that you need to install the libgmp3-dev
package:
$ sudo apt install libgmp3-dev
On Thu, 1 Nov 2018 at 16:46, Rhodri James wrote:
On 01/11/2018 14:40, jacob m wrote:
Hi guys
I have some problems with gmpy2 module installation - pip install gmpy2
doesn't work for me
[image: image.png]
How to fix it?
Hi Jacob,
I'm afraid the screen-shot that you sent was automatically stripped off
by the mailing list. Could you copy the error text (preferably
cut-and-paste the whole thing rather than retyping!)? The more
information you can give us, the better the advice will be from the
people who actually understand pip :-)
--
Rhodri James *-* Kynesim Ltd
--
https://mail.python.org/mailman/listinfo/python-list
--
Rhodri James *-* Kynesim Ltd
--
https://mail.python.org/mailman/listinfo/python-list
Re: Asyncio tasks getting cancelled
This weird mixing was actually a side effect of me quickly coming up with a small example, sorry for the confusion. I just saw, actually using the same loop gets rid of the behavior in this case and now I'm not sure about my assertions any more. Yet it still looks like asyncio doen'st keep strong references. Hmm. The original problem prompting my question is here: https://github.com/ldo/dbussy/issues/13 Also this is a python 3.7.1 on a current Fedora 29. > > Usually, when you're already running into a coroutine, you're using > > "await other_coroutine()" instead of > > "asyncio.create_task(other_coroutine())". > > Which is not accurate. What Ian said is accurate. One indeed may need to > schedule a coroutine in some situation. I just assumed that's wasn't > what the OP intended to do given the name of the "main" function. > > I also said: > > > But anyway, I highly recommend you to use the "await other_coroutine()" > > syntax I talked about earlier. It may even fix the issue (90% chance). > > This should indeed fix the issue, but this is definitely not what one is > looking for if one really want to _schedule_ a coroutine. Which is what I want in this case. Scheduling a new (long-running) task as a side effect, but returning early oneself. The new task can't be awaited right there, because the creating one should return already. > If the goal here is for the task created by main() to complete before > the loop exits, then main() should await it, and not just create it > without awaiting it. So if this happens somewhere deep in the hirarchy of your application you would need some mechanism to pass the created tasks back up the chain to the main function? Greetings. -- https://mail.python.org/mailman/listinfo/python-list
Re: Overwhelmed by the Simplicity of Python. Any Recommendation?
On 11/4/18 8:55 AM, Mike C wrote: Same here. Debugging in Python is annoying, I like to step through my code line by line, it's impossible to do it with object-oriented programming language. Also, there's no good REPL IDE. Spyder barely works with some basic features. PyCharm, the most popular, takes too long to start, and you have to setup folders and directories EVERY SINGLE TIME at startup. I've never been a fan of IDEs, but a code editor window on the left and the IPython QtConsole on the right is a pretty efficient way to blaze through code. -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. -- https://mail.python.org/mailman/listinfo/python-list
Re: Asyncio tasks getting cancelled
On 05/11/2018 16:38, [email protected] wrote: > I just saw, actually > using the same loop gets rid of the behavior in this case and now I'm > not sure about my assertions any more. It's fixing the issue because you're running loop with the run_forever(). As Ian and myself pointed out, using both asyncio.run() and loop.run_forever() is not what you are looking for. > Yet it still looks like asyncio > doen'st keep strong references. You may want to look at PEP 3156. It's the PEP defining asyncio. Ian made a good explanation about why your loop wasn't running the coroutine scheduled from main(). >> This should indeed fix the issue, but this is definitely not what one is >> looking for if one really want to _schedule_ a coroutine. > > Which is what I want in this case. Scheduling a new (long-running) task > as a side effect, but returning early oneself. The new task can't be > awaited right there, because the creating one should return already. > >> If the goal here is for the task created by main() to complete before >> the loop exits, then main() should await it, and not just create it >> without awaiting it. > > So if this happens somewhere deep in the hirarchy of your application > you would need some mechanism to pass the created tasks back up the > chain to the main function? Sorry, I don't get what is you're trying to express. Either you await for a coroutine, which means the calling one is "paused" until the awaited coroutine finish its job, either you schedule a coroutine, which means it will be ran once the already stacked events/coroutines on the loop are all ran. In your case, you probably want to change > loop = asyncio.get_event_loop() > asyncio.run(main()) > loop.run_forever() to > loop = asyncio.get_event_loop() > loop.create_task(main()) > loop.run_forever() - Léo [1] https://www.python.org/dev/peps/pep-3156/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Asyncio tasks getting cancelled
On Mon, Nov 5, 2018 at 8:41 AM wrote: > > > But anyway, I highly recommend you to use the "await other_coroutine()" > > > syntax I talked about earlier. It may even fix the issue (90% chance). > > > > This should indeed fix the issue, but this is definitely not what one is > > looking for if one really want to _schedule_ a coroutine. > > Which is what I want in this case. Scheduling a new (long-running) task > as a side effect, but returning early oneself. The new task can't be > awaited right there, because the creating one should return already. If you want to do this in the asyncio.run main coroutine, then that seems like a problematic design. Once the main coroutine returns, the event loop should be considered no longer running, and any still pending callbacks or futures won't resolve. > > If the goal here is for the task created by main() to complete before > > the loop exits, then main() should await it, and not just create it > > without awaiting it. > > So if this happens somewhere deep in the hirarchy of your application > you would need some mechanism to pass the created tasks back up the > chain to the main function? I haven't used asyncio.run yet myself, so take all this with a grain of salt, but it seems to me that anything that you want to resolve before the event loop terminates should be awaited either directly or indirectly by the main coroutine. From the documentation: """ This function always creates a new event loop and closes it at the end. It should be used as a main entry point for asyncio programs, and should ideally only be called once. """ So I think part of the idea with this is that the asyncio.run main coroutine is considered the main function of your async app. Once it returns, the program should be effectively done. For example, maybe the main coroutine spins up a web server and returns when the web server shuts down. If that doesn't suit your program, for instance there's no core task to await, but you want to schedule a lot of things that need to resolve and that the main coroutine has no way to know about, then it may be the case that asyncio.run is not right for your use case and you should use loop.run_forever() instead. You'll still need some criteria for figuring out when to exit though, and it seems to me that whatever that is you could just bundle it up in a coroutine and await it from main. -- https://mail.python.org/mailman/listinfo/python-list
Re: Asyncio tasks getting cancelled
On Mon, Nov 05, 2018 at 01:57:56PM -0700, Ian Kelly wrote: > > Which is what I want in this case. Scheduling a new (long-running) task > > as a side effect, but returning early oneself. The new task can't be > > awaited right there, because the creating one should return already. > > If you want to do this in the asyncio.run main coroutine, then that > seems like a problematic design. Once the main coroutine returns, the > event loop should be considered no longer running, and any still > pending callbacks or futures won't resolve. This is only true for the small example I provided. In the actual code this is somewhere deep in the hirarchy. > > > If the goal here is for the task created by main() to complete before > > > the loop exits, then main() should await it, and not just create it > > > without awaiting it. > > > > So if this happens somewhere deep in the hirarchy of your application > > you would need some mechanism to pass the created tasks back up the > > chain to the main function? > > I haven't used asyncio.run yet myself, so take all this with a grain > of salt, but it seems to me that anything that you want to resolve > before the event loop terminates should be awaited either directly or > indirectly by the main coroutine. From the documentation: > > """ > This function always creates a new event loop and closes it at the > end. It should be used as a main entry point for asyncio programs, and > should ideally only be called once. > """ > > So I think part of the idea with this is that the asyncio.run main > coroutine is considered the main function of your async app. Once it > returns, the program should be effectively done. For example, maybe > the main coroutine spins up a web server and returns when the web > server shuts down. Again sorry for the confusion, but I don't think this is an issue with restarting loops, as this isn't happening in my application. For context: https://github.com/ldo/dbussy/issues/13 https://gist.github.com/tu500/3232fe03bd1d85b1529c558f920b8e43 It really feels like asyncio is loosing strong references to scheduled tasks, as excplicitly keeping them around helps. Also, the error messages I'm getting are the ones from here: https://github.com/python/cpython/blob/16c8a53490a22bd4fcde2efaf4694dd06ded882b/Lib/asyncio/tasks.py#L145 Which indicates that the tasks actually weren't even started at all? > If that doesn't suit your program, for instance there's no core task > to await, but you want to schedule a lot of things that need to > resolve and that the main coroutine has no way to know about, then it > may be the case that asyncio.run is not right for your use case and > you should use loop.run_forever() instead. You'll still need some > criteria for figuring out when to exit though, and it seems to me that > whatever that is you could just bundle it up in a coroutine and await > it from main. Though not really related with my actual problem, so getting off topic, but I can imagine an architecture where that would be "There aren't any running tasks any more." or even "Never." Also, I may be overlooking things, but I haven't found a way to add a task before calling run_forever(), as asyncio will then say the loop isn't running yet. So I'm not sure how you would jumpstart in that case. -- https://mail.python.org/mailman/listinfo/python-list
Re: Asyncio tasks getting cancelled
On Tue, Nov 06, 2018 at 12:45:03AM +0100, [email protected] wrote: > Also, I may be overlooking things, but I haven't found a way to add a > task before calling run_forever(), as asyncio will then say the loop > isn't running yet. So I'm not sure how you would jumpstart in that case. Ok, I was confused there. It doesn't work by using asyncio.create_task but it will with loop.create_task. Sorry for the noise. -- https://mail.python.org/mailman/listinfo/python-list
Re: Asyncio tasks getting cancelled
On Mon, Nov 05, 2018 at 01:57:56PM -0700, Ian Kelly wrote: > > Which is what I want in this case. Scheduling a new (long-running) task > > as a side effect, but returning early oneself. The new task can't be > > awaited right there, because the creating one should return already. > > If you want to do this in the asyncio.run main coroutine, then that > seems like a problematic design. Once the main coroutine returns, the > event loop should be considered no longer running, and any still > pending callbacks or futures won't resolve. This is only true for the small example I provided. In the actual code this is somewhere deep in the hirarchy. > > > If the goal here is for the task created by main() to complete before > > > the loop exits, then main() should await it, and not just create it > > > without awaiting it. > > > > So if this happens somewhere deep in the hirarchy of your application > > you would need some mechanism to pass the created tasks back up the > > chain to the main function? > > I haven't used asyncio.run yet myself, so take all this with a grain > of salt, but it seems to me that anything that you want to resolve > before the event loop terminates should be awaited either directly or > indirectly by the main coroutine. From the documentation: > > """ > This function always creates a new event loop and closes it at the > end. It should be used as a main entry point for asyncio programs, and > should ideally only be called once. > """ > > So I think part of the idea with this is that the asyncio.run main > coroutine is considered the main function of your async app. Once it > returns, the program should be effectively done. For example, maybe > the main coroutine spins up a web server and returns when the web > server shuts down. Again sorry for the confusion, but I don't think this is an issue with restarting loops, as this isn't happening in my application. For context: https://github.com/ldo/dbussy/issues/13 https://gist.github.com/tu500/3232fe03bd1d85b1529c558f920b8e43 It really feels like asyncio is loosing strong references to scheduled tasks, as excplicitly keeping them around helps. Also, the error messages I'm getting are the ones from here: https://github.com/python/cpython/blob/16c8a53490a22bd4fcde2efaf4694dd06ded882b/Lib/asyncio/tasks.py#L145 Which indicates that the tasks actually weren't even started at all? > If that doesn't suit your program, for instance there's no core task > to await, but you want to schedule a lot of things that need to > resolve and that the main coroutine has no way to know about, then it > may be the case that asyncio.run is not right for your use case and > you should use loop.run_forever() instead. You'll still need some > criteria for figuring out when to exit though, and it seems to me that > whatever that is you could just bundle it up in a coroutine and await > it from main. Though not really related with my actual problem, so getting off topic, but I can imagine an architecture where that would be "There aren't any running tasks any more." or even "Never." Also, I may be overlooking things, but I haven't found a way to add a task before calling run_forever(), as asyncio will then say the loop isn't running yet. So I'm not sure how you would jumpstart in that case. -- https://mail.python.org/mailman/listinfo/python-list
Re: Asyncio tasks getting cancelled
On Mon, Nov 5, 2018 at 6:20 PM wrote: > Again sorry for the confusion, but I don't think this is an issue with > restarting loops, as this isn't happening in my application. Sure, I wasn't talking about restarting loops so much as strategies for making sure that everything has completed in the first place. > For context: > https://github.com/ldo/dbussy/issues/13 > https://gist.github.com/tu500/3232fe03bd1d85b1529c558f920b8e43 > > It really feels like asyncio is loosing strong references to scheduled > tasks, as excplicitly keeping them around helps. Also, the error > messages I'm getting are the ones from here: > https://github.com/python/cpython/blob/16c8a53490a22bd4fcde2efaf4694dd06ded882b/Lib/asyncio/tasks.py#L145 > Which indicates that the tasks actually weren't even started at all? No, it indicates that it was cleaned up (likely because the program exited) before the task completed. Which likely implies that the loop exited without waiting for it. From the stack trace, you're using loop.run_until_complete(main()). Like asyncio.run, that only runs until the specific thing you pass it completes, which might wait on other things or might not. Anything else that's still pending is going to be left up in the air unless you subsequently restart the same event loop. > Though not really related with my actual problem, so getting off topic, > but I can imagine an architecture where that would be "There aren't any > running tasks any more." or even "Never." Well, "never" is easy; that's just loop.run_forever(). "No more running tasks" is trickier; I'm not sure how one might do that without collecting all the tasks (at least, all the top-level tasks) oneself. Implementing a custom task factory might help. -- https://mail.python.org/mailman/listinfo/python-list
Re: Asyncio tasks getting cancelled
On Mon, Nov 05, 2018 at 07:15:04PM -0700, Ian Kelly wrote: > > For context: > > https://github.com/ldo/dbussy/issues/13 > > https://gist.github.com/tu500/3232fe03bd1d85b1529c558f920b8e43 > > > > It really feels like asyncio is loosing strong references to scheduled > > tasks, as excplicitly keeping them around helps. Also, the error > > messages I'm getting are the ones from here: > > https://github.com/python/cpython/blob/16c8a53490a22bd4fcde2efaf4694dd06ded882b/Lib/asyncio/tasks.py#L145 > > Which indicates that the tasks actually weren't even started at all? > > No, it indicates that it was cleaned up (likely because the program > exited) before the task completed. Which likely implies that the loop > exited without waiting for it. From the stack trace, you're using > loop.run_until_complete(main()). Like asyncio.run, that only runs > until the specific thing you pass it completes, which might wait on > other things or might not. Anything else that's still pending is going > to be left up in the air unless you subsequently restart the same > event loop. What I meant was, the error message is specific to futures in the 'PENDING' state. Which should be set to 'RUNNING' before any actions occur. So it appears the tasks weren't started at all. Also the cleanup happens instantly, ie before the loop exits (as the main loop has an asyncio.sleep timeout. Printing a message also clearly shows this happens way before main returns. -- https://mail.python.org/mailman/listinfo/python-list
