[issue42928] adafruit_ads1x15.ads1115 not comaptible with globals()[pkg_trunc] = importlib.import_module(pkg_trunc)
New submission from John Brearley : The first attached script ada_dbg1.py.txt which uses simple hardcoded import statements, the creation of ads1115 objects work fine. The second attached script ada_dbg2.py.txt uses a loop to import a variable list of packages, which allows for more graceful error handling & messaging. globals()[pkg_trunc] = importlib.import_module(pkg_trunc) The high level observation is that board, busio & adafruit_extended_bus objects continue to be correctly created after the import loop is done. Its just adafruit_ads1x15.ads1115 that does not like this import method. The same line of code creating ads10 gets error: Traceback (most recent call last): File "ada_dbg2.py", line 33, in ads10 = adafruit_ads1x15.ads1115.ADS1115(i2c_1, address=0x48) AttributeError: module 'adafruit_ads1x15' has no attribute 'ads1115' When I look at the global variables, from ads_dbg1.py, I see: 'adafruit_ads1x15': , When I look at the global variables, from ads_dbg2.py, I see: 'adafruit_ads1x15.ads1115': , So I tried doing the import without the .ads1115 appended, & got: 'adafruit_ads1x15': , But I still get an error. Traceback (most recent call last): File "ada_dbg2.py", line 33, in ads10 = adafruit_ads1x15.ads1115.ADS1115(i2c_1, address=0x48) AttributeError: module 'adafruit_ads1x15' has no attribute 'ads1115' I went through all 8 permutations of with/without the .ads1115 in the importlib.import_module statement, no luck. Comments? -- components: Interpreter Core files: ada_dbg1.py messages: 385069 nosy: jbrearley priority: normal severity: normal status: open title: adafruit_ads1x15.ads1115 not comaptible with globals()[pkg_trunc] = importlib.import_module(pkg_trunc) type: behavior versions: Python 3.9 Added file: https://bugs.python.org/file49741/ada_dbg1.py ___ Python tracker <https://bugs.python.org/issue42928> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42928] adafruit_ads1x15.ads1115 not comaptible with globals()[pkg_trunc] = importlib.import_module(pkg_trunc)
Change by John Brearley : Added file: https://bugs.python.org/file49742/ada_dbg2.py ___ Python tracker <https://bugs.python.org/issue42928> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42928] adafruit_ads1x15.ads1115 not comaptible with globals()[pkg_trunc] = importlib.import_module(pkg_trunc)
Change by John Brearley : Added file: https://bugs.python.org/file49743/print_data.py ___ Python tracker <https://bugs.python.org/issue42928> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42928] adafruit_ads1x15.ads1115 not comaptible with globals()[pkg_trunc] = importlib.import_module(pkg_trunc)
John Brearley added the comment: Hi Guido: So the Adafruit CircuitPython forum has already said they aren't really sure this is their issue or not, see: https://github.com/adafruit/Adafruit_CircuitPython_ADS1x15/issues/66 What else would I need to do to get this looked at by people knowledgeable in importlib.import_module? Regards, John Brearley 613-259-5622 (Home) From: Guido van Rossum Sent: 2021-01-14 11:03 AM To: brear...@bell.net Subject: [issue42928] adafruit_ads1x15.ads1115 not comaptible with globals()[pkg_trunc] = importlib.import_module(pkg_trunc) Guido van Rossum added the comment: Sorry, this is not a help forum. Please try a user forum. Probably you have to find one with people using the Adafruit modules (and CircuitPython?). -- nosy: +gvanrossum resolution: -> third party stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue42928> ___ -- ___ Python tracker <https://bugs.python.org/issue42928> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31880] subprocess process interaction with IDLEX GUI causes pygnuplot silent failures
John Brearley added the comment: I retested with Python 3.6.4 upgrades and the issue no longer occurs. You may want to close this issue. -- ___ Python tracker <https://bugs.python.org/issue31880> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33000] IDLEX GUI consumes all RAM for scrollback buffer, uses 161Bytes / character stored
New submission from John Brearley :
While running a tensorflow script in the IDLEX GUI that runs for 8 million
steps and produce 2 lines stdout per step, my PC used all 16GB RAM and crashed
the python process, not to mention messed up other apps, like Firefox & Norton
AntiVirus. While the RAM was recovered, Firefox started responding, but Norton
Antivirus didn’t, so the PC had to be rebooted.
The issue is easily reproduced with the short print loop that dumps 20K lines
of stdout, at 171 characters / line on the IDLEX GUI window. When the script is
run in the IDLEX GUI, the Windows Task Manager shows the python process start
at 19MB RAM consumption, then grows to 569MB RAM consumption. If I run the
script a second time in the same IDLEX GUI window, it grows to 1.1GB RAM
consumption.
So 20K lines off output at 171 characters / line (“i: n” prefix + 2 * 80
byte string + newline) = 3.4M total characters stored in the scrollback buffer.
The delta memory consumed was 569MB – 19MB = 550MB. The RAM consumed /
character is 550MB / 3.4M = 161 bytes / character. This seems excessively
inefficient.
I now understand how the tensorflow script would stop after 550K iterations and
the 550K lines of stdout in the IDLEX GUI would consume all 16GB RAM on my PC.
BTW, when I run the same test script in the WinPython command prompt window, it
only consumes 4MB RAM while it runs. However the scrollback buffer is limited
to 10K lines, wrapped at the 80 character mark, so much less data saved.
I haven’t found any options in IDLEX GUI to limit the scrollback buffer size.
My request is to review the scrollback memory storage algorithms. If nothing
can be done to improve them, then please add a circular buffer to limit the
memory consumption.
# Print loop to test memory consumption in Python IDLEX GUI.
s1 = "0123456789"
s2 = s1+s1+s1+s1+s1+s1+s1+s1
for i in range(2):
print("i:", i, s2, s2)
I am using Python 3.6.4 on Windows 7 PC, Intel i7-4770S, 3.1GHz, 16GB RAM.
--
assignee: terry.reedy
components: IDLE
messages: 313263
nosy: jbrearley, terry.reedy
priority: normal
severity: normal
status: open
title: IDLEX GUI consumes all RAM for scrollback buffer, uses 161Bytes /
character stored
type: resource usage
versions: Python 3.6
___
Python tracker
<https://bugs.python.org/issue33000>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33000] IDLEX GUI consumes all RAM for scrollback buffer, uses 161Bytes / character stored
John Brearley added the comment: Hi Terry: I am exploring the value of a language specific editor and runtime environment. Its definitely a large step up from Windows Notepad and Gnome Gedit. Perhaps some notes in the IDLEX documentation regarding the development versus production runtime usages would be in order? Perhaps in your Win10 environment you get the memory back when you close the shell window. In Win7, you dont get the memory back until you close both windows. -- ___ Python tracker <https://bugs.python.org/issue33000> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33000] IDLEX GUI consumes all RAM for scrollback buffer, uses 161Bytes / character stored
John Brearley added the comment: Hi Terry: The icon on my Win 7 desktop points to: "C:\WinPython\IDLEX (Python GUI).exe". This was part of the 430MB installer file WinPython-64bit-3.6.4.0Qt5b4.exe from https://sourceforge.net/projects/winpython. I attached a screen shot of IDLEX window & help about. -- Added file: https://bugs.python.org/file47471/Python_IDLEX.png ___ Python tracker <https://bugs.python.org/issue33000> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31679] pydot missing write, write_png, etc
New submission from John Brearley : I have successfully installed Graphviz tool from http://graphviz.org, updated my PATH variable appropriately and can generate .PNG files using Python module Graphviz with WinPython 3.6.1. However, I cannot get anywhere using the pydot V1.2.3 module in the same environment. The online help suggests that there there are write & write_png members but I can find no trace of them. My sample script is attached. The error I get is: Traceback (most recent call last): File "ML_ex2.py", line 109, in graph.write(path=out_fn, format="png") AttributeError: 'list' object has no attribute 'write' -- components: Extension Modules files: ML_ex2.py messages: 303632 nosy: jbrearley priority: normal severity: normal status: open title: pydot missing write, write_png, etc type: behavior versions: Python 3.6 Added file: https://bugs.python.org/file47186/ML_ex2.py ___ Python tracker <https://bugs.python.org/issue31679> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31880] subprocess process interaction with IDLEX GUI causes pygnuplot silent failures
New submission from John Brearley : There is an interesting interaction between the IDLEX GUI and subprocess module that causes pygnuplot silent failures. The example.py script below works fine when run from the WinPython Command Prompt.exe terminal window. The script will popup a new window from gnuplot with the desired graph, and in the same directory save the generated raw data as example.out and the generated graph as example.pdf. If you erase the generated files and then run the same script via the IDLEX GUI, there is no graph window created and only the raw data example.out file is created. There are no error messages. The PyGnuplot module sets up a subprocess.Popen pipe as persistant connection to the gnuplot.exe utility. This allows the top level script to send multiple commands to gnuplot to compose the desired graph, and then finally save the file to disk. This all works fine when run from the WinPython Command Prompt.exe terminal window. However something subtle is breaking when the same script is run from the IDLEX GUI environment. It is not at all obvious if the subprocess module is not as forgiving as it needs to be of the IDLEX GUI input or if the IDLEX GUI is breaking the rules somewhere. I will start by asking the subprocess module team to comment. I did try adding some trace code to the PyGnuplot.py module. In particular I turned on stdout=subprocess.PIPE and stderr=subprocess.PIPE. I did proc.poll() before/after the command is sent to gnuplot. Interestingly, when I did proc.communicate(timeout=0) to do an immediate read for any stdout/err data, this seems to cause the subsequent write to the pipe.stdin to fail, saying the file is already closed. In another learning exercise script for subprocess, the communicate() method does not seem to interfere with the pipe behavior. This issue is quite repeatable on demand. I set up a second PC and reproduced the issue on demand on the second PC. I am using WinPython 3.6.1 on Windows 7 with gnuplot 5.2.0 on both PC. Here are the links to the various components needed to setup the environment to reproduce this issue: 1) gnuplot 5.2.0 https://sourceforge.net/projects/gnuplot/files/gnuplot/5.2.0/ 2) pygnuplot 0.10.0 https://pypi.python.org/pypi/PyGnuplot/0.10.0 3) There is a one-line fix to PyGnuplot.py needed https://github.com/benschneider/PyGnuplot/blob/master/PyGnuplot.py 4) example.py script https://github.com/benschneider/PyGnuplot/blob/master/example.py WinPython 3.6.1 installer comes with subprocess.py as part of the package, no version info that I can find. When installing gnuplot there is an advanced option to check on that will automatically update your PATH variable. If you dont do this, then you must manully add "C:\gnuplot\bin" (or whatever the directory is) to your PATH variable. To check if gnuplot is correctly installed, from a DOS prompt terminal window, type "gnuplot -V". You should get a one line response "gnuplot 5.2 patchlevel 0". -- components: Interpreter Core, Windows messages: 305073 nosy: jbrearley, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: subprocess process interaction with IDLEX GUI causes pygnuplot silent failures type: behavior versions: Python 3.6 ___ Python tracker <https://bugs.python.org/issue31880> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31880] subprocess process interaction with IDLEX GUI causes pygnuplot silent failures
John Brearley added the comment:
Additonal testing shows that the subprocess.run command will reliably interact
directly with gnuplot, either from the IDLEX GUI or the Python terminal window.
import subprocess
def run_cmd(cmd):
print("run_cmd cmd:", cmd)
# MUST explicitly ask for stdout, stderr. timeout is in seconds
p1 = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
timeout=20)
# print("run_cmd p1:", p1, type(p1))
print("run_cmd p1.stdout:", p1.stdout, type(p1.stdout),
p1.stdout.decode("utf-8"))
print("run_cmd p1.stderr:", p1.stderr, type(p1.stderr),
p1.stderr.decode("utf-8"))
print("run_cmd p1.returncode:", p1.returncode, type(p1.returncode))
cmd = "gnuplot.exe "+self+"_candles.gnu"
run_cmd(cmd)
--
___
Python tracker
<https://bugs.python.org/issue31880>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31880] subprocess process interaction with IDLEX GUI causes pygnuplot silent failures
John Brearley added the comment: The owner of PyGnuplot figured out that for Python 3.4+ that a flush on stdin is needed. IDLEX GUI now runs example.py and my own test code correctly. proc.stdin.flush() # send the command in python 3.4+ This leaves the interesting behavior of IDLEX GUI. What is it doing differently re stdin from the command line terminal behavior? There is probably something to be learned here, if someone wants to dig into it. -- ___ Python tracker <https://bugs.python.org/issue31880> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16920] multiprocessing.connection listener gets MemoryError on recv
New submission from John Brearley: Using a multiprocessing.connection listener, I can accept an incoming socket OK, but when I do conn.recv(), I get memory error. The attached script mpl_bug.py will readily reproduce the issues on WinXP & WinVista, see sample output below: c:\>python mpl_bug.py main server listening on host port 10500 main created simple_client simple_client connecting to host localhost port 10500 main conn: waiting for data simple_client sending: b'abcd' simple_client waiting for data Traceback (most recent call last): File "mpl_bug.py", line 61, in data=conn.recv() ;# Memory Error occurs here <<<== == File "c:\python33\lib\multiprocessing\connection.py", line 251, in recv buf = self._recv_bytes() File "c:\python33\lib\multiprocessing\connection.py", line 405, in _recv_bytes return self._recv(size) File "c:\python33\lib\multiprocessing\connection.py", line 380, in _recv chunk = read(handle, remaining) MemoryError -- files: mpl_bug.py messages: 179577 nosy: jbrearley priority: normal severity: normal status: open title: multiprocessing.connection listener gets MemoryError on recv versions: Python 3.3 Added file: http://bugs.python.org/file28673/mpl_bug.py ___ Python tracker <http://bugs.python.org/issue16920> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16920] multiprocessing.connection listener gets MemoryError on recv
Changes by John Brearley : -- versions: +Python 3.2 ___ Python tracker <http://bugs.python.org/issue16920> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16920] multiprocessing.connection listener gets MemoryError on recv
John Brearley added the comment: In V3.2.2.3, the conn.accept() was failing to resolve the socket address. -- ___ Python tracker <http://bugs.python.org/issue16920> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16920] multiprocessing.connection listener gets MemoryError onrecv
John Brearley added the comment: Hi Richard: Thanks for the update. Yes, the multiprocessing.communication.Client works much better. The residual issue left here is wether Python is vulnerable to a DOS attack. If someone used regular sockets deliberately, they could crash multiprocessing server code deliberately. Any chance of doing a real message length check against the embedded message length check? Might not hurt for documentation to state that raw sockets are not supported, that you must use the client. Regards, John Brearley 613-259-5622 (H) -- title: multiprocessing.connection listener gets MemoryError on recv -> multiprocessing.connection listener gets MemoryError onrecv ___ Python tracker <http://bugs.python.org/issue16920> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16955] multiprocessing.connection poll() always returns false
New submission from John Brearley: In the attached multiprocessing.connection simple client script, the poll() method is always returning false. If I force the temp variable to 1, the recv()method happily gets the incoming data. I had hoped to use the poll() method to implement a timeout error, to avoid the blocking nature of the client. -- files: socket_client_mp.py messages: 179885 nosy: jbrearley priority: normal severity: normal status: open title: multiprocessing.connection poll() always returns false versions: Python 3.3 Added file: http://bugs.python.org/file28717/socket_client_mp.py ___ Python tracker <http://bugs.python.org/issue16955> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16955] multiprocessing.connection poll() always returns false
John Brearley added the comment: Hi Rchard: Thanks very much. The wait() method works fine. Regards, John Brearley 613-259-5622 (H) -- ___ Python tracker <http://bugs.python.org/issue16955> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16920] multiprocessing.connection listener gets MemoryErroronrecv
John Brearley added the comment: Hi Richard: Thanks for pointers on other methods. I am coming from a TCL background, and learning Python. I have gone through regular sockets, select, asyncore, sockserv, threading and multiprocessing modules. Only multiprocessing seems to be able to use more than a single CPU and the listener is nicely shared between proceses. Is there another module that that would be better suited spreading tasks around CPUS and communicating between hosts? I am trying to understand your reservations about using them for communication over a network Yes, if I was doing a production server, I would turn on authentication. Regards, John Brearley 613-259-5622 (H) -- title: multiprocessing.connection listener gets MemoryError onrecv -> multiprocessing.connection listener gets MemoryErroronrecv ___ Python tracker <http://bugs.python.org/issue16920> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue16966] Publishing multiprocessing listener code
New submission from John Brearley: Hi Richard: I have published my multiprocessing server & client scripts on the 2 web sites shown below in the hopes that they will help others learning this module. I haven't seen anyplace on the python.org web site that might be suitable for a copy. Are people encouraged to post sample scripts on python.org? http://codereview.stackexchange.com/questions/20516/sample-python-multiprocessin g-connection-server http://www.tek-tips.com/viewthread.cfm?qid=1701937 Regards, John Brearley 613-259-5622 (H) -- messages: 179968 nosy: jbrearley priority: normal severity: normal status: open title: Publishing multiprocessing listener code ___ Python tracker <http://bugs.python.org/issue16966> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
