pip module not found

[David John]

Hi,
I recently have been experiencing issues with the pip installation module.
I have python version 3.11 installed. I've checked the directory installed
in the systems variables window and nothing is amiss. Kindly assist.

Regards,
David
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pip module not found

[Mats Wichmann]

On 5/12/23 00:42, David John wrote:

Hi,
I recently have been experiencing issues with the pip installation module.


How?  Please be explicit or nobody can answer your question.



--
https://mail.python.org/mailman/listinfo/python-list

Re: pip module not found

[Thomas Passin]

On 5/12/2023 11:18 AM, Thomas Passin wrote:

On 5/12/2023 2:42 AM, David John wrote:

Hi,
I recently have been experiencing issues with the pip installation 
module.
I have python version 3.11 installed. I've checked the directory 
installed

in the systems variables window and nothing is amiss. Kindly assist.


It would be useful if you told us what operating system you are using 
and how you installed Python.


Many if not most Linux distributions do not include pip by default. 
Usually the package manager as a version to install.  On systems based 
on Debian, you can install pip with:


sudo apt install python3-pip

On others, you will have to look around in the package manager or search 
on line.


As a last resort, if you cannot find an OS package manager way to 
install pip, you find out how from here:


https://pip.pypa.io/en/stable/installation/

As the link says, you can run from a command line:

 -m ensurepip --upgrade

NOTE: instead of , use the command that launches the right 
version of python on your system  On Windows, this is usually py.  On 
Linux, it is usually python3.


On Linux, if you want tkinter, you may have to install it with the 
package manager too.  On Debian-related systems:


sudo apt-get install python3-tk

For the Yum package manager:

yum install tkinter

You may also need to install ImageTk:

sudo apt-get install python3-pil.imagetk  (Debian-based)

On Centos/Red Hat derived systems, you will also need to install

python3-pillow
python3-pillow-tk



--
https://mail.python.org/mailman/listinfo/python-list

Re: pip module not found

[Barry]

> On 12 May 2023, at 18:31, Thomas Passin  wrote:
> 
> On 5/12/2023 11:18 AM, Thomas Passin wrote:
>>> On 5/12/2023 2:42 AM, David John wrote:
>>> Hi,
>>> I recently have been experiencing issues with the pip installation module.
>>> I have python version 3.11 installed. I've checked the directory installed
>>> in the systems variables window and nothing is amiss. Kindly assist.
>> It would be useful if you told us what operating system you are using and 
>> how you installed Python.
>> Many if not most Linux distributions do not include pip by default. Usually 
>> the package manager as a version to install.  On systems based on Debian, 
>> you can install pip with:
>> sudo apt install python3-pip
>> On others, you will have to look around in the package manager or search on 
>> line.
>> As a last resort, if you cannot find an OS package manager way to install 
>> pip, you find out how from here:
>> https://pip.pypa.io/en/stable/installation/
>> As the link says, you can run from a command line:
>>  -m ensurepip --upgrade
>> NOTE: instead of , use the command that launches the right version 
>> of python on your system  On Windows, this is usually py.  On Linux, it is 
>> usually python3.
> 
> On Linux, if you want tkinter, you may have to install it with the package 
> manager too.  On Debian-related systems:
> 
> sudo apt-get install python3-tk
> 
> For the Yum package manager:
> 
> yum install tkinter
> 
> You may also need to install ImageTk:
> 
> sudo apt-get install python3-pil.imagetk  (Debian-based)
> 
> On Centos/Red Hat derived systems, you will also need to install
> 
> python3-pillow
> python3-pillow-tk
> 

PIP not PIL is the topic right?

We still need OP to tell us which OS and where python came from.

Barry

> 
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pip module not found

[Thomas Passin]

On 5/12/2023 1:45 PM, Barry wrote:




On 12 May 2023, at 18:31, Thomas Passin  wrote:

On 5/12/2023 11:18 AM, Thomas Passin wrote:

On 5/12/2023 2:42 AM, David John wrote:
Hi,
I recently have been experiencing issues with the pip installation module.
I have python version 3.11 installed. I've checked the directory installed
in the systems variables window and nothing is amiss. Kindly assist.

It would be useful if you told us what operating system you are using and how 
you installed Python.
Many if not most Linux distributions do not include pip by default. Usually the 
package manager as a version to install.  On systems based on Debian, you can 
install pip with:
sudo apt install python3-pip
On others, you will have to look around in the package manager or search on 
line.
As a last resort, if you cannot find an OS package manager way to install pip, 
you find out how from here:
https://pip.pypa.io/en/stable/installation/
As the link says, you can run from a command line:
 -m ensurepip --upgrade
NOTE: instead of , use the command that launches the right version of 
python on your system  On Windows, this is usually py.  On Linux, it is usually 
python3.


On Linux, if you want tkinter, you may have to install it with the package 
manager too.  On Debian-related systems:

sudo apt-get install python3-tk

For the Yum package manager:

yum install tkinter

You may also need to install ImageTk:

sudo apt-get install python3-pil.imagetk  (Debian-based)

On Centos/Red Hat derived systems, you will also need to install

python3-pillow
python3-pillow-tk



PIP not PIL is the topic right?

We still need OP to tell us which OS and where python came from.

Barry


Sure, Pip is the topic. But on Linux, once you have it installed, Tk may 
not be far behind, so I thought I'd mention the info since I've been 
bitten by it myself, and it's not obvious what to do.


By throwing out a number of common possibilities I was hoping to cover 
the OP's situation - and help others who didn't write in - I've been 
there.   And yes, of course it would be better to have the OP's detailed 
information.


Let's hope that if the OP does succeed we will get to know of it.

--
https://mail.python.org/mailman/listinfo/python-list

Re: pip module not found

[Thomas Passin]

On 5/12/2023 2:42 AM, David John wrote:

Hi,
I recently have been experiencing issues with the pip installation module.
I have python version 3.11 installed. I've checked the directory installed
in the systems variables window and nothing is amiss. Kindly assist.


It would be useful if you told us what operating system you are using 
and how you installed Python.


Many if not most Linux distributions do not include pip by default. 
Usually the package manager as a version to install.  On systems based 
on Debian, you can install pip with:


sudo apt install python3-pip

On others, you will have to look around in the package manager or search 
on line.


As a last resort, if you cannot find an OS package manager way to 
install pip, you find out how from here:


https://pip.pypa.io/en/stable/installation/

As the link says, you can run from a command line:

 -m ensurepip --upgrade

NOTE: instead of , use the command that launches the right 
version of python on your system  On Windows, this is usually py.  On 
Linux, it is usually python3.



--
https://mail.python.org/mailman/listinfo/python-list

Re: Do subprocess.PIPE and subprocess.STDOUT sametime

[Horst Koiner]

Using asyncio for this is a good possibility I was not aware of.

My best try with asyncio was:
import asyncio

async def run_command():
# Create subprocess
process = await asyncio.create_subprocess_exec(
'./test.sh',
stdout=asyncio.subprocess.PIPE,  # Redirect stdout to a pipe
stderr=asyncio.subprocess.PIPE  # Redirect stderr to a pipe
)

# Read stdout and stderr asynchronously
captured_output = b''
async for line in process.stdout:
print(line.decode().strip())
captured_output += line
async for line in process.stderr:
print(line.decode().strip())
captured_output += line

await process.wait()
print(captured_output)


# Run the asyncio event loop
asyncio.run(run_command())


This fulfills all my requirements. A nice to have would that the 
captured_output has not to be constructed with += 's but with a final seek(0) 
and read() of process.stdout. But I didn't find anything how to rewind the 
stream, that i can read the whole output again.
Another question is, if this solution is deadlock proof.

Thank you all for the already very valuable input!

Greetings,
Horst
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pip module not found

[Barry]

> On 12 May 2023, at 21:59, Thomas Passin  wrote:
> 
> On 5/12/2023 2:42 AM, David John wrote:
>> Hi,
>> I recently have been experiencing issues with the pip installation module.
>> I have python version 3.11 installed. I've checked the directory installed
>> in the systems variables window and nothing is amiss. Kindly assist.
> 
> It would be useful if you told us what operating system you are using and how 
> you installed Python.
> 
> Many if not most Linux distributions do not include pip by default. Usually 
> the package manager as a version to install.

From what i see the fedora/redhat/centos world includes the batteries.
The debian/ubuntu world take batteries out.



>  On systems based on Debian, you can install pip with:
> 
> sudo apt install python3-pip
> 
> On others, you will have to look around in the package manager or search on 
> line.
> 
> As a last resort, if you cannot find an OS package manager way to install 
> pip, you find out how from here:
> 
> https://pip.pypa.io/en/stable/installation/
> 
> As the link says, you can run from a command line:
> 
>  -m ensurepip --upgrade
> 
> NOTE: instead of , use the command that launches the right version of 
> python on your system  On Windows, this is usually py.  On Linux, it is 
> usually python3.
> 
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Do subprocess.PIPE and subprocess.STDOUT sametime

[Chris Angelico]

On Sat, 13 May 2023 at 07:21, Horst Koiner  wrote:
>
> Using asyncio for this is a good possibility I was not aware of.
>
> My best try with asyncio was:
> import asyncio
>
> async def run_command():
> # Create subprocess
> process = await asyncio.create_subprocess_exec(
> './test.sh',
> stdout=asyncio.subprocess.PIPE,  # Redirect stdout to a pipe
> stderr=asyncio.subprocess.PIPE  # Redirect stderr to a pipe
> )
>
> # Read stdout and stderr asynchronously
> captured_output = b''
> async for line in process.stdout:
> print(line.decode().strip())
> captured_output += line
> async for line in process.stderr:
> print(line.decode().strip())
> captured_output += line
>
> await process.wait()
> print(captured_output)
>
>
> # Run the asyncio event loop
> asyncio.run(run_command())
> 
>
> This fulfills all my requirements. A nice to have would that the 
> captured_output has not to be constructed with += 's but with a final seek(0) 
> and read() of process.stdout. But I didn't find anything how to rewind the 
> stream, that i can read the whole output again.
> Another question is, if this solution is deadlock proof.
>

No it's not, but the best part is, it's really close to! Asynchronous
I/O is perfect for this: you need to wait for any of three events
(data on stdout, data on stderr, or process termination). So here it
is as three tasks:

captured_output = b""
async def collect_output(stream):
global captured_output
async for line in stream:
print(line.decode().strip())
captured_output += line

(You can play around with other ways of scoping this, I'm just using a
global for simplicity)

Inside run_command, you can then spawn three independent tasks and
await them simultaneously. Once all three finish, you have your
captured output, and the process will have terminated.

This would then be guaranteed deadlock-proof (and, if you needed to
feed data on stdin, you could do that with a fourth task and it'd
still be deadlock-proof, even if it's more than one pipe buffer of
input), since all the pipes are being managed concurrently.

Even cooler? You can scale this up to multiple processes by calling
run_command more than once as separate tasks!

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Multithreading? How?

[Diego Souza]

Hi there,

I hope this e-mail is still on time for you. I have implemented this
architecture a few times, and they all work fine nowadays. However, your
question made me review it and create a small gist.

I suggest you create a thread for every output and input connection. This
makes it easier to focus on reading or writing inside a given object. For
example, in your project, I would separate it into IotReader, IotWriter,
MqttReader, and MqttWriter. Another thing I do to avoid manipulating
multiple locks, semaphores, and so on is to create a central event loop.
For every event that comes from IotReader or MqttReader, I would pack it
into an event and send it to a central thread. This is the main gateway
thread, and I would call it Gateway.

I don't know if you have ever programmed in Android. But the Android
framework uses a similar approach to processing data. Whenever you need to
process something, you start a new 'Thread', and when you need to present
the result in the interface you dispatch events until the main thread is
notified and updates the corresponding Views. The point here is: never do
any extensive processing in the main thread as it is going delay. You will
probably not do it now, but if you ever need it, make a pool of workers to
process this and keep the Gateway free. Replace the Threads with
multiprocessing.Process, as well, as Python lacks true multithreading.

Regarding thread/process communication, I like to implement this using
Queues. The Gateway class would have a main_queue to receive events from
IotReader and MqttReader. IotWriter and MqttWriter have a particular queue
as well. Whenever the Gateway needs to send something to either of them, it
just needs to reference their respective queues, which I wrap inside a
method, for simplicity.

Another benefit of this architecture is the ability to scale to more
connections easily. In the past, I have used this strategy to schedule
tasks for up to about 20 devices (each with an input and output thread). I
believe it could go higher, but I haven't needed to. There are fully
distributed architectures more suitable for hundreds and thousands of
connections, but this is likely not what you need now.

The following is a possible implementation for the IotReader. You need to
replace the AnySerialReader class and its read method with the
initialization of your own Bus wrapper. The read method must have a timeout
parameter if you want to cancel the operation properly. The terminate
method is used to terminate the program properly.


































*class IotReader(Thread):def __init__(self, queue_master,
name='IotReader'):super().__init__()
self.queue_master = queue_masterself.queue = Queue()
self.done  = Falseself.name   = name
  self.start()def terminate(self):self.done = True
def run(self):log.info (f"Starting thread for
{self.name }")serial_reader =
AnySerialReader('Serial' + self.name )log.info
(f"Serial reader for {self.name }
initialized")while not self.done:try:
  data = serial_reader.read(timeout=1)
if data is None:continue
self.queue_master.put(('on_iot_event', data))except:
traceback.print_exc(file=sys.stdout)
log.warning("Terminating IotReader")serial_reader.terminate()*


The following is a possible implementation for IotWriter. It adds a method
named send that adds new tasks to the queue. The main loop, running inside
the thread, waits for these events and calls write in AnySerialWriter. This
may be a slow operation, the connection may be down, and we need to
reconnect, etc. This is why we need a thread for the output message as well.









































*class IotWriter(Thread):def __init__(self, name='IotWriter'):
super().__init__()self.queue  = Queue()self.done
= Falseself.name    = name
self.start()def terminate(self):self.done = True
self.queue.put( ('terminate', None) )def send(self, data):
self.queue.put( ('write_message', data) )def run(self):
log.info (f"Starting thread for {self.name
}")serial_writer = AnySerialWriter('Serial' +
self.name )log.info (f"Serial
writer for {self.name } initialized")
while not self.done:try:action, data =
self.queue.get()if action == 'terminate':
  breakelif action ==
'write_message':serial_writer.write(data)
  else:log.error(f'Unknown action for
IotWriter - action={action}, data={data}')except:
  traceback.print_exc(file=sys.stdout)
log.warning("