Re: Didn't understand the output of the following Python 3 code with reduce function?

[Shivlal Sharma]

On Saturday, 29 August 2020 at 02:47:56 UTC+5:30, Ben Bacarisse wrote:
Thanks you all, I was really confused in this code.
> Shivlal Sharma  writes: 
> 
> > I have seen this code on one of competative programming site but I 
> > didn't get it, Why output is 9? 
> > 
> > from functools import * 
> > 
> > def ADDS(a,b): 
> > return a+1 
> > nums = [1, 2, 3, 4, 5, 6, 7, 8, 9] 
> > add = reduce(ADDS, nums) 
> > print(add) 
> > 
> > output: 9
> Hint: 
> 
> reduce(f, [e1, e2]) is f(e1, e2) 
> reduce(f, [e1, e2, e3]) is f(f(e1, e2), e3) 
> reduce(f, [e1, e2, e3, e4]) is f(f(f(e1, e2), e3), e4) 
> 
> Replace f with a function that adds one to its first argument. Does 
> that help? 
> 
> -- 
> Ben.
-- 
https://mail.python.org/mailman/listinfo/python-list

Error in lambda function..!

[Shivlal Sharma]

from functools import*
nums = [1, 2, 3, 4, 5, 6, 7, 8, 9]
add = reduce(lambda a : a + 1, nums)
print(add)

error: -
TypeError Traceback (most recent call last)
 in ()
  1 from functools import*
  2 nums = [1, 2, 3, 4, 5, 6, 7, 8, 9]
> 3 add = reduce(lambda a : a + 1, nums)
  4 print(add)

TypeError: () takes 1 positional argument but 2 were given
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Error in lambda function..!

[Peter Otten]

Shivlal Sharma wrote:

> from functools import*
> nums = [1, 2, 3, 4, 5, 6, 7, 8, 9]
> add = reduce(lambda a : a + 1, nums)
> print(add)
> 
> error: -
> TypeError Traceback (most recent call
> last)  in ()
>   1 from functools import*
>   2 nums = [1, 2, 3, 4, 5, 6, 7, 8, 9]
> > 3 add = reduce(lambda a : a + 1, nums)
>   4 print(add)
> 
> TypeError: () takes 1 positional argument but 2 were given

Read the error message: your lambda accepts ony one argument, but reduce 
tries to pass two to it. That you don't need the second argument doesn't 
mean that you can it omit it, so:

>>> from functools import reduce
>>> reduce(lambda a, b: a + 1, range(1, 10))
9


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Error in lambda function..!

[Rob Cliffe via Python-list]

On 29/08/2020 08:58, Shivlal Sharma wrote:

from functools import*
nums = [1, 2, 3, 4, 5, 6, 7, 8, 9]
add = reduce(lambda a : a + 1, nums)
print(add)

error: -
TypeError Traceback (most recent call last)
 in ()
   1 from functools import*
   2 nums = [1, 2, 3, 4, 5, 6, 7, 8, 9]
> 3 add = reduce(lambda a : a + 1, nums)
   4 print(add)

TypeError: () takes 1 positional argument but 2 were given

When you write
    lambda a :
you are defining a function that takes **one** argument, viz. a.
It's the same as if you wrote:
    def myfunc(a): return a+1
and then
    add = reduce(myfunc, nums)
But reduce always calls your supplied function with **two** arguments.  
Hence the error.

I don't know exactly what you are trying to do, but if you wrote
    add = reduce(lambda a,b: a+1, nums)
or equivalently
    def myfunc(a,b): return a+1
    add = reduce(myfunc, nums)
reduce would call your lambda function succesively with arguments
    (1,2)    # which would return 1+1 i.e. 2
    (2,3)    # use previous result (2) and next number in the sequence 
(3); returns 2+1 i.e. 3

    (3,4)    # use previous result (3) and next number in the sequence (4)

    .
    (8,9)    # returns 8+! i.e. 9
and finally 9 would be printed.

If you want to watch what is happening try
    def myfunc(a,b):
        print(f"Myfunc({a},{b})")
        return a+1
    add = reduce(myfunc, nums)

Try
    from functools import *
    help(reduce)
for an explanation of how reduce works.  Note that the first sentence 
starts "Apply a function of two arguments"

Best wishes
Rob Cliffe

--
https://mail.python.org/mailman/listinfo/python-list

Re: ABC with abstractmethod: kwargs on Base, explicit names on implementation

[Samuel Marks]

So there is no way to get a AOT error when the two functions aren't
implemented, if the two functions have different required arguments on
implementors?

To paint this picture for why I need this, say the first is:

class Base(ABC):
@abstractclass
def train(self, epochs):
asset epochs is not None and epochs > 0

…and the implementation is:

class TensorFlow(Base):
def train(self, learning_rate, epochs, callbacks, metrics, loss, optimizer):
super(Base, self).__init__(epochs=epochs)

[+ docstrings and PEP484 types; excluded here for concision]

So how do I enable this use-case? - Obviously it doesn't make sense to
include these on the base class, and giving them default values
probably doesn't make sense either.

You're saying I shouldn't be using ABC for this. So what should I be using?
The requirement I'm trying to enforce is that each implementor has a
`train` callable attached.
Preferably with one required field (but this is just a nice-to-have).

BTW: I have enabled an argument inheritance use-case through creating
a library with the builtin `ast` module. So now you can have `def
train_c(self, config)` with config referring to an instance of a class
which inherits from a base config class. See here for the
implementation https://github.com/SamuelMarks/doctrans

However if I only have functions which accept an instance of a class
as the argument, then that will make it less user-friendly to the API
caller. So I really am looking for handling both interfaces in a
straightforward manner.

Thanks for your suggestions,

Samuel Marks
Charity  | consultancy
 | open-source  |
LinkedIn 

On Fri, Aug 28, 2020 at 3:51 AM Dieter Maurer  wrote:
>
> Samuel Marks wrote at 2020-8-27 15:58 +1000:
> >The main thing I want is type safety. I want Python to complain if the
> >callee uses the wrong argument types, and to provide suggestions on
> >what's needed and info about it.
> >
> >Without a base class I can just have docstrings and type annotations
> >to achieve that.
> >
> >What can I use that will require all implementers to have a minimum of
> >the same properties and arguments, but also allow them to add new
> >properties and arguments?
>
> A main paradigm of object oriented programming is the
> ability to use a derived class instance with knowledge only
> about the base class. This implies that in many cases, you
> need not know the concrete class because any instance of a derived
> class must have the essential behavior of the base class instances.
>
> This paradigm imposes limitations on the allowable signature changes.
> An overriding method may add further parameters but all those
> must have default values - otherwise, the use with base class knowledge
> only would cause errors.
>
> > Preferably I would like this all to happen before compile/interpret
> time.
>
> Use a "lint" version to check compatibilty.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: ABC with abstractmethod: kwargs on Base, explicit names on implementation

[Dieter Maurer]

Samuel Marks wrote at 2020-8-29 19:14 +1000:
>So there is no way to get a AOT error when the two functions aren't
>implemented, if the two functions have different required arguments on
>implementors?
>
>To paint this picture for why I need this, say the first is:
>
>class Base(ABC):
>@abstractclass
>def train(self, epochs):
>asset epochs is not None and epochs > 0
>
>…and the implementation is:
>
>class TensorFlow(Base):
>def train(self, learning_rate, epochs, callbacks, metrics, loss, 
> optimizer):
>super(Base, self).__init__(epochs=epochs)
>
>[+ docstrings and PEP484 types; excluded here for concision]
>
>So how do I enable this use-case? - Obviously it doesn't make sense to
>include these on the base class, and giving them default values
>probably doesn't make sense either.
>
>You're saying I shouldn't be using ABC for this. So what should I be using?

What is your purpose to use a base class in the first place --
and especially one where `train` has this signature?

You signal with this base class, that it makes sense to
call `train` with a single positional argument.
But `train` in the derived class cannot be called in this way.

Base classes are there to capture common features of several
related classes. In your example above, this is not the case.
Do not use base classes in this way.

>The requirement I'm trying to enforce is that each implementor has a
>`train` callable attached.
>Preferably with one required field (but this is just a nice-to-have).

Read the Python documentation about metaclasses and the special methods
related to class derivation. Both approaches would allow you
to check whatever you want.

> ...
>However if I only have functions which accept an instance of a class
>as the argument, then that will make it less user-friendly to the API
>caller. So I really am looking for handling both interfaces in a
>straightforward manner.

Any system makes some things easy and other things difficult.
Try to stay with the essential paradigms underlaying the system --
in order to use the easy things whenever possible.

In your case, this means that the signature of an overriding
method must be quite similar to that of the overridden method.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: ABC with abstractmethod: kwargs on Base, explicit names on implementation

[Samuel Marks]

Putting Liskov substitution principal to one side, I had a suggestion
to follow PEP3102, and do `def train(self, *, epochs):`…


It's a rather simple suggestion that I just might take aboard.

In response to Dieter:
My purpose for using a base class is so that the highest level
interface—say a CLI, GUI, REST and RPC API—can all be configured in
one place, but enable the greatest possible divergence from a
parameter standpoint.

An alternative here [in this domain] would be to create a new
Keras/scikit.learn. I.e., one consistent interface that is to be
implemented in full by each framework.

The reason I don't want to take this alternative is manyfold, that I
don't need to get in here (happy to start a new thread if you're
interested).

Two major advantages of this approach are:
0. Implementers can be [pretty much] as divergent as they want, with
different default and required parameters; and even semantics [though
I do have `assert`s to force `method0` to be executed before
`method1`];
1. Implementers don't need to create their own CLIs, GUIs, REST and RPC APIs

Two major disadvantages:
0. Parameters aren't known ahead of time, so you can do the whole
`Animal` [duck type] trick where `Animal` could actually be `Dog` or
`Horse` [making the obvious Base `ABC` & `abstractmethod` approach
'wrong']
1. Each implementer framework can maintain wildly different internal
APIs, making more hardcore integrations—in a multi-ML sense—far more
difficult

Samuel Marks
Charity  | consultancy
 | open-source  |
LinkedIn 

On Sat, Aug 29, 2020 at 8:24 PM Dieter Maurer  wrote:
>
> Samuel Marks wrote at 2020-8-29 19:14 +1000:
> >So there is no way to get a AOT error when the two functions aren't
> >implemented, if the two functions have different required arguments on
> >implementors?
> >
> >To paint this picture for why I need this, say the first is:
> >
> >class Base(ABC):
> >@abstractclass
> >def train(self, epochs):
> >asset epochs is not None and epochs > 0
> >
> >…and the implementation is:
> >
> >class TensorFlow(Base):
> >def train(self, learning_rate, epochs, callbacks, metrics, loss, 
> > optimizer):
> >super(Base, self).__init__(epochs=epochs)
> >
> >[+ docstrings and PEP484 types; excluded here for concision]
> >
> >So how do I enable this use-case? - Obviously it doesn't make sense to
> >include these on the base class, and giving them default values
> >probably doesn't make sense either.
> >
> >You're saying I shouldn't be using ABC for this. So what should I be using?
>
> What is your purpose to use a base class in the first place --
> and especially one where `train` has this signature?
>
> You signal with this base class, that it makes sense to
> call `train` with a single positional argument.
> But `train` in the derived class cannot be called in this way.
>
> Base classes are there to capture common features of several
> related classes. In your example above, this is not the case.
> Do not use base classes in this way.
>
> >The requirement I'm trying to enforce is that each implementor has a
> >`train` callable attached.
> >Preferably with one required field (but this is just a nice-to-have).
>
> Read the Python documentation about metaclasses and the special methods
> related to class derivation. Both approaches would allow you
> to check whatever you want.
>
> > ...
> >However if I only have functions which accept an instance of a class
> >as the argument, then that will make it less user-friendly to the API
> >caller. So I really am looking for handling both interfaces in a
> >straightforward manner.
>
> Any system makes some things easy and other things difficult.
> Try to stay with the essential paradigms underlaying the system --
> in order to use the easy things whenever possible.
>
> In your case, this means that the signature of an overriding
> method must be quite similar to that of the overridden method.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Video file to subtitles file

[Muskan Sanghai]

On Friday, August 28, 2020 at 10:52:57 PM UTC+5:30, Barry wrote:
> > On 28 Aug 2020, at 17:37, Muskan Sanghai  wrote:
> > 
> > On Friday, August 28, 2020 at 12:27:25 AM UTC+5:30, Barry Scott wrote: 
>  On 27 Aug 2020, at 18:00, Muskan Sanghai  wrote: 
> >>> 
> >>> I would be really thankful if someone can suggest me how can I generate 
> >>> subtitles file (srt format) from a video or audio without using Google 
> >>> cloud and AWS. 
> >> What do you know about how subtitles work with video? Do you mean you want 
> >> to extract the bitmap subtitle data from a MPEG video? 
> >> 
> >> Barry 
> >> 
> >> 
> >> 
> >>> -- 
> >>> https://mail.python.org/mailman/listinfo/python-list 
> >>> 
> > 
> > Thank you Barry for your reply, 
> > I just know the basics and I want to extract subtitles from a MPEG video 
> > and then put the subtitles in that same video. Subtitles can be of any 
> > format but it should be convenient for the entire procedure.
> It seems you are looking for an App to do this work? 
> I searched the web and saw this. 
> 
> https://www.openshot.org/ 
> 
> I have not used this app, maybe it’s a starting point for you. 
> 
> Barry 
> 
> > -- 
> > https://mail.python.org/mailman/listinfo/python-list 
> >
I actually want to create an app or website which can do this.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Video file to subtitles file

[Muskan Sanghai]

On Friday, August 28, 2020 at 10:59:29 PM UTC+5:30, Chris Angelico wrote:
> On Sat, Aug 29, 2020 at 3:24 AM Barry  wrote: 
> > 
> > 
> > 
> > > On 28 Aug 2020, at 17:37, Muskan Sanghai  wrote: 
> > >
> > > On Friday, August 28, 2020 at 12:27:25 AM UTC+5:30, Barry Scott wrote: 
> >  On 27 Aug 2020, at 18:00, Muskan Sanghai  wrote: 
> > >>> 
> > >>> I would be really thankful if someone can suggest me how can I generate 
> > >>> subtitles file (srt format) from a video or audio without using Google 
> > >>> cloud and AWS. 
> > >> What do you know about how subtitles work with video? Do you mean you 
> > >> want to extract the bitmap subtitle data from a MPEG video? 
> > >> 
> > >> Barry 
> > >> 
> > >> 
> > >> 
> > >>> -- 
> > >>> https://mail.python.org/mailman/listinfo/python-list 
> > >>> 
> > > 
> > > Thank you Barry for your reply, 
> > > I just know the basics and I want to extract subtitles from a MPEG video 
> > > and then put the subtitles in that same video. Subtitles can be of any 
> > > format but it should be convenient for the entire procedure. 
> >
> > It seems you are looking for an App to do this work? 
> > I searched the web and saw this. 
> > 
> > https://www.openshot.org/ 
> > 
> > I have not used this app, maybe it’s a starting point for you. 
> > 
> > Barry 
> >
> Not familiar with Openshot, but it's worth looking into. 
> Alternatively, I'd definitely recommend ffmpeg for anything like this 
> sort of job. But if you actually need to OCR something, then you may 
> need to do some scripting work. I don't have code to offer you, but it 
> would involve FFMPEG to lift the images, something like Tesseract to 
> do the actual OCRing, and then you'd write the rest of it yourself in 
> Python. 
> 
> Other than that, this probably is something best done with a dedicated 
> movie editing tool, not Python. Use what exists. 
> 
> ChrisA
I want to extract subtitles from a MPEG video (which does not have any previous 
subtitles) and then add them to the same video . So is it possible to do it 
with some scripting. Actually I tried to find the same features in FFMPEG but 
couldn't.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Video file to subtitles file

[Barry Scott]

> On 29 Aug 2020, at 12:51, Muskan Sanghai  wrote:
> 
> On Friday, August 28, 2020 at 10:59:29 PM UTC+5:30, Chris Angelico wrote:
>> On Sat, Aug 29, 2020 at 3:24 AM Barry > > wrote: 
>>> 
>>> 
>>> 
 On 28 Aug 2020, at 17:37, Muskan Sanghai  wrote: 
 
 On Friday, August 28, 2020 at 12:27:25 AM UTC+5:30, Barry Scott wrote: 
>>> On 27 Aug 2020, at 18:00, Muskan Sanghai  wrote: 
>> 
>> I would be really thankful if someone can suggest me how can I generate 
>> subtitles file (srt format) from a video or audio without using Google 
>> cloud and AWS. 
> What do you know about how subtitles work with video? Do you mean you 
> want to extract the bitmap subtitle data from a MPEG video? 
> 
> Barry 
> 
> 
> 
>> -- 
>> https://mail.python.org/mailman/listinfo/python-list 
>> 
 
 Thank you Barry for your reply, 
 I just know the basics and I want to extract subtitles from a MPEG video 
 and then put the subtitles in that same video. Subtitles can be of any 
 format but it should be convenient for the entire procedure. 
>>> 
>>> It seems you are looking for an App to do this work? 
>>> I searched the web and saw this. 
>>> 
>>> https://www.openshot.org/ 
>>> 
>>> I have not used this app, maybe it’s a starting point for you. 
>>> 
>>> Barry 
>>> 
>> Not familiar with Openshot, but it's worth looking into. 
>> Alternatively, I'd definitely recommend ffmpeg for anything like this 
>> sort of job. But if you actually need to OCR something, then you may 
>> need to do some scripting work. I don't have code to offer you, but it 
>> would involve FFMPEG to lift the images, something like Tesseract to 
>> do the actual OCRing, and then you'd write the rest of it yourself in 
>> Python. 
>> 
>> Other than that, this probably is something best done with a dedicated 
>> movie editing tool, not Python. Use what exists. 
>> 
>> ChrisA
> I want to extract subtitles from a MPEG video (which does not have any 
> previous subtitles)

If it has no subtitles there is nothing to extract?

> and then add them to the same video .
> So is it possible to do it with some scripting. Actually I tried to find the 
> same features in FFMPEG but couldn't.

ffmpreg I doubt has the APIs you need for this.

Have a look at GStreamer, which has pytohn APIs. I know it can handle subtitles 
because the company
I used to work for paid for subtitle support to be added to GStreamer for a 
product
I worked on a few years ago.

I recall that in MPEG subtitles are RLE encoded bitmaps with timing and 
position data.
Which allows the player to show this bitmap at position X, Y starting at T0 and 
remove at t1 etc.
You have to track multiple subtitles at the same time.

You should be able to extract the subtitle bit maps and timing data with modest 
work.
You could use OCR technology to turn the subtitles into text.

Barry


> -- 
> https://mail.python.org/mailman/listinfo/python-list 
> 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Video file to subtitles file

[Muskan Sanghai]

On Saturday, August 29, 2020 at 6:44:34 PM UTC+5:30, Barry Scott wrote:
> > On 29 Aug 2020, at 12:51, Muskan Sanghai  wrote: 
> > 
> > On Friday, August 28, 2020 at 10:59:29 PM UTC+5:30, Chris Angelico wrote:
> >> On Sat, Aug 29, 2020 at 3:24 AM Barry  >> > wrote: 
> >>> 
> >>> 
> >>> 
>  On 28 Aug 2020, at 17:37, Muskan Sanghai  wrote: 
>  
>  On Friday, August 28, 2020 at 12:27:25 AM UTC+5:30, Barry Scott wrote: 
> >>> On 27 Aug 2020, at 18:00, Muskan Sanghai  wrote: 
> >> 
> >> I would be really thankful if someone can suggest me how can I 
> >> generate subtitles file (srt format) from a video or audio without 
> >> using Google cloud and AWS. 
> > What do you know about how subtitles work with video? Do you mean you 
> > want to extract the bitmap subtitle data from a MPEG video? 
> > 
> > Barry 
> > 
> > 
> > 
> >> -- 
> >> https://mail.python.org/mailman/listinfo/python-list 
> >> 
>  
>  Thank you Barry for your reply, 
>  I just know the basics and I want to extract subtitles from a MPEG video 
>  and then put the subtitles in that same video. Subtitles can be of any 
>  format but it should be convenient for the entire procedure. 
> >>> 
> >>> It seems you are looking for an App to do this work? 
> >>> I searched the web and saw this. 
> >>> 
> >>> https://www.openshot.org/ 
> >>> 
> >>> I have not used this app, maybe it’s a starting point for you. 
> >>> 
> >>> Barry 
> >>> 
> >> Not familiar with Openshot, but it's worth looking into. 
> >> Alternatively, I'd definitely recommend ffmpeg for anything like this 
> >> sort of job. But if you actually need to OCR something, then you may 
> >> need to do some scripting work. I don't have code to offer you, but it 
> >> would involve FFMPEG to lift the images, something like Tesseract to 
> >> do the actual OCRing, and then you'd write the rest of it yourself in 
> >> Python. 
> >> 
> >> Other than that, this probably is something best done with a dedicated 
> >> movie editing tool, not Python. Use what exists. 
> >> 
> >> ChrisA 
> > I want to extract subtitles from a MPEG video (which does not have any 
> > previous subtitles)
> If it has no subtitles there is nothing to extract?
> > and then add them to the same video . 
> > So is it possible to do it with some scripting. Actually I tried to find 
> > the same features in FFMPEG but couldn't.
> ffmpreg I doubt has the APIs you need for this. 
> 
> Have a look at GStreamer, which has pytohn APIs. I know it can handle 
> subtitles because the company 
> I used to work for paid for subtitle support to be added to GStreamer for a 
> product 
> I worked on a few years ago. 
> 
> I recall that in MPEG subtitles are RLE encoded bitmaps with timing and 
> position data. 
> Which allows the player to show this bitmap at position X, Y starting at T0 
> and remove at t1 etc. 
> You have to track multiple subtitles at the same time. 
> 
> You should be able to extract the subtitle bit maps and timing data with 
> modest work. 
> You could use OCR technology to turn the subtitles into text. 
> 
> Barry 
> 
> 
> > -- 
> > https://mail.python.org/mailman/listinfo/python-list 
> > 
Actually the video is in MP4 format and does not have subtitles in it. But I 
will surely try to use GStreamer as you have suggested.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Video file to subtitles file

[Chris Angelico]

On Sat, Aug 29, 2020 at 11:15 PM Barry Scott  wrote:
> > On 29 Aug 2020, at 12:51, Muskan Sanghai  wrote:
> > On Friday, August 28, 2020 at 10:59:29 PM UTC+5:30, Chris Angelico wrote:
> >> Not familiar with Openshot, but it's worth looking into.
> >> Alternatively, I'd definitely recommend ffmpeg for anything like this
> >> sort of job. But if you actually need to OCR something, then you may
> >> need to do some scripting work. I don't have code to offer you, but it
> >> would involve FFMPEG to lift the images, something like Tesseract to
> >> do the actual OCRing, and then you'd write the rest of it yourself in
> >> Python.
> >>
> >> Other than that, this probably is something best done with a dedicated
> >> movie editing tool, not Python. Use what exists.
> >>
> >> ChrisA
> > I want to extract subtitles from a MPEG video (which does not have any 
> > previous subtitles)
>
> If it has no subtitles there is nothing to extract?
>
> I recall that in MPEG subtitles are RLE encoded bitmaps with timing and 
> position data.
> Which allows the player to show this bitmap at position X, Y starting at T0 
> and remove at t1 etc.
> You have to track multiple subtitles at the same time.
>
> You should be able to extract the subtitle bit maps and timing data with 
> modest work.
> You could use OCR technology to turn the subtitles into text.

That's what I was thinking of. I have a separate project that involves
grabbing image frames from the subtitles track, running them through
Tesseract (for OCR), and attempting to intelligently parse two
concurrent tracks of subtitles. Probably more complicated than needed
here though.

I don't understand the OP's request though. Extract subtitles when
there aren't any?

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 3 how to convert a list of bytes objects to a list of strings?

[Grant Edwards]

On 2020-08-27, Marco Sulla  wrote:
> Are you sure you want `str()`?
>
 str(b'aaa')
> "b'aaa'"
>
> Probably you want:
>
> map(lambda x: x.decode(), bbb)

If you're an old Scheme or Lisp programmer. :)

This is probably the more usual way to spell it:

   sss = [x.decode() for x in bbb]



-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 3 how to convert a list of bytes objects to a list of strings?

[Grant Edwards]

On 2020-08-27, Chris Green  wrote:

> bbb = [b'aaa', b'bbb', b'ccc']
> sss = []
> for i in range(0, 2):
> sss.append(str(bbb[i])
>
> but that does seem a bit clumsy.  Is there a better way?

sss = [str(s) for s in bbb]

--
Grant

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 3 how to convert a list of bytes objects to a list of strings?

[Grant Edwards]

On 2020-08-28, Chris Green  wrote:

> Maybe I shouldn't but Python 2 has been managing to do so for several
> years without any issues.  I know I *could* put the exceptions in a
> bucket somewhere and deal with them separately but I'd really rather
> not.

Then just leave it as bytes and do whatever processing you used to do
on the bytes instead of trying to converting it to a string.  What is
it you're doing that you think requires it be converted to a string?

--
Grant



-- 
https://mail.python.org/mailman/listinfo/python-list

Threading plus multiprocessing plus cv2 error

[John O'Hagan]

Dear list

Thanks to this list, I haven't needed to ask a question for
a very long time, but this one has me stumped.

Here's the minimal 3.8 code, on Debian testing:

-
from multiprocessing import Process
from threading import Thread
from time import sleep
import cv2

def show(im, title, location):
cv2.startWindowThread()
cv2.namedWindow(title)
cv2.moveWindow(title, *location)
cv2.imshow(title, im)
sleep(2) #just to keep window open

im1 = cv2.imread('/path/to/image1')
im2 = cv2.imread('/path/to/image2')

Thread(target=show, args=(im1, 'im1', (600,0))).start()
sleep(1)
Process(target=show, args=(im2, 'im2', (0, 0))).start()
-

Here's the error:

-
[xcb] Unknown sequence number while processing queue
[xcb] Most likely this is a multi-threaded client and XInitThreads has
not been called 
[xcb] Aborting, sorry about that.
python3: ../../src/xcb_io.c:260: poll_for_event: Assertion
`!xcb_xlib_threads_sequence_lost' failed.
-

There's no error without the sleep(1), nor if the Process is started
before the Thread, nor if two Processes are used instead, nor if two
Threads are used instead. IOW the error only occurs if a Thread is
started first, and a Process is started a little later.

Any ideas what might be causing the error?

Thanks.

--

John
-- 
https://mail.python.org/mailman/listinfo/python-list

FW: pip not recognized

[gpostle500]

Sent from my Verizon, Samsung Galaxy smartphone
 Original message From: geoff postle  
Date: 8/27/20  12:08 PM  (GMT-08:00) To: python-list@python.org Subject: pip 
not recognized Hey just downloaded the 3.8.5 and when I search for pip in cmd 
it doesn’t show up and says not recognized. Trying to figure out why that is 
since I didn’t change the default selection and left the box with pip checked  
Sent from Mail for Windows 10 
-- 
https://mail.python.org/mailman/listinfo/python-list

Proposal: Ignore all empty entries within $PYTHONPATH

[David A. Wheeler]

All: I propose that all empty entries with $PYTHONPATH be ignored.
This would eliminate some accidents that can lead to security problems.
I thought it'd be easiest to explain by drafting a PEP, so please see
this first draft below.

Thanks! Comments welcome.

--- David A. Wheeler

=

PEP: 
Title: Ignore all empty entries within ``$PYTHONPATH``
Author: David A. Wheeler 
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 27-Aug-2020
Python-Version: 3.7.10
Post-History: 27-Aug-2020


Abstract


Currently, common ways to set ``$PYTHONPATH`` also unexpectedly add the
current directory as an entry.  As a result, users have an increased
risk of unintentionally running malicious code.

This PEP proposes that all empty entries within ``$PYTHONPATH`` be ignored,
instead of being treated as the current directory.
In the rare cases where the current directory is intended, "." (or
even better its absolute path) can continue to be used instead.
This small change eliminates an easily-made and subtle mistake that can
lead to a security vulnerability.

Motivation
==

The essay "Never Run 'python' In Your Downloads Folder" by Glyph
`Never Run`_
points out that the way Python currently implements ``$PYTHONPATH``
easily leads to users unintentionally running malicioous code.
The problem is that "most places that [recommend using] PYTHONPATH
recommend adding things to it like so":

``export PYTHONPATH="/new/useful/stuff:$PYTHONPATH"``

As the essay notes, "this idiom has a critical flaw:
the first time it's invoked, if ``$PYTHONPATH`` was previously
either empty or un-set, this then includes an empty string,
which resolves to the current directory..."
As a result, someone who executed this line above will quietly execute
Python libraries in whatever their current directory happens to be,
and that could quickly lead to running malicious code.

It could be argued that this is fine, because the shell's ``$PATH``
does the same thing.
After all, the current documentation for ``$PYTHONPATH`` says that it
augments "the default search path for module files.  The format is
the same as the shell’s PATH: one or more directory pathnames
separated by os.pathsep (e.g. colons on Unix or semicolons on
Windows)."
`Cmdline`_

But this argument ignores a key difference: the shell's ``$PATH`` practically
*always* has an initial non-empty value, while ``$PYTHONPATH`` typically
starts with an *empty* value.
This means that the same patterns that are normally safe with ``$PATH``
(because ``$PATH`` is non-empty) are *dangerous* with ``$PYTHONPATH``
`Hacker News`_.
Once ``$PYTHONPATH`` is set with a dangerous value, it is likely to stay
dangerous after having other values appended.

A far safer approach is for Python to simply skip all empty entries
within ``$PYTHONPATH``.
In this situation, a ``$PYTHONPATH`` with the value "``:spam::eggs:``"
would be treated the same as "``spam:eggs``".
Empty entries are almost never intended.
They are also widely confusing, in part because they are not obvious.
Users who truly want to include the runtime current directory
can use "." instead, but in almost all cases they will want the
absolute path to the current directory anyway.

By ignoring all empty entries, Python users will be quietly protected
from this mistake, making Python just a little easier to use securely.

Setting ``$PYTHONPATH`` is less common (due to virtualenvs), but it's
still in use and is a useful mechanism.
It'd be best to quietly interpret empty entries in ``$PYTHONPATH``
as unintentional mistakes and ignore them, instead of quietly
enabling security vulnerabilities.

Implementing this is trivial. I have created this PEP because this
is technically a behavioral change in a long-present mechanism.


References
==

.. _Never Run: 
https://glyph.twistedmatrix.com/2020/08/never-run-python-in-your-downloads-folder.html
 by Glyph
.. _Cmdline: https://docs.python.org/3/using/cmdline.html
.. _Hacker News: https://news.ycombinator.com/item?id=24250418


Copyright
=

This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End:
-- 
https://mail.python.org/mailman/listinfo/python-list

Can't use interpreter

[Joshua Sharma]

I have downloaded python 3.7.6, but I can't seem to use it in PyCharm. I
have subscribed.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 3 how to convert a list of bytes objects to a list of strings?

[Chris Green]

Dennis Lee Bieber  wrote:
> On Fri, 28 Aug 2020 12:26:07 +0100, Chris Green  declaimed the
> following:
> 
> 
> 
> >Maybe I shouldn't but Python 2 has been managing to do so for several
> >years without any issues.  I know I *could* put the exceptions in a
> >bucket somewhere and deal with them separately but I'd really rather
> >not.
> >
> 
> In Python2 "string" IS BYTE-STRING. It is never UNICODE, and ignores
> any encoding.
> 
> So, for Python3, the SAME processing requires NOT USING "string" 
> (which
> is now Unicode) and ensuring that all literals are b"stuff", and using the
> methods of the bytes data type.
> 
Now I'm beginning to realise that *this* may well be what I need to
do, after going round in several convoluted circles! :-)

Thank you!

-- 
Chris Green
·
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 3 how to convert a list of bytes objects to a list of strings?

[Chris Green]

Chris Green  wrote:
> Dennis Lee Bieber  wrote:
> > On Fri, 28 Aug 2020 12:26:07 +0100, Chris Green  declaimed 
> > the
> > following:
> > 
> > 
> > 
> > >Maybe I shouldn't but Python 2 has been managing to do so for several
> > >years without any issues.  I know I *could* put the exceptions in a
> > >bucket somewhere and deal with them separately but I'd really rather
> > >not.
> > >
> > 
> > In Python2 "string" IS BYTE-STRING. It is never UNICODE, and ignores
> > any encoding.
> > 
> > So, for Python3, the SAME processing requires NOT USING "string" 
> > (which
> > is now Unicode) and ensuring that all literals are b"stuff", and using the
> > methods of the bytes data type.
> > 
> Now I'm beginning to realise that *this* may well be what I need to
> do, after going round in several convoluted circles! :-)
> 
However the problem appears to be that internally in Python 3 mailbox
class there is an assumption that it's being given 'ascii'.  Here's
the error (and I'm doing no processing of the message at all):-

Traceback (most recent call last):
  File "/home/chris/.mutt/bin/filter.py", line 102, in 
mailLib.deliverMboxMsg(dest, msg, log)
  File "/home/chris/.mutt/bin/mailLib.py", line 52, in deliverMboxMsg
mbx.add(msg)
  File "/usr/lib/python3.8/mailbox.py", line 603, in add
self._toc[self._next_key] = self._append_message(message)
  File "/usr/lib/python3.8/mailbox.py", line 758, in _append_message
offsets = self._install_message(message)
  File "/usr/lib/python3.8/mailbox.py", line 830, in _install_message
self._dump_message(message, self._file, self._mangle_from_)
  File "/usr/lib/python3.8/mailbox.py", line 215, in _dump_message
gen.flatten(message)
  File "/usr/lib/python3.8/email/generator.py", line 116, in flatten
self._write(msg)
  File "/usr/lib/python3.8/email/generator.py", line 181, in _write
self._dispatch(msg)
  File "/usr/lib/python3.8/email/generator.py", line 214, in _dispatch
meth(msg)
  File "/usr/lib/python3.8/email/generator.py", line 432, in
_handle_text
super(BytesGenerator,self)._handle_text(msg)
  File "/usr/lib/python3.8/email/generator.py", line 249, in
_handle_text
self._write_lines(payload)
  File "/usr/lib/python3.8/email/generator.py", line 155, in
_write_lines
self.write(line)
  File "/usr/lib/python3.8/email/generator.py", line 406, in write
self._fp.write(s.encode('ascii', 'surrogateescape'))
UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-3: 
ordinal not in range(128)

Any message with other than ASCII in it is going to have bytes >128
unless it's encoded some way to make it 7-bit and that's not going to
happen in the general case.

-- 
Chris Green
·
-- 
https://mail.python.org/mailman/listinfo/python-list

Silly question, where is read() documented?

[Chris Green]

Well it sounds a silly question but I can't find the documentation for
read().  It's not a built-in function and it's not documented with
(for example) the file type object sys.stdin.

So where is it documented?  :-)

-- 
Chris Green
·
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Silly question, where is read() documented?

[Ian Hobson]

https://docs.python.org/3/tutorial/inputoutput.html#methods-of-file-objects

(It is in the top result returned by Google, searching for
Python read documentation)

On 29/08/2020 17:18, Chris Green wrote:

Well it sounds a silly question but I can't find the documentation for
read().  It's not a built-in function and it's not documented with
(for example) the file type object sys.stdin.

So where is it documented?  :-)



--
Ian Hobson


--
This email has been checked for viruses by AVG.
https://www.avg.com

--
https://mail.python.org/mailman/listinfo/python-list

Re: Can't use interpreter

[Mats Wichmann]

On 8/28/20 5:02 AM, Joshua Sharma wrote:
> I have downloaded python 3.7.6, but I can't seem to use it in PyCharm. I
> have subscribed.
> 

See the PyCharm instructions for setting an interpreter, which can be
default, and per-project.

Unless there was more to this question that you didn't ask...


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: FW: pip not recognized

[Mats Wichmann]

On 8/27/20 2:18 PM, gpostle500 wrote:
> 
> Sent from my Verizon, Samsung Galaxy smartphone
>  Original message From: geoff postle  
> Date: 8/27/20  12:08 PM  (GMT-08:00) To: python-list@python.org Subject: pip 
> not recognized Hey just downloaded the 3.8.5 and when I search for pip in cmd 
> it doesn’t show up and says not recognized. Trying to figure out why that is 
> since I didn’t change the default selection and left the box with pip checked 
>  Sent from Mail for Windows 10 
> 

Use

py -m pip


pip-the-command goes in a different directory on Windows, which probably
didn't get added to your PATH - but you probably don't want to do that
anyway.

e.g. if Python was in

\Users\geoff\AppData\Local\Progrms\Python38

then the pip command will be in

\Users\geoff\AppData\Local\Progrms\Python38\Scripts


But use it as a module - you have a better chance of package installs
staying in sync with the version of Python you use, eventually there
will probably be several of those

-- 
https://mail.python.org/mailman/listinfo/python-list

Finally fixed my Python 2 to Python 3 e-mail problem

[Chris Green]

I've started a new thread because this relates to two or three threads
I started here over the past few days.

First, thank you everyone for all the help and suggestions.


I've finally fixed the problem, it was due to sys.stdin.read()
returning a string object in Python 3 as opposed to bytes in Python 2.

So, the code that reads an incoming message was:-

msg = mailbox.mboxMessage(sys.stdin.read())

... and msg eventually gets passed into mailbox.mbox.add(msg)

This works fine in Python 2 but fails with a "can't encode" error in
Python 3 (deep down in the mailbox code).


The cause of the problem is that Python 3's sys.stdin.read() returns a
string, so changing the above line to:-

msg = mailbox.mboxMessage(sys.stdin.buffer.read()) 

fixes the problem.


It took me quite a while to find the answer, mainly because I was
thinking the wrong way round, trying to make my message into a string,
whereas what I needed to do was keep it as a 'byte string' (I still
think there should be a better name for that!).

Again, thank you everyone for all the help, it did get me there in the
end!

-- 
Chris Green
·
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Where read() is documented

[Chris Green]

Stefan Ram  wrote:
> Chris Green  writes:I can't find the documentation for
> >read().  It's not a built-in function and it's not documented with
> >(for example) the file type object sys.stdin.
> 
> |read() (asyncio.StreamReader method), 894
> |read() (chunk.Chunk method), 1385
> |read() (codecs.StreamReader method), 164
> |read() (configparser.ConfigParser method), 537
> |read() (http.client.HTTPResponse method), 1276
> |read() (imaplib.IMAP4 method), 1291
> |read() (in module os), 578
> |read() (io.BufferedIOBase method), 622
> |read() (io.BufferedReader method), 625
> |read() (io.RawIOBase method), 621
> |read() (io.TextIOBase method), 626
> |read() (mimetypes.MimeTypes method), 1146
> |read() (mmap.mmap method), 1053
> |read() (ossaudiodev.oss_audio_device method), 1388
> |read() (ssl.MemoryBIO method), 1024
> |read() (ssl.SSLSocket method), 1005
> |read() (urllib.robotparser.RobotFileParser method), 1268
> |read() (zipfile.ZipFile method), 499
> Index of "The Python Library Reference, Release 3.9.0a3"
> 
> 
But none of those is the documentation for read(), they're just places
that refer to read().

-- 
Chris Green
·
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 3 how to convert a list of bytes objects to a list of strings?

[Richard Damon]

On 8/29/20 11:50 AM, Chris Green wrote:
> Chris Green  wrote:
>> Dennis Lee Bieber  wrote:
>>> On Fri, 28 Aug 2020 12:26:07 +0100, Chris Green  declaimed 
>>> the
>>> following:
>>>
>>>
>>>
 Maybe I shouldn't but Python 2 has been managing to do so for several
 years without any issues.  I know I *could* put the exceptions in a
 bucket somewhere and deal with them separately but I'd really rather
 not.

>>> In Python2 "string" IS BYTE-STRING. It is never UNICODE, and ignores
>>> any encoding.
>>>
>>> So, for Python3, the SAME processing requires NOT USING "string" 
>>> (which
>>> is now Unicode) and ensuring that all literals are b"stuff", and using the
>>> methods of the bytes data type.
>>>
>> Now I'm beginning to realise that *this* may well be what I need to
>> do, after going round in several convoluted circles! :-)
>>
> However the problem appears to be that internally in Python 3 mailbox
> class there is an assumption that it's being given 'ascii'.  Here's
> the error (and I'm doing no processing of the message at all):-
>
> Traceback (most recent call last):
>   File "/home/chris/.mutt/bin/filter.py", line 102, in 
> mailLib.deliverMboxMsg(dest, msg, log)
>   File "/home/chris/.mutt/bin/mailLib.py", line 52, in deliverMboxMsg
> mbx.add(msg)
>   File "/usr/lib/python3.8/mailbox.py", line 603, in add
> self._toc[self._next_key] = self._append_message(message)
>   File "/usr/lib/python3.8/mailbox.py", line 758, in _append_message
> offsets = self._install_message(message)
>   File "/usr/lib/python3.8/mailbox.py", line 830, in _install_message
> self._dump_message(message, self._file, self._mangle_from_)
>   File "/usr/lib/python3.8/mailbox.py", line 215, in _dump_message
> gen.flatten(message)
>   File "/usr/lib/python3.8/email/generator.py", line 116, in flatten
> self._write(msg)
>   File "/usr/lib/python3.8/email/generator.py", line 181, in _write
> self._dispatch(msg)
>   File "/usr/lib/python3.8/email/generator.py", line 214, in _dispatch
> meth(msg)
>   File "/usr/lib/python3.8/email/generator.py", line 432, in
> _handle_text
> super(BytesGenerator,self)._handle_text(msg)
>   File "/usr/lib/python3.8/email/generator.py", line 249, in
> _handle_text
> self._write_lines(payload)
>   File "/usr/lib/python3.8/email/generator.py", line 155, in
> _write_lines
> self.write(line)
>   File "/usr/lib/python3.8/email/generator.py", line 406, in write
> self._fp.write(s.encode('ascii', 'surrogateescape'))
> UnicodeEncodeError: 'ascii' codec can't encode characters in position 
> 0-3: ordinal not in range(128)
>
> Any message with other than ASCII in it is going to have bytes >128
> unless it's encoded some way to make it 7-bit and that's not going to
> happen in the general case.
>
When I took a quick look at the mailbox class, it said it could take a
'string', or a 'message'. It may well be that the string option assumes
ASCII. You may need to use the message parsing options of message to
convert messages with extended characters into the right format. This is
one of the cases where Python 2's non-strictness made things easier, but
also much easier to get wrong if not careful. Python 3 is basically
making you do more work to make sure you are doing it right.

-- 
Richard Damon

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Where read() is documented

[MRAB]

On 2020-08-29 17:48, Chris Green wrote:

Stefan Ram  wrote:

Chris Green  writes:I can't find the documentation for
>read().  It's not a built-in function and it's not documented with
>(for example) the file type object sys.stdin.

|read() (asyncio.StreamReader method), 894
|read() (chunk.Chunk method), 1385
|read() (codecs.StreamReader method), 164
|read() (configparser.ConfigParser method), 537
|read() (http.client.HTTPResponse method), 1276
|read() (imaplib.IMAP4 method), 1291
|read() (in module os), 578
|read() (io.BufferedIOBase method), 622
|read() (io.BufferedReader method), 625
|read() (io.RawIOBase method), 621
|read() (io.TextIOBase method), 626
|read() (mimetypes.MimeTypes method), 1146
|read() (mmap.mmap method), 1053
|read() (ossaudiodev.oss_audio_device method), 1388
|read() (ssl.MemoryBIO method), 1024
|read() (ssl.SSLSocket method), 1005
|read() (urllib.robotparser.RobotFileParser method), 1268
|read() (zipfile.ZipFile method), 499
Index of "The Python Library Reference, Release 3.9.0a3"



But none of those is the documentation for read(), they're just places
that refer to read().

There's no read() function. What you're referring to are the 'read' 
methods of various classes.


If you open a file in text mode, you'll get an instance of 
TextIOWrapper, which inherits .read from TextIOBase.


If you open a file in binary mode, you'll get an instance of 
BufferedReader, which has a .read method.


Multiple classes, each with its own 'read' method.

sys.stdin is an instance of TextIOWrapper, so for that you should look 
at the methods of TextIOWrapper.

--
https://mail.python.org/mailman/listinfo/python-list

Aw: Re: Python 3 how to convert a list of bytes objects to a list of strings?

[Karsten Hilbert]

> However the problem appears to be that internally in Python 3 mailbox
> class there is an assumption that it's being given 'ascii'.

Do you really _need_ the mailbox class ? From what you've
written so far my understanding was that you receive data
(bytes) and want to append that to a file (which happens
to be an mbox).

Can't you "just do that" ?

IOW, read the bytes, open the file, dump the bytes, close the file ?

Karsten
-- 
https://mail.python.org/mailman/listinfo/python-list

Aw: Re: Video file to subtitles file

[Karsten Hilbert]

> I want to extract subtitles from a MPEG video (which does not have any 
> previous subtitles) and then add them to the same video .

I am not sure I parse the above: You want to *extract* subtitles
from a video which *does not have* subtitles ?

I have a feeling you will need to rephrase your objective
to get better help.

Karsten

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Aw: Re: Python 3 how to convert a list of bytes objects to a list of strings?

[Richard Damon]

On 8/29/20 3:31 PM, Karsten Hilbert wrote:
>> However the problem appears to be that internally in Python 3 mailbox
>> class there is an assumption that it's being given 'ascii'.
> Do you really _need_ the mailbox class ? From what you've
> written so far my understanding was that you receive data
> (bytes) and want to append that to a file (which happens
> to be an mbox).
>
> Can't you "just do that" ?
>
> IOW, read the bytes, open the file, dump the bytes, close the file ?
>
> Karsten

Just appending a message as a raw file to a mailbox, doesn't properly
add it as a new message. You need to add a From: line to the front, and
then go through the message and alter any line that begins as "From:"
(and possibly any line that begins with something like ">From:" or
">>From:" depending on which mailbox format is being used. There may be
a few other small details that needs to happen to.

-- 
Richard Damon

-- 
https://mail.python.org/mailman/listinfo/python-list

Aw: Re: Re: Python 3 how to convert a list of bytes objects to a list of strings?

[Karsten Hilbert]

> Just appending a message as a raw file to a mailbox, doesn't properly
> add it as a new message. You need to add a From: line to the front, and
> then go through the message and alter any line that begins as "From:"
> (and possibly any line that begins with something like ">From:" or
> ">>From:" depending on which mailbox format is being used. There may be
> a few other small details that needs to happen to.

I see, thanks.

Karsten
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 3 how to convert a list of bytes objects to a list of strings?

[Cameron Simpson]

On 29Aug2020 16:50, Chris Green  wrote:
>However the problem appears to be that internally in Python 3 mailbox
>class there is an assumption that it's being given 'ascii'.  Here's
>the error (and I'm doing no processing of the message at all):-
>
>Traceback (most recent call last):
>  File "/home/chris/.mutt/bin/filter.py", line 102, in 
>mailLib.deliverMboxMsg(dest, msg, log)
>  File "/home/chris/.mutt/bin/mailLib.py", line 52, in deliverMboxMsg
>mbx.add(msg)
[...]

Here is the entire save-to-mbox code form my own mailfiler:

text = M.as_string(True).replace('\nFrom ', '\n>From ')
with open(folderpath, "a") as mboxfp:
  mboxfp.write(text)

where M is the current message, a Message object.

Note that this does _not_ assume ASCII output. The process here is:

- transcribe the message to a Python 3 str (so Unicode code points)
- replace embedded "From " to protect the mbox format
- open the mbox for append - the _default_ encoding is utf-8
- write the message in utf-8 because of the open mode

This sidesteps the library you're using which may well do something 
ASCII based. And it has _never_ failed for me.

Cheers,
Cameron Simpson 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Where read() is documented

[Joe Pfeiffer]

Chris Green  writes:

> Stefan Ram  wrote:
>> Chris Green  writes:I can't find the documentation for
>> >read().  It's not a built-in function and it's not documented with
>> >(for example) the file type object sys.stdin.
>> 
>> |read() (asyncio.StreamReader method), 894
>> |read() (chunk.Chunk method), 1385
>> |read() (codecs.StreamReader method), 164
>> |read() (configparser.ConfigParser method), 537
>> |read() (http.client.HTTPResponse method), 1276
>> |read() (imaplib.IMAP4 method), 1291
>> |read() (in module os), 578
>> |read() (io.BufferedIOBase method), 622
>> |read() (io.BufferedReader method), 625
>> |read() (io.RawIOBase method), 621
>> |read() (io.TextIOBase method), 626
>> |read() (mimetypes.MimeTypes method), 1146
>> |read() (mmap.mmap method), 1053
>> |read() (ossaudiodev.oss_audio_device method), 1388
>> |read() (ssl.MemoryBIO method), 1024
>> |read() (ssl.SSLSocket method), 1005
>> |read() (urllib.robotparser.RobotFileParser method), 1268
>> |read() (zipfile.ZipFile method), 499
>> Index of "The Python Library Reference, Release 3.9.0a3"
>> 
>> 
> But none of those is the documentation for read(), they're just places
> that refer to read().

There is no single read() method, so there can be no single place to
find its documentation.  To take one of Stefan's examples,

https://docs.python.org/3/library/io.html?highlight=io%20bufferedreader#io.BufferedReader.read

says

read([size])
Read and return size bytes, or if size is not given or negative, until
EOF or if the read call would block in non-blocking mode.

That's the documentation.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Silly question, where is read() documented?

[Cameron Simpson]

Also:

https://docs.python.org/3/library/io.html#io.TextIOBase.read
https://docs.python.org/3/library/io.html#io.BufferedIOBase.read

Found by going to:

https://docs.python.org/3/
https://docs.python.org/3/genindex.html
https://docs.python.org/3/genindex-R.html

and finding the links to "read".

Personally, I fetch the Python docs every so often from:

https://docs.python.org/3/download.html

I fetch the "HTML" version, unpack it on my machine, and put a link to 
the "index.html" file on my Desktop. Instant, offline-ready, Python docs 
on my machine. Really snappy, because my browser's pulling from the 
local filesystem.

Cheers,
Cameron Simpson 

On 29Aug2020 17:33, Ian Hobson  wrote:
>https://docs.python.org/3/tutorial/inputoutput.html#methods-of-file-objects
>
>(It is in the top result returned by Google, searching for
>Python read documentation)
>
>On 29/08/2020 17:18, Chris Green wrote:
>>Well it sounds a silly question but I can't find the documentation for
>>read().  It's not a built-in function and it's not documented with
>>(for example) the file type object sys.stdin.
>>
>>So where is it documented?  :-)
>
>-- Ian Hobson
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Silly question, where is read() documented?

[Terry Reedy]

On 8/29/2020 12:18 PM, Chris Green wrote:

Well it sounds a silly question but I can't find the documentation for
read().  It's not a built-in function and it's not documented with
(for example) the file type object sys.stdin.


sys.stdin is of no particular type, but must at least have a .read method.


So where is it documented?  :-)


>>> import sys; sys.stdin
should give a hint.  In the standard REPL,

<_io.TextIOWrapper name='' mode='r' encoding='utf-8'>

As others said, actually look in the io module doc.  You might spend 
some time reading the doc to get an idea of what is going on.  If you 
run from IDLE, you currently get



(I should make that more like the REPL answer.)

--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: Video file to subtitles file

[Christian Gollwitzer]

Am 29.08.20 um 13:51 schrieb Muskan Sanghai:

I want to extract subtitles from a MPEG video (which does not have any previous 
subtitles)


I'm still not sure I get it. "Extract" subtitles, when they are NOT 
there? Can it be, by any chance, that you are talking about speech 
recognition? I.e., you want a software which understands the spoken word 
in the movie sound and turns that into text, which can be shown as 
subtitles? Like the "auto-generated" subtitles which youtube offers for 
some videos.


If so, it is a complex task and will not work overly well. I defer to 
the experts if there are any usable speech recognitino engines for this 
task.


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: Threading plus multiprocessing plus cv2 error

[Stephane Tougard via Python-list]

On 2020-08-29, Dennis Lee Bieber  wrote:
>   Under Linux, multiprocessing creates processes using fork(). That means
> that, for some fraction of time, you have TWO processes sharing the same
> thread and all that entails (if it doesn't overlay the forked process with
> a new executable, they are sharing the thread until the thread exits).
> same error condition even with the sleep(1) in place.

I'm not even that makes sense, how 2 processes can share a thread ?


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Threading plus multiprocessing plus cv2 error

[Chris Angelico]

On Sun, Aug 30, 2020 at 4:01 PM Stephane Tougard via Python-list
 wrote:
>
> On 2020-08-29, Dennis Lee Bieber  wrote:
> >   Under Linux, multiprocessing creates processes using fork(). That 
> > means
> > that, for some fraction of time, you have TWO processes sharing the same
> > thread and all that entails (if it doesn't overlay the forked process with
> > a new executable, they are sharing the thread until the thread exits).
> > same error condition even with the sleep(1) in place.
>
> I'm not even that makes sense, how 2 processes can share a thread ?
>

They can't. However, they can share a Thread object, which is the
Python representation of a thread. That can lead to confusion, and
possibly the OP's error (I don't know for sure, I'm just positing).

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Video file to subtitles file

[Muskan Sanghai]

On Sunday, August 30, 2020 at 10:57:00 AM UTC+5:30, Christian Gollwitzer wrote:
> Am 29.08.20 um 13:51 schrieb Muskan Sanghai:
> > I want to extract subtitles from a MPEG video (which does not have any 
> > previous subtitles)
> I'm still not sure I get it. "Extract" subtitles, when they are NOT 
> there? Can it be, by any chance, that you are talking about speech 
> recognition? I.e., you want a software which understands the spoken word 
> in the movie sound and turns that into text, which can be shown as 
> subtitles? Like the "auto-generated" subtitles which youtube offers for 
> some videos. 
> 
> If so, it is a complex task and will not work overly well. I defer to 
> the experts if there are any usable speech recognitino engines for this 
> task. 
> 
> Christian
Yes, this is what I exactly want to do. I want to create a software which 
understands the spoken word in the movie sound and turns that into text.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Video file to subtitles file

[Chris Angelico]

On Sun, Aug 30, 2020 at 4:11 PM Muskan Sanghai  wrote:
>
> On Sunday, August 30, 2020 at 10:57:00 AM UTC+5:30, Christian Gollwitzer 
> wrote:
> > Am 29.08.20 um 13:51 schrieb Muskan Sanghai:
> > > I want to extract subtitles from a MPEG video (which does not have any 
> > > previous subtitles)
> > I'm still not sure I get it. "Extract" subtitles, when they are NOT
> > there? Can it be, by any chance, that you are talking about speech
> > recognition? I.e., you want a software which understands the spoken word
> > in the movie sound and turns that into text, which can be shown as
> > subtitles? Like the "auto-generated" subtitles which youtube offers for
> > some videos.
> >
> > If so, it is a complex task and will not work overly well. I defer to
> > the experts if there are any usable speech recognitino engines for this
> > task.
> >
> > Christian
> Yes, this is what I exactly want to do. I want to create a software which 
> understands the spoken word in the movie sound and turns that into text.
>

I recommend looking into CMU Sphinx then. I've used that from Python.

The results are highly entertaining.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Video file to subtitles file

[Muskan Sanghai]

On Sunday, August 30, 2020 at 11:46:15 AM UTC+5:30, Chris Angelico wrote:
> On Sun, Aug 30, 2020 at 4:11 PM Muskan Sanghai  wrote: 
> > 
> > On Sunday, August 30, 2020 at 10:57:00 AM UTC+5:30, Christian Gollwitzer 
> > wrote: 
> > > Am 29.08.20 um 13:51 schrieb Muskan Sanghai: 
> > > > I want to extract subtitles from a MPEG video (which does not have any 
> > > > previous subtitles) 
> > > I'm still not sure I get it. "Extract" subtitles, when they are NOT 
> > > there? Can it be, by any chance, that you are talking about speech 
> > > recognition? I.e., you want a software which understands the spoken word 
> > > in the movie sound and turns that into text, which can be shown as 
> > > subtitles? Like the "auto-generated" subtitles which youtube offers for 
> > > some videos. 
> > > 
> > > If so, it is a complex task and will not work overly well. I defer to 
> > > the experts if there are any usable speech recognitino engines for this 
> > > task. 
> > > 
> > > Christian 
> > Yes, this is what I exactly want to do. I want to create a software which 
> > understands the spoken word in the movie sound and turns that into text. 
> >
> I recommend looking into CMU Sphinx then. I've used that from Python. 
> 
> The results are highly entertaining. 
> 
> ChrisA
Okay I will try it, thank you.
-- 
https://mail.python.org/mailman/listinfo/python-list