Re: Confusing textwrap parameters, and request for RE help

[Chris Angelico]

On Tue, Mar 24, 2020 at 4:36 AM Dieter Maurer  wrote:
>
> Chris Angelico wrote at 2020-3-23 06:00 +1100:
> >When using textwrap.fill() or friends, setting break_long_words=False
> >without also setting break_on_hyphens=False has the very strange
> >behaviour that a long hyphenated word will still be wrapped.
>
> Having worked with `TeX`, I am familiar that hyphens indicate
> word locations favorable to line breaks. Therefore, I understand
> that one wants a way to control whether or not breaks are allowed
> at hyphens, even if words should not be broken normally.
>
> Apparently, "break_on_hyphens" adds hyphen as a word delimiter.
> With this notion, the behavior can be understood.

Well um... yes. I think we know that hyphens do indicate word-split
points. That's not really in question. But if I say "don't break long
words", and then long words get broken at hyphens, that's surprising.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Confusing textwrap parameters, and request for RE help

[Karsten Hilbert]

On Tue, Mar 24, 2020 at 08:08:31PM +1100, Chris Angelico wrote:

> Well um... yes. I think we know that hyphens do indicate word-split
> points. That's not really in question.

I know you don't mean it like that, but it sounds equally
future-proof like "all text is ASCII".

Karsten
--
GPG  40BE 5B0E C98E 1713 AFA6  5BC0 3BEA AC80 7D4F C89B
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Confusing textwrap parameters, and request for RE help

[Chris Angelico]

On Tue, Mar 24, 2020 at 8:19 PM Karsten Hilbert  wrote:
>
> On Tue, Mar 24, 2020 at 08:08:31PM +1100, Chris Angelico wrote:
>
> > Well um... yes. I think we know that hyphens do indicate word-split
> > points. That's not really in question.
>
> I know you don't mean it like that, but it sounds equally
> future-proof like "all text is ASCII".
>

Sure, but I'm not really disputing that part. You can disable it anyway.

I'm trying to figure out a way to handle URLs, and that's something
that has its own governing standard, so the meanings of characters
like hyphens is well defined.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Confusing textwrap parameters, and request for RE help

[Karsten Hilbert]

On Tue, Mar 24, 2020 at 08:20:32PM +1100, Chris Angelico wrote:

> > > Well um... yes. I think we know that hyphens do indicate word-split
> > > points. That's not really in question.
> >
> > I know you don't mean it like that, but it sounds equally
> > future-proof like "all text is ASCII".
>
> Sure, but I'm not really disputing that part. You can disable it anyway.
>
> I'm trying to figure out a way to handle URLs, and that's something
> that has its own governing standard, so the meanings of characters
> like hyphens is well defined.

Agreed.

More to the point, if anyone knows how to typographically
properly handle this it's the TeX community. Here's a thread:

https://tex.stackexchange.com/questions/3033/forcing-linebreaks-in-url

URL shorteners work but suffer from link rot.

Ellipsised URLs plus footnote would work but don't lend
themselves to textwrap() very well (but perhaps to each
individual application of textwrap()).

Karsten
--
GPG  40BE 5B0E C98E 1713 AFA6  5BC0 3BEA AC80 7D4F C89B
-- 
https://mail.python.org/mailman/listinfo/python-list

How to create an Excel app that runs Python?

[farayao . ds]

I have the following scenario:

I have created lots of python files that I use to calculate a Cashflow model, 
when I run these files I get a beautiful pandas DataFrame that contains my 
final model. My mission is to show this table to the rest of the company in a 
friendly format ("Excel") and they need to be able to generate this table 
themselves from Excel, using the Python script I created ("the idea is that 
they open an excel file that has some type of dashboard and a form that they 
need to fill in, once they fill it in and press "Go" or "Run" or something like 
that, these parameters will be sent to the Python script to generate this final 
pandas DataFrame. Finally this DataFrame needs to be displayed/pasted in the 
Excel sheet that they initially opened)


The problems:

The company needs to be able to run this model, but the users, apart from me, 
use Excel and they can't open up a Jupyter notebook or VSC to run the code. 
They can't even install Python on their machines (or at least that is not ideal)


My Attempts:

I am currently using "xlwings" to run Python within Excel, although it requires 
that the user has Python and xlwings installed and has added the xlwings 
"Addin", which is not an ideal solution at all.

I am thinking about using Dash, I don't know if it is what I need since I 
haven't looked into it that much yet (I have been trying to make it work on 
xlwings first)



If you have any suggestions on how to do this, I would really appreciate that.

Thank you
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Intermittent bug with asyncio and MS Edge

[Frank Millman]

On 2020-03-23 1:56 PM, Frank Millman wrote:

On 2020-03-23 12:57 PM, Chris Angelico wrote:

On Mon, Mar 23, 2020 at 8:03 PM Frank Millman  wrote:


On 2020-03-22 12:11 PM, Chris Angelico wrote:
On Sun, Mar 22, 2020 at 8:30 PM Frank Millman  
wrote:


On 2020-03-22 10:45 AM, Chris Angelico wrote:


If you can recreate the problem with a single socket and multiple
requests, that would be extremely helpful. I also think it's highly
likely that this is the case.



I am working on a stripped-down version, but I realise there are a few
things I have not grasped.

Hope you don't mind, but can you scan through what follows and tell me
if I am on the right lines?


No probs!



[...]

Really appreciate the one-on-one tuition. I am learning a lot!




If this all makes sense, I should write two versions of the client
program, one using a single connection, and one using a pool of 
connections.




Possibly! I think you'll most likely see that one of those behaves
perfectly normally, and you only trigger the issue in the other. So
you could move forward with just one test program.



Well, I have got the first one working - single connection - and so far 
it has not gone wrong.


However, it is difficult to be sure that I am comparing apples with 
apples. I have written my test server to handle 'Keep-Alive' correctly, 
but as I mentioned earlier, my live program closes the connection after 
each transfer. So now I have to make my test server do the same, and 
change my test client to react to that and re-open the connection each 
time. I will make the changes and see how that behaves.


Of course now I am in the murky waters of trying to second-guess how 
Edge reacts to that. Presumably that is where Wireshark will be useful. 
I will keep you posted.


Here is a progress report.

I decided to concentrate on using Wireshark to detect the difference 
between a Python3.7 session and a Python3.8 session. Already I can see 
some differences.


There is only one version of my program. I am simply running it with 
either 'py -3.7 ' or 'py -3.8'. And I am ignoring Chrome at this stage, 
as it is only that Edge shows the problem.


First point - Python3.7 also shows a lot of [RST, ACK] lines. My guess 
is that this is caused by my 'protocol violation' of sending a 
'Keep-Alive' header and then closing the connection. Python3.7 does not 
suffer from dropping files, so I now think this is a sidetrack. I will 
fix my program when this is all over, but for now I don't want to touch it.


When I send the response to the initial connection, I write a status 
line, then multiple header lines, then an html file. I then close the 
connection. Python3.7 sends a packet with the status, then a separate 
packet for each header, then a packet with the file. Python3.8 sends a 
packet with the status, then merges everything else into a single packet 
and sends it in one go. I just mention this as an indication that quite 
a lot has changed between my two versions of Python.


I have one frustration with Wireshark. I will mention it in case anyone 
has a solution.


I can see that Edge opens multiple connections. I am trying to track the 
activity on each connection separately. I can export the data to csv, 
which makes it easier to work on. But while the TCP lines include the 
source and destination ports, the HTTP lines do not, so I don't know 
which connection they belong to. If I view the data in Wireshark's gui 
it does show the ports, so the data is there somewhere. Does anyone know 
how to include it in the csv output?


That's all for now. I will keep you posted.

Frank

--
https://mail.python.org/mailman/listinfo/python-list

Re: Intermittent bug with asyncio and MS Edge

[Frank Millman]

On 2020-03-24 1:54 PM, Frank Millman wrote:

On 2020-03-23 1:56 PM, Frank Millman wrote:

I have one frustration with Wireshark. I will mention it in case anyone 
has a solution.


I can see that Edge opens multiple connections. I am trying to track the 
activity on each connection separately. I can export the data to csv, 
which makes it easier to work on. But while the TCP lines include the 
source and destination ports, the HTTP lines do not, so I don't know 
which connection they belong to. If I view the data in Wireshark's gui 
it does show the ports, so the data is there somewhere. Does anyone know 
how to include it in the csv output?




I solved my Wireshark problem by exporting the data as text. A much more 
wordy format, and fiddly to parse, but the info I am looking for is there.


I am getting some interesting results now, such as a second GET 
interrupting an existing session. However, my brain is now frazzled, so 
I will continue tomorrow.


Frank
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to create an Excel app that runs Python?

[Souvik Dutta]

You might try pyqt5 if you want to make a custom GUI and also if you have
tha time to do so.

On Tue, Mar 24, 2020, 4:45 PM  wrote:

> I have the following scenario:
>
> I have created lots of python files that I use to calculate a Cashflow
> model, when I run these files I get a beautiful pandas DataFrame that
> contains my final model. My mission is to show this table to the rest of
> the company in a friendly format ("Excel") and they need to be able to
> generate this table themselves from Excel, using the Python script I
> created ("the idea is that they open an excel file that has some type of
> dashboard and a form that they need to fill in, once they fill it in and
> press "Go" or "Run" or something like that, these parameters will be sent
> to the Python script to generate this final pandas DataFrame. Finally this
> DataFrame needs to be displayed/pasted in the Excel sheet that they
> initially opened)
>
>
> The problems:
>
> The company needs to be able to run this model, but the users, apart from
> me, use Excel and they can't open up a Jupyter notebook or VSC to run the
> code. They can't even install Python on their machines (or at least that is
> not ideal)
>
>
> My Attempts:
>
> I am currently using "xlwings" to run Python within Excel, although it
> requires that the user has Python and xlwings installed and has added the
> xlwings "Addin", which is not an ideal solution at all.
>
> I am thinking about using Dash, I don't know if it is what I need since I
> haven't looked into it that much yet (I have been trying to make it work on
> xlwings first)
>
>
>
> If you have any suggestions on how to do this, I would really appreciate
> that.
>
> Thank you
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to create an Excel app that runs Python?

[oliver]

Use the win32com library to interact with Excel via COM. Although the
learning curve is a little steeper it's not too bad and works great. I used
this technique to control ARENA with Python through its COM API to run a
whack of simulations (1000-1), then used python to gather the results,
and used win32com from python to finally inject the data into Excel and get
it to generate charts and tables that could be used by stakeholders without
access to python.

Oliver Schoenborn
  || Cloud Application Engineer, Owner || Sentian Cloud Computing Inc.
  || Ottawa, ON, Canada || +1-613-552-4466 (mobile)
  || oli...@sentianse.com




On Tue, Mar 24, 2020 at 7:16 AM  wrote:

> I have the following scenario:
>
> I have created lots of python files that I use to calculate a Cashflow
> model, when I run these files I get a beautiful pandas DataFrame that
> contains my final model. My mission is to show this table to the rest of
> the company in a friendly format ("Excel") and they need to be able to
> generate this table themselves from Excel, using the Python script I
> created ("the idea is that they open an excel file that has some type of
> dashboard and a form that they need to fill in, once they fill it in and
> press "Go" or "Run" or something like that, these parameters will be sent
> to the Python script to generate this final pandas DataFrame. Finally this
> DataFrame needs to be displayed/pasted in the Excel sheet that they
> initially opened)
>
>
> The problems:
>
> The company needs to be able to run this model, but the users, apart from
> me, use Excel and they can't open up a Jupyter notebook or VSC to run the
> code. They can't even install Python on their machines (or at least that is
> not ideal)
>
>
> My Attempts:
>
> I am currently using "xlwings" to run Python within Excel, although it
> requires that the user has Python and xlwings installed and has added the
> xlwings "Addin", which is not an ideal solution at all.
>
> I am thinking about using Dash, I don't know if it is what I need since I
> haven't looked into it that much yet (I have been trying to make it work on
> xlwings first)
>
>
>
> If you have any suggestions on how to do this, I would really appreciate
> that.
>
> Thank you
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Confusing textwrap parameters, and request for RE help

[Dieter Maurer]

Karsten Hilbert wrote at 2020-3-24 10:34 +0100:
>On Tue, Mar 24, 2020 at 08:20:32PM +1100, Chris Angelico wrote:
> ...
>> I'm trying to figure out a way to handle URLs, and that's something
>> that has its own governing standard, so the meanings of characters
>> like hyphens is well defined.
>
>Agreed.
>
>More to the point, if anyone knows how to typographically
>properly handle this it's the TeX community. Here's a thread:
>
>   https://tex.stackexchange.com/questions/3033/forcing-linebreaks-in-url

Page based output devices are more difficult: they have a fixed width.

When the result targets a screen, horizontal scrolling is possible
(even if it is not very convenient). There, preventing urls from being
split may be an option.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to create an Excel app that runs Python?

[Paolo G. Cantore]

On Tue, Mar 24, 2020, 4:45 PM  wrote:


I have the following scenario:

I have created lots of python files that I use to calculate a Cashflow
model, when I run these files I get a beautiful pandas DataFrame that
contains my final model. My mission is to show this table to the rest of
the company in a friendly format ("Excel") and they need to be able to
generate this table themselves from Excel, using the Python script I
created ("the idea is that they open an excel file that has some type of
dashboard and a form that they need to fill in, once they fill it in and
press "Go" or "Run" or something like that, these parameters will be sent
to the Python script to generate this final pandas DataFrame. Finally this
DataFrame needs to be displayed/pasted in the Excel sheet that they
initially opened)


The problems:

The company needs to be able to run this model, but the users, apart from
me, use Excel and they can't open up a Jupyter notebook or VSC to run the
code. They can't even install Python on their machines (or at least that is
not ideal)


My Attempts:

I am currently using "xlwings" to run Python within Excel, although it
requires that the user has Python and xlwings installed and has added the
xlwings "Addin", which is not an ideal solution at all.

I am thinking about using Dash, I don't know if it is what I need since I
haven't looked into it that much yet (I have been trying to make it work on
xlwings first)



If you have any suggestions on how to do this, I would really appreciate
that.

Thank you
--
https://mail.python.org/mailman/listinfo/python-list


The scenario you describe should be fulfilled by a web application.

I'm sure your users are also able to use a browser, not only Excel.

The HTML forms management offers the GUI capabilities to enter the 
parameters. The processing steps could be as follows:


1. Enter the url for the application in question and display the form.
2. Enter the needed data in the form.
3. Process the data (with lots of your python scripts) and show the 
result in the browser.
4. If all is ok, download the data sheet in Excel format (using 
DataFrame.to_excel)


--
https://mail.python.org/mailman/listinfo/python-list

Re: Intermittent bug with asyncio and MS Edge

[Barry Scott]

> On 24 Mar 2020, at 11:54, Frank Millman  wrote:
> 
> On 2020-03-23 1:56 PM, Frank Millman wrote:
>> On 2020-03-23 12:57 PM, Chris Angelico wrote:
>>> On Mon, Mar 23, 2020 at 8:03 PM Frank Millman  wrote:
 
 On 2020-03-22 12:11 PM, Chris Angelico wrote:
> On Sun, Mar 22, 2020 at 8:30 PM Frank Millman  wrote:
>> 
>> On 2020-03-22 10:45 AM, Chris Angelico wrote:
> 
> If you can recreate the problem with a single socket and multiple
> requests, that would be extremely helpful. I also think it's highly
> likely that this is the case.
> 
 
 I am working on a stripped-down version, but I realise there are a few
 things I have not grasped.
 
 Hope you don't mind, but can you scan through what follows and tell me
 if I am on the right lines?
>>> 
>>> No probs!
>>> 
>> [...]
>> Really appreciate the one-on-one tuition. I am learning a lot!
>>> 
 If this all makes sense, I should write two versions of the client
 program, one using a single connection, and one using a pool of 
 connections.
 
>>> 
>>> Possibly! I think you'll most likely see that one of those behaves
>>> perfectly normally, and you only trigger the issue in the other. So
>>> you could move forward with just one test program.
>>> 
>> Well, I have got the first one working - single connection - and so far it 
>> has not gone wrong.
>> However, it is difficult to be sure that I am comparing apples with apples. 
>> I have written my test server to handle 'Keep-Alive' correctly, but as I 
>> mentioned earlier, my live program closes the connection after each 
>> transfer. So now I have to make my test server do the same, and change my 
>> test client to react to that and re-open the connection each time. I will 
>> make the changes and see how that behaves.
>> Of course now I am in the murky waters of trying to second-guess how Edge 
>> reacts to that. Presumably that is where Wireshark will be useful. I will 
>> keep you posted.
> 
> Here is a progress report.
> 
> I decided to concentrate on using Wireshark to detect the difference between 
> a Python3.7 session and a Python3.8 session. Already I can see some 
> differences.
> 
> There is only one version of my program. I am simply running it with either 
> 'py -3.7 ' or 'py -3.8'. And I am ignoring Chrome at this stage, as it is 
> only that Edge shows the problem.
> 
> First point - Python3.7 also shows a lot of [RST, ACK] lines. My guess is 
> that this is caused by my 'protocol violation' of sending a 'Keep-Alive' 
> header and then closing the connection. Python3.7 does not suffer from 
> dropping files, so I now think this is a sidetrack. I will fix my program 
> when this is all over, but for now I don't want to touch it.

Yes your protocol violation is why you see [RST, ACK].

I'm confused you know that the code has a critical bug in it and you have not 
fixed it?
Just send "Connection: close" and I'd assume all will work.

If my last message was not clear let me know what is unclear.

It not interesting that you get away with it in 3.7 and not 3.8 because the 
code is wrong.

> When I send the response to the initial connection, I write a status line, 
> then multiple header lines, then an html file. I then close the connection. 
> Python3.7 sends a packet with the status, then a separate packet for each 
> header, then a packet with the file. Python3.8 sends a packet with the 
> status, then merges everything else into a single packet and sends it in one 
> go. I just mention this as an indication that quite a lot has changed between 
> my two versions of Python.
> 
> I have one frustration with Wireshark. I will mention it in case anyone has a 
> solution.
> 
> I can see that Edge opens multiple connections. I am trying to track the 
> activity on each connection separately. I can export the data to csv, which 
> makes it easier to work on. But while the TCP lines include the source and 
> destination ports, the HTTP lines do not, so I don't know which connection 
> they belong to. If I view the data in Wireshark's gui it does show the ports, 
> so the data is there somewhere. Does anyone know how to include it in the csv 
> output?

There is a option to follow a single TCP or HTTP connection in Wireshark.
Right click on one pf the packets then choose Follow/HTTP stream.

The way to share the data is as a PCAP file that allows someone else to look at 
the capture
with tools like wireshark and others.

> 
> That's all for now. I will keep you posted.

Barry

> 
> Frank
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list 
> 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Confusing textwrap parameters, and request for RE help

[DL Neil via Python-list]

On 23/03/20 8:00 AM, Chris Angelico wrote:

When using textwrap.fill() or friends, setting break_long_words=False
without also setting break_on_hyphens=False has the very strange
behaviour that a long hyphenated word will still be wrapped. I
discovered this as a very surprising result when trying to wrap a
paragraph that contained a URL, and wanting the URL to be kept
unchanged:


I dropped textwrap years ago. Which policy likely shows my/my 
applications' bias.


Today it feels like an anachronism because it is comes from the era of 
fixed-width fonts and line-lengths denominated in characters*. The issue 
is that it was designed to re-define 'white space' and to enable the 
conversion of text 'wrapped' in one (fixed) format, to suit another. 
With the arrival?predominance of proportional-width fonts, the skills of 
hyphenation have started to go the way of cursive hand-writing 
[substitute any number of grumpy, old man regrets/favorite complaints, 
here].



Beyond such syntactic concerns, textwrap applies no semantic meaning to 
its text-content. For that we have to move to markup languages (HTML is 
my bias - but (largely) presumes screen presentation). Python seems to 
prefer reST (eg Sphinx). See also, markdown.



However, it's annoying if one is 'stuck' within a spec and some list 
smart-alec comes along saying 'change the tool'...


Your idea of sub-classing (as I'm sure YOU know, textwrap is but a 
convenience-function) struck me as clever-thinking! Could textwrap's 
'final format' be caught just before 'return', enabling a post-process 
to undo anything textwrap has done, and (re-)format the URLs to spec, or 
to treat textwrap's output as a template and 'inject' the URL 
appropriately? If not a sub-class, a decorator?


My idea (being more simple-minded than you!), would be to partition the 
text (yes, am alluding to the Python str.method):

- textwrap the 'early text',
- treat the URL as a string using the required convention,
- textwrap the 'later text', and
- str.join() the three components/partitions afterwards.

Both likely 'force' the URL to occupy a line of its own, and thus create 
some odd-looking results!



* that said, my terminal windows and SublimeText are all configured to 
use fixed-width fonts - seems easier on the eyes! Similarly, the 
mainframe legacy of punched-card input is alive-and-well in Python 
'standards', which talk of a 79- or 80-character line-width. Please 
don't take me by the ear and wash-out my mouth with soap and water!

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Confusing textwrap parameters, and request for RE help

[Chris Angelico]

On Wed, Mar 25, 2020 at 8:04 AM DL Neil via Python-list
 wrote:
>
> On 23/03/20 8:00 AM, Chris Angelico wrote:
> > When using textwrap.fill() or friends, setting break_long_words=False
> > without also setting break_on_hyphens=False has the very strange
> > behaviour that a long hyphenated word will still be wrapped. I
> > discovered this as a very surprising result when trying to wrap a
> > paragraph that contained a URL, and wanting the URL to be kept
> > unchanged:
>
> I dropped textwrap years ago. Which policy likely shows my/my
> applications' bias.
>
> Today it feels like an anachronism because it is comes from the era of
> fixed-width fonts and line-lengths denominated in characters*. The issue
> is that it was designed to re-define 'white space' and to enable the
> conversion of text 'wrapped' in one (fixed) format, to suit another.
> With the arrival?predominance of proportional-width fonts, the skills of
> hyphenation have started to go the way of cursive hand-writing
> [substitute any number of grumpy, old man regrets/favorite complaints,
> here].

Terminals still use somewhat-fixed-width fonts, so it's still
reasonably appropriate - until there's some sort of indentation level
escape code.

> Your idea of sub-classing (as I'm sure YOU know, textwrap is but a
> convenience-function) struck me as clever-thinking! Could textwrap's
> 'final format' be caught just before 'return', enabling a post-process
> to undo anything textwrap has done, and (re-)format the URLs to spec, or
> to treat textwrap's output as a template and 'inject' the URL
> appropriately? If not a sub-class, a decorator?

Hmm. Very VERY interesting idea, and one I hadn't thought of. Thank you.

> My idea (being more simple-minded than you!), would be to partition the
> text (yes, am alluding to the Python str.method):
> - textwrap the 'early text',
> - treat the URL as a string using the required convention,
> - textwrap the 'later text', and
> - str.join() the three components/partitions afterwards.
>
> Both likely 'force' the URL to occupy a line of its own, and thus create
> some odd-looking results!

The use-case here is a Twitter client I'm building. It works in the
terminal. I would very much like NOT to build any sort of GUI for it.
Since tweets often contain URLs, it's important to render them
correctly. The display style is to have "@Username: " at the start of
the first line, and the same number of spaces on subsequent lines,
which creates a very readable display. (Also, quoting retweets show
the original tweet indented underneath, giving a bit more
indentation.) Unfortunately, forcing every URL onto its own line would
make the display a bit too vertical for my liking; often there's a
very short URL (eg someone's Twitch link) that doesn't want to be
split across.

Actually the ultimate solution would be the not-yet-standardized
protocol for effectively showing hypertext on the console, where the
abbreviated text could actually be made clickable as the full text.
But that requires more help from the terminal emulator, unless I'm
just misreading the examples (it's supposed to work in gnome-terminal
but I couldn't get it to behave). Or alternatively, as mentioned, a
way to say to the terminal emulator, "please indent this text by at
least this amount" (where the amount would most likely be specified as
a number of characters, but in theory could be millimeters instead).

For now, though, all I can do is rewrap URLs. And at the moment, what
I've done is just block all long words from being wrapped AND block
words from being wrapped at hyphens, when really what I actually want
is to say "https://..."; becomes unbreakable, and leave all the
rest unchanged. Hence the question about the regex.

Currently, the regex splits a long line of text into a series of
words, including hyphen points. What I want is to assert "this ain't a
word split point, because the word starts [a-z]+:// and is thus a
URL". But that might be beyond the flexibility of REs.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to create an Excel app that runs Python?

[Grant Edwards]

On 2020-03-24, oliver  wrote:

> Use the win32com library to interact with Excel via COM.

Huh?  I thought that the users have no way of running a local Python
app.

--
Grant


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Confusing textwrap parameters, and request for RE help

[DL Neil via Python-list]

On 25/03/20 10:30 AM, Chris Angelico wrote:

On Wed, Mar 25, 2020 at 8:04 AM DL Neil via Python-list
 wrote:


On 23/03/20 8:00 AM, Chris Angelico wrote:

When using textwrap.fill() or friends, setting break_long_words=False
without also setting break_on_hyphens=False has the very strange
behaviour that a long hyphenated word will still be wrapped. I
discovered this as a very surprising result when trying to wrap a
paragraph that contained a URL, and wanting the URL to be kept
unchanged:



Your idea of sub-classing (as I'm sure YOU know, textwrap is but a
convenience-function) struck me as clever-thinking! Could textwrap's
'final format' be caught just before 'return', enabling a post-process
to undo anything textwrap has done, and (re-)format the URLs to spec, or
to treat textwrap's output as a template and 'inject' the URL
appropriately? If not a sub-class, a decorator?


Hmm. Very VERY interesting idea, and one I hadn't thought of. Thank you.


A pleasure to be able to offer even such a small 'something', by way of 
return!




My idea (being more simple-minded than you!), would be to partition the
text (yes, am alluding to the Python str.method):
- textwrap the 'early text',
- treat the URL as a string using the required convention,
- textwrap the 'later text', and
- str.join() the three components/partitions afterwards.

Both likely 'force' the URL to occupy a line of its own, and thus create
some odd-looking results!


The use-case here is a Twitter client I'm building. It works in the
terminal. I would very much like NOT to build any sort of GUI for it.


+1



Since tweets often contain URLs, it's important to render them
correctly. The display style is to have "@Username: " at the start of
the first line, and the same number of spaces on subsequent lines,
which creates a very readable display. (Also, quoting retweets show
the original tweet indented underneath, giving a bit more
indentation.) Unfortunately, forcing every URL onto its own line would
make the display a bit too vertical for my liking; often there's a
very short URL (eg someone's Twitch link) that doesn't want to be
split across.

Actually the ultimate solution would be the not-yet-standardized
protocol for effectively showing hypertext on the console, where the
abbreviated text could actually be made clickable as the full text.
But that requires more help from the terminal emulator, unless I'm
just misreading the examples (it's supposed to work in gnome-terminal
but I couldn't get it to behave). Or alternatively, as mentioned, a
way to say to the terminal emulator, "please indent this text by at
least this amount" (where the amount would most likely be specified as
a number of characters, but in theory could be millimeters instead).

For now, though, all I can do is rewrap URLs. And at the moment, what
I've done is just block all long words from being wrapped AND block
words from being wrapped at hyphens, when really what I actually want
is to say "https://..."; becomes unbreakable, and leave all the
rest unchanged. Hence the question about the regex.

Currently, the regex splits a long line of text into a series of
words, including hyphen points. What I want is to assert "this ain't a
word split point, because the word starts [a-z]+:// and is thus a
URL". But that might be beyond the flexibility of REs.



As you observe, the problem with terminal emulators is the extent of 
their emulation and the degree of adoption of their 'extended features'!


My concern grows because of the need (I assume) for the URL to be 
'clickable', not merely 'complete' and 'unadorned' (no added hyphen 
rendering it useless).


Despite the almost-irresistible urge to prove that I'm a class-y guy 
[albeit with a warped sense of humor], I'm warming to the 'partition' 
suggestion:

- the output from textwrap.wrap() is a list of strings,
- we are talking about fixed-length lines denominated in characters,
- locating a[ll] URI (from a sub-set of schemes, eg "http", "https", 
...) is a well-worn path.


Thus (simplified to assume the presence of exactly one URI!!!):
- textwrap the first partition
- len( URI )
- it becomes trivial to ascertain if the URL might append the last line 
(of the first 'wrapped' partition)

- failing that, the URI must start a new line (by defn)
- if it is 'too long', ie would be wrapped by textwrap, treat it separately,
- conversely, prepend the URI to the third partition,
- text-wrap the third partition,
- assemble the tweet-display.

Opinion to the contrary, I am not a twit. I fear missing something in 
those subtleties...

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Confusing textwrap parameters, and request for RE help

[Chris Angelico]

On Wed, Mar 25, 2020 at 9:37 AM DL Neil via Python-list
 wrote:
>
> As you observe, the problem with terminal emulators is the extent of
> their emulation and the degree of adoption of their 'extended features'!
>
> My concern grows because of the need (I assume) for the URL to be
> 'clickable', not merely 'complete' and 'unadorned' (no added hyphen
> rendering it useless).

A complete URL is clickable. If I print out the string
"https://example.com/"; then it can be right-clicked and copied, or
sent to a browser, or whatever. (Technically I don't need it to be
"clickable", but the terminal emulator has to recognize it as a link.)
FTR I'm using XFCE and xfce4-terminal, although I think everything
applies to most of the popular terminal emulators.

The problem is that a long URL will wrap. So my options are:

1) Let it wrap, violating the indentation
2) Break it, which stops it from being clickable
3) Get help from the terminal emulator

> Despite the almost-irresistible urge to prove that I'm a class-y guy
> [albeit with a warped sense of humor], I'm warming to the 'partition'
> suggestion:
> - the output from textwrap.wrap() is a list of strings,
> - we are talking about fixed-length lines denominated in characters,
> - locating a[ll] URI (from a sub-set of schemes, eg "http", "https",
> ...) is a well-worn path.

I'm not bothered too much by specific schemes. If a word matches the
regex ^[a-z]+:// then I'll call it a URL.

> Thus (simplified to assume the presence of exactly one URI!!!):
> - textwrap the first partition
> - len( URI )
> - it becomes trivial to ascertain if the URL might append the last line
> (of the first 'wrapped' partition)
> - failing that, the URI must start a new line (by defn)
> - if it is 'too long', ie would be wrapped by textwrap, treat it separately,
> - conversely, prepend the URI to the third partition,
> - text-wrap the third partition,
> - assemble the tweet-display.
>

Hmm. So, basically, manual reimplementation of parts of textwrap. I'll
hold that thought for later... if it's possible to just change the
regex and keep the code unchanged, I'll prefer that, but this is a
possibility. Thanks.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

yield from

[Dan Stromberg]

Some time ago, when I first heard about yield from, it wasn't faster than a
for loop yielding.

Has that improved?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: yield from

[Chris Angelico]

On Wed, Mar 25, 2020 at 10:08 AM Dan Stromberg  wrote:
>
> Some time ago, when I first heard about yield from, it wasn't faster than a
> for loop yielding.
>
> Has that improved?

It's not about speed. It's about correctness, and the way it delegates
everything, not just "for x in iter: yield x". Here's what it's
actually equivalent to:

https://www.python.org/dev/peps/pep-0380/#formal-semantics

It'll be a LOT more efficient than writing all that out manually. For
the simple case where you really can just loop and yield, sure, though
IMO it's still clearer to yield-from than to loop-yield.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How does the super type present itself and do lookups?

[Greg Ewing]

On 23/03/20 7:40 pm, Adam Preble wrote:

I don't doubt what you got from the source, but I am trying to figure
out how I could have inferred that from the code I was trying. It
looks like child_instance.__getattribute__ ==
child_instance.super().__getattribute__.


Don't feel too bad, it took me a while to figure out what was
happening here myself!

I think the problem is that the act of looking up __getattribute__
on the super object invokes the super object's magic lookup
machinery, and ends up giving a misleading result.

It's clearer if you look for __getattribute__ directly on the
*class* of the super object. I tried this experiment:

class C:

a = "a in C"

class D(C):

a = "a in D"

def getsuper(self):
return super()

d = D()
s = d.getsuper()
print("type(d).__getattribute__ =", type(d).__getattribute__)
print("type(s).__getattribute__ =", type(s).__getattribute__)

Result:

type(d).__getattribute__ = objects>
type(s).__getattribute__ = objects>


If you try this with __getattr__ you find that super objects
don't even have one:

print("type(s).__getattr__ =", type(s).__getattr__)

gives

AttributeError: type object 'super' has no attribute '__getattr__'

Another way to see this is to look at the class dict of the super
object:

print(list(type(s).__dict__.keys()))

gives

['__repr__', '__getattribute__', '__get__', '__init__', '__new__', 
'__thisclass__', '__self__', '__self_class__', '__doc__']


from which it's clear that the super object overrides __getattribute__
but not __getattr__.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: Intermittent bug with asyncio and MS Edge

[Frank Millman]

On 2020-03-24 8:39 PM, Barry Scott wrote:




On 24 Mar 2020, at 11:54, Frank Millman  wrote:


I decided to concentrate on using Wireshark to detect the difference between a 
Python3.7 session and a Python3.8 session. Already I can see some differences.

There is only one version of my program. I am simply running it with either 'py 
-3.7 ' or 'py -3.8'. And I am ignoring Chrome at this stage, as it is only that 
Edge shows the problem.

First point - Python3.7 also shows a lot of [RST, ACK] lines. My guess is that 
this is caused by my 'protocol violation' of sending a 'Keep-Alive' header and 
then closing the connection. Python3.7 does not suffer from dropping files, so 
I now think this is a sidetrack. I will fix my program when this is all over, 
but for now I don't want to touch it.


Yes your protocol violation is why you see [RST, ACK].

I'm confused you know that the code has a critical bug in it and you have not 
fixed it?
Just send "Connection: close" and I'd assume all will work.



Well, the reason is simply that I wanted to understand why my code that 
worked all the way from 3.4 through 3.7 stopped working in 3.8. I 
realise that my code is faulty, but I still wanted to know what the 
trigger was that caused the bug to appear.


From my testing with Wireshark, I can see that both Edge and Chrome 
create 20 connections to GET 20 files. The difference seems to be that 
Chrome does not attempt to re-use a connection, even though both client 
and server have sent Keep-Alive headers. Edge does attempt to re-use the 
connection.


The difference between 3.7 and 3.8 is that 3.7 sends the data in 
separate packets for the status, each header, and then each chunk, 
whereas 3.8 sends the whole lot in a single packet.


My guess is that 3.7 is slower to send the files, so Edge starts up all 
20 connections before it has finished receiving the first one, whereas 
with 3.8, by the time it has opened a few connections the first file has 
been received, so it tries to re-use the same connection to receive the 
next one. By then I have closed the connection. If I am right, it is 
surprising that my program worked *some* of the time.


The same reasoning would explain why it worked when connecting from a 
remote host. There would be enough delay to force it into the same 
behaviour as 3.7.


It has been an interesting ride, and I have learned a lot. I will now 
look into fixing my program. The easy fix is to just send 'Connection: 
Close', but I will do it properly and implement 'Keep-Alive'.


Thanks all

Frank
--
https://mail.python.org/mailman/listinfo/python-list

Re: Intermittent bug with asyncio and MS Edge

[Chris Angelico]

On Wed, Mar 25, 2020 at 5:14 PM Frank Millman  wrote:
> My guess is that 3.7 is slower to send the files, so Edge starts up all
> 20 connections before it has finished receiving the first one, whereas
> with 3.8, by the time it has opened a few connections the first file has
> been received, so it tries to re-use the same connection to receive the
> next one. By then I have closed the connection. If I am right, it is
> surprising that my program worked *some* of the time.

That is definitely plausible. It would also mean that, depending on
exact browser settings, you might see the same problem crop up in
other situations. Definitely worth the time to investigate this one,
since you now know the ACTUAL reason and can solve the ACTUAL bug.

> It has been an interesting ride, and I have learned a lot. I will now
> look into fixing my program. The easy fix is to just send 'Connection:
> Close', but I will do it properly and implement 'Keep-Alive'.

Yep, and honestly, popping a quick "Connection: Close" isn't a
terrible solution. I have one app where I had a similar weird problem
with keep-alive, and made the opposite decision to you: rather than
sink in the hours to figure out the actual cause, I just slapped in a
"Connection: Close" and moved on. Part of the problem is that my issue
only shows up on HTTPS connections and not on plain HTTP ones, which
makes it extremely fiddly to wireshark (since all the traffic I care
about is encrypted, unless through some miracle the bug is visible
when it has no symptoms). But hey, that's what TODO files are for

Thank you for including us on your debugging journey. To anyone else
out there with a bizarre problem, I recommend reading over this thread
to get an idea of how to get help - Frank demonstrated a willingness
to sink his own time into this, a comprehensive writeup of all useful
symptoms, and a truly interesting problem report.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list