[Tutor] Python Help
Hello! I have a Mac and idle refuses to quit no matter what I try to do. What should I do? Thank you! -Janelle ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Python Help
On 14/04/15 02:39, Janelle Harb wrote: Hello! I have a Mac and idle refuses to quit no matter what I try to do. What should I do? You can start by giving us some specific examples of things you did that didn't work. Starting with the obvious: 1) Did you click the little close icon? 2) Did you use the Quit option in the menus? 3) Did you try a Force Quit? 4) Did you run kill from the terminal? 5) Did you try rebooting? 6) Did you hit it with a club hammer? Without any idea of what you did we can't suggest anything specific. It might also help if you tell us which OS version, which Python version, and whether this has always been the case or if its something that only happened recently. If the latter, what were you doing with IDLE immediately before the problem arose? -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Regular expression on python
Steven D'Aprano wrote:
> On Mon, Apr 13, 2015 at 02:29:07PM +0200, jarod...@libero.it wrote:
>> Dear all.
>> I would like to extract from some file some data.
>> The line I'm interested is this:
>>
>> Input Read Pairs: 2127436 Both Surviving: 1795091 (84.38%) Forward
>> Only Surviving: 17315 (0.81%) Reverse Only Surviving: 6413 (0.30%)
>> Dropped: 308617 (14.51%)
>
>
> Some people, when confronted with a problem, think "I know, I'll
> use regular expressions." Now they have two problems.
> -- Jamie Zawinski
>
> I swear that Perl has been a blight on an entire generation of
> programmers. All they know is regular expressions, so they turn every
> data processing problem into a regular expression. Or at least they
> *try* to. As you have learned, regular expressions are hard to read,
> hard to write, and hard to get correct.
>
> Let's write some Python code instead.
>
>
> def extract(line):
> # Extract key:number values from the string.
> line = line.strip() # Remove leading and trailing whitespace.
> words = line.split()
> accumulator = [] # Collect parts of the string we care about.
> for word in words:
> if word.startswith('(') and word.endswith('%)'):
> # We don't care about percentages in brackets.
> continue
> try:
> n = int(word)
> except ValueError:
> accumulator.append(word)
> else:
> accumulator.append(n)
> # Now accumulator will be a list of strings and ints:
> # e.g. ['Input', 'Read', 'Pairs:', 1234, 'Both', 'Surviving:', 1000]
> # Collect consecutive strings as the key, int to be the value.
> results = {}
> keyparts = []
> for item in accumulator:
> if isinstance(item, int):
> key = ' '.join(keyparts)
> keyparts = []
> if key.endswith(':'):
> key = key[:-1]
> results[key] = item
> else:
> keyparts.append(item)
> # When we have finished processing, the keyparts list should be empty.
> if keyparts:
> extra = ' '.join(keyparts)
> print('Warning: found extra text at end of line "%s".' % extra)
> return results
>
>
>
> Now let me test it:
>
> py> line = ('Input Read Pairs: 2127436 Both Surviving: 1795091'
> ... ' (84.38%) Forward Only Surviving: 17315 (0.81%)'
> ... ' Reverse Only Surviving: 6413 (0.30%) Dropped:'
> ... ' 308617 (14.51%)\n')
> py>
> py> print(line)
> Input Read Pairs: 2127436 Both Surviving: 1795091 (84.38%) Forward
> Only Surviving: 17315 (0.81%) Reverse Only Surviving: 6413 (0.30%)
> Dropped: 308617 (14.51%)
>
> py> extract(line)
> {'Dropped': 308617, 'Both Surviving': 1795091, 'Reverse Only Surviving':
> 6413, 'Forward Only Surviving': 17315, 'Input Read Pairs': 2127436}
>
>
> Remember that dicts are unordered. All the data is there, but in
> arbitrary order. Now that you have a nice function to extract the data,
> you can apply it to the lines of a data file in a simple loop:
>
> with open("255.trim.log") as p:
> for line in p:
> if line.startswith("Input "):
> d = extract(line)
> print(d) # or process it somehow
The tempter took posession of me and dictated:
>>> pprint.pprint(
... [(k, int(v)) for k, v in
... re.compile(r"(.+?):\s+(\d+)(?:\s+\(.*?\))?\s*").findall(line)])
[('Input Read Pairs', 2127436),
('Both Surviving', 1795091),
('Forward Only Surviving', 17315),
('Reverse Only Surviving', 6413),
('Dropped', 308617)]
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Regular expression on python
On Tue, Apr 14, 2015 at 10:00:47AM +0200, Peter Otten wrote:
> Steven D'Aprano wrote:
> > I swear that Perl has been a blight on an entire generation of
> > programmers. All they know is regular expressions, so they turn every
> > data processing problem into a regular expression. Or at least they
> > *try* to. As you have learned, regular expressions are hard to read,
> > hard to write, and hard to get correct.
> >
> > Let's write some Python code instead.
[...]
> The tempter took posession of me and dictated:
>
> >>> pprint.pprint(
> ... [(k, int(v)) for k, v in
> ... re.compile(r"(.+?):\s+(\d+)(?:\s+\(.*?\))?\s*").findall(line)])
> [('Input Read Pairs', 2127436),
> ('Both Surviving', 1795091),
> ('Forward Only Surviving', 17315),
> ('Reverse Only Surviving', 6413),
> ('Dropped', 308617)]
Nicely done :-)
I didn't say that it *couldn't* be done with a regex. Only that it is
harder to read, write, etc. Regexes are good tools, but they aren't the
only tool and as a beginner, which would you rather debug? The extract()
function I wrote, or r"(.+?):\s+(\d+)(?:\s+\(.*?\))?\s*" ?
Oh, and for the record, your solution is roughly 4-5 times faster than
the extract() function on my computer. If I knew the requirements were
not likely to change (that is, the maintenance burden was likely to be
low), I'd be quite happy to use your regex solution in production code,
although I would probably want to write it out in verbose mode just in
case the requirements did change:
r"""(?x)(?# verbose mode)
(.+?): (?# capture one or more character, followed by a colon)
\s+ (?# one or more whitespace)
(\d+) (?# capture one or more digits)
(?: (?# don't capture ... )
\s+ (?# one or more whitespace)
\(.*?\) (?# anything inside round brackets)
)?(?# ... and optional)
\s* (?# ignore trailing spaces)
"""
That's a hint to people learning regular expressions: start in verbose
mode, then "de-verbose" it if you must.
--
Steve
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Regular expression on python
Steven D'Aprano wrote:
> On Tue, Apr 14, 2015 at 10:00:47AM +0200, Peter Otten wrote:
>> Steven D'Aprano wrote:
>
>> > I swear that Perl has been a blight on an entire generation of
>> > programmers. All they know is regular expressions, so they turn every
>> > data processing problem into a regular expression. Or at least they
>> > *try* to. As you have learned, regular expressions are hard to read,
>> > hard to write, and hard to get correct.
>> >
>> > Let's write some Python code instead.
> [...]
>
>> The tempter took posession of me and dictated:
>>
>> >>> pprint.pprint(
>> ... [(k, int(v)) for k, v in
>> ... re.compile(r"(.+?):\s+(\d+)(?:\s+\(.*?\))?\s*").findall(line)])
>> [('Input Read Pairs', 2127436),
>> ('Both Surviving', 1795091),
>> ('Forward Only Surviving', 17315),
>> ('Reverse Only Surviving', 6413),
>> ('Dropped', 308617)]
>
> Nicely done :-)
>
> I didn't say that it *couldn't* be done with a regex.
I didn't claim that.
> Only that it is
> harder to read, write, etc. Regexes are good tools, but they aren't the
> only tool and as a beginner, which would you rather debug? The extract()
> function I wrote, or r"(.+?):\s+(\d+)(?:\s+\(.*?\))?\s*" ?
I know a rhetorical question when I see one ;)
> Oh, and for the record, your solution is roughly 4-5 times faster than
> the extract() function on my computer.
I wouldn't be bothered by that. See below if you are.
> If I knew the requirements were
> not likely to change (that is, the maintenance burden was likely to be
> low), I'd be quite happy to use your regex solution in production code,
> although I would probably want to write it out in verbose mode just in
> case the requirements did change:
>
>
> r"""(?x)(?# verbose mode)
> (.+?): (?# capture one or more character, followed by a colon)
> \s+ (?# one or more whitespace)
> (\d+) (?# capture one or more digits)
> (?: (?# don't capture ... )
> \s+ (?# one or more whitespace)
> \(.*?\) (?# anything inside round brackets)
> )?(?# ... and optional)
> \s* (?# ignore trailing spaces)
> """
>
>
> That's a hint to people learning regular expressions: start in verbose
> mode, then "de-verbose" it if you must.
Regarding the speed of the Python approach: you can easily improve that by
relatively minor modifications. The most important one is to avoid the
exception:
$ python parse_jarod.py
$ python3 parse_jarod.py
The regex for reference:
$ python3 -m timeit -s "from parse_jarod import extract_re as extract"
"extract()"
10 loops, best of 3: 18.6 usec per loop
Steven's original extract():
$ python3 -m timeit -s "from parse_jarod import extract_daprano as extract"
"extract()"
1 loops, best of 3: 92.6 usec per loop
Avoid raising ValueError (This won't work with negative numbers):
$ python3 -m timeit -s "from parse_jarod import extract_daprano2 as extract"
"extract()"
1 loops, best of 3: 44.3 usec per loop
Collapse the two loops into one, thus avoiding the accumulator list and the
isinstance() checks:
$ python3 -m timeit -s "from parse_jarod import extract_daprano3 as extract"
"extract()"
1 loops, best of 3: 29.6 usec per loop
Ok, this is still slower than the regex, a result that I cannot accept.
Let's try again:
$ python3 -m timeit -s "from parse_jarod import extract_py as extract"
"extract()"
10 loops, best of 3: 15.1 usec per loop
Heureka? The "winning" code is brittle and probably as hard to understand as
the regex. You can judge for yourself if you're interested:
$ cat parse_jarod.py
import re
line = ("Input Read Pairs: 2127436 "
"Both Surviving: 1795091 (84.38%) "
"Forward Only Surviving: 17315 (0.81%) "
"Reverse Only Surviving: 6413 (0.30%) "
"Dropped: 308617 (14.51%)")
_findall = re.compile(r"(.+?):\s+(\d+)(?:\s+\(.*?\))?\s*").findall
def extract_daprano(line=line):
# Extract key:number values from the string.
line = line.strip() # Remove leading and trailing whitespace.
words = line.split()
accumulator = [] # Collect parts of the string we care about.
for word in words:
if word.startswith('(') and word.endswith('%)'):
# We don't care about percentages in brackets.
continue
try:
n = int(word)
except ValueError:
accumulator.append(word)
else:
accumulator.append(n)
# Now accumulator will be a list of strings and ints:
# e.g. ['Input', 'Read', 'Pairs:', 1234, 'Both', 'Surviving:', 1000]
# Collect consecutive strings as the key, int to be the value.
results = {}
keyparts = []
for item in accumulator:
if isinstance(item, int):
key = ' '.join(keyparts)
keyparts = []
if key.endswith(':'):
key = key[:-1]
results[key] = item
else:
keyparts.append(item)
# When we have finished process
[Tutor] where is the wsgi server root?
I set up a simple python wsgi server on port 8000, which works, but where the heck is the server root? Is there a physical server root I can simply put a python program in, as I put a html program into the wampserver root, and see it in a browser on localhost:port 8000, or do I need to do a bit more reading ;') I just cut and pasted the server setup code from the docs, and assumed a server would have a physical root as my local wampserver does. I don't want to fool with django right now - just do the simplest thing to see if I can get python on a web page since I'm used to making them. Or if the root is virtual how do I set up a physical root? Or am I totally confused? -- Jim ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] where is the wsgi server root?
On 14/04/15 17:56, Jim Mooney wrote: simplest thing to see if I can get python on a web page since I'm used to making them. Or if the root is virtual how do I set up a physical root? I'm guessing; but I'd expect it to be the current directory when you started the server. try adding a print( os,getcwd() ) And see if the result is your root. Or you could try reading the documents...they might tell you! -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] where is the wsgi server root?
On Tue, Apr 14, 2015 at 12:56 PM, Jim Mooney wrote: > I set up a simple python wsgi server on port 8000, which works, but where > the heck is the server root? Is there a physical server root I can simply > put a python program in, as I put a html program into the wampserver root, > and see it in a browser on localhost:port 8000, or do I need to do a bit > more reading ;') I just cut and pasted the server setup code from the > docs, and assumed a server would have a physical root as my local > wampserver does. I don't want to fool with django right now - just do the > simplest thing to see if I can get python on a web page since I'm used to > making them. > > Or if the root is virtual how do I set up a physical root? > > Or am I totally confused? > > -- > Jim Here is a link about wsgi: https://code.google.com/p/modwsgi/ It isn't like php. You can put your code anywhere. > ___ > Tutor maillist - Tutor@python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor -- Joel Goldstick http://joelgoldstick.com ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Regular expression on python
On 14/04/15 13:21, Steven D'Aprano wrote: although I would probably want to write it out in verbose mode just in case the requirements did change: r"""(?x)(?# verbose mode) (.+?): (?# capture one or more character, followed by a colon) \s+ (?# one or more whitespace) (\d+) (?# capture one or more digits) (?: (?# don't capture ... ) \s+ (?# one or more whitespace) \(.*?\) (?# anything inside round brackets) )?(?# ... and optional) \s* (?# ignore trailing spaces) """ That's a hint to people learning regular expressions: start in verbose mode, then "de-verbose" it if you must. New one on me. Where does one find out about verbose mode? I don't see it in the re docs? I see an re.X flag but while it seems to be similar in purpose yet it is different to your style above (no parens for example)? -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Regular expression on python
On 15/04/2015 00:49, Alan Gauld wrote: On 14/04/15 13:21, Steven D'Aprano wrote: although I would probably want to write it out in verbose mode just in case the requirements did change: r"""(?x)(?# verbose mode) (.+?): (?# capture one or more character, followed by a colon) \s+ (?# one or more whitespace) (\d+) (?# capture one or more digits) (?: (?# don't capture ... ) \s+ (?# one or more whitespace) \(.*?\) (?# anything inside round brackets) )?(?# ... and optional) \s* (?# ignore trailing spaces) """ That's a hint to people learning regular expressions: start in verbose mode, then "de-verbose" it if you must. New one on me. Where does one find out about verbose mode? I don't see it in the re docs? I see an re.X flag but while it seems to be similar in purpose yet it is different to your style above (no parens for example)? https://docs.python.org/3/library/re.html#module-contents re.X and re.VERBOSE are together. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Regular expression on python
On Wed, Apr 15, 2015 at 12:49:26AM +0100, Alan Gauld wrote: > New one on me. Where does one find out about verbose mode? > I don't see it in the re docs? > > I see an re.X flag but while it seems to be similar in purpose > yet it is different to your style above (no parens for example)? I presume it is documented in the main docs, but I actually found this in the "Python Pocket Reference" by Mark Lutz :-) All of the regex flags have three forms: - a numeric flag with a long name; - the same numeric flag with a short name; - a regular expression pattern. So you can either do: re.compile(pattern, flags) or embed the flag in the pattern. The flags that I know of are: (?i) re.I re.IGNORECASE (?L) re.L re.LOCALE (?M) re.M re.MULTILINE (?s) re.S re.DOTALL (?x) re.X re.VERBOSE The flag can appear anywhere in the pattern and applies to the whole pattern, but it is good practice to put them at the front, and in the future it may be an error to put the flags elsewhere. When provided as a separate argument, you can combine flags like this: re.I|re.X -- Steve ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Regular expression on python
On 2015-04-14 16:49, Alan Gauld wrote: New one on me. Where does one find out about verbose mode? I don't see it in the re docs? This is where I go whenever I find myself having to (re)learn the details of regex: https://docs.python.org/3/howto/regex.html I believe a '2' can be substituted for the '3' but I've not found any difference between the two. (I submit this not so much for Alan (tutor) as for those like me who are learning.) ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
