OH MY GOD! HUMANS HAVE ORIGINS IN THE DEVONIAN!
== >BREAKING HOGWASH! == > TALK.ORIGINS RECENTLY ANNOUNCED THAT HUMANS HAVE ORIGINS IN THE DEVONIAN. > THIS RECENT RECOGNITION OF THE GREATEST FACT IN EARTH'S HISTORY IS PROFOUND TO http://www.talkorigins.org/ CREDIBILITY. (Also see http://www.ediacara.org/) > DAVID IAIN GREIG, TALK.ORIGINS DICTATOR WAS RELUCTANT BUT FINALLY ANNOUNCED IN A PAPER IN http://www.sciencemag.org/ THAT THRINAXODON WAS RIGHT, AND HUMANS DO HAVE ORIGINS IN THE DEVONIAN! > === EVIDENCE THAT HUMANS LIVED IN THE DEVONIAN: https://groups.google.com/group/sci.bio.paleontology/browse_thread/thread/6f501c469c7af24f# https://groups.google.com/group/sci.bio.paleontology/browse_thread/thread/3aad75c16afb0b82# http://thrinaxodon.wordpress.com/ === THRINAXODON ONLY HAD THIS TO SAY: "I..I...I...Can't believe it. This completely disproved Darwinian orthodoxy." === THE BASTARDS AT THE SMITHSONIAN, AND THE LEAKEY FOUNDATION ARE ERODING WITH FEAR. === THESE ASSHOLES ARE GOING TO DIE: THOMAS AQUINAS; ALDOUS HUXLEY; BOB CASANVOVA; SkyEyes; DAVID IAIN GRIEG; MARK ISAAK; JOHN HARSHAM; RICHARD NORMAN; DR. DOOLITTLE; CHARLES DARWIN; MARK HORTON; ERIK SIMPSON; HYPATIAB7; PAUL J. GANS; JILLERY; WIKI TRIK; THRINAXODON; PETER NYIKOS; RON OKIMOTO; JOHN S. WILKINS === THRINAXODON WAS SCOURING ANOTHER DEVONIAN FOSSIL BED, AND FOUND A HUMAN SKULL, AND A HUMAN FEMUR. HE ANALYSED THE FINDS, AND SAW THAT THEY WERE NOT NORMAL ROCKS. THESE WERE FOSSILIZED BONES. THEY EVEN HAD TOOTH MARKS ON THEM. SO, THRINAXODON BROUGHT THEM TO THE LEAKEY FOUNDATION, THEY UTTERLY DISMISSED IT, AND SAID, "We want to keep people thinking that humans evolved 2 Ma." THRINAXODON BROUGHT HIS SWORD, AND SAID, "SCIENCE CORRECTS ITSELF." RICHARD LEAKEY SAID, "That is a myth, for people to believe in science." THRINAXODON PLANS TO BRING DOOM TO SCIENCE, ITSELF. THRINAXODON IS NOW ON REDDIT -- ---Thrinaxodon -- https://mail.python.org/mailman/listinfo/python-list
blist in standard library (was Re: Balanced trees)
On 15/03/2014 01:13, Joshua Landau wrote: On 8 March 2014 20:37, Mark Lawrence wrote: I've found this link useful http://kmike.ru/python-data-structures/ I also don't want all sorts of data structures added to the Python library. I believe that there are advantages to leaving specialist data structures on pypi or other sites, plus it means Python in a Nutshell can still fit in your pocket and not a 40 ton articulated lorry, unlike the Java equivalent. The thing we really need is for the blist containers to become stdlib (but not to replace the current list implementation). The rejected PEP (http://legacy.python.org/dev/peps/pep-3128/) misses a few important points, largely in how the "log(n)" has a really large base: random.choice went from 1.2µs to 1.6µs from n=1 to n=10⁸, vs 1.2µs for a standard list. Further, it's worth considering a few advantages: * copy is O(1), allowing code to avoid mutation by just copying its input, which is good practice. * FIFO is effectively O(1), as the time just about doubles from n=1 to n=10⁸ so will never actually branch that much. There is still a speed benefit of collections.deque, but it's much, much less significant. This is very useful when considering usage as a multi-purpose data structure, and removes demand for explicit linked lists (which have foolishly been reimplemented loads of times). * It reduces demand for trees: * There are efficient implementations of sortedlist, sortedset and sorteddict. * Slicing, slice assignment and slice deletion are really fast. * Addition of lists is sublinear. Instead of "list(itertools.chain(...))", one can add in a loop and end up *faster*. I think blist isn't very popular not because it isn't really good, but because it isn't a specialised structure. It is, however, almost there for almost every circumstance. This can help keep the standard library clean, especially of tree data structures. Here's what we kill: * Linked lists and doubly-linked lists, which are scarily popular for whatever reason. Sometimes people claim that collections.deque isn't powerful enough for whatever they want, and blist will almost definitely sate those cases. * Balanced trees, with blist.sortedlist. This is actually needed right now. * Poor performance in the cases where a lot of list merging and pruning happens. * Most uses of bisect. * Some instances where two data structures are used in parallel in order to keep performance fast on disparate operations (like `x in y` and `y[i]`). Now, I understand there are downsides to blist. Particularly, I've looked through the "benchmarks" and they seem untruthful. Further, we'd need a maintainer. Finally, nobody jumps at blists because they're rarely the obvious solution. Rather, they attempt to be a different general solution. Hopefully, though, a stdlib inclusion could make them a lot more obvious, and support in some current libraries could make them feel more at home. I don't know whether this is a good idea, but I do feel that it is more promising and general than having a graph in the standard library. I'd raise this on python-dev or python ideas as a result of reading PEP 3128. Specifically the second paragraph states Raymond Hettinger's sage advice:- "Depending on its success as a third-party module, it still has a chance for inclusion in the collections module. The essential criteria for that is whether it is a superior choice for some real-world use cases. I've scanned my own code and found no instances where BList would have been preferable to a regular list. However, that scan has a selection bias because it doesn't reflect what I would have written had BList been available. So, after a few months, I intend to poll comp.lang.python for BList success stories. If they exist, then I have no problem with inclusion in the collections module. After all, its learning curve is near zero -- the only cost is the clutter factor stemming from indecision about the most appropriate data structure for a given task." -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence --- This email is free from viruses and malware because avast! Antivirus protection is active. http://www.avast.com -- https://mail.python.org/mailman/listinfo/python-list
'complex' function with string argument.
Hi everyone!
This is regarding the inbuilt 'complex' function. The python docs say:
"Note: When converting from a string, the string must not contain whitespace
around the central + or - operator. For example, complex('1+2j') is fine, but
complex('1 + 2j') raises ValueError."
Why is this so? Why can't spaces be allowed everywhere and simply ignored? I
went through the source and it did not seem like this was an internal
requirement, so it seems like a design choice. Is there any reason why spaces
in the middle would be a problem?
Jayanth Koushik
--
https://mail.python.org/mailman/listinfo/python-list
Re: intersection, union, difference, symmetric difference for dictionaries
In article , mauro wrote: > Dictionaries and sets share a few properties: >- Dictionaries keys are unique as well as sets items >- Dictionaries and sets are both unordered >- Dictionaries and sets are both accessed by key >- Dictionaries and sets are both mutables > >So I wonder why operations such us intersection, union, difference, >symmetric difference that are available for sets and are not available >for dictionaries without going via key dictviews. This is a plain bad idea. Dictionaries correspond to the mathematical concept of a mapping. A mapping (or a function) is a set in math, as everything is a set. It is a subset of the product set of two set A and B where there is exactly one pair for each a in A. No sane mathematician talks about unions, intersections etc. of those sets, though clearly they are well defined. OTOH there is a very rich vocabulary specific for the properties of functions. So dear mauro do as everybody does, as soon as you've invented something useful related to dicts, you'll discover that it correspond to an age old mathematical concept. It is unwise not to borrow its name. Those old geezers, Chebychov, Euler, Laplace, Fourier had their marbles in a row. It is hard to outsmart them. Groetjes Albert -- Albert van der Horst, UTRECHT,THE NETHERLANDS Economic growth -- being exponential -- ultimately falters. albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst -- https://mail.python.org/mailman/listinfo/python-list
Sharing: File Reader Generator with & w/o Policy
hi folks, I am posting to share a File Reader Generator which I have
been playing with, that simplifies reading of text files on-demand:
like log files, config files, small record flat data-bases, &c.
I have two generators to share, one with & one without "policy".
The idea is to have the generator open and close the file (with error
checking: try-finish block) and then maintain its state for on-demand
reading either into memory (as list or dict) or for in-line processing.
I will demonstrate the generators here, and then post the code
following. The generator will be reading a path+filename of a local disk
file and printing it as in this simple case without policy:
>>> from my_utils import *
>>> for record in fName(path+"my_fox"):
print(record)
The quick brown fox jumped
over the lazy dog's tail.
Now is the time for all
good women to come to the
aid of computer science!
>>>
The second generator adds "policy" to the generator processing and
yields tuples, rather than strings. Each tuple contains the record
number (from zero), and record length (minus the line end), and the
record itself (stripped of the line end):
>>>
>>> for record in fnName(path+"my_fox"):
print(record)
(0, 26, 'The quick brown fox jumped')
(1, 25, "over the lazy dog's tail.")
(2, 0, '')
(3, 23, 'Now is the time for all')
(4, 25, 'good women to come to the')
(5, 24, 'aid of computer science!')
>>>
>>>
I will now share the source by allowing the fName(filename) utility
to expose itself. Enjoy:
>>>
>>> for record in fName(path+"my_utils.py"):
print(record)
#-
# fName(filename) generator: file reader iterable
#-
def fName(filename):
try:
fh = open(filename, 'r')
except FileNotFoundError as err_code:
print (err_code)
else:
while True:
linein = fh.readline()
if (linein!=''):
yield(linein.strip('\n'))
else:
break
fh.close()
finally:
None
#-
# fnName(filename) generator: file reader iterable
#-
def fnName(filename):
try:
fh = open(filename, 'r')
except FileNotFoundError as err_code:
print (err_code)
else:
line_count = 0
while True:
linein = fh.readline()
if (linein!=''):
lineout = linein.strip('\n')
length = len(lineout)
yield((line_count, length, lineout))
line_count+=1
else:
break
fh.close()
finally:
None
#-
# {next util}
#-
>>>
mark h harris
--
https://mail.python.org/mailman/listinfo/python-list
test
test -- https://mail.python.org/mailman/listinfo/python-list
Re: Sharing: File Reader Generator with & w/o Policy
On 2014-03-15 21:38, Mark H Harris wrote:
hi folks, I am posting to share a File Reader Generator which I have
been playing with, that simplifies reading of text files on-demand:
like log files, config files, small record flat data-bases, &c.
I have two generators to share, one with & one without "policy".
The idea is to have the generator open and close the file (with error
checking: try-finish block) and then maintain its state for on-demand
reading either into memory (as list or dict) or for in-line processing.
I will demonstrate the generators here, and then post the code
following. The generator will be reading a path+filename of a local disk
file and printing it as in this simple case without policy:
>>> from my_utils import *
>>> for record in fName(path+"my_fox"):
print(record)
The quick brown fox jumped
over the lazy dog's tail.
Now is the time for all
good women to come to the
aid of computer science!
>>>
The second generator adds "policy" to the generator processing and
yields tuples, rather than strings. Each tuple contains the record
number (from zero), and record length (minus the line end), and the
record itself (stripped of the line end):
>>>
>>> for record in fnName(path+"my_fox"):
print(record)
(0, 26, 'The quick brown fox jumped')
(1, 25, "over the lazy dog's tail.")
(2, 0, '')
(3, 23, 'Now is the time for all')
(4, 25, 'good women to come to the')
(5, 24, 'aid of computer science!')
>>>
>>>
I will now share the source by allowing the fName(filename) utility
to expose itself. Enjoy:
>>>
>>> for record in fName(path+"my_utils.py"):
print(record)
#-
# fName(filename) generator: file reader iterable
#-
def fName(filename):
try:
fh = open(filename, 'r')
except FileNotFoundError as err_code:
print (err_code)
else:
while True:
linein = fh.readline()
if (linein!=''):
yield(linein.strip('\n'))
else:
break
fh.close()
finally:
None
I don't like how it always swallows the exception, so you can't tell
whether the file doesn't exist or exists but is empty, and no way to
specify the file's encoding.
Why do you have the 'finally' clause with 'None' in it? Instead of None
you should have 'pass', or, better yet, omit the clause entirely.
You can also shorten it somewhat:
def fName(filename):
try:
with open(filename, 'r') as fh:
for linein in fh:
yield linein.strip('\n')
except FileNotFoundError as err_code:
print(err_code)
[snip]
--
https://mail.python.org/mailman/listinfo/python-list
Re: Sharing: File Reader Generator with & w/o Policy
On 3/15/14 4:56 PM, MRAB wrote:
I don't like how it always swallows the exception, so you can't tell
whether the file doesn't exist or exists but is empty, and no way to
specify the file's encoding.
Yes, the error handling needs more robustness/ and instead of printing
the errcode, my actual model on system will log it.
Why do you have the 'finally' clause with 'None' in it? Instead of None
you should have 'pass', or, better yet, omit the clause entirely.
Its a stub-in really, and that's all at this point. The 'finally'
happens regardless of whether the exception occurs, and I don't need
anything there yet, just don't want to forget it.
I've been playing around with wrapping generators within generators for
readability and simplicity. Like this, where I'm going to wrap the
fnName(filename) generator within a getnumline(filename) wrapper:
>>> from my_utils import *
>>> def getnumline(filename):
for record in fnName(filename):
yield(record)
>>> line = getnumline("my_foxy")
>>>
>>> next(line)
(0, 26, 'The quick brown fox jumped')
>>>
>>> next(line)
(1, 25, "over the lazy dog's tail.")
>>>
Or this, where I put it all in memory as a dict:
>>> d1={}
>>> for line in getnumline("my_foxy"):
d1[line[0]]=(line[1], line[2])
>>> for key in d1:
print (d1[key])
(26, 'The quick brown fox jumped')
(25, "over the lazy dog's tail.")
(0, '')
(23, 'Now is the time for all')
(25, 'good women to come to the')
(24, 'aid of computer science!')
>>>
marcus
--
https://mail.python.org/mailman/listinfo/python-list
Re: Sharing: File Reader Generator with & w/o Policy
On 3/15/14 4:56 PM, MRAB wrote:
def fName(filename):
try:
with open(filename, 'r') as fh:
for linein in fh:
yield linein.strip('\n')
except FileNotFoundError as err_code:
print(err_code)
[snip]
The "with" confuses me because I am not sure specifically what happens
in the context manager. I'm taking it for granted in this case that
__exit__() closes the file?
I am finding many examples of file handling using the context manager,
but none so far that wrap into a generator; more often and file object.
Is there a preference for file object over generator?
marcus
--
https://mail.python.org/mailman/listinfo/python-list
Re: Sharing: File Reader Generator with & w/o Policy
On 3/15/14 4:56 PM, MRAB wrote:
You can also shorten it somewhat:
Thanks, I like it... I shortened the fnName() also:
#-
# fn2Name(filename) generator: file reader iterable
#-
def fn2Name(filename):
try:
with open(filename, 'r') as fh:<=== can you tell me
line_count = 0
for linein in fh:
lineout = linein.strip('\n')
length = len(lineout)
yield((line_count, length, lineout))
line_count+=1
except FileNotFoundError as err_code:
print(err_code)
#-
... where I can go to find out (for specific contexts) what the
__init__() and __exit__() are actually doing, like for instance in this
case does the filename get closed in __exit__(), and also if errors
occur does the file close automatically? thanks
marcus
--
https://mail.python.org/mailman/listinfo/python-list
Re: Sharing: File Reader Generator with & w/o Policy
On 16/03/2014 01:06, Mark H Harris wrote:
On 3/15/14 4:56 PM, MRAB wrote:
You can also shorten it somewhat:
Thanks, I like it... I shortened the fnName() also:
#-
# fn2Name(filename) generator: file reader iterable
#-
def fn2Name(filename):
try:
with open(filename, 'r') as fh:<=== can you tell me
line_count = 0
for linein in fh:
lineout = linein.strip('\n')
length = len(lineout)
yield((line_count, length, lineout))
line_count+=1
except FileNotFoundError as err_code:
print(err_code)
#-
... where I can go to find out (for specific contexts) what the
__init__() and __exit__() are actually doing, like for instance in this
case does the filename get closed in __exit__(), and also if errors
occur does the file close automatically? thanks
marcus
Start here
http://docs.python.org/3/library/stdtypes.html#context-manager-types
--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.
Mark Lawrence
---
This email is free from viruses and malware because avast! Antivirus protection
is active.
http://www.avast.com
--
https://mail.python.org/mailman/listinfo/python-list
Re: test
> On Mar 15, 2014, at 14:24, Mark H Harris wrote: > > test Pass -- https://mail.python.org/mailman/listinfo/python-list
Re: Sharing: File Reader Generator with & w/o Policy
On Sat, 15 Mar 2014 16:38:18 -0500, Mark H Harris wrote:
> hi folks, I am posting to share a File Reader Generator which I have
> been playing with, that simplifies reading of text files on-demand: like
> log files, config files, small record flat data-bases, &c.
Reading from files is already pretty simple. I would expect that it will
be harder to learn the specific details of custom, specialised, file
readers that *almost*, but not quite, do what you want, than to just
write a couple of lines of code to do what you need when you need it.
Particularly for interactive use, where robustness is less important than
ease of use.
> I have two generators to share, one with & one without "policy".
What's "policy"?
> The idea is to have the generator open and close the file (with error
> checking: try-finish block) and then maintain its state for on-demand
> reading either into memory (as list or dict) or for in-line processing.
>
> I will demonstrate the generators here, and then post the code
> following. The generator will be reading a path+filename of a local disk
> file and printing it as in this simple case without policy:
>
> >>> from my_utils import *
> >>> for record in fName(path+"my_fox"):
> print(record)
>
> The quick brown fox jumped
> over the lazy dog's tail.
What's "fName" mean? "File name"? That's a horribly misleading name,
since it *takes* a file name as argument, it doesn't return one. That
would be like renaming the len() function to "list", since it takes a
list as argument. Function and class names should be descriptive, giving
at least a hint as to what they do.
It looks to me that this fName just iterates over the lines in a file,
which makes it pretty close to just:
for line in open(path + "my_fox"):
print(line)
> The second generator adds "policy" to the generator processing and
> yields tuples, rather than strings. Each tuple contains the record
> number (from zero), and record length (minus the line end), and the
> record itself (stripped of the line end):
I presume that "record" here means "line", rather than an actual record
from a flat file with fixed-width fields, or some delimiter other than
newlines.
for i, line in enumerate(open(pathname + "my_fox")):
print((i, len(line), line))
> >>> for record in fnName(path+"my_fox"):
> print(record)
What's "fnName" mean? Perhaps "filename name"? "function name"? Again,
the name gives no hint as to what the function does.
> def fName(filename):
> try:
> fh = open(filename, 'r')
> except FileNotFoundError as err_code:
> print (err_code)
For interactive use, this is *just barely* acceptable as a (supposedly)
user-friendly alternative to a stack trace.
[Aside: I don't believe that insulating programmers from tracebacks does
them any favours. Like the Dark Side of the Force, hiding errors is
seductively attractive, but ultimately harmful, since error tracebacks
are intimidating to beginners but an essential weapon in the battle
against buggy code. But reading tracebacks is a skill programmers have to
learn. Hiding tracebacks does them no favours, it just makes it harder
for them to learn good debugging skills, and encourages them to treat
errors as *something to hide* rather than *something to fix*.]
But as a reusable tool for use in non-interactive code, this function
fails badly. By capturing the exception, it makes it painfully difficult
for the caller to have control over error-handling. You cannot let the
exception propagate to some other part of the application for handling;
you cannot log the exception, or ignore it, or silently swallow the
exception and try another file. The fName function makes the decision for
you: it will print the error to standard output (not even standard
error!) no matter what you want. That's the very essence of *user-
hostile* for library code.
Worse, it's inconsistent! Some errors are handled normally, with an
exception. It's only FileNotFoundError that is captured and printed. So
if the user wants to re-use this function and do something with any
exceptions, she has to use *two* forms of error handling:
(1) wrap it in try...except handler to capture any exception other
than FileNotFoundError; and
(2) intercept writes to standard out, capture the error message, and
reverse-engineer what went wrong.
instead of just one.
> else:
> while True:
> linein = fh.readline()
> if (linein!=''):
> yield(linein.strip('\n'))
> else:
> break
> fh.close()
Apart from stripping newlines, which is surely better left to the user
(what if they need to see the newline? by stripping them automatically,
the user cannot distinguish between a file which ends with a newline
character and one which does not), this part is just a re-invention of
the existing wheel. File objects are already iterable, and yield
Re: Sharing: File Reader Generator with & w/o Policy
On 3/15/14 8:32 PM, Mark Lawrence wrote: Start here http://docs.python.org/3/library/stdtypes.html#context-manager-types Thanks Mark. I have three books open, and that doc, and wading through. You might like to know (as an aside) that I'm done with gg. Got back up here with a real news reader and server. All is good that way. gg has not been stable over the past three weeks, and this weekend it completely quit working. It looks like this reader|client handles the line wrapping correctly. whoohoo. marcus -- https://mail.python.org/mailman/listinfo/python-list
Clearing out handlers in logging?
The current (2.7; maybe 3.x?) logging module doesn't have any sort of "clear out all the current handlers" method. I can hack it by doing log = logging.getLogger() # get the root logger del log.handlers[:]# reach inside and nuke 'em log.addHandler(...)# install the one(s) I want but it feels a little dirty to reach into logging.root.handlers since there are other methods for adding/removing handlers. However, as best I can tell, to remove a handler, you already need to have it saved somewhere. Is there a better way to do this, or do I just suck it up and deal with the (potentially thread-ignorant, as it doesn't lock) hack? Thanks, -tkc -- https://mail.python.org/mailman/listinfo/python-list
Re: Sharing: File Reader Generator with & w/o Policy
On 3/15/14 9:01 PM, Steven D'Aprano wrote:
Reading from files is already pretty simple. I would expect that it will
be harder to learn the specific details of custom, specialised, file
readers that *almost*, but not quite, do what you want, than to just
write a couple of lines of code to do what you need when you need it.
Particularly for interactive use, where robustness is less important than
ease of use.
Yes. What I'm finding is that I'm coding the same 4-6 lines of code
with every file open (I do want error handling, at least for
FileNotFoundError) and I only want it to be two lines, read the file
into a list with error handling.
What's "policy"?
That's part of what I personally struggle with (frequently) is do I
place the policy in the generator, or do I handle it on the outside. For
instance, I normally strip the line-end and I want to know the record
lengths. I also may want to know the record number from arrival
sequence. This policy can be handled in the generator; although, I could
have handled it outside too.
for i, line in enumerate(open(pathname + "my_fox")):
print((i, len(line), line))
I like it... and this is where I've always been, when I finally said to
myself, yuk. yes, it technically works very well. But, its ugly. And I
don't mean its technically ugly, I mean its aesthetically ugly and not
user-easy-to-read. (I know that's all subjective)
for line in getnumline(path+"my_foxy")):
print(line)
In this case getnumline() is a generator wrapper around fName(). It of
course doesn't do anything different than the two lines you listed, but
it is immediately easier to tell what is happening; even if you're not
an experienced python programmer.
[Aside: I don't believe that insulating programmers from tracebacks does
them any favours.
Yes. I think you're right about that. But what if they're not
programmers; what if they're just application users that don't have a
clue what a trace-back is, and just want to know that the file does not
exist? And right away they realize that, oops, I spelled the filename
wrong. Yeaah, I struggle with this as I'm trying to simplify, because
personally I want to see the trace back info.
Worse, it's inconsistent! Some errors are handled normally, with an
exception. It's only FileNotFoundError that is captured and printed. So
if the user wants to re-use this function and do something with any
exceptions, she has to use *two* forms of error handling:
Yes. The exception handling needs to handle all normal errors.
(1) wrap it in try...except handler to capture any exception other
than FileNotFoundError; and
(2) intercept writes to standard out, capture the error message, and
reverse-engineer what went wrong.
Ok.
Apart from stripping newlines, which is surely better left to the user
(what if they need to see the newline? by stripping them automatically,
the user cannot distinguish between a file which ends with a newline
character and one which does not), this part is just a re-invention of
the existing wheel. File objects are already iterable, and yield the
lines of the file.
Yes, this is based on my use case, which never needs the line-ends, in
fact they are a pain. These files are variable record length and the
only thing the newline is used for is delimiting the records.
def fnName(filename):
for count, line in enumerate(fName(filename)):
yield (count, len(line), line)
I like this, thanks! enumerate and I are becoming friends.
I like this case philosophically because it is a both | and. The policy
is contained in the wrapper generator using enumerate() and len()
leaving the fName() generator to produce the line.
And you are right about another thing, I just want to use this thing
over and over.
for line in getnumline(filename):
{whatever}
There does seem to be just one way of doing this (file reads) but
there are actually many ways of doing this. Is a file object really
better than a generator, are there good reasons for using the generator,
are there absolute cases for using a file object?
marcus
--
https://mail.python.org/mailman/listinfo/python-list
Re: test
On 3/15/14 8:48 PM, Travis Griggs wrote: On Mar 15, 2014, at 14:24, Mark H Harris wrote: test Pass Thanks Travis. I hated doing that. I have been having a fit with gg, and its taken just a little time to get a real news-reader client for posting. What a fit. Google does really well with some things, but gg is not one of them, IMHO. marcus -- https://mail.python.org/mailman/listinfo/python-list
Re: Sharing: File Reader Generator with & w/o Policy
On Sun, Mar 16, 2014 at 2:34 PM, Mark H Harris wrote:
> And you are right about another thing, I just want to use this thing over
> and over.
>
> for line in getnumline(filename):
> {whatever}
>
>There does seem to be just one way of doing this (file reads) but there
> are actually many ways of doing this. Is a file object really better than a
> generator, are there good reasons for using the generator, are there
> absolute cases for using a file object?
I recommend you read up on the Rule of Three. Not the comedic
principle - although that's worth knowing about too - but the
refactoring rule. [1]
As a general rule, code should be put into a function when it's been
done three times the same way. It depends a bit on how similar the
versions are, of course; having two places where the exact same thing
is done might well be enough to refactor, and sometimes you need to
see four or five places doing something only broadly similar before
you can figure out what the common part is, but most of the time,
three usages is the point to give it a name.
There's a cost to refactoring. Suddenly there's a new primitive on the
board - a new piece of language. If you can't give it a good name,
that's potentially a high cost. Splitting out all sorts of things into
generators when you could use well-known primitives like enumerate
gets expensive fast - what's the difference between fName and fnName?
I certainly wouldn't be able to call that, without actually looking
them up.
Let your use-cases justify your refactoring.
ChrisA
[1] https://en.wikipedia.org/wiki/Rule_of_three_(computer_programming)
--
https://mail.python.org/mailman/listinfo/python-list
Re: Sharing: File Reader Generator with & w/o Policy
On 3/15/14 10:48 PM, Chris Angelico wrote:
There's a cost to refactoring. Suddenly there's a new primitive on the
board - a new piece of language . . . Splitting out all sorts of things into
generators when you could use well-known primitives like enumerate
gets expensive fast {snip}
[1] https://en.wikipedia.org/wiki/Rule_of_three_(computer_programming)
Very good to remember. I am finding the temptation to make all kinds of
generators (as you noted above). Its just that the python generator
makes it so easy to define a function that maintains state between calls
(of next() in this case) and so its also so easy to want to use them...
almost forgetting about primitives!
And the rule of three is one of those things that sneaks up on oneself.
I have actually coded about seven (7) such cases when I discovered that
they were all identical. I am noticing that folks code the same file
reader cases "with open() as fh: yadda yadda" and I've noticed that
they are all pretty close to the same. Wouldn't it be nice to have one
simpler getline() or getnumline() name that does this one simple thing
once and for all. But as simple as it is, it isn't. Well, as you say,
use cases need to determine code refactoring.
The other thing I'm tempted to do is to find names (even new names) that
read like English closely (whatever I mean by that) so that there is no
question about what is going on to a non expert.
for line in getnumline(file):
{whatever}
Well, what if there were a project called SimplyPy, or some such, that
boiled the python language down to a (Rexx like) or (BASIC like) syntax
and usage so that ordinary folks could code out problems (like they did
in 1964) and expert users could use it too including everything else
they know about python? Would it be good?
A SimplyPy coder would use constructs similar to other procedural
languages (like Rexx, Pascal, even C) and without knowing the plethora
of Python intrinsics could solve problems, yet not be an "expert".
SimplyPy would be a structured subset of the normal language for
learning and use (very small book/tutorial/ think the Rexx handbook, or
the K&R).
Its a long way off, and I'm just now experimenting. I'm trying to get my
hands around context managers (and other things). This is an idea I got
from Anthony Briggs' Hello Python! (forward SteveHolden) from Manning
books. Its very small, lite weight, handles real work, but--- its still
too big. I am wanting to condense it even further, providing the minimal
basic core language as an end application product rather than the
"expert" computer science language that will run under it.
or, over it, as you like.
(you think this is a nutty idea?)
marcus
--
https://mail.python.org/mailman/listinfo/python-list
Re: Sharing: File Reader Generator with & w/o Policy
On Sun, Mar 16, 2014 at 3:47 PM, Mark H Harris wrote:
> On 3/15/14 10:48 PM, Chris Angelico wrote:
>>
>> There's a cost to refactoring. Suddenly there's a new primitive on the
>> board - a new piece of language . . . Splitting out all sorts of things
>> into
>>
>> generators when you could use well-known primitives like enumerate
>> gets expensive fast {snip}
>>
>>
>> [1] https://en.wikipedia.org/wiki/Rule_of_three_(computer_programming)
>
>
> Very good to remember. I am finding the temptation to make all kinds of
> generators (as you noted above). Its just that the python generator makes it
> so easy to define a function that maintains state between calls (of next()
> in this case) and so its also so easy to want to use them... almost
> forgetting about primitives!
General rule of thumb: Every object in the same namespace should be
readily distinguishable by name alone. And if doing that makes your
names so long that the function signature is longer than the function
body, it might be better to not have that as a function :)
Also, I'd consider something code smell if a name is used in only one
place. Maybe not so much with local variables, as there are other
reasons to separate things out, but a function that gets called from
only one place probably doesn't need to exist at top-level. (Of
course, a published API will often seem to have unused or little-used
functions, because they're being provided to the caller. They don't
count.)
> And the rule of three is one of those things that sneaks up on oneself. I
> have actually coded about seven (7) such cases when I discovered that they
> were all identical. I am noticing that folks code the same file reader cases
> "with open() as fh: yadda yadda" and I've noticed that they are all pretty
> close to the same. Wouldn't it be nice to have one simpler getline() or
> getnumline() name that does this one simple thing once and for all. But as
> simple as it is, it isn't. Well, as you say, use cases need to determine
> code refactoring.
If getline() is doing nothing that the primitive doesn't, and
getnumline is just enumerate, then they're not achieving anything
beyond shielding you from the primitives.
> The other thing I'm tempted to do is to find names (even new names) that
> read like English closely (whatever I mean by that) so that there is no
> question about what is going on to a non expert.
>
> for line in getnumline(file):
> {whatever}
The trouble is that your idea of getnumline(file) might well differ
from someone else's idea of getnumline(file). Using Python's
primitives removes that confusion - if you see enumerate(file), you
know exactly what it's doing, even in someone else's code.
> Well, what if there were a project called SimplyPy, or some such, that
> boiled the python language down to a (Rexx like) or (BASIC like) syntax and
> usage so that ordinary folks could code out problems (like they did in 1964)
> and expert users could use it too including everything else they know about
> python? Would it be good?
>
> A SimplyPy coder would use constructs similar to other procedural languages
> (like Rexx, Pascal, even C) and without knowing the plethora of Python
> intrinsics could solve problems, yet not be an "expert".
>
> SimplyPy would be a structured subset of the normal language for learning
> and use (very small book/tutorial/ think the Rexx handbook, or the K&R).
>
> Its a long way off, and I'm just now experimenting. I'm trying to get my
> hands around context managers (and other things). This is an idea I got from
> Anthony Briggs' Hello Python! (forward SteveHolden) from Manning books. Its
> very small, lite weight, handles real work, but--- its still too big. I am
> wanting to condense it even further, providing the minimal basic core
> language as an end application product rather than the "expert" computer
> science language that will run under it.
>
> or, over it, as you like.
>
> (you think this is a nutty idea?)
To be quite frank, yes I do think it's a nutty idea. Like most nutty
things, there's a kernel of something good in it, but that's not
enough to build a system on :)
Python is already pretty simple. The trouble with adding a layer of
indirection is that you'll generally be limiting what the code can do,
which is usually a bad idea for a general purpose programming
language, and also forcing you to predict everything the programmer
might want to do. Or you might have an "escape clause" that lets the
programmer drop to "real Python"... but as soon as you allow that, you
suddenly force the subsequent reader to comprehend all of Python,
defeating the purpose.
We had a discussion along these lines a little while ago, about
designing a DSL [1] for window creation. On one side of the debate was
"hey look how much cleaner the code is if I use this DSL", and on the
other side was "hey look how much work you don't have to do if you
just write code directly". The more cushioning between the programmer
and the language, th
Re: Sharing: File Reader Generator with & w/o Policy
On 3/16/14 12:41 AM, Chris Angelico wrote:
Good stuff Chris, and thanks for the footnotes, I appreciate it.
If getline() is doing nothing that the primitive doesn't, and
getnumline is just enumerate, then they're not achieving anything
beyond shielding you from the primitives.
Yes. getline(fn) is returning the raw line minus the newline \n.
And getnumline(fn) is 1) creating a name that is easily recognizable,
and 2) shielding the 'user' from the primitives; yup.
The trouble is that your idea of getnumline(file) might well differ
from someone else's idea of getnumline(file). Using Python's
primitives removes that confusion
I am seeing that; esp for folks used to seeing the primitives; don't
want confusion.
To be quite frank, yes I do think it's a nutty idea. Like most nutty
things, there's a kernel of something good in it, but that's not
enough to build a system on :)
Thanks for your candor. I appreciate that too. Well, like I said,
I'm just experimenting with the idea right now, just playing around
really. In the process I'm coming more up-to-speed with python3.3 all
the time. :)
Python is already pretty simple.
statement == True
We had a discussion along these lines a little while ago, about
designing a DSL [1] for window creation. On one side of the debate was
"hey look how much cleaner the code is if I use this DSL", and on the
other side was "hey look how much work you don't have to do if you
just write code directly".
Was that on python-dev, or python-ideas, or here? I'd like to read
through it sometime.
Well just for grins, here is the updated my_utils.py for compare with
where I started tonight, ending like before with the code:
>>>
>>> for line in getline(path+"my_foxy"):
print (line)
The quick brown fox jumped
over the lazy dog's tail.
Now is the time for all
good women to come to the
aid of computer science!
>>> for line in getnumline(path+"my_foxy"):
print (line)
(0, 26, 'The quick brown fox jumped')
(1, 25, "over the lazy dog's tail.")
(2, 0, '')
(3, 23, 'Now is the time for all')
(4, 25, 'good women to come to the')
(5, 24, 'aid of computer science!')
>>> for line in getline(path+"my_utils.py"):
print (line)
#-
# __fOpen__(filename) generator: file open internal
#-
def __fOpen__(filename):
try:
with open(filename, 'r') as fh:
for linein in fh:
yield linein.strip('\n')
except FileNotFoundError as err_code:
print(err_code)
# think about error handling, logging
finally:
pass
#-
# getnumline(filename) generator: enumerated file reader
#-
def getnumline(filename):
for count, line in enumerate(__fOpen__(filename)):
yield((count, len(line), line))
#-
# getline(filename) generator: raw file reader iterable
#-
def getline(filename):
for line in __fOpen__(filename):
yield(line)
#-
# {next util}
#-
>>>
--
https://mail.python.org/mailman/listinfo/python-list
Re: Sharing: File Reader Generator with & w/o Policy
On Sun, Mar 16, 2014 at 5:19 PM, Mark H Harris wrote: > On 3/16/14 12:41 AM, Chris Angelico wrote: >> To be quite frank, yes I do think it's a nutty idea. Like most nutty >> things, there's a kernel of something good in it, but that's not >> enough to build a system on :) > > >Thanks for your candor. I appreciate that too. Well, like I said, I'm > just experimenting with the idea right now, just playing around really. In > the process I'm coming more up-to-speed with python3.3 all the time. :) Good, glad you can take it the right way :) Learning is not by doing whatever you like and being told "Oh yes, very good job" like in kindergarten. Learning is by doing something (or proposing doing something) and getting solid feedback. Of course, that feedback may be wrong - your idea might be brilliant even though I believe it's a bad one - and you need to know when to stick to your guns and drive your idea forward through the hail of oncoming ... okay, this metaphor's getting a bit tangled in its own limbs... okay, the meta-metaphor is getting... alright I'm stopping now. >> We had a discussion along these lines a little while ago, about >> designing a DSL [1] for window creation. On one side of the debate was >> "hey look how much cleaner the code is if I use this DSL", and on the >> other side was "hey look how much work you don't have to do if you >> just write code directly". > >Was that on python-dev, or python-ideas, or here? I'd like to read > through it sometime. Was here on python-list: https://mail.python.org/pipermail/python-list/2014-January/664617.html https://mail.python.org/pipermail/python-list/2014-January/thread.html#664617 The thread rambled a bit, but if you like reading, there's some good content in there. You'll see some example code from my Pike MUD client, Gypsum, and Steven D'Aprano and I discuss it. If you'd rather skip most of the thread and just go to the bits I'm talking about, here's my explanation of the Pike code: https://mail.python.org/pipermail/python-list/2014-January/665286.html And here's Steven's take on it: https://mail.python.org/pipermail/python-list/2014-January/665356.html And keep reading from there. TL;DR: It's not perfect as a DSL, but it's jolly good as something that is already there and takes no effort. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
