Re: [Tutor] lists, name semantics
On 18Apr2015 22:03, Bill Allen wrote: On Apr 18, 2015 4:11 PM, "boB Stepp" wrote: On Sat, Apr 18, 2015 at 3:28 PM, Bill Allen wrote: > On Apr 18, 2015 7:50 AM, "Peter Otten" <__pete...@web.de> wrote: >> You can test your newfound knowledge by predicting the output of the >> following script: >> >> a = [1, ["x", "y"], 3] >> b = a[:] >> >> a[1][1] = "hello!" >> >> print(a) # [1, ['x', 'hello!'], 3] >> print(b) # what will that print? >> >> Think twice before you answer. What is copied, what is referenced? > print(b) will print the original copy of a which b now references which is > [1, ["x", "y"], 3] Uh, oh! You should have checked your work in the interpreter before replying! Peter is being very tricky!! (At least for me...) Look again at that list inside of a list and... [...] Ok, just tried it out. In this example b=a and b=a[:] seem to yield the same results even after the change to a, which I do not understand. Should not b be a copy of a and not reflect the change? Because is it a _shallow_ copy. Doing this: b = a[:] produces a new list, referenced by "b", with the _same_ references in it as "a". a[1] is a reference to this list: ["x", "y"] b[1] is a reference to the _same_ list. So "a[:]" makes what is called a "shallow" copy, a new list with references to the same deeper structure. So this: a[1][1] = "hello!" affects the original x-y list, and both "a" and "b" reference it, so printing either shows the change. By contrast, this: a[1] = ["x", "hello"] _replaces_ the reference in "a" with a reference to a new, different, list. Sometimes you want a "deep" copy, where "b" would have got a copy of the iriginal x-y list. See the "copy" module's "deepcopy" function, which supplies this for when it is needed: https://docs.python.org/3/library/copy.html#copy.deepcopy Cheers, Cameron Simpson Draw little boxes with arrows. It helps. - Michael J. Eager ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] lists, name semantics
On 18Apr2015 23:26, boB Stepp wrote: On Sat, Apr 18, 2015 at 11:08 PM, Cameron Simpson wrote: Sometimes you want a "deep" copy, where "b" would have got a copy of the iriginal x-y list. See the "copy" module's "deepcopy" function, which supplies this for when it is needed: https://docs.python.org/3/library/copy.html#copy.deepcopy In this reference, part of it states: "Two problems often exist with deep copy operations that don’t exist with shallow copy operations: Recursive objects (compound objects that, directly or indirectly, contain a reference to themselves) may cause a recursive loop. Because deep copy copies everything it may copy too much, e.g., administrative data structures that should be shared even between copies." If I am understanding things correctly, should not that last sentence read instead: "...structures that should *not* be shared even between copies." ??? No, the text is correct. Suppose you have a graph of objects where some single object A is referenced in mulitple places: /--> O1 v G < A \--> O2 ^ If I make a deepcopy of "G" I want this: /--> O1copy v Gcopy < Acopy \--> O2copy ^ i.e. a _single_ copy "Acopy" of "A", again referenced from 2 places in the copied graph. Without deepcopy()'s object tracking you would get something like this: /--> O1copy > Acopy1 Gcopy < \--> O2copy > Acopy2 i.e. two distinct copies of "A". Changes to "Acopy1" would not affect "Acopy2". Cheers, Cameron Simpson They said it couldn't be done/they said nobody could do it/ But he tried the thing that couldn't be done!/He tried - and he couldn't do it. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] lists, name semantics
On 19Apr2015 15:09, Cameron Simpson wrote: On 18Apr2015 23:26, boB Stepp wrote: On Sat, Apr 18, 2015 at 11:08 PM, Cameron Simpson wrote: Sometimes you want a "deep" copy, where "b" would have got a copy of the iriginal x-y list. See the "copy" module's "deepcopy" function, which supplies this for when it is needed: https://docs.python.org/3/library/copy.html#copy.deepcopy In this reference, part of it states: "Two problems often exist with deep copy operations that don’t exist with shallow copy operations: Recursive objects (compound objects that, directly or indirectly, contain a reference to themselves) may cause a recursive loop. Because deep copy copies everything it may copy too much, e.g., administrative data structures that should be shared even between copies." If I am understanding things correctly, should not that last sentence read instead: "...structures that should *not* be shared even between copies." ??? No, the text is correct. Um, my explaination was incomplete. The first sentence quoted explains why it is necessary for deepcopy keep track of copied objects in order to correctly duplicate objects just once instead of twice or more, let alone in an unbounded way if there is a loop (A->B->C->A). It does this with a "memo" dictionary of already copied objects so it can know them when it sees them again. The second sentence "Because deep copy copies everything it may copy too much, e.g., administrative data structures that should be shared even between copies" is an issuewhich is addressed lower down when mentioning the __copy_ and __deepcopy__ methods. And it is not necessarily obvious. Suppose you've got a data structure of objects, which should broadly be copied. However, _internally_, these objects may use some external facility. Further, suppose that facility can't be copied; perhaps it is a reference to a databse or something built around something like that. Like this: G -> A -> B -> _open_database When you deepcopy that data sctructure you want to copy everything, but _not_ copy the external facility object. So for the above example, after the deepcopy you want this: Gcopy -> Acopy -> Bcopy -> _open_database i.e. the same reference to the database, but copies of "A" and "B". If you give "G"'s class a .__deepcopy__ method, then that will be called by deepcopy() to make a copy of "G" by calling "G.__deepcopy__(memodict)". And G's class will define its __deepcopy__ method to copy "A" and "B" but not "_open_database". Most classes do not need this and deepcopy() just copies everything. Does this clarify things? Cheers, Cameron Simpson No wonder these courtroom dramas are so popular. Half the population are lawyers, judges, or some other kind of criminal.- Jake Vest 1986 ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] introspection
On 21Apr2015 09:59, Alex Kleider wrote: On 2015-04-20 22:21, Danny Yoo wrote: What's supposed to happen in this situation? ## class Person(object): def __init__(self): pass j = Person() john = j jack = j ## What single name should we get back from the single Person object here? "j", "john", or "jack"? I was hoping that it would be possible to create a function that would do the following: def my_name(some_object): return some_object.__name__ so, using what you entered above.. my_name(j) 'j' my_name(john) 'john' my_name(jack) 'jack' But I see what I think you and others have been trying to explain to me: that the expression some_object.__name__, if it existed, would indeed be schizophrenic since it would be an attribute of the object, not the name(s) to which it is bound. But it would not be schizophrenic to write a function that returned a name arbitrarily, by inspecting locals(). It depends whether you only need a name, or if you need "the" name. Write yourself a "find_name_from_locals(local_map, value)" function that reports. That will (a) get you a partial answer and (b) cement in your mind what is possible and why. Easy and useful! Cheers, Cameron Simpson Technique will get you through times of no strength a lot better than strength will get you through times of no technique.- the Nealomatic ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] calling a method directly
On 21Apr2015 09:03, alan.ga...@btinternet.com wrote: On 21/04/15 05:19, Jim Mooney wrote: Is there any difference between these two since they give the same result, and when is the second preferred? x = 'ABE' x.lower() 'abe' str.lower(x) 'abe' They are essentially the same method being accessed in two different ways. The first via the instance, the second via the class. Though only because 'x' is directly a str. Not some other class. In Jim's example we _know_ 'x' is a str. In general one is less sure, and often one hopes not to care! It's a bit like when you call a superclass method in OOP: class C: ... def f(s): print 'in C' ... class D(C): ... def f(s): ... C.f(s) ... print 'and D' ... def g(s): print 'only D' ... In the first line of D.f() you invoke C's foo method by referring to C and passing the local self as the object. Worth pointing out that this is almost always the only time you cant to call via the class directly. And that it is actually common in subclass method implementations, particularly __init__, where on must often go: class D(C): def __init__(self, blah...): # initialise the common "C" class related stuff C.__init__(self, ...) # intialise the specific "D" class related stuff ... D-specific-stuff-here ... [...] There are very few cases where the class version is preferred, you nearly always use the instance technique. Indeed. Not least because (a) you don't always know or even care what class an object is and (b) objects themselves can have callable attributes. Cheers, Cameron Simpson Applications Programming is for dullards who can't do Systems Programming. - David Rose, d...@werple.apana.org.au ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Is it possible to "backport" the datetime module of Python 3.3 to Python 3.2?
On 21Apr2015 22:50, Steven D'Aprano wrote: On Tue, Apr 21, 2015 at 12:31:53PM +0100, Mark Lawrence wrote: Python is very strong in guaranteeing backward compatibility, so why not copy the 3.3 pure Python code to your 3.2 setup and see what happens? Normally backwards compatibility refers to the other way: 3.3 will run 3.2 code. To have 3.2 run 3.3 code is *forward compatibility*. Yes, but it can depend on where you stand. Mark was talking about taking the 3.3 datetime module code and running it on 3.2. So in this context, he is suggesting that the _code_ in the 3.3 datetime module is probably backward compatible - it has presents features but does not depend on 3.3 aspects of the language, and therefore may well run on 3.2. Cheers, Cameron Simpson The most annoying thing about being without my files after our disc crash was discovering once again how widespread BLINK was on the web. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Please disable “digest mode” before participating
On 26Apr2015 15:20, Dave Angel wrote: On 04/26/2015 08:07 AM, Ben Finney wrote: That doesn't reply to the original message. So it lacks the threading provided by the “In-Reply-To” field, since you're not replying to the original message. [...] When I used to subscribe here using digest, and reading with Thunderbird, I'd see each message as an attachment. To reply to one, I first opened the appropriate attachment, and did a Reply-List to it. WOrked well. I took a quick look, for another forum which I monitor using the digest, and it looks like the same approach would still work. For occasional posting. Did you check the digest for Mesaage-ID headers for each message item? Most digests lack them and as such, replies cannot be stitched together correctly. You may have been using a mailer with a "group messages by subject" (mis?)feature, which may have merely made it look like it worked well. Cheers, Cameron Simpson I swear to god, officer, I'm fixing this bridge. Just go divert traffic. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Please disable “digest mode” before participating
On 26Apr2015 22:07, Ben Finney wrote: Jim Mooney writes: On 25 April 2015 at 18:03, Ben Finney wrote: > Digest mode should only ever be used if you know for certain you will > never be responding to any message. That brings up a great shortcut if you use gmail. If you select some text before reply that's All that is sent. That doesn't reply to the original message. So it lacks the threading provided by the “In-Reply-To” field, since you're not replying to the original message. It also lacks [...] Yes, but it _does_ point an easy way for people from top-posting do-not-trim-quotes cultures to adopt the technophile's trim-quotes-for-relevance style which is so useful in discussion (versus chatter). Apple Mail also supports this handy "use the selected text for the context quote" technique and I think a number of the other OSX based mail programs. It doesn't cut it for multipoint quote1-reply1 quote2-reply2 messages, but it is a great jumping off point for people new to the method. _If_ the OP has abandoned digest mode, Jim's pointer is very useful. Before then I agree that it is far less so, but that is because digests are very unfortunate. Cheers, Cameron Simpson Never belong to any party, always oppose privileged classes and public plunderers, never lack sympathy with the poor, always remain devoted to the public welfare, never be satisfied with merely printing news, always be drastically independent, never be afraid to attack wrong, whether by predatory plutocracy or predatory poverty. - Joseph Pulitzer, 1907 Speech ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Function works one time then subsequently fails
On 28Apr2015 20:58, Jim Mooney Py3winXP wrote: This is really puzzling me. I'm parsing a string to do some simple math operations and practice tossing functions around. My parser works on the first run, then it continually fails on the same input. [...] numbers = [] [...] def parse_string(math_string): """Input: A math string with a verbal or mathematical operation and two valid numbers to operate on. Extra numbers and operations are ignored. Output: A tuple containing a function corresponding to the operation and the two numbers. Returns None on failure. """ operation = None tokens = math_string.split() for token in tokens: if token in operations: operation = operations[token] elif test_number(token) != None: numbers.append(test_number(token)) if len(numbers) > 1: break [...] At a first glance numbers is a global. It is reset to [] at program start, but never again. So you're appending to it forever. I have not investigated further as to how that affects your program's flow. Cheers, Cameron Simpson There are two ways of dealing with this problem: one is complicated and messy, and the other is simple and very elegant. We don't have much time left, so I'll just show you the complicated and messy way. - Richard Feynman, 1981 ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Function works one time then subsequently fails
On 28Apr2015 22:27, Jim Mooney Py3winXP wrote: On 28 April 2015 at 21:27, Cameron Simpson wrote: At a first glance numbers is a global. It is reset to [] at program start, but never again. So you're appending to it forever. I have not investigated further as to how that affects your program's flow. Took you less time to find that than me. I, of course, realized I forgot to empty the number list on each time through the parse loop, After I posted ;') Seems to work okay otherwise, although criticism of Pythonicity is welcome. I just wanted to practice tossing functions around. I'll put it into a user input loop and work it. It won't catch everything. I have to study regexes for that. Basicly, you should almost never have a global variable. There is a host of special circumstances where globals solve some specific problems. But by and large, don't do it without a very concrete reason. If numbers had been local to the parse function you would never have been bitten by this. There's no need for it to be global; you can return the operator and the operands (numbers) easily from the parse function. At this point I'm starting to lose track and have to think of better ways to organize so I recall and understand what I'm doing. I know there are general tuts on that but that's just reading. Is there a python-specific tut on it where I could follow actual code along with the interpreter to reinforce things? Or, what is the official word for organize-so-you-don't-forget-or-get-confused so I can search google with it? "functional decomposition"? One rule of thumb is that functions should be short. That way you can generally keep their purpose and internal oparations in your head in one go, and the whole thing also fits on your screen. As with all things, sometimes that cannot be reasonably achieved, but it is usually so. We can pick over your code as well if you like. Should we? Cheers, Cameron Simpson If this experiment we're doing works, then I will follow up and push it as hard as possible. And if it doesn't work, I will write a science-fiction novel where it does work. It's a win-win situation. - John Cramer on his experiment for possible cuasality violation ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] How to use Git from Windows PC for files on Solaris machine where Git cannot be installed?
On 29Apr2015 12:12, boB Stepp wrote: ... (3) install git if needed ... It seems Git is needed, but I am not allowed to install it on the Solaris workstation. So is there a way around this? What do you mean by "install"? Bear in mind that while you may be forbidden from installing git in the main areas (/usr, /usr/local, whatever), you may be free to install it in your own home directory. It is just an executable... Cheers, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Fwd: Function works one time then subsequently fails
On 29Apr2015 08:34, Jim Mooney Py3.4.3winXP wrote:
On 28 April 2015 at 22:40, Cameron Simpson wrote:
We can pick over your code as well if you like. Should we?
Sure, but let me include the full working program after fixup. Although I
still have to write another program to feed it random problems to work it
out. It's a bit spacey, but I ran it through the pep8 program and it
suggested that. I think two spaces between such tiny functions is overkill,
though.
"""
Takes the name of a binary math operation and two numbers from input,
repeatedly, and displays the results until done
"""
def add(a, b):
return a + b
def subtract(a, b):
return b - a
def minus(a, b):
return a - b
These could all do with docstrings. add() is pretty obvious, but the
distinction between minus() and subtract() could do with elaboration.
Also, I would be inclined to define minus() in terms of subtract(), not because
it is faster but because it establishes an equivalence.
def multiply(a, b):
return a * b
def divide(a, b):
return a / b
operations = {'add': add, '+': add, 'plus': add, 'subtract': subtract,
'subtracted': subtract,
'-': minus, 'minus': minus, 'multiply': multiply, '*':
multiply, 'multiplied': multiply,
'times': multiply, 'divide': divide, '/': divide, 'divided':
divide}
I'd make this much more vertical, like this:
operations = {'add': add,
'+': add,
'plus': add,
'subtract': subtract,
'subtracted': subtract,
Easier to maintain, easier to read, easier to modify, and if you use version
control you will get much more meaningful diff output with changes, which aids
code review (which more or less equates to readability and maintainability).
def test_number(astring):
"""
Input: A string that should represent a valid int or float. Output:
An int or float on success. None on failure.
"""
for make_type in (int, float):
try:
return make_type(astring)
except ValueError:
pass
return None
If this is a test function, it should return True or False.
Other have already remarked on this in other threads: the Pythonic way to do
this is usually to attempt the conversion and let an exception get out:
def numberify(astring):
return float(astring)
Consider the calling code. What are you going to do with it. I see you probe
first, which makes some sense in lexical analysis:
def parse_string(math_string):
"""Input: A math string with a verbal or mathematical operation
and two valid numbers to operate on. Extra numbers and operations
are ignored. Output: A tuple containing a function corresponding
to the operation and the two numbers. Returns None on failure.
"""
operation = None
tokens = math_string.split()
numbers = []
for token in tokens:
if token in operations:
operation = operations[token]
elif test_number(token) != None:
numbers.append(test_number(token))
if len(numbers) > 1:
break
if operation is None or len(numbers) < 2:
return None
else:
return operation, numbers[0], numbers[1]
As a lexical exercise probing (test then act) is pretty common and normal.
However, an alternative way to structure this is the "ask for forgiveness"
approach:
try:
operation = operations[token]
except KeyError:
value = numberify(token)
numbers.append(value)
This will raise ValueError if it is neither an operation nor a number.
The practice of raising an exception permits a mode of programming where both
your code and the caller can broadly write the code as though values are
generally valid, and avoids a lot error catching verbiage that obscures the
core logic.
In that vein, you would also modify the function to raise an exception
(typically ValueError) for other invalid math_strings. The advantage here is
that the caller can use your function like this:
operation, *arguments = parse_string(math_string)
and not perform any special testing for getting None back. Instead, the caller
can plow on as though the math_string was valid as well. An exception can
bubble up to a suitable catch point, such as the main loop which reads strings
from the user.
instructions = '''Enter two numbers and one of the four basid math
operations,
either mathematical or verbal. i.e. 3 + 2, 12 divided by 14, 10 minus 4,
etc.
Enter done to quit.
'''
try:
user_input = input(instructions)
while True:
if user_input == 'done':
break
While most programs will accept a special token
Re: [Tutor] How to use Git from Windows PC for files on Solaris machine where Git cannot be installed?
On 29Apr2015 22:10, boB Stepp wrote: On Wed, Apr 29, 2015 at 9:40 PM, Cameron Simpson wrote: On 29Apr2015 12:12, boB Stepp wrote: ... (3) install git if needed ... It seems Git is needed, but I am not allowed to install it on the Solaris workstation. So is there a way around this? What do you mean by "install"? Bear in mind that while you may be forbidden from installing git in the main areas (/usr, /usr/local, whatever), you may be free to install it in your own home directory. It is just an executable... On the smart enterprise where we (now) do our clinical planning they are very strict: no installing any external software; no accessing the Internet; no email; etc. Not all centers adhere to this level of strictness, but ours does. [...] But I am reluctant to do anything that I am not allowed to replicate on smart enterprise. I am trying to keep it as close to identical to the actual clinical environment as I can. And fair enough too. I frequently wish our developers' dev environments more rigorously modelled the target environment. Perhaps this is misguided on my part? Not necessarily. But I would be inclined to distinguish your working tools from the target environment. For example, I commend you making your dev environment as close to identical to the actual clinical environment as you can. But I wouldn't deprive yourself of tools that enhance your code management practices without affecting that environment. For example, there is much to be said for tracking your changes with a tool like git or hg. It encourages your to annotate your changes, and lets you review history to see when and why some change occurred, especially if a new bug has arisen. Distributed version control systems like git and hg also make it easier to work on distinct changes to the same code as separate work processes; of course the price for such a workflow is that you need to merge the two things later and ensure that they do not interact badly. If you're serious about keeping the two environments very similar, you could always do your _coding_ on a third platform (a laptop or whatever) using your desired tools and _deploy_ the code to the dev (and later, prod) environments from there, keeping the dev environment pristine. By "deploy", that can be as simple as copying the code to the target. Just thoughts; of course you must choose practices that suit your needs and plans. Cheers, Cameron Simpson Nothing is impossible for the man who doesn't have to do it. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Python property()
As Alan and I disagree on this, to a degree, I thought I would follow up to add another perspective. Remember that it is _only_ perspective; Alan's opinions are sound and have good reasons. So are mine:-) We place different values on the costs and benefits here. On 17May2015 00:27, alan.ga...@btinternet.com wrote: On 16/05/15 20:25, daaku gee wrote: I think I understand what's going on here. What I don't understand is if/when to use property instead of a normal attribute. 1. Is 'property' used widely? Do you use it? No, Yes, occasionally For me: somewhat, and yes. [...snip...] 2. Would you use it only when you need to validate or calculate the value to be assigned to an attribute or is there any other place for 'property()' to be used? Those are the main use cases for me. Calculating values can be an important one though, especially if it avoids the need for lots of methods to update some attribute like size. You can calculate it at runtime and expose the method to look like a data item. But that in itself can be done by a method, the real driver to a property is when I'm using the class in a polymorphic relationship with other classes that do have a data attribute and I want mixed access with a common interface. Here we differ, at least in where we draw the line. I am very fond of the "property to calculate a value". It makes for more readable code: foo.volume() + bah.volume() versus foo.volume + bah.volume with less syntactic noise. The "foo" and "bah" objects here might compute their volume from their more basic attributes, perhaps length, height etc. For Alan, using a property here provides no fundamental advantage over a method call, especially since a property _is_ a method call underneath. He feels that it breaks the OO model and as he mentions above, makes polymorphic use harder because different types of objects might readily provide a .volume() method but not a property. For me, when the property such as .volume is (a) a basic thing trivially and _repeatably_ computed from the other attributes and (b) cheap (usually O(1) to compute, not requiring complex and expensive operations) and (c) having no side effects such as creating files etc then I may often choose to use a property. To me, the method/property distinction is in how the value it thought of: A "computed", potentially complex or changing thing. This is a method. A "direct" and simple thing, almost free to compute and (usually) stable. This may be a property. If you need to pass parameters other than "self", obviously it must be a method. 3. If there is value in learning and using property, kindly explain it or point me to a document/writeup/book that I should read to get a better understanding. Oddly enough I'm currently reviewing a book that addresses this quite well in a chapter on properties. But it won't be published till later in the year so not much help to you just now. Personally I wouldn't worry about them until you think you've found a real use for them. There are plenty other things to be doing first! I'm in agreement here. Become comfortable with methods and using them. Until that is second nature you won't have much feel for when you may choose to use a property. Cheers, Cameron Simpson Heavier-than-air flying machines are impossible. --Lord Kelvin, president, Royal Society, 1895. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] ls *.py[co] >> .hidden
On 21May2015 17:52, Laura Creighton wrote: If you keep appending the results of ls to your .hidden file it will grow to enormous size. [...] sort -u .hidden >newhidden #or whatever you want to call it mv newhidden .hidden Or: sort -u -o .hidden .hidden Cheers, Cameron Simpson [...] post-block actions should be allowed everywhere, not just on subroutines. The ALWAYS keyword was agreed upon as a good way of doing this, although POST was also suggested. This lead to the semi-inevitable rehash of the try- catch exception handling debate. According to John Porter, "There is no try, there is only do. :-)" - from the perl6 development discussion ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Is there a way to use "with" across suite boundaries?
On 22May2015 22:56, Jim Mooney Py3.4.3winXP wrote:
'''I was using with open...:, but I'm printing a header in one function,
calling a looping
function to print detail lines, then returning to the calling function to
print
the footer. But that didn't work since the with statement only seems to work
with the lexical suite and the file wasn't open in the detail print
function. Is there
a way around this, other than passing the file as I have here? Also, is it a
good idea to pass a file handle like that or is there a better way?
It is just fine. Example:
with open("blah", "w") as blahfp:
print("header", file=blahfp)
print_details(blahfp, ...)
print("footer", file=blahfp)
def print_details(fp, blah_info):
... loop using blah_info to write data to "fp"
Cheers,
Cameron Simpson
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Yielding from a with block
On 27May2015 23:27, Oscar Benjamin wrote:
On 23 May 2015 at 10:52, Peter Otten <__pete...@web.de> wrote:
import csv
I'm just wondering what other people think about this. Should code
like make_lines below be discouraged?
In my opinion, no.
def make_lines():
with open('co2-sample.csv') as co2:
yield htbegin
for linelist in csv.reader(co2):
yield from fprint(linelist)
yield htend
def fprint(linelist):
yield ("{}{}{}"
"{}{}{}\n").format(*linelist)
yield '\n'
There's a fundamental contradiction between the with and yield
statements. With means "don't exit this block without running my
finalisation routine". Yield means "immediately exit this block
without doing anything (including any finalisation routines) and then
allow anything else to occur".
Not really. Yield passes a return value back to the iterator user and _pauses_
execution of the generator. The generator's execution context is _still_ inside
the "with".
Certainly if the user/consumer of the generator does not consume all the way to
the end then the "with" exit code won't fire. At least in a timely fashion;
might it fire during the cleanup phase? I'd think so.
However, the common circumstance is that the consumer does run to the end of
the iterator, and therefore the "with" exit process _will_ run when because
that is a standard part of the generator's execution.
BTW, I think including the fprint() function here is a furphy; it doesn't
affect the issue discussed.
Using yield inside a with statement like this renders the with
statement pointless:
This is not true.
either way we're really depending on __del__ to
execute the finalisation code. So it with or without the with
statement it will work fine on CPython. On other implementations it
may or may not close the file with or without the with statement. IOW
the with statement is not able to provide the guarantees normally
expected of it when used this way.
If make_lines() above runs to completion then the "with" exit process will also
run and everything will be fine.
It is only when make_lines() is not completely consumed that the file may not
be closed in a timely fashion.
Why? Because the consumer does not get to see end-of-iterator until
make_lines() actually returns, and that directly implies completion of the
"with" exit phase.
It's usually fairly trivial to rearrange things so that this doesn't happen:
def wrap_header_footer(fin):
yield htbegin
for linelist in csv.reader(fin):
yield from fprint(linelist)
yield htend
I would not encourage this kind of thing except in quite special circumstance.
For your example, failure to completely consume make_ines() will result in
invalid HTML output (missing closing "htend", for example). So the result of
the entire process will be invalid anyway, with the leaked open file just
another side effect.
Normally (ignoring unbounded iterators), the same scenario will apply: you
always iterate to the end of somehting like your example because to do
otherwise will usually result in an invalid/incorrect final result. So while
technically the risk you describe exists, in practice it will almost never
occur unless the larger program outcome also results in failure of some kind.
Cheers,
Cameron Simpson
I object to doing things that computers can do. - Olin Shivers
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] How to close a Tkinter window from a different thread?
On 27May2015 16:42, boB Stepp wrote: Python 2.4.4, Solaris 10 I realize that this is a bit old, but I got severely sidetracked! ~(:>) [...] On Mon, Apr 20, 2015 at 2:10 AM, Alan Gauld wrote: I would suggest forgetting about windows and think about the processes that create them. Use the OS tools (via the os module) to see if the first process is still running. If so kill the process - which will in turn kill the window. Is this a reasonable way to determine if a given pid is still active? I found this at: http://stackoverflow.com/questions/568271/how-to-check-if-there-exists-a-process-with-a-given-pid import os def check_pid(pid): """ Check For the existence of a unix pid. """ try: os.kill(pid, 0) except OSError: return False else: return True I realize that this will also return True if I don't have permissions to kill, or some such. Yes it is reasonable and normal. However the clean way to do it involves an extra test: try: os.kill(pid, 0) except except OSError as e: if e.errno == errno.EPERM: return True return False else: return True Do bear in mind that process ids repeat. So on a busy system (or if your own program is very long lived and the original process exited a long time ago) you might be polling that pid as used by something unrelated. However, if you started the process directly and it has not been wait()ed for, then the pid will not be available for reuse and you will be ok. You can find the process based on its name or based on its PID which you could store in a file somewhere (like in /tmp?) I can implement this, but thinking about this anew has led to a concern. Hypothetically, if I check for the existence of a running process based on this stored pid, what is the likelihood that the pid will be reassigned to something other than one of my program's windows left open? As discussed above. Possible but unlikely. And if you are the creator of the original process you can control the situation. Cheers, Cameron Simpson Processes are like potatoes.- NCR device driver manual ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Generate Prime Numbers
On 29May2015 18:48, alan.ga...@btinternet.com wrote: On 29/05/15 16:28, Mirage Web Studio wrote: def IsDivisibleBy3(number):#string variable v=0 for c in number: v=v+int(c) if v%3==0: return True else: return False def IsDivisibleBy3(number):#string variable return not int(number) % 3 To illustrate that there isn't just One Right Way, I would code the above like this: def IsDivisibleBy3(number): #string variable return int(number) % 3 == 0 Alan's code relies on "not" using an expressions "nonzero"ness as a boolean value. I _much_ prefer to directly state the test instead of relying on magic numeric=>boolean effects. The only other remark I would make is that it is probably a ba idea to write this function to take a string. It expresses a numerics test - make it take a number! Do the string conversion-to-int outside the function before it is called. def IsDivisibleBy7(number):#string variable See above, but maybe better still def isDivisibleByN(number, N): return not int(number)%N Indeed, but again I would write an outright numeric expression and not rely on "not(numeric-expression)". def IsPrime(number): Google for the "sieve of eratosthenes" Or look it up on wikipedia. That will give one of several possible improvements to your algorithm. Plus, it is incredibly easy to code the SoE in Python. So easy that I once did it in about a minute in a Code Wars session. It is a lovely algorithm which is easy to remember. primelist=[] for i in range (11,20,2): number=str(i) print "checking ",i if IsPrime(number): Note, you are starting with a number, then converting it to a string then in your functions converting it back to a number. That's crazy! Indeed! Numbers throughout if at all possible. Cheers, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] unittest with random population data
On 30May2015 12:16, Sydney Shall wrote:
Following advice from you generous people, I have chosen a project >that
interests me, to develop some knowledge of python.
My projest is a simulation of a biological population.
I have a base class and a simulation function, which uses instances of
the class.
This, after many months of work and lots of advice, now seems to work
well. It generates sensible data and when I write a small test program
it gives sensible output.
Now I want to learn to use unittest.
I have written a unittest class which works OK.
But the problem I have is that because I use the random module to
populate my initial arrays, my data is not strictly predictable even
though I am using seed(0). So the tests return many *fails* because
the numbers are not exactly correct, although they are all rather
close, consistent with the sigma value I have chosen for the spread of
my population. I do of course use *almostEqual* and not *Equal*.
First of all, several people have posted suggestions for getting identical
results on every run.
However, there is another approach, which you might consider. (And use in
addition, not inseadt of, the reproducable suggestions).
It is all very well to have a unit test that runs exactly the same with a test
set of data - it lets you have confidence that algorithm changes do not change
the outcome. But on for that data set.
You say that your results are "all rather close, consistent with the sigma
value I have chosen for the spread of my population". I would advocate making
some "contraint" tests that verify this property for _any_ input data set.
Then you can run with random and _changing_ input data sets to verify that your
code produces the expected _kind_ of results with many data sets.
So you would have one test which ran with a fixed data set which confirms
preidctable unchanging results. And you have other tests with run with randomly
chosen data and confirms that outcomes fall within the parameters you expect.
You can apply those checks ("outcome in range") to both sets of tests.
As an exmaple, I have a few classes which maintain data structures which are
sensitive to boundary conditions. The glaring example is a numeric range class
which stores contiguous ranges efficiently (a sequence of (low,high) pairs). It
has a few add/remove operations which are meant to maintain that sequence on
ordered minimal form. cutting and merging adjacent ranges is very easy to get
wrong, very sensitive to off-by-one logic errors.
So my tests for this class include some random tests which do random
unpredictable add/remove operations, and run a consistency check on the object
after each operation. This gives me good odds of exercising some tricky
sequence which I have not considered explicitly myself.
You can see the test suite here:
https://bitbucket.org/cameron_simpson/css/src/tip/lib/python/cs/range_tests.py
It has a bunch of explicit specific tests up the top, and then the random
consistency test down the bottom as "test30random_set_equivalence".
Cheers,
Cameron Simpson
MS-Word is Not a document exchange format - Jeff Goldberg
http://www.goldmark.org/netrants/no-word/attach.html
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] unittest with random population data
[ I've taken this discussion back to the tutor list. - Cameron ] On 01Jun2015 18:14, Sydney Shall wrote: On 31/05/2015 03:00, Cameron Simpson wrote: You say that your results are "all rather close, consistent with the sigma value I have chosen for the spread of my population". I would advocate making some "contraint" tests that verify this property for _any_ input data set. Then you can run with random and _changing_ input data sets to verify that your code produces the expected _kind_ of results with many data sets. [...] range class which stores contiguous ranges efficiently (a sequence of (low,high) pairs). It has a few add/remove operations which are meant to maintain that sequence on ordered minimal form. cutting and merging adjacent ranges is very easy to get wrong, very sensitive to off-by-one logic errors. So my tests for this class include some random tests which do random unpredictable add/remove operations, and run a consistency check on the object after each operation. This gives me good odds of exercising some tricky sequence which I have not considered explicitly myself. You can see the test suite here: https://bitbucket.org/cameron_simpson/css/src/tip/lib/python/cs/range_tests.py I have studied the material you indicated and it has been most helpful. I think that I have understood the principle involved, but I have had some problem implementing it. The range tests are mostly clear to me but there is one aspect I cannot follow. You use in this suite imports from Range, including Range, overlap, spans and Span. Are these programs that you have written? If so, are they specific to your set up or are they generic? If so, is it possible to obtain these programs? The "cs.range" is a module of my own. As you might imagine, cs.range_tests has the entire purpose of testing cs.range. cs.range is here: https://bitbucket.org/cameron_simpson/css/src/tip/lib/python/cs/range.py Feel free. Since it in turn has some dependencies the easiest way to get it is to use "pip" to install it, as pip will also fetch and insteall its dependencies. But if you just want something to read, fetch and enjoy. I have established a very primitive test suite based on your cs.range >notions and it works fine, but it would be better, I am sure, to do it >properly. There are many other people whose test writing ability surpasses mine. But you're welcome to learn anything you can from mine, and to ask questions. Finally, I have one comment for the respected Moderator, if he is not out on a walk in the highlands in this cold and wet weather. I have taken the liberty of raising my problem here rather than elsewhere, because I have observed that the biological and bio-medical community, who always come late to new notions, is now rapidly discovering python. A great deal of work in these fields involve either stochastic simulations or statistical problems of analysis. The latter are more or less straight-forward, but the simulations are not. You might ask a separate question on the python-l...@python.org about simulations. It has a wider audience than the tutor list and may well include people doing simulation work, or who know where to look. Thanks for all the help. You people are a model of how we could perhaps civilize humanity. Nah. We might all be striving to be a model of how humanity might be when civilised though... Cheers, Cameron Simpson You my man are a danger to society and should be taken out of society for all our sakes. As to what is done to you once removed I couldn't care less. - Roy G. Culley, Unix Systems Administrator ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] string delimiters
On 03Jun2015 14:16, Alex Kleider wrote: On 2015-06-03 12:53, Alan Gauld wrote: ... If this is really about parsing dates and times have you looked at the datetime module and its parsing/formatting functions (ie strptime/strftime)? I asssume strftime gets its name from 'string from time.' What about strptime? How did that get its name? No, they both come from the standard C library functions of the same names, being "(f)ormat a time as a string" and "(p)arse a time from a string". The shape of the name is because they're "str"ing related functions, hence the prefix. See "man 3 strptime" and "man 3 strftime". Cheers, Cameron Simpson A program in conformance will not tend to stay in conformance, because even if it doesn't change, the standard will. - Norman Diamond ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] string delimiters
On 03Jun2015 17:35, richard kappler wrote:
Perhaps the better way for me to have asked this question would have been:
How can I find the location within a string of every instance of a
character such as ']'?
With the str.find method!
s = 'a]b]c'
pos = s.find(']')
while pos >= 0:
print("pos =", pos)
pos = s.find(']', pos + 1)
Obviously you could recast that as a generator funciton yielding positions for
general purpose use.
Cheers,
Cameron Simpson
Serious error.
All shortcuts have disappeared.
Screen. Mind. Both are blank.
- Haiku Error Messages
http://www.salonmagazine.com/21st/chal/1998/02/10chal2.html
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] line iteration in a file
On 03Jun2015 22:37, richard kappler wrote: Figured out the string delimiters problem, thanks for all the help. Now I've run into another. I've used the re.finditer that I think it was Peter suggested. So I have: for line in file: s = line t = [m.start() for m in re.finditer(r"]", s)] q = len(t) which works fine, in testing it finds the number and position of the ]'s in any line I throw at it. I then wrote a series of if/elif statements based on q, in other words if q == 1: do something elif q == 2: do something else elif q == 3: do a third thing else: pass as I looked through enough example to figure out that the most ]'s I can have is 3, but the pass is there just in case. I keep getting a list index out of range error, and my best guess is that it's because t and q are set on the first line read, not each line read, is that right? If not, what might be the problem and either way, how do I fix it? Please post a self contained example (i.e. small complete code, not snippets) and a transcribe of the full error message with stack backtrace. What you have above is not enough to figure out what is going wrong. If what you display above is accurate then t and q are set for every line read. Another remark, what is the use of your "else: pass" code? Normally one would put some action here, such as raising an exception for the unhandled value or issuing a warning. Cheers, Cameron Simpson My computer always does exactly what I tell it to do but sometimes I have trouble finding out what it was that I told it to do. - Dick Wexelblat ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
tutor@python.org
On 11Jun2015 18:55, alan.ga...@btinternet.com wrote: On 11/06/15 14:25, Michelle Meiduo Wu wrote: I read some document and found Pytest can be used to write simpler code compared with using unittest in Python. Yes, but PyTest (and unittest) are for testing units of Python code. So they really only apply if your application is written in Python. Though if you like the framework, you could write very thin Python wrappers calling something else, and test them. Might be more trouble thatn it is worth though. And in any case you should really have written all the unit tests long before completing your application. Ah, the S-word :-) Cheers, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] method conflict?
On 02Jul2015 17:39, Jim Mooney Py3.4.3winXP wrote:
Okay, it appears the method in a class has its own ID, but all
instantiations of that method have identical IDs. But what happens if we
have a huge number of instantiations trying to access the identical method
at the same time?
Um, they all get given access to the method? I'm not sure what you think should
be happening here:
class MyClass:
def setdata(self, data):
self.data = data
def getdata(self):
print(self.data)
MyClass.setdata
id(MyClass.setdata)
40676568
f = MyClass()
g = MyClass()
Two instances of the class.
id(f.setdata)
43576616
id(g.setdata)
43576616
Two references to the same method obtained from the class.
d = MyClass()
id(d.setdata)
43576616
And a third.
Steven D'Aprano posted a decent explaination in your other thread ("Are the
methods in a class copied or just linked to?"); it would have been helpful if
your post had stayed in that thread - it is essentially the same discussion.
So all your "d.setdata" expressions consult various places in order to find the
method definition, and they all find it in the class. And so you are given the
same method every time, and thus the same id.
Cheers,
Cameron Simpson
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] String Problem
On 06Jul2015 15:44, Crusier wrote:
Dear All,
I have used the urllib.request and download some of the information from a
site.
I am currently using Python 3.4. My program is as follows:
import urllib.request
response = urllib.request.urlopen('
http://www.hkex.com.hk/eng/ddp/Contract_Details.asp?PId=175')
saveFile = open('HKEX.txt','w')
saveFile.write(str(response.read()))
saveFile.close()
And the result is as follows:
d align="right"> - 0
[...]
Please let me know how to deal with this string. I hope I could put onto a
table first. Eventually, I am hoping that I can able to put all this
database. I need some guidance of which area of coding I should look into.
Look into the BeautifulSoup library, which will parse HTML. That will let you
locate the TABLE element and extract the content by walking the rows (TR) and
cells (TD).
Start here:
http://www.crummy.com/software/BeautifulSoup/bs4/doc/
You can install bs4 using pip, or in other ways:
http://www.crummy.com/software/BeautifulSoup/bs4/doc/#installing-beautiful-soup
Cheers,
Cameron Simpson
30 years ago, I made some outrageous promises about AI. I didn't deliver.
Neither did you. This is all your fault. - Marvin Minsky, IJCAI'91 (summary)
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Socket Module
On 12Jul2015 22:36, Nym City wrote:
Hello,
I am working on a 2 individual programs. In the first program, I am taking in a
list of domains in .csv format and want to get the associated IPs.
In the second program, I want to import in a list of IP .csv and than get the
associated domains out.
Eventually I want to get both of the above programs to write out to the same
.csv file that they are reading from- in the next column. But I am not there
yet.
Doing research I came across the following
example:http://python.about.com/od/pythonstandardlibrary/qt/dnscheck.htm
From the link above, doing domain -> IP would be easier to begin with. Here are
the two variations of my code:
Variation 1:
import csv
import socket
domains = open('top500domains.csv', 'r')
for domain in domains:
You're getting a whole line here, including the traling newline. Add this line:
print("domain=%r" % (domain,))
at the start of the loop to see this.
domain = socket.gethostbyname(str(domains))
There's no point converting a line to str; a line is a str. If you're wokring
from example code I suppose you might have mistyped "strip".
Anyway, you're passing a string ending in a newline to gethostbyname. It will
not resolve.
print(domains + "\n")
For the code above, I receive the following error on run: socket.gaierror:
[Errno -2] Name or service not known.
As the exception suggests. When debugging issues like these, _always_ put in
print() calls ahead of the failing function to ensure that you are passing the
values you think you're passing.
Variation 2:
import csv
import socketdomains = []
with open('top500domains.csv', 'r') as f:
for line in f:
line = line.strip()
domains.append(line)
hostbyname_ex = socket.gethostbyname_ex(str(domains))
gethostbyname_ex takes a single hostname string. You are passing a list of
strings.
print(hostbyname_ex)
I receive the same error as the first variation.
Please share your advice and let me know what I am doing wrong. The
top500domains.csv list is attached.
Plenty of mailing list software strips attachments. In future, unless the file
is large, just append the contents below your message (with some explaination).
Cheers,
Cameron Simpson
Life is uncertain. Eat dessert first. - Jim Blandy
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] _ vs. _name vs. __name vs. name_ vs. __name__ usages
e to indicate their flavour. That works just fine. But when I want to access the object, "type" is also a natural name for the parameter: def check_type(self, type): return self.type == type While that works, it means you have lost access to the predefined Python type() function inside that bit of code. Which can be bad. Even if you do not use the function, it makes your code confusing for other programmers (== you several months later) because they have an expectation of its meaning. Way of addressing this all involve just using another name. Common ways include "name_", "n" (trivial short term variable) or a misspelling. For example, saying "klass" instead of "class". 5) __name__ is meant to be used only by the creators of Python for their special built-in methods, such as __init__, __new__, etc. Yes, but in the sense that they have predefined meanings. If you are implementing a mapping (a class that has methods and access like a dictionary), you will need to define methods using these names to implement the API. So don't make up your own! Are my understandings above correct or flawed? Yes. For (3), it seems to me that one would normally be able to use the simpler _name construction from (2). What would be a best-practice example of when name mangling *should* be used? Well, people rarely use _name as a function parameter, unless it is a private/secret parameter for special internal use. So I would write this: def check_type(self, type_): return self.type == type_ instead of using "_type". because "type_" here is a _public_ parameter for people to use. Using "_type" would mislead. Likewise, it seems that normally (4) should never be needed, though I have a feeling that I have seen something in tkinter recently that suggests some exceptions, but I cannot (yet) bring it to mind. (4) is never required, but is often convenient. And for (5), surely I should never violate this one? It seems that in some future edition of Python they might add any particular __name__ that I might try to use presently in their future version of Python (however miniscule that possibility might actually be). Yes: don't make up new __name__s. But you will _implement_ some __name__s at some point. Cheers, Cameron Simpson On the one hand I knew that programs could have a compelling and deep logical beauty, on the other hand I was forced to admit that most programs are presented in a way fit for mechanical execution, but even if of any beauty at all, totally unfit for human appreciation. - Edsger W. Dijkstra ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Mailbox
On 29Jul2015 21:55, ltc.hots...@gmail.com wrote:
I have a second and unrelated question:
I tried to work backward to see if there is a logic error associated with a
variable is being skipped over:
#top of code, initialize variable
output_list = ["default"]
Curious: why not just make this an empty list: []
[...snip...]
count = 0
fname = raw_input("Enter file name: ")
if len(fname) < 1 : fname = "mbox-short.txt"
for line in fname:
line = line.strip()
if not line.startswith('From '): continue
line = line.split()
count = count + 1
print len(line)
fh = open(fname)
print "There were", count, "lines in the file with From as the first word"
[...snip...]
My first observation would be that "count = count + 1" is not inside the loop.
That means that it fires just once, _after_ the loop. So it will always be 1.
Also, what is the purpose of this line:
line = line.split()
Cheers,
Cameron Simpson
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] String Attribute
On 31Jul2015 17:21, Ltc Hotspot wrote: Mark: Is this any better, message sent from GMail? Regards, Hal Looks better to me. Cheers, Cameron Simpson On Fri, Jul 31, 2015 at 5:02 PM, Mark Lawrence wrote: On 31/07/2015 19:57, ltc.hots...@gmail.com wrote: I believe that this is the third time that you've been asked to do something about the amount of whitespace that you're sending to this list. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] scratching my head
On 02Aug2015 23:01, ALAN GAULD wrote:
On 02/08/15 22:44, Clayton Kirkwood wrote:
for dir_path, directories, files in os.walk(main_dir):
for file in files:
#print( " file = ", file)
# if( ("(\.jpg|\.png|\.avi|\.mp4)$") not in file.lower() ):
Python sees that as a single string. That string is not in your filename.
#if( (".jpg" or ".png" or ".avi" or ".mp4" ) not in file.lower()
[...]
But you could use a loop:
found = False
for s in (".jpg",".png",".avi",".mp4"):
found = test or (s in file.lower())
if not found: ...
if( ".jpg" not in file.lower() and
".png" not in file.lower() and
".avi" not in file.lower() and
".mp4" not in file.lower() ):
Whether that's any better than your combined test is a moot point.
Alan has commented extensively on the logic/implementation errors. I have a
suggestion.
Personally I'd be reaching for os.path.splitext. Untested example below:
from os.path import splitext
for dir_path, directories, files in os.walk(main_dir):
for file in files:
prefix, ext = splitext(file)
if ext and ext[1:].lower() in ('jpg', 'png', 'avi', 'mp4'):
which I think is much easier to read.
BTW, I'd be using the variable names "filename" and "filenames" instead of
"file" and "files": in python 2 "file" is a builtin function (though long
deprecated by "open()") and in any case I'd (personally) expect such a name to
be an _open_ file. As opposed to "filename", which is clearer.
Cheers,
Cameron Simpson
Rudin's Law:
If there is a wrong way to do something, most people will do it every time.
Rudin's Second Law:
In a crisis that forces a choice to be made among alternative courses of
action, people tend to choose the worst possible course.
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] scratching my head
On 02Aug2015 16:15, Clayton Kirkwood wrote:
Behalf Of Cameron Simpson
Sent: Sunday, August 02, 2015 3:35 PM
[...]
Personally I'd be reaching for os.path.splitext. Untested example below:
from os.path import splitext
for dir_path, directories, files in os.walk(main_dir):
for file in files:
prefix, ext = splitext(file)
if ext and ext[1:].lower() in ('jpg', 'png', 'avi', 'mp4'):
which I think is much easier to read.
BTW, I'd be using the variable names "filename" and "filenames" instead of
"file" and "files": in python 2 "file" is a builtin function (though long
deprecated by "open()") and in any case I'd (personally) expect such a
name
to be an _open_ file. As opposed to "filename", which is clearer.
Thanks, that should also help a lot. Now time to look at splitext, and the
ext and ext[1:.
The "[1:]" is because "ext" will include the dot.
I appreciate your comments also about the variable names.
Any comments on the problems lower in the file?
Maybe you'd better reraise these problems again explicitly.
Cheers,
Cameron Simpson
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] scratching my head
On 03Aug2015 08:12, Laura Creighton wrote: I think people are giving you sub-optimal advice. Python has a module in the standard library for doing exactly what you want to do -- match files with certain extensions. See: https://docs.python.org/2/library/fnmatch.html It's unix style file matching, but I am fairly certain this works on windows also. I don't have a windows machine to test and make sure. That depends. This is the tutor list; we're helping Clayton debug his code as an aid to learning. While it's good to know about the facilities in the standard library, pointing him directly at fnmatch (which I'd entirely forgotten) is the "give a man a fish" approach to help; a magic black box to do the job for him. Besides, I'm not sure fnmatch is much better for his task than the more direct methods being discussed. Cheers, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] scratching my head - still
On 05Aug2015 12:46, Steven D'Aprano wrote: On Tue, Aug 04, 2015 at 05:52:15PM -0700, Clayton Kirkwood wrote: As seen below (closely), some filenames are not being removed while others are, such as in the first stanza, some pdfs are removed, some aren't. In the second stanza, Thumbs.db makes it through, but was caught in the first stanza. (Thanks for those who have proffered solutions to date!) I see no logic in the results. What am I missing??? You are modifying the list of files while iterating over it, which plays all sorts of hell with the process. Watch this: [... detailed explaination ...] The lesson here is that you should never modify a list while iterating over it. Instead, make a copy, and modify the copy. What Steven said. Yes indeed. Untested example suggestion: all_filenames = set(filenames) for filename in filenames: if .. test here ...: all_filenames.remove(filename) print(all_filenames) You could use a list instead of a set and for small numbers of files be fine. With large numbers of files a set is far faster to remove things from. Cheers, Cameron Simpson In the desert, you can remember your name, 'cause there ain't no one for to give you no pain. - America ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Fraction Class HELP ME PLEASE!
On 06Aug2015 16:55, Quiles, Stephanie wrote:
I need to do the following assignment. I need to know how do i hard code an
example for each of the operators I am implementing? What i have so far is
below? He said he does not care if we plug in some numbers or if we have user
input numbers, however I am unsure of how to write a program that tests each
operator? Or can i write one that tests all of them? I don’t know where to
start with this. Please help!
For someone who doesn't know where to start, you seem to have a lot of decent
looking code. Is the code below all yours? If so, a good start.
Also, please try to preserve the indentation when pasting in code; indentation
is critical in Python as you know and incorrect indentation, when not an
outright syntax error, can be a source of bugs. I'll presume your code was
originally indented correctly.
Note that in this list we _do_ like code to be pasted inline where it can be
seen directly and commented on like any other part of the message text. We're
not big on attachments or links to external things (other than doco citations
like your reference to the operators Python page).
You may implement as many of these operators as you like (such as isub,
itruediv, etc.) You MUST indicate in your header information which operators
you are implementing, and you MUST hard code an example for each.
"Header information" is usually an opening comment. So perhaps start the
program with text like this:
# Here is a Fraction class implementing variaous operators. It currently
# supports:
#
# + (addition)
# - (subtraction)
# * (multiplication)
# / (true division)
#
and so forth.
Regarding examples and tests, one workable approach to to make your program
work as a "main program". The idea is that is invoked as a main program it will
run the examples or tests.
As your code sits now it could be used as a module - someone could place it
into python's library tree and import it, and use your Fraction class.
However, you can of course run it directly at the command prompt, eg:
% python your-fraction-file.py
When you do that, the code is still imported as a module but with the special
name '__main__'. So what a lot of python modules do is have a "main" function
which is to be fired only in the command line circumstance, such as:
def main():
F1 = Fraction(1, 2) # 1/2
F2 = Fraction(2, 3) # 2/3
print("F1 =", F1)
print("F2 =", F2)
print("F1 + F2 =", F1 + F2)
print("F1 - F2 =", F1 - F2)
print("F1 * F2 =", F1 * F2)
In order to make your program work as both an importable module which defines
the Fraction class but does not run the main function and also as a main
program which defines the class and then runs the main function you put this
piece of boilerplate code on the very bottom of the program:
if __name__ == '__main__':
main()
which checks if you're invoking it directly from the command line. If so, run
the main function.
Hopefully that gives you an idea about hardwiring examples also.
Regarding tests, you might write a simple function like this:
def check(label, computed, expected):
ok = computed == expected
if ok:
print(label, 'OK', computed, '==', expected)
else:
print(label, 'BAD', computed, '!=', expected)
return ok
Then you might consider modifying your main program to run tests instead of
bare examples, replacing:
print("F1 + F2 =", F1 + F2)
with:
check("F1 + F2", F1 + F2, Fraction(7, 6))
Because check() returns whther the check was ok, you might even count the
number of failures for some kind of report:
fail_count = 0
...
if not check("F1 + F2", F1 + F2, Fraction(7, 6)):
fail_count += 1
... more checks ...
print(fail_count, "failures")
and so forth.
This also gets you test code so that you can test your own program for
correctness before submitting your assignment.
Feel free to return to this list with updated code and new questions.
Cheers,
Cameron Simpson
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Fraction Class HELP ME PLEASE!
method.
All the __name__ methods and attributes in Python are considered part of the
language, and typically and implicitly called to implement things like
operators (i.e. F1+F2 calls F1.__ad__(F2)).
You should not make up __name__ methods: they are reserved for the language.
There are at least two downsides/risks here: first that you think this will be
used, when it will not - the code will never be run and you will wonder why
your call does not behave as you thought it should. The second is that some
furture update to the language will define that name and it will not mean what
you meant when you used it. Now your code _will_ run, but not do what a user
expects!
BTW, the jargon for the __name__ names is "dunder": .__add__ is a "dunder
method"; derived from "double underscore", which I hope you'd agree is a
cumbersome term.
def main():
F1 = Fraction(1,2)
F2 = Fraction(2,3)
print("F1 = ", F1)
print("F2 = ", F2)
print() adds a space between the comma separate items, you don't need to
include one inside the quotes.
print("Add Fractions: F1 + F2=", Fraction.__add__(F1, F2))
Any reason you've not fired this implicitly? Like this:
print("Add Fractions: F1 + F2=", F1 + F2)
Actually, you can make a case for doing both in your main program to show that
they do the same thing.
print("Subtract Fractions: F1 - F2=", Fraction.__sub__(F1, F2))
print("Multiply Fractions: F1 * F2=", Fraction.__mul__(F1, F2))
print("True Division with Fractions: F1 / F2=", Fraction.__truediv__(F1, F2))
print("Exponentiation with Fractions: F1 // F2=", Fraction.__pow__(F1, F2))
Shouldn't this be "**"?
print("Is F1 Greater than F2?:", Fraction.__gt__(F1, F2))
print("Is F1 less than F2?:", Fraction.__lt__(F1, F2))
print("Is F1 Equal to F2?:", Fraction.__eq__(F1, F2))
print("Is F1 different than F2?:", Fraction.__ne__(F1, F2))
print ("Is F1 same as F2?:", Fraction.__is__(F1, F2))
Here you want to avoid this. Firstly, Python has an "is" operator, and it does
not have a dunder method. Secondly, in what way does your __is__ differe from
__eq__?
print("Is:", Fraction.__iadd__(F1, F2))
if __name__ == '__main__':
main()
Otherwise this is all looking promising. Does it run correctly for you?
Cheers,
Cameron Simpson
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Fraction Class HELP ME PLEASE!
... ''' F1 = ... ... When you put a string inside a function or class, immediately under the "def" or "class" line, that string is called a docstring as it is intended as documentation for that function or class. The Python language actually collects it and saves it as the .__doc__ attribute of the function or class, and you can get it back later. In the interactive Python interpreter (which you get when you just run the command "python"), you can call the help() function on something and have the docstring recited back at you. So at the python prompt you can type: help(main) and be told about it. By putting docstrings on your Fraction class and also on each of its methods you make that information available via help() later. Otherwise, good luck. Cheers, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Pep 8, about indentation
On 06Aug2015 11:03, ALAN GAULD wrote: On 06/08/15 04:40, Steven D'Aprano wrote: Um, the oldest standard for TABs is *eight* spaces, not 3 or 4, Yes, but that comes from the days of mechanical typewriters not any study of code comprehension. [...snip...] I beg to differ a little. Old _teletypes_ and terminals used 8 character TAB spacing. Mechanical typewriters had slidable lugs to position to totally arbitrary spots with a lever motion triggered by a key labelled "Tab", which was very useful for typing tabular information, hence the name. There's no "8" in there! My personal habit is to have my terminals, editors such as vi, pagers like less etc interpret the TAB byte code to mean align to the next 8 column position. That is the common default meaning. However, when _editing_ I tell vi that when I press TAB it is to insert enough SPACE characters to get to the next 4 column position. In this way I have the convenience of a TAB key to advance rapidly when needed, and my code comes out composed entirely with spaces so that my spacing in not mispresented to others. In short, I use the TAB _key_ to aid typing. But I avoid embedded TAB characters into my text, because they are subject to arbitrary interpreation by others. Cheers, Cameron Simpson I had a *bad* day. I had to subvert my principles and kowtow to an idiot. Television makes these daily sacrifices possible. It deadens the inner core of my being.- Martin Donovan, _Trust_ ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Pep 8, about indentation
On 07Aug2015 12:19, Alex Kleider wrote: On Aug 7, 2015 1:18 AM, Cameron Simpson wrote: However, when _editing_ I tell vi that when I press TAB it is to insert enough SPACE characters to get to the next 4 column position. How do you do that? I've got vim set up so a CTRL-T while in insert mode does that (and CTRL-D does the opposite) but don't know how to make use of the tab key. ^T and ^D have done that since time immemorial in vi, no special setup required. I pretty much never use ^T but I do use ^D to back out an indent level. These are my settings relevant to tabs and indenting: set autoindent set expandtab set shiftwidth=2 set tabstop=4 In order: autoindent: start the next line's text on the same indent as this one expandtab: write spaces instead of a TAB character shiftwidth: how far the < and > shift-text operations move tabstop: the multiple used to tabstops - every 4 columns for me Happy to post my other settings should anyone care. Cheers, Cameron Simpson Swiftkey, [...] teaches one the lamentable lesson that most English speakers start most sentences, phrases and sub-clauses with 'I'. One does not use the word often oneself and it therefore strikes one as a little unfair that one's texts and emails so often end up littered with implications of egocentricity. - Stephen Fry http://www.stephenfry.com/2012/04/03/four-and-half-years-on/9/ ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] [off-topic] vi/vim editor settings (was: Pep 8, about indentation)
On 07Aug2015 21:50, Alex Kleider wrote: On 2015-08-07 20:56, Cameron Simpson wrote: autoindent: start the next line's text on the same indent as this one expandtab: write spaces instead of a TAB character shiftwidth: how far the < and > shift-text operations move tabstop: the multiple used to tabstops - every 4 columns for me Thanks! Here is what I've got in my ~/.vimrc file: set autoindent set shiftwidth=4 set expandtab set textwidth=72 set scrolljump=2 set scrolloff=2 I'll add set tabstop=4 Happy to post my other settings should anyone care. ... and yes, I for one certainly do "care." I'd very much like to see your other settings. Ok. Here are my other vi settings which I think useful (it seems I have plenty which aren't). Vim specific settings follow after that. set autowrite Automatically save the file when switching to another. Saves tedious :w and also worrying about whether I should, not to mention all those "file modified" warnings. set ignorecase Saves fiddling with shift keys. set nowrapscan Searches do not resume from the top of the file if nothing found below the current position. I used to have this on, but now consider this more informative. set optimize set redraw I forget, probably dates from editing over a 300baud modem. set report=1 Threshold for reporting changed lines. set showmatch Highlight matching open/close bracket. Handy for nested stuff. And C code. set noterse Longer warnings and other messages. set writeany Trust me, I'm a doctor. (I'm not.) map! \c c...@zip.com.au map! \C Cameron Simpson <\c> Insert mode macros to recite my email, short and long. I've got matching ones for my partner's email, very often CCed or attributed. map! :stop a Suspend the editor, just as with ^Z in the shell. map! a Redraw the screen then resume inserting. map! I map! i map! A map! lli Emacs mode motion: start of line, left, end of line, right, resume insert mode. map # :n # z. Edit the previous file. Very very handy. map g Gz. Go to line and position it in the centre of the screen. map q ZZ Save and quit. map t :ta Commence "go to tag". map Y y$ Yank to end of line. map 1G}-:.r!exec /dev/tty; readline -B 'Attach: ' IAttach: My uber-convenient "attach file" macro. In mutt (my mailer) an "Attach:" header gets turned into a real attachment. So this macro runs a bit of shell to issue the prompt "Attach: " and accept the filename you typ, _with filename completion!_. And inserts that as an Attach: header. When I leave the editor, mutt adds the file to the attachments for this message. set bg=dark My terminals have a dark background. syntax reset syntax on I like my syntax highlighting. set wildmenu wildmode=list:longest Make vim's filename completion act more like zsh's, which I find far more ergonomic than bash's. Essentially: complete as much as you can and _immediately_ show me the remainder without an extra TAB keystroke. set ignorecase set smartcase Case insensitive searches unless there is an uppercase letter in the search string. set incsearch Show the first match in the file as you type the search regexp. set backupcopy=yes I prefer to overwrite the original file instead of making a new one and cutting it in with rename(). The former edits the file I intended to edit. The latter breaks hardlinks and has to do silly hoop jumping with symlinks. Gah! Do what I say, not what you think I want! set noswapfile set swapsync= Vim's fetish for .swp files drives me mad, especially if one loses remote sessions regularly or terminal emulators die or one coworks with others in the same directories or simply closes one's laptop one a whim or moves around. I save files very regularly anyway, so I pretty much never lose a significant edit. set nofsync Trust the filesystem, the filesystem is your friend. Once I've handed the file to the OS (i.e. written it) I do not want to shout at the OS: not get that onto the disc, now! Especially when fsync() means sync(0 on many filesystems i.e. write _all_ pending disc I/O before returning, often with a huge and visible hit to the system behaviour. set scrolloff=2 Like you, keep some screen context. set t_Co=256 Yes, my terminal have lots of colours. set ttyfast Better display instead of lower bandwidth display. set spellfile=~/rc/vim/spellfile.utf-8.add Additional words to augument the spelling table. set spelllang=en_gb British English. set nospell Spell checking off by default. let g:loaded_netrw = 1 I forget. Anyone? map :set hlsearch! nospell! F3 turns off highlighting. and when co
Re: [Tutor] Help
On 09Aug2015 19:10, Shannon Callahan wrote:
Sorry for my lack of clear explanation. Yes, I am having problem to install
python into my mac laptop.
-I am trying to install python into my mac laptop
Macs normally ship with Python installed already, version 2.7.
If you need a specific version, for example Python 3, you can fetch installers
for the current versions here:
3.4.3: https://www.python.org/downloads/release/python-343/
2.7.10: https://www.python.org/downloads/release/python-2710/
-I am trying to learn how to execute the possible finished code
Generally you execute python code from the command line by handing it the
filename or module name, eg:
python filename.py
python -m module.name
-I am trying to create a code so I can transfer it into SD card so I can
put it into raspberry pi.
You will need a text editor. There are any number of things.
Regarding the transfer, your raspberry should have an ethernet socket. You'll
need to plug it into your network. Then you can scp the files to the raspberry
without mucking around with SD cards.
-I am trying to have it able to hear like three specific commands such as
"hey, hello, sir".
-I am trying to execute the code that able to make a connect between
raspberry pi to breadboard to ringer component.
I cannot help you with these. What degree of programming knowledge or
experience do you have?
To answer your questions;
- what you are trying to do; mimic sound detector from specific trigger
words
- what you have done so far; Struggle to download python and create code
Download links above. You will have to pick an editor. The Mac offers the
textedit app and the usual command line based editors. Make sure you're using a
text editor, which makes plain text files. Not a word processor.
- what result you hope to get; able to make code to execute detect trigger
words that cause a ring.
- the actual result you have got.; none.
Start by making sure you have a working Python. Write an utterly trivial Python
program and run it. For example, make a file "foo.py" containing:
print("Hello world!")
and run it:
python foo.py
Cheers,
Cameron Simpson
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Help
On 10Aug2015 18:07, ALAN GAULD wrote: PS. What is SDSU? Google told me San Diego State University. Cheers, Cameron Simpson To be positive: To be mistaken at the top of one's voice. Ambrose Bierce (1842-1914), U.S. author. The Devil's Dictionary (1881-1906). ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Help
On 10Aug2015 11:40, Alex Kleider wrote: On 2015-08-10 11:13, Emile van Sebille wrote: On 8/10/2015 10:07 AM, Alan Gauld wrote: PS. What is SDSU? San Diego State University I'd guess. Emile South Dakota State is the other possibility. They seem to addreviate to sdstate. At least as a domain name, while San Diego State University actually uses sdsu. Maybe the race _is_ always to the swift. Cheers, Cameron Simpson Airplanes are interesting toys but of no military value. --Marechal Ferdinand Foch, Professor of Strategy, Ecole Superieure de Guerre. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] a few question about my evolving program
11-05-28 17.13.38.jpg', '2011-05-28 17.26.37.jpg', '2012-02-02
20.16.46.jpg', '218.JPG', 'honda accident 001.jpg', 'honda accident
002.jpg', 'honda accident 003.jpg', 'honda accident 004.jpg', 'honda
accident 005.jpg', 'honda accident 006.jpg', 'honda accident 007.jpg',
'Image (1).jpg', 'Image.jpg', 'IMG.jpg', 'IMG3.jpg', 'IMG00040.jpg',
'IMG00058.jpg', 'IMG_0003.jpg', 'IMG_0004.jpg', 'IMG_0005.jpg',
'IMG_0007.jpg', 'IMG_0008.jpg', 'IMG_0009.jpg', 'IMG_0010.jpg', 'Mak diploma
handshake.jpg', 'New Picture.bmp', 'temp 121.jpg', 'temp 122.jpg', 'temp
220.jpg', 'temp 320.jpg', 'temp 321.jpg', 'temp 322.jpg', 'temp 323.jpg',
'temp 324.jpg', 'temp 325.jpg', 'temp 326.jpg', 'temp 327.jpg', 'temp
328.jpg', 'temp 329.jpg', 'temp 330.jpg', 'temp 331.jpg', 'temp 332.jpg',
'temp 333.jpg', 'temp 334.jpg', 'temp 335.jpg', 'temp 336.jpg', 'temp
337.jpg', 'temp 338.jpg', 'temp 339.jpg', 'temp 340.jpg', 'temp 341.jpg',
'temp 342.jpg', 'temp 343.jpg']
if current_filename in target_directory_file_list:
print("IN! (unexpected!)")
else:
print("NOT IN")
If the small program works correctly, that may point you to the issue in your
larger program.
Cheers,
Cameron Simpson
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] a few question about my evolving program
n of what correct results should be. For that, we always want the original code of that smallest example rather than an anecdote. We want to be sure that what we're inspecting is what you're inspecting. The other thing we often want is the original problem (which you've described); often people come with their specific solution to a larger problem. Which is fine as long far as it goes, but sometimes the larger problem has a better, less cumbersome solution. Cheers, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Python help
On 13Aug2015 10:34, ALAN GAULD wrote: On 13/08/15 02:01, IDN3 wrote: [...snip...] Condition: If c is less than 5, then the loop will continue; else, it will end. 3. *Problem 2:*Print a string variable that states the number of loops required to meet the condition for Problem 1. This is a simple bit of math. You don't need to run the loop to figure out the answer, just write an equation. Maybe so, but for an arbitrarily weird condition running the loop may be the way to go. Therefore I would imagine he is being asked to print how many times that loop ran. So he should just print that value after the loop finishes (i.e outside the loop). My attempt below. I used a while loop even though the question is saying IF/THEN/ELSE. You obviously did not send the whole question. I don't see any mention of if/then/else? Actually, the question's "Condition:" section is written in exactly those terms. To my knowledge loops in Python have to be while/for. Also, it seems like the question is missing some direction, but I could be wrong. The way I would read your "Cndition:" requirement is that it is describing what kind of decision must be made every time the loop commences. It is not telling you to use Pythons "if" statement. So just putting the correct condition at the top of the "while" loop is what you want. Cheers, Cameron Simpson If you lie to the compiler, it will get its revenge.- Henry Spencer ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Searching through files for values
On 13Aug2015 16:48, Jason Brown wrote: I'm trying to search for list values in a set of files. The goal is to generate a list of lists that can later be sorted. I can only get a match on the first value in the list: contents of value_file: value1 value2 value3 ... The desired output is: file1 value1 file1 value2 file2 value3 file3 value1 ... Bit it's only matching on the first item in vals, so the result is: file1 value1 file3 value1 The subsequent values are not searched. Rhat is because the subsequent values are never loaded: filenames = [list populated with filenames in a dir tree] vals = [] value_file = open(vars) for i in value_file: vals.append(i.strip()) value_file.close() You close value_file inside the loop i.e. immediately after the first value. Because the file is closed, the loop iteration stops. You need to close it outside the loop (after all the values have been loaded): value_file = open(vars) for i in value_file: vals.append(i.strip()) value_file.close() It is worth noting that a better way to write this is: with open(vars) as value_file: for i in value_file: vals.append(i.strip()) Notice that there is no .close(). The "with" construct is the pynthon syntax to use a context manager, and "open(vars)" returns an open file, which is also a context manager. A context manager has enter and exit actions which fire unconditionally at the start and end of the "with", even if the with is exited with an exception or a control like "return" or "break". The benefit of this is after the "with", the file will _always" get closed. It is also shorter and easier to read. for file_list in filenames: with open(file_list) as files: for items in vals: for line in files: if items in line: print file_list, line I would remark that "file_list" is not a great variable name. Many people would read it as implying that its value is a list. Personally I would have just called it "filename", the singular of your "filenames". Cheers, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] SQLite, Python and SQL injection attacks
On 14Aug2015 13:40, boB Stepp wrote: I was just looking at the sqlite3 docs at https://docs.python.org/3/library/sqlite3.html?highlight=sqlite#module-sqlite3 and found the following cheery news: "Usually your SQL operations will need to use values from Python variables. You shouldn’t assemble your query using Python’s string operations because doing so is insecure; it makes your program vulnerable to an SQL injection attack ..." There followed this recommendation: "Instead, use the DB-API’s parameter substitution. Put ? as a placeholder wherever you want to use a value, and then provide a tuple of values as the second argument to the cursor’s execute() method..." I have to be honest -- I would have fallen into this potential trap if I had not read this. It is not clear to me yet how the recommendation avoids this issue. Does the placeholder enforce some sort of type checking so that arbitrary SQL strings will be rejected? Well, better to say that it transcribes the values correctly, possibly with some type checking. You run the same risk constructing shell command lines too, which is why "shell=True" is generally discourages with subprocess.Popen. So if you have: SELECT FROM tablename WHERE columnvalue = ? and you have it a python string like "foo;bah", the SQL API will take care of quoting the string so that the ";" is inside the quotes. Likewise if the string contains SQL end of string markers (quotes). And if the value cannot be transcribed the API should raise an exception. IN this way you know that the structure of the query has been preserved correctly. _And_ you do not need to worry about quoting values (or otherwise transcribing them) correctly; that is a solved and debugged problem. You code is simpler and robust. Cheers, Cameron Simpson The Fano Factor, where theory meets reality. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] try and file existence
On 14Aug2015 18:28, Clayton Kirkwood wrote:
try:
fp = open( user_preferences )
except( PermissionError ):
else:
with open(user_preferences ) as f:
I originally only had the bottom open statement. Ran but file didn't exist,
and my run failed with file doesn't exist. I figured I'd check to see if the
file existed. This is one of those situations where a search of
documentation for fd_exist (which I thought I'd seen once), or exist turns
up either nothing or nothing relevant. I finally found that the try: clause
with the open statement might help and I copied the snippet to my code. I am
getting an indentation error: expected an indent block. What is wrong, and
what is the best way to find out if a file exists?
In purely syntactic terms you need some code in the suite under the "except"
clause, and you don't want the brackets:
try:
fp = open( user_preferences )
except PermissionError as e:
print("open(%r) fails: %s" % (user_preferences, e))
else:
with open(user_preferences ) as f:
In logical terms, the "with" is not wanted - you're opening the file again.
Leaving aside the logical issue there, this structure (test then operate) is
also racy: suppose the file has its attributes changed or is removed between
the first open and the second.
Next: you're catching PermissionError. That normally means that you have not
got rights for opening the file; you will get a different exception if the file
does not exist. You're being too precise if you want both.
But maybe you don't. You need to decide
Finally, the usual Python pattern is not to catch exceptions _at all_ unless
you have a deliberate polciy plan for what to do. Normally you would have some
function looking like this:
def read_prefs(filename):
prefs = {}
with open(filename) as fp:
... read preferences, save in prefs dict for example ...
return prefs
If the file is missing or you have no permissions, that will raise an
exception. Let it!
Then in an outer layer of your program you might catch the exception, where you
can make a policy decision because you have a wider view of what is going on:
try:
prefs = read_prefs(prefs_filename)
except FileNotFoundError as e:
print("warning: file not found: %r: %s" % (prefs_filename, e))
# proceed with empty preferences, not an error
prefs = {}
This bit of code catches _only_ FileNotFoundError on the (presumed) policy that
a missing preferences file is ok - your program will proceed with default
behaviours - but any _other_ kind of exception is not expected - let your
program abort! Do not proceed!
Cheers,
Cameron Simpson
Capitalism is the extraordinary belief that the nastiest of men, for the
nastiest of reasons, will somehow work for the benefit of us all.
- John Maynard Keynes
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] try and file existence
On 15Aug2015 15:20, Clayton Kirkwood wrote: Behalf Of Laura Creighton [..] To: boB Stepp In a message of Sat, 15 Aug 2015 14:24:21 -0500, boB Stepp writes: >I understand your points, but wonder then what is the intended use for >os.path.exists()? That is, in what types of circumstances would it be >both appropriate and safe to use? If you want to locate dangling symlinks, os.path.exists will return False, so the symlink is there, but the file it pointed to is long gone. Can't you do that with os.path.open() and get a value in os.path.status? (I think that is the thing to call) Open does more that os.stat (which is what os.path.exists uses underneath). There are plenty of times you will want to know a file exists but not have permission to open it. Also, open can have side effects if the target file is a device or a socket/pipe. Always use the smallest thing you can to achieve an effect: stat is smaller than open. Cheers, Cameron Simpson Q: How many user support people does it take to change a light bulb? A: We have an exact copy of the light bulb here and it seems to be working fine. Can you tell me what kind of system you have? ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] variable naming conventions
On 15Aug2015 18:24, D Wyatt wrote: It seems every book I read these days uses camel case for variable names in Python. I was once told that using underscores is preferred. Is there a preference in the Python community or does it really matter? I'd like to instill good habits while I'm learning. PEP 8 is lowercase with underscores for normal variables. Class names tend to be CamelCase. I try to follow this. One advantage in staying with this is that you share this convention with the stdlib and with a lot of other Python code, which make it easier for you to read because it aligns with your own habits. Ideally, anyway. If you're not invested in another style, and not working in someone else's codebase with its own conventions, try PEP 8. Cheers, Cameron Simpson Oh, what tangled webs we weave, when first we practice to deceive. And when we've practiced for awhile, How we do improve our style!- Dorothy Parker ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Using lambda
On 24Aug2015 12:08, rakesh sharma wrote: I am beginner in pythonI see the use of lambda has been for really simple ones as in the numerous examples over the net.Why cant we use lambda in another one like g = lambda x: (lambda y: y + 1) + 1 when I am able to do that in two lines >>> h = lambda x: x + 1 >>> h(12) 13 >>> y = lambda x: h(x) + 1 >>> y(1) 3 Hi, First, please include more whitespace in your posts to make them easier to read. If you are includining line breaks etc, I think something is eating them. Regarding your question, you can do what you ask. You suggestion was: g = lambda x: (lambda y: y + 1) + 1 The thing you've missed is that a lambda is a function; you need to call it. Your example only defines a lambda but doesn't call it. Try this: g = lambda x: (lambda y: y + 1)(x) + 1 which passes x through to the inner lambda: >>> g(12) 14 So yes, you can do it. But as you can see it doesn't make for very readable code. Lambdas are ok for small expressions. When things get more complex it is worth breaking them up. Cheers, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Query regarding loop problem
On 29Aug2015 22:37, Martin Mwaka wrote:
I would be grateful for some help please. I have recently started learning
python and I am attemping to write a programme where a user enters a number
between 1 and 5, and the computer generates random numbers until an
equivalent number is achieved. The code (see below) runs, but the else
part of the loop does not run when the computer generates an equivalent
number. I have tried to resolve this a number of ways and have run the code
under a debugger, but cannot work out why the else part is not running. I
would be grateful for your help / guidance with this.
[...]
import random
computerNumber = 0
myNumber = input("Input a number between 1 and 5: ")
print ("Your chosen number is: ", myNumber)
computerNumber = input("Press enter to prompt the computer to enter a
number: ")
while computerNumber != 0:
if myNumber != computerNumber:
computerNumber = random.randint(1,5)
print ("Your chosen number is ", myNumber,": Computer number is: ",
computerNumber)
print ("Numbers do not match.")
prompt = input("Press enter to prompt the computer to enter a
number: ")
else:
print ("MyNumber is ", str(myNumber),": Computer number is: ",
str(computerNumber))
print ("We have a match.")
Looks like you assign the new number to "prompt" instead of to "myNumber".
Cheers,
Cameron Simpson
In theory, there is no difference between theory and practice.
In practice, there is. - Yogi Berra
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] How to parse a mailing list thread?
On 19Sep2015 21:46, chandan kumar wrote: I am looking for a python module which i can use to parse mailing thread and extract some information from it. Any pointer regarding that would be helpful. You should describe where the email messages are stored. I'll presume you have obtained the messages. Construct a Message object from each message text. See the email.message module: https://docs.python.org/3/library/email.message.html#module-email.message Every message has a Message-ID: header which uniquely identifies it. Replies to that message have that id in the In_Reply-To: header. (If you're parsing usenet newsgroup messages, you want the References: header - personally I consult both.) The complete specification of an email message is here: http://tools.ietf.org/html/rfc2822 and the email.message module (and the other email.* modules) makes most of it easily available. If you need to parse email addresses import the "getaddresses" function from the "email.utils" module. Constuct a graph connecting messages with the replies. You're done! Cheers, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Exception Handling
On 03Oct2015 00:51, ALAN GAULD wrote:
On 02/10/15 23:57, Nym City via Tutor wrote:
socket.gaierror: [Errno 11004] getaddrinfo failed
...
for name in ListOfHostNames:
try:
ResolveHostname = socket.gethostbyname(name)
print(ResolveHostname)
newFile.write(ResolveHostname + "\n")
print(ResolveHostname)
except socket.herror as e:
newFile.write("No resolution available for %s" % (name) + "\n")
You are catching herror but your code is resulting in gaierror.
Add socket.gaierror to your except line.
except (socket.herror, socket.gaierror):
newFile.write("No resolution available for %s" % (name) + "\n")
see if that works
Just a followon remark: always try to print the exception when you're not
certain of what it will be or what to do with it. So I'd augument Alan's code
like this:
except (socket.herror, socket.gaierror) as e:
newFile.write("No resolution available for %s: %s" % (name, e) + "\n")
Cheers,
Cameron Simpson
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Mutable data type in python
On 02Oct2015 22:22, Anshu Kumar wrote: When we invoke the same function inside a function (recursive function), i want to keep track of some data across all function invocation so i should have shareable type but built in data types string, int are not helping as they are immutable they are changed while next invocation. I could use lists which are mutable but sometimes i find it not suitable for example when i need only single number or string. Please provide some more context, perhaps with an example of what you're trying to do. What follows is general advice which may not be what you need. Anyway... What you use depends very much on what you need (trite, but true). Numbers and strings are not themselfs mutable. I'm presuming you want some kind of shared state that continues over the recursion, for example a record of what nodes have already been visiting while traversing a graph with loops: if you follow a loop you want to stop instead of circling/recursing forever. When I do this I usually form the function somewhat like the below. Let's suppose we're walking a graph whose nodes have a ".children" attribute which is a list of child nodes. And that the graph might have loops (this node might be a child or subchild of itself). def traverse(node, seen=None): ''' Traverse the graph from `node`, printing each node's name. `seen`: if not None, a set of nodes already visited. ''' if seen is None: seen = set() seen.add(node) print(node.name) for child in node.children: if child not in seen: traverse(child, seen) So the paramater "seen" is your mutable object keeping track of which nodes have been visited. You add this node to "seen", then visit any children which are not in "seen". You pass "seen" to the traverse() of each child, so all the function calls share the same "seen" object. So in this case we're using a set. There's any number of different things you might use depending on what you need to keep track of. Cheers, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] how to unittest cli input
On 10Oct2015 17:41, Alex Kleider wrote:
I'm trying to follow a test driven development paradigm (using
unittest) but can't figure out how to test functions that collect
info from the command line such as the following.
Aside: I'd say "the standard input" , not "the command line"; to me the latter
connotes to command line arguments fro sys.argv.
Anyway, ...
I would supply test data either as a string in the unit tests or as a file kept
with the source tree eg:
os.path.join(os.path.dirname(__file__), 'test_data.txt')
and then parameterise the input source in you functions. For example:
def collect_data(src=None):
if src is None:
src = sys.stdin
and supply src.
However, you'r eusing input(), which unconditionally uses stdin and stdout. In
that circumstance I'd consider this:
def collect_data(src=None, out=None):
if src is None:
src = sys.stdin
if out is None:
out = sys.stdout
ostdin = sys.stdin
sys.stdin = src
ostdout = sys.stdout
sys.stdout = out
ret = {}
ret['first'] = input("Enter your first name: ")
... etc ...
sys.stdout = ostdout
sys.stdin = ostdin
Note that this is not thread safe because sys.stdin is a global, but it should
work for testing.
Anyway, perhap that gives you some way forward.
Cheers,
Cameron Simpson
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] how to unittest cli input
On 11Oct2015 09:29, Alex Kleider wrote:
On 2015-10-10 18:10, Cameron Simpson wrote:
However, you'r eusing input(), which unconditionally uses stdin and
stdout. In that circumstance I'd consider this:
[... temporarily replace stdin and stdout with test data ...]
Yes indeed, and thank you for your input.
Here's where I'm going with your suggestion:
[...]
test_data = 'test_src.txt'
def data_collection_wrapper(collect, source=None):
"""
"""
if source:
Minor remark: I would write "if src is not None:". In principle the empty
string is also "falsey" like None, making your plain "if src:" slightly
unreliable. Be precise!
ostdin = sys.stdin
ostdout = sys.stdout
src = open(source, 'r')
sys.stdin = src
out = open('/dev/null', 'w') # Dump the prompts.
sys.stdout = out
ret = collect()
if source:
src.close()
out.close()
sys.stdin = ostdin
sys.stdout = ostdout
return ret
def collect_data():
ret = {}
ret['first'] = input("Enter your first name: ")
ret['last'] = input("Enter your last name: ")
ret['phone'] = input("Your mobile phone #: ")
return ret
That looks like what I had in mind.
If you expect to do this with several functions you could write a context
manager to push new values for stdin and stdout, call a function and restore.
The "contextlib" stdlib module provides a convenient way to write trivial
context managers using the "@contextmanager" decorator, which wraps a generator
function which does the before/after steps. Have a read. I'd be inclined to
write something like this (untested):
import sys
from contextlib import contextmanager
@contextmanager
def temp_stdinout(src, dst):
ostdin = sys.stdin
ostdout = sys.stdout
sys.stdin = src
sys.stdout = dst
yield None
sys.stdin = ostdin
sys.stdout = ostdout
and then in your test code:
with open(source) as src:
with open('/dev/null', 'w') as dst:
with temp_stdinout(src, dst):
ret = collect()
This has several benefits. Primarily, a context manager's "after" code _always_
runs, even if an exception is raise in the inner section. This means that the
files are always closed, and the old stdin and stdout always restored. This is
very useful.
You'll notice also that an open file is a context manager which can be used
with the "with" statement: it always closes the file.
You also asked (off list) what I meant by parameterisation. I mean that some of
your difficult stems from "collect_data" unconditionally using stdin and
stdout< and that you can make it more flexible by supplying the input and
output as paramaters to the function. Example (not finished):
def collect_data(src, dst):
ret = {}
ret['first'] = input("Enter your first name: ")
ret['last'] = input("Enter your last name: ")
ret['phone'] = input("Your mobile phone #: ")
return ret
Now, the input() builtin always uses stdin and stdout, but it is not hard to
write your own:
def prompt_for(prompt, src, dst):
dst.write(prompt)
dst.flush()
return src.readline()
and use it in collect_data:
def collect_data(src, dst):
ret = {}
ret['first'] = prompt_for("Enter your first name: ", src, dst)
ret['last'] = prompt_for("Enter your last name: ", src, dst)
ret['phone'] = prompt_for("Your mobile phone #: ", src, dst)
return ret
You can also make src and dst optional, falling back to stdin and stdout:
def collect_data(src=None, dst=None):
if src is None:
src = sys.stdin
if dst is None:
dst = sys.stdout
ret = {}
ret['first'] = prompt_for("Enter your first name: ", src, dst)
ret['last'] = prompt_for("Enter your last name: ", src, dst)
ret['phone'] = prompt_for("Your mobile phone #: ", src, dst)
return ret
Personally I would resist that in this case because the last thing you really
want in a function is for it to silently latch onto your input/output if you
forget to call it with all its arguments/parameters. Default are better for
things that do not have side effects.
def main():
print(collect_data()) # < check that user input works
# then check that can test can be automated >
print(data_collection_wrapper(collect_data,
src=test_data))
if __name__ == "__main__":
main()
Perhaps data_collection_wrapper could be made into a decorator (about
which I am still pretty naive.)
It'll take more studying on m
Re: [Tutor] Guidance on using custom exceptions please
On 13Oct2015 10:37, Steven D'Aprano wrote: On Mon, Oct 12, 2015 at 02:55:43PM +, David Aldrich wrote: Consider a 'send' method that sends a message to another system via a socket. This method will wait for a response before returning. There are two possible error conditions: [...] So, my question is, what's the pythonic way of doing this? Should I subclass RuntimeError for each possible error condition? E.g.: class MessageTimeoutError(RuntimeError): pass class IllegalResponseError(RuntimeError): pass I don't think you should be subclassing RuntimeError at all. I'm not quite sure what exception you should subclass, but I am confident it shouldn't be RuntimeError. Help on class RuntimeError in module exceptions: class RuntimeError(StandardError) Unspecified run-time error. I tend to use RuntimeError for program logic errors (code paths that shoudn't happen, like not handling an option combination). I would not use it for this. Since you are working with sockets, I think a socket error might be most useful: import socket # which I expect you are already doing class MessageTimeoutError(socket.error): pass class IllegalResponseError(socket.error): pass Or possibly inherit from the same exception class that socket.error inherits from: IOError. I have mixed feeling about this; I feel that socket.error or IOError should reflect actual OS level or IO subsystem errors, not higher level program discrepancies such as invalid packet data. I would be included to subclass StandardError. Antipattern example: I discovered yesterday that PILLOW raises OSError when it doesn't recognise an image file's content. I consider that a bad choice; I'd prefer ValueError probably - there's been no OS level failure like lack of permission to open the file. On that basis, I recommend either just raising a ValueError for an invalid packet or subclassing it. I'm not certain that you actually need MessageTimeoutError since the socket module itself already defines a socket.timeout error that will be raised on a timeout. Just re-use that. That seems reasonable, if he's using the recv timeout facility. Cheers, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Beautiful Soup
On 12Oct2015 21:21, Crusier wrote:
I am using Python 3.4. I am trying to do some web scraping at this moment.
I got stuck because there is an IndexError: list index out of range if I
put stock_code = (18). My desire output is that the script is able to
detect print out the recent price whether it is up, down or unchanged.
Just a remark: you write:
var = (value)
a lot. It's ok, but needless and also slightly prone to turning into a tuple if
there's a stray comma. Anyway...
It would be helpful to have the exact error message. You say "there is an
IndexError: list index out of range", but that can occur in several places in
your code. Normally Python will print a stack trace indicating the specific
place in your code where the exception occurred.
Please always include the complete error message when asking for help.
[...]
print('Now is trading at UP', item.select('.up2')[0].text)
Everywhere you have [0] you may get an IndexError if the select returns an
empty list, because there will be no element 0.
elif item.select('.down2') == item.select('.down2'):
I'm curious: how can this test ever be false?
Cheers,
Cameron Simpson
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Guidance on using custom exceptions please
On 13Oct2015 13:43, David Aldrich wrote: Thanks for all the answers to my question, they were all helpful. I have one more question, which regards style. Suppose my 'send' method is in its own module: TxControl, along with the custom exceptions: TxControl.py: class MessageTimeoutError(Exception): pass class UndefinedMessageTypeError(Exception): pass def send(msg) etc. Then it seems that an importer of that module must include the module name when referencing the exceptions: import TxControl try: send(msg) except (TxControl.MessageTimeoutError, TxControl.UndefinedMessageTypeError) as err: # Exception processing Including 'TxControl' seems a bit tedious, and is even worse if TxControl imports exceptions from yet another module and allows them to pass up the stack. Without ignoring others' arguments for keeping the full names, you can also go: from TxControl import send, MessageTimeoutError, UndefinedMessageTypeError and just use them unqualified like any other name you might import. The core issue, to me, is: are the exception names nice and descriptive (they look ok to me) and is the module you're using them in not using other exceptions of similar name and purpose (i.e. how confusing might the unqualified named be)? Also bear in mind that you can do this: from TxControl import send as tx_send, MessageTimeoutError as TxTimeoutError, UndefinedMessageTypeError as TxUndefinedMessageType which gets you more descriptive names for local use. If course, if these exceptions are widely used in many contexts (many other code pieces) then you might want to stick with the original names for consistency. Cheers, Cameron Simpson Go not to the elves for counsel, for they will say both no and yes. - Frodo, The Fellowship of the Ring ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Create complex dictionary
On 22Oct2015 23:19, jarod...@libero.it wrote:
Hi!!I would like to prepare a dictionary with complex structure:
complex = {name ="value",surname="po",age=poi)
Well, in Python you'd write the above like this:
complex = {'name': "value", 'surname': "po", 'age': poi}
What is the most pythonic way to build a dictionary of dictionary?
Your description is a bit vague, but it sounds like a dictionary or
dictionaries is a reasonable way to do it. Example:
records = {}
complex = {'name': "value", 'surname': "po", 'age': poi}
records['key1'] = complex
complex = {'name': "value2", 'surname': "xy", 'age': poi2}
records['key2'] = complex
Nothing wrong with that if it fits what you actually need to do.
Cheers,
Cameron Simpson
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] noob python question
On 25Oct2015 16:16, bob5487 wrote: Reading *Learn Python the Hard Way* Zed Shaw All good so far but got to Ex 42 dealing with classes... I get a TypeError: getattr(): attribute name must be a string when I run the program... I looked online, python reference manual 3rd edition, etc... I was wondering if anyone has experience with this book because I hit a road block As always with this list, please post the code you're trying. Even if we had the book (I do not, personally), we don't know if your failing code matches the code in the book, or if the code in the book is correct. So please post the code, and the complete error message your computer gives you. Also not that getattr takes a string, per the error message. So to fetch the .foo attribute from some object "o" you would write: attribute = getattr(o, 'foo') Note the quotes. CHeers, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] How do I (idiomatically) determine when I'm looking at the last entry in a list?
On 28Oct2015 14:48, Flynn, Stephen (L & P - IT)
wrote:
Python 3.
I'm iterating through a list and I'd like to know when I'm at
the end of the said list, so I can do something different. For example
list_of_things = ['some', 'special', 'things']
for each_entry in list_of_things:
print(each_entry)
if each_entry == list_of_things[-1]: # do something special to
last entry
...etc
Is this the idiomatic way to detect you're at the last entry in a list
as you iterate through it?
If it really is a list then enumerate is your friend.
list_of_things = ['some', 'special', 'things']
last_index = len(list_of_things) - 1
for index, each_entry in enumerate(list_of_things):
print(each_entry)
if index == last_index:
... special stuff for the last index ...
For context, I'm working my way through a (csv) file which describes
some database tables. I'm building the Oracle DDL to create that table
as I go. When I find myself building the last column, I want to finish
the definition with a ");" rather than the usual "," which occurs at the
end of all other column definitions...
This is a bit different, in that you are probably not using a list: you don't
know how long the sequence is.
I build things like that this way:
fp.write('CREATE TABLE wibble\n(')
sep = '\n '
for item in items:
fp.write(sep)
fp.write(... column definition for item ...)
sep = ',\n '
fp.write('\n);\n')
i.e. instead of printing the separator _after_ each item, print it _before_.
That way you can special case the first occasion and use a comma for each
successive occasion.
Cheers,
Cameron Simpson
Why is it whatever we don't understand is called a 'thing'? - "Bones" McCoy
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Unable to retreive the stock code
On 16Nov2015 15:41, Crusier wrote:
I am currently trying to download the stock code. I am using Python
3.4 and the code is as follows:
[...]
for link in soup.find_all("a"):
stock_code = re.search('/d/d/d/d/d', "1" )
print(stock_code, '', link.text)
[...]
I am trying to retrieve the stock code from here:
1
or from a href.
Well it looks like you have all the needed libraries. You're doing a few things
wrong above. Firstly, to match a digit you need "\d", not "/d". Secondly, your
use of "re.search" searches the literal string "1" instead of, presumably,
the value of link.text. Thirdly, the return from "re.search" is not a stock
code but a "match object"; I would not call it "stock_code" but
"stock_code_match". That will contain a reference to the stock code; since
your regexp matches the stock code then stock_code_match.group(0) will return
the actual matched text, the stock code.
Fix that stuff and see where you are.
Cheers,
Cameron Simpson
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Python 3.5 console logging
On 30Nov2015 16:50, Tyler Smithers wrote:
I am doing a project for my school and i am trying to find out how to make
a event log. But every where i look all it has is just making a text
document and it having in their what they put in their script.
That is normally what an event log it; a text file recording significant
events. This can be done simply with print() calls or more elaborately via the
logging module.
But i cant
find out how to code python into recording everything that happens when i
run the program. And can you please help me soon because i don't have much
time. Thank you!
When you say everything, what do you mean? Normally people don't want that
("everything" might mean every python code line traversed, or every internal
python opcode executed, etc). Normally that is an insane amount of details, and
people normally just want to know significant events (user logged in, did this,
was refused that, files opened/closed or stuff like that).
You'll need to elaborate a bit more on what you're trying to achieve here.
Cheers,
Cameron Simpson
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Python 3.5 console logging
On 01Dec2015 11:31, Laura Creighton wrote: He may be looking for something like the logging facility of Gnu screen. Or the venerable "script" command. But I suspect he wants something else. Cheers, Cameron Simpson The worst tyrannies were the ones where a governance required its own logic on every embedded node. - Vernor Vinge ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Beginner: Socket object and packets
On 05Dec2015 13:21, Marc Eymard wrote:
Hi tutor,
I am trying to locate the first blank line in the first received packet when
pinging an internet server using a socket object.
First up: everything ALan already said.
Next:
Note that the HTTP response need not all be in a single packet, though that is
not your problem.
My assumption is there will be a mandatory blank line right after the
http headers in accordance with the http protocol.
There certainly should be.
Consider the following:
import socket
mysock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
mysock.connect( ('www.py4inf.com/code/', 80) )
mysock.send('GET http://www.py4inf.com/code/' + ' HTTP/1.0\n\n')
data_str = mysock.recv(700)
My question:
Why is the following statement False when there is an actual blank
line in the received packet:
'\n\n' in data
1: You will be getting bytes backup from the server (obviously so in Python 3
and implicitly in Python 2).
2: The HTTP protocol, like most internet text protocols, ends lines with the
bytes '\r\n', not just '\n'.
Therefore you should expect the bytes '\r\n\r\n' in the response data.
However, you should have discovered this already by doing some debugging. Since
you're clearly not getting the response you expected, the very first step on
your part should be to print our the received data, for example by adding:
print(repr(data_str))
after your recv() call. Then you could inspect the received data and probably
have seen the '\r' characters.
Cheers,
Cameron Simpson
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] reading an input stream
On 24Dec2015 13:54, richard kappler wrote: I have to create a script that reads xml data over a tcp socket, parses it and outputs it to console. Not so bad, most of which I already know how to do. I know how to set up the socket, though I am using a file for development and testing, am using lxml and have created an xslt that does what I want with the xml, and it outputs it to console. What I'm not really sure of, each xml 'message' is preceeded by an STX (\x02) and ends with an ETX (\x03). These 'messages' (Danny, are you noting I don't say -lines- anymore? :-) ) need to be parsed and output whole as opposed to partial. My concern is, there will actually be numerous machines sending data to the tcp socket, so it's entirely likely the messages will come in fragmented and the fragments will need to be held until complete so they can be sent on whole to the parser. While this is the job of tcp, my script needs to I think what I need to do would be analogous to (pardon if I'm using the wrong terminology, at this poing in the discussion I am officially out of my depth) sending the input stream to a buffer(s) until the ETX for that message comes in, shoot the buffer contents to the parser while accepting the next STX + message fragment into the buffer, or something analogous. Any guidance here? Since a TCP stream runs from one machine to another (may be the same machine); presumably your actually have multiple TCP streams to manage, and at the same time as otherwise you could just process one until EOF, then the next and so on. Correct? My personal inclination would start a Thread for each stream, and have that thread simple read the stream extracting XML chunks, and then .put each chunk on a Queue used by whatever does stuff with the XML (accept chunk, parse, etc). If you need to know where the chunk came from, .put a tuple with the chunk and some context information. Does that help you move forward? Cheers, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Read from large text file, find string save 1st string of each line where it appeared.
On 29Dec2015 03:12, Steven D'Aprano wrote: On Mon, Dec 28, 2015 at 04:50:05PM +0530, sutanu bhattacharya wrote: suppose 61746245 is my searching string. so o/p will be [...] I don't understand the question. What is "o/p"? "output" Cheers, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] method, type?
On 05Jan2016 20:58, Alex Kleider wrote:
#!/usr/bin/env python3
# OS: Ubuntu 10.4LTS
# My code:
class JournalLineItem(object):
"""
"""
def __init__(self, account, debit_or_credit, amount):
self.account = account
self.debit_or_credit = debit_or_credit
self.amount = float(amount)
def show(self):
return ("ACNT: {} ${:0.2f} {}".
format(self.account, self.amount, self.debit_or_credit))
def get_line_item(text):
return JournalLineItem(*text.split())
def test():
print(
JournalLineItem.get_line_item("2435 Dr 25.33").show())
if __name__ == "__main__":
test()
myquestion = """
What kind of a method/function is get_line_item?
As written, it is an instance menthod that _thinks_ it is a static method. Not
that that was what you intended :-)
From what I've read (and not fully understood)
static methods and class methods must have
@staticmethod and @classmethod on the line above them.
As written, you could put @staticmethod above it - it is an ordinary function
that resides in the class to associate its functionality with the class.
However, what you are probably better off with is @classmethod, for reasons I
will explain below.
Instance methods expect their first parameter to be the instance (self) and
class methods expect their first parameter to be the class. This is arranged
implicitly when you call the method via an instance or class.
get_line_item works as I wanted but it's clearly not the
usual type of method and I don't know how to categorize it.
It is confused. There is nothing in the Python language that requires the
instance to be called "self", that is just convention. So your method thinks
that the instance is "text", and works on that. So you could do this:
jli = JournalLineItem(account, debit_or_credit, amount)
jli.get_line_item()
and the call would go ahead, but fail because a JournalLineItem has no .split
method. And you can't do this:
jli = JournalLineItem(account, debit_or_credit, amount)
jli.get_line_item("2435 Dr 25.33")
because, since it is an instance method, Python implicitly passes in the
instance (jli) as the first parameter, so the call is equivalent to:
JournalLineItem.get_line_item(jil, "2435 Dr 25.33")
but the function only expected a single parameter. Badness ensues.
It's an instance creator- is there a better term?
Generally these are called factories. Normally you would either write this as a
class method:
@classmethod
def get_line_item(cls, text):
return cls(*text.split())
which will get JournalLineItem when you call it as in your example:
item = JournalLineItem.get_line_item("2435 Dr 25.33")
but will get the right subclass if you subclass JournalLineItem:
class SpecialJournalLineItem(JournalLineItem):
...
special_item = SpecialJournalLineItem.get_line_item("2435 Dr 25.33")
Alternatively, if you never expect to subclass this, you could just declare it
as a normal function outside the class entirely:
def get_line_item(cls, text):
return JournalLineItem(*text.split())
Often these functions or methods are called from_blah, so the outside-the-class
function might be called "JournalLineItem_from_text", and the inside the class
@classmethod one might be called "from_text". So you might write:
jli = JournalLineItem.from_text("2435 Dr 25.33")
which reads well and allows you to subclass JournalLineItem later. Therefore I
would recommend the @classmethod approach. And stylisticly, I would put that up
the top, just under __init__.
Hoping this helps rather than confuses,
Cameron Simpson
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] method, type?
On 06Jan2016 18:57, ALAN GAULD wrote: On 06/01/16 14:46, Steven D'Aprano wrote: It would be reasonable to make this a factory function declared in the global module level. But I think making it a classmethod is better. I won't put up much of an argument here. In C++ or Java I'd definitely say use a class method. But it seems to me that I've seen more factory functions in modules than class method factories. But that's not been a detailed analysis just a gut feel for what seems idiomatic in Python. It also seems easier for beginners to grasp than the more esoteric notion of class methods. Most of my factories are global functions, but that is mere historic artifact. As a direct consequence of explaining this to Alex Kleider I'm moving aggressively to using @classmethod for this, naming them ".from_*" unless I have some amazing reason not to. ... ensures that if you subclass the class, you automatically get a valid constructor as well. Wouldn't it return an instance of the superclass rather than the sub class? You'd need to override it wouldn't you? No, when you call: SubClass.from_blah(...) where from_blah is defined in the superclass, the leading "cls" parameter is the subclass because you invoked it through the subclass, just as you get the leading "self" instance parameter when you invoke an instance method through the instance. That's one reason that @classmethod is so nice here. Cheers, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] method, type?
On 07Jan2016 00:06, ALAN GAULD wrote: On 06/01/16 23:00, Cameron Simpson wrote: ... ensures that if you subclass the class, you automatically get a valid constructor as well. Wouldn't it return an instance of the superclass rather than the sub class? You'd need to override it wouldn't you? No, when you call: SubClass.from_blah(...) where from_blah is defined in the superclass, the leading "cls" parameter is the subclass because you invoked it through the subclass, Yes, but my point is that constructors by their nature tend to be very specific. That wasn't what you said; because cls comes in as a parameter you will inherently get the subclass. To your point: I don't think factory methods are necessarily very different in a subclass. Sometimes? Often? I think it is pretty variable. In principle factory functions need not vary much more than any other method. And like any other method, if it is special then it needs an override. For example a constructor that fetches data from a database will embed a SQL query that is table specific and so may be completely wrong for a sub class (unless it somehow shares the superclass's table). Or if the table name (or table definition from some mapping) is a parameter of the subclass. I would try to make it so myself. Similarly reading from a network it will only pull the fields necessary for the superclass from the message. Indeed the trigger message it sends to the server to initiate the transfer may well have the class details included. So although you might wind up with an object whose type is subclass, its details may be totally wrong or even fail to operate (if the table has no corresponding ID for example). Shrug. So in these cases you either need to have per-class parameters or have to override the factory methods as required. Like anything else. I don't think this is an argument against @classmethod factory functions, merely an argument against inattention when subclassing. It would certainly work for a few cases but I suspect most real-world constructors will fail unless overridden. Other class methods are much more likely to work OK. (For example those maintaining a cache of instances or simply keeping a count) I'm just dubious about constructors because they are so specific about where/how they obtain their data. (Unless you get very clever about how you use cls to access meta-data, or maintain some kind of config mapping (possibly as class attributes) to tell the constructor what to do. I agree it won't also work, or work smoothly. But parameters or metadata from the class/subclass may be quite viable - that is the great convenience of the cls parameter to class methods. Once all this stuff starts getting too hard to look after you're probably in either don't-subclass territory, or providing common methods/mechanisms with mixins (i.e. a lookalike class instead of a subclass). Cheers, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] reading an input stream
On 07Jan2016 12:14, richard kappler wrote:
On Thu, Jan 7, 2016 at 12:07 PM, James Chapman wrote:
From an architectural POV I'd have a few listener threads that upon
receipt would spawn (or take from a pool is a better approach) a worker
thread to process the received data.
As would I.
That's the plan, if I'm understanding you correctly. We've brainstormed the
threading, haven't written any of it yet.
The code you've posted should be fine for testing a single connection.
I'd be doing 2 things to what you posted, myself:
- use plain old .read to collect the data and assemble the XML packets
- decouple your XML parsing from the collection and packet parsing
To the first, I suspect that when you have our packets arriving rapidly you are
either dropping data because the data overflows your 8192 recv size or you're
getting multiple logical packets stuffed into a buffer:
recv #1:
\x02xml...\x03\x02partial-xml
recv #2:
tail-of-previous-xml\x03\x02more-xml...
which would definitiely get your XML parser unhappy.
Instead, gather the data progressively and emit XML chunks. You've got a TCP
stream - the TCPServer class will do an accept and handle you an _unbuffered_
binary stream file from which you can just .read(), ignoring any arbitrary
"packet" sizes. For example (totally untested) using a generator:
def xml_extractor(fp):
''' Read a continuous stream of bytes from `fp`, yield bytes to be parsed
elsewhere. An arbitrary size of 8192 bytes is used to request more data; it
doesn't say anything about any underlying network packet size.
'''
# a (buffer, offset) pair of ungathered data
buffer = b''
offset = 0
# locate start of XML chunk
while True:
if offset >= len(buffer):
buffer = fp.read1(8192)
offset = 0
if not buffer:
# EOF: exit generator
return
# examine the next byte
b = buffer[offset]
offset += 1
if b == 0x02:
# opening delimiter
break
warning("discard byte 0x%02x", b)
# gather XML chunk
chunks = []
while True:
endpos = buffer.find(b'\x03', offset)
if endpos < 0:
# no delimiter, collect entire chunk
chunks.append(buffer[offset:])
offset = len(buffer)
else:
# collect up to the delimiter
chunks.append(buffer[offset:endpos])
offset = endpos + 1
break
# keep collecting...
if offset >= len(buffer):
buffer = fp.read1(8192)
offset = 0
if not buffer:
error("EOF: incomplete final XML packet found: %r", b''.join(chunks))
return
# yield the XML bytes
yield b''.join(chunks)
chunks = None # release chunks so memory can be freed promptly
This reads bytes into a buffer and locates the 0x02...0x03 boundaries and
yields the bytes in between. Then your main stream decoder just looks like
this:
for xml_bytes in xml_extractor(fp):
# decode the bytes into a str
xml_s = xml_bytes.decode('utf-8')
... pass xml_s to your XML parser ...
All of this presumes you have a binary file-like object reading from your TCP
stream. And since we're suggesting you spawn a Thread per connection, I'm
suggesting you use the TCPServer class from the socketserver module, using its
ThreadingMixin. That gets you a threading TCP server.
Query: do the cameras connect to you, or do you connect to the cameras? I'm
presuming the former.
So the surround framework would create a TCPServer instance listening on your
ip:port, and have a handler method which is given a "request" parameter by
TCPServer. That object has a .rfile property which is a read-only binary stream
for reading from the socket, and _that_ is what we refer to as `fp` in the code
above.
Setting up the TCPServer is pretty simple. Lifting the essential bits from some
code of my own (again, untested):
from socketserver import TCPServer, ThreadingMixIn, StreamRequestHandler
class MyServer(ThreadingMixIn, TCPServer):
def __init__(self, bind_addr):
TCPServer.__init__(self, bind_addr, MyRequestHandler)
class MyRequestHandler(StreamRequestHandler):
def handle(self):
fp = self.rfile
for xml_bytes in xml_extractor(fp):
# decode the bytes into a str
xml_s = xml_bytes.decode('utf-8')
... pass xml_s to your XML parser ...
# start the server
S = MyServer( ("hostname", ) )
S.serve_forever()
One critical bit in the above is the use of .read1() in the xml_extractor
function: that calls the underlying stream's .read() method at most once, so
that it behaves like a UNIX read() call and may return a "short" read - less
than the maximum supplied. This is what you need to return data as soon as it
is r
Re: [Tutor] reading an input stream
On 08Jan2016 08:52, Cameron Simpson wrote: [...] Instead, gather the data progressively and emit XML chunks. You've got a TCP stream - the TCPServer class will do an accept and handle you an _unbuffered_ binary stream file from which you can just .read(), ignoring any arbitrary "packet" sizes. For example (totally untested) using a generator: [...] Just a few followup remarks: This is all Python 3, where bytes and strings are cleanly separated. You've got a binary stream with binary delimiters, so we're reading binary data and returning the binary XML in between. We separately decode this into a string for handing to your XML parser. Just avoid Python 2 altogether; this can all be done in Python 2 but it is not as clean, and more confusing. The socketserver module is... annoyingly vague about what the .rfile property gets you. It says a "a file-like object". That should be a nice io.BytesIO subclass with a .read1() method, but conceivably it is not. I'm mentioning this because I've noticed that the code I lifted the TCPServer setup from seems to make a BytesIO from whole cloth by doing: fp = os.fdopen(os.dup(request.fileno()),"rb") You'd hope that isn't necessary here, and that request.rfile is a nice BytesIO already. In xml_extractor, the "# locate start of XML chunk" loop could be better by using .find exactly as in the "# gather XML chunk"; I started with .read(1) instead of .read1(8192), which is why it does things byte by byte. Cheers, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] reading an input stream
On 07Jan2016 17:22, richard kappler wrote: On Thu, Jan 7, 2016 at 5:07 PM, Cameron Simpson wrote: Just a few followup remarks: This is all Python 3, where bytes and strings are cleanly separated. You've got a binary stream with binary delimiters, so we're reading binary data and returning the binary XML in between. We separately decode this into a string for handing to your XML parser. Just avoid Python 2 altogether; this can all be done in Python 2 but it is not as clean, and more confusing. Love to, can't. Splunk uses 2.7 so that's what we have to work with. That will not change in the forseeable future. Doing other homework right now, but will more closely review this and the other posts that have come in since I left work later tonight or first thing in the morning. Ok. You should still be ok, but things like bs[0] == 0x02 will need to be bs[0] == '\x02' and so forth, because you get str objects back from reads. The rest of the suggested code should still broadly work. Cheers, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] basic threading question
On 07Jan2016 14:26, richard kappler wrote: See previous posts on 'looping generator' for details about the code and project. The brief version, I am reading and parsing a data stream through a socket, several actually. Each new connection spawns a thread that reads and parses. Should the client close, I want the thread to terminate. Everything I've read says you don't kill threads, but if the client closes, the socket closes, and the thread is just sitting there, hanging. If the socket.recv returns 0 bytes, the docs tell me that means the client closed and therefore the server socket closes as well. If I do something in the thread target function like: data_chunks = connection.recv(8192) if len(data_chunks) == 0: break len(data_chunks)==0 tells me the socket is closed, but does the break kill the thread? How do I prove that if it does? The break doesn't kill the thread, but if the recv loop is all your thread's main function does, then sure: when the function exits, the thread exits. For example: T = Thread(target=gather_func) T.start() ... def gather_func(): while True: data_chunks = connection.recv(8192) if len(data_chunks) == 0: break ... do stuff with data_chunks ... Then you break exits the loop. Then the function returns since there's no more code in the function after the loop. And then the Thread exits because the function it was running has finished. There is no need to kill the Thread here. You can wait for the Thread with T.wait() and probe it with T.is_alive(); see the threading module documentation. BTW, I still recommend you drop use of .recv() and use .read1() to assemble packets. See your main discussion. Cheers, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] method, type?
On 08Jan2016 10:19, Steven D'Aprano wrote: [...] "factory" methods (typically called '.from_*') can be: Maybe I should have said "often" instead of "typically", if I said "typically". I think they read well that way and there are several stdlib functions named this way as a precedent. I'm aiming for the notion "make a Foo from this or that or something else (3 distinct methods/functions, all taking arguments a bit different from the core __init__)". 1. a normal function outside the class, or 2. a class method (would allow subclassing.) "Factory methods" just means a method which you, the creator or author, thinks of as a factory. What's a factory? A function or method which takes a bunch of arguments and creates something useful. It's a pretty vague definition, because it's a pretty vague term. It's mostly vague if you take the "everything is an object in Python" stance. Which is strictly true, but it is often useful to be thinking of a factory function as an alternative to the bare ClassName(args-for-__init__) constructor, where those arguments are not convenient. Such as Alex's JournalLineItem construction from a line of text. [...] "alternative constructor" (what Petter Otten and Steven DAprano call it,) Alternative in the sense of "not the default", that is all. would be best placed immediately after __init__. *shrug* It doesn't matter where you put it inside the class. That is entirely a matter of personal taste. I suggested this as a style thing (of course, inherently a matter of personal taste:-) I like functions with the same purpose to be textually close together. [...] [Alan Gauld] recommends making it a factory function (defined at the module level, outside the class.) That's a matter of personal taste, and one which I happen to disagree with. Look at the design of the built-in classes like dict. We have dict.fromkeys(), not a global fromkeys() function. I'm largely with Steven here rather than Alan, partly because a classmethod subclasses nicely (with all the caveats Alan alluded to - if you do this then your subclasses _may_ need to override the extra constructor just as they may need to override other methods), and partly because it keeps the constructor inside the class definition, which I find conceptually tidier. Steven DAprano calls it a Python3 regular function/ a Python2 broken method and mentions the Descriptor protocol and how 'methods' are initially simply functions that are then converted to methods (bound) as required. In my case it would be an 'unbound' method which works in 3 but not in Python2. Cameron Simpson indicated that putting @staticmethod above my 'method' would be OK (although not preferred.) Present or absent, my method still functions the same way. Only because you're just calling it from the class. As soon as you create an instance and call the method from that, you'll see why it is broken :-) Aye. While we're on what staticmethod and classmethod accomplish, we could stop treating them like magic. Have you (alex) written any decorators? They are functions which accept a function definition and return a wrapper function with tweaked behaviour. So the notation: @foo def func1(blah): defines "func1" and then calls "foo(func1)". "foo" returns a new function definition, and the class binds _that_ definition to its "func1" method. So... The table provided by Peter Otten (very helpful:) - invoked with | @staticmethod | @classmethod| no decorator -- class| args unchanged | class as 1st arg | args unchanged instance | args unchanged | class as 1st arg | inst as 1st arg --- It suggests that use of the @staticmethod serves to protect one should the 'method' be called via an instance rather than the class. Has it any other effect? Yes, to confuse people into thinking they should be using staticmethod when what they really should use is classmethod :-) Now consider what @staticmethod achieves: it causes a normal method to be called as though it were a global function i.e. without the normally implied "self" parameter. So we could write our own: def staticmethod(func): def method(self, *a, **kw): return func(*a, **kw) return method As described above, this effectively installs the "method" function as the class's actual method, and that function's whole purpose is simply to _discard_ the self parameter and call the original function without "self". Once tha
Re: [Tutor] method, type?
On 08Jan2016 00:18, ALAN GAULD wrote: My only point of difference here, I think, is the definition of a constructor. I consider a constructor to be the creator of object instances, which makes the only default Python constructor the __new__() since the __init__() is only an initializer. Me too. I was deliberately avoiding the term "constructor", then let myself get sucked into using it just now because you get a new instance of your target class out of the factory function/method. But they are better thought of as wrappers for the class' real constructor. For this sloppiness, my apologies, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Syntax error
On 07Jan2016 13:15, Sarah Rasco wrote: Alan - I realized I did that right after I sent this email. However, I can't run it in the windows or python prompts. Here is my document: [image: Inline image 1] When I try to run it in the windows prompt I get: [image: Inline image 2] [...] Hi Sarah, The tutor list and the main python-list strips attachments, so we cannot see your images. Please cut/paste the text as text instead of using screenshots. I also tried to run it in my windows command prompt. I put in cd C:\python and it gave me the python prompt. I think you're misreading the Windows command prompt which recites the current folder. This: C:\python> is still the Window command prompt. It is also probably not where you should do your python work unless you personally made this folder specially. Where did you save your "hello.py" file? That is probably where you should cd. Then, when I tried to open the file by typing python hello.py, I was given a syntax error again. Does anyone have any suggestions as to what the problem could be? Please cut/paste the text of your hello.py file and a complete transcript of the syntax error (all the lines). Thanks, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Organizing files
On 11Jan2016 13:51, Rene Werner wrote: right now I am working on a couple of programming-related challenges. The challenges are sorted into problem sets, where each set contains a number of problems. Because a lot of these problems rely on code that is often the same, I have put these parts into a seperate file, util.py, and I simply do from util import * Remark: you are better off importing only specific names: from util import this, that, the_other otherwise you pollute your namespace to lots of junk. While you've only got one module your version works, but it stops being useful pretty soon after you start importing more things. in each solution. I have my solutions organized in different folders, one for each problem set: set1/prob1.py set1/prob2.py set2/prob3.py and so on. So far, I keep symlinking util.py in each set folder, so e.g. set1/util.py is a symlink to ../util.py. The same goes for set2, set3 and so on. This works, but I get the feeling that it is clumsy and could be done better. What is the usual way in python do realize this in a way so that I don't have to symlink util.py for every problem set, but still have it available? The normal way is to have your own module library and modify $PYTHONPATH to consult it. You might have a $HOME/python_modules directory with util.py inside it (or whatever other modules). Put: PYTHONPATH=$HOME/python_modules export PYTHONPATH in your environment. Then Python will know to look in your python_modules directory for modules ahead of other places and you will not need the symlinks (which are fragile anyway). Cheers, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] s.insert(i, x) explanation in docs for Python 3.4 confusing to me
On 15Jan2016 22:20, boB Stepp wrote: At https://docs.python.org/3.4/library/stdtypes.html#sequence-types-list-tuple-range it states: "s.insert(i, x) inserts x into s at the index given by i (same as s[i:i] = [x])" I find this confusing. First, at the interpreter, whenever I type in: things [0, 'Hmm...', 3, 'WhackABunny', 6, '?'] things[-1:-1] [] things[0:0] [] I always get an empty list, which is actually what I was expecting, so I do not see how s[i:i] can ever equal [x]. It isn't an equality test (==), it is an assignent. It is saying "set the zero length sequence at index i to the one element sequence [x]". The second thing I find puzzling is the docs say x is inserted at position i, while in the interpreter: help(list.insert) Help on method_descriptor: insert(...) L.insert(index, object) -- insert object before index The "...insert object before index" makes sense to me, but "...inserts x into s at the index given by i..." does not because: Personally I'd rather it said "insert object at index". For "before" I'd need something longer, like "insert object before the elements from index onward". things.insert(-1, 'What the heck?!?') things [0, 'Hmm...', 3, 'WhackABunny', 6, 'What the heck?!?', '?'] "...at the index..." to me would mean that 'What the heck?!?' should become the last item in the list. Again, the interpreter help gave what I was expecting. To me it means "insert 'x' so that its index is 'i'". Am I just being dense or are the docs in this instance confusing? They may be a bit confusing, though I do think you're misreading the "=" bit at the top. Cheers, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] s.insert(i, x) explanation in docs for Python 3.4 confusing to me
On 15Jan2016 23:05, boB Stepp wrote:
On Fri, Jan 15, 2016 at 10:53 PM, Cameron Simpson wrote:
things.insert(-1, 'What the heck?!?')
things
[0, 'Hmm...', 3, 'WhackABunny', 6, 'What the heck?!?', '?']
"...at the index..." to me would mean that 'What the heck?!?' should
become the last item in the list. Again, the interpreter help gave
what I was expecting.
To me it means "insert 'x' so that its index is 'i'".
But that's my point! In my example x (here 'What the heck?!?') is
*not* at index i (here, -1). Instead it winds up at index -2. But
this fits in perfectly with the interpreter help, since it winds up
*before* index i (-1).
Ah, but -1 isn't the "real" index. It is a convenient value for computing the
real index if you want to figure things out from the end of the list instead of
the start. In your example above, the real index is 5. As you would get from
things.index('?') before the insert. So your insert really means:
things.insert(5, 'What the heck?!?')
Cheers,
Cameron Simpson
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] s.insert(i, x) explanation in docs for Python 3.4 confusing to me
On 16Jan2016 18:43, boB Stepp wrote: This led me to try: mylist[:None] [100, 200, 300, 400, 500] So, in effect, None is acting as a place holder for that final position in slices. Also, I would never have thought to be able to use a logical "or" inside an index in Peter's "[:-i or None]". Yah, like the default value for many missing parameters. When you don't need an expression after the ":" you can of course write: mylist[:] much like writing a function "def f(x, y=None)"; None is a sentinel value - specially recognised as nor in the normal domain for that value. Cheers, Cameron Simpson Q: How does a hacker fix a function which doesn't work for all of the elements in its domain? A: He changes the domain. - Rich Wareham ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] s.insert(i, x) explanation in docs for Python 3.4 confusing to me
On 16Jan2016 22:42, Alex Kleider wrote:
On 2016-01-16 18:02, Cameron Simpson wrote:
much like writing a function "def f(x, y=None)"; None is a sentinel
value - specially recognised as nor in the normal domain for that
value.
Can you please clarify the last bit:
"specially recognised as nor in the normal domain for that value."
s/nor/not/
The domain of a function (or value) is the set of valid values it may take. The
range is the set of values it may produce.
A sentinel value is a special value you may encounter in a data set (or as a
value) which is _not_ part of the normal domain; often it indicates the end of
a sequence (hence the name, like a border guard). In Python the special value
None is often used as a sentinel value. Since all names have _some_ value, if
you need to indicate that this name "isn't set" or "doesn't specify a valid
value", you need a sentinal value, often "None".
So the common idiom for default values in Python function definitions:
def func(a, b, c=None):
if c is None:
c = default-value-for-c
...
indicates that the parameters "a" and "b must be supplied, and "c" is optional.
If not supplied it will have the value "None". This is a sentinel value so that
"func" can distinguish a valid value from a value not supplied.
You can also use them to indicate the end of data in some sense. If you're
looking at a text file as lines of printable characters, the newline character
at the end of the line could be considered a sentinel. Also consider a Queue:
there's no notion of "closed" in the stdlib version, so one might put a None on
the queue to indicate to the consumer that there will be no more items.
It isn't always None. Sometimes you may want to pass None as normal data, or
you have no control over what is passed around. In that case you might need to
make a unique sentinel value entirely for your object. The normal way to do
this is simply to make a new object:
sentinel = object()
This is a minimal Python object which nobody else is using. Its value is
nothing special (or even meaningful), so you merely want to check whether what
you've got is that particular object, using "is". Untested example sketch:
class SomethingLikeAQueue:
def __init__(self,..):
self.things = []
# private value to use as a sentinel, unique per instance
self._sentinel = object()
... whatever else ...
def put(self, value):
# public method to put a new value
if value is self._sentinel:
raise ValueError("you may not put the sentinel value")
self._put(value)
def _put(self, value):
# private method accepting any value including the sentinel
# we will use receipt of the sentinel to process the things and stop
# further acceptance of more things
if value is self._sentinel:
things = self.things
self.things = None # will make the .append blow up
... process things here maybe ...
else:
things.append(value)
def close(self):
# send the sentinel to indicate no more things
self._put(self._sentinel)
Cheers,
Cameron Simpson
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] s.insert(i, x) explanation in docs for Python 3.4 confusing to me
On 17Jan2016 10:49, Alex Kleider wrote:
Can you please clarify the last bit:
"specially recognised as nor in the normal domain for that value."
s/nor/not/
May I trouble you further by specifically asking about 's/nor/not/'- I don't
get what that's about.
Ah. Ed, sed, vi, vim speak. Substitute: replace "nor" with "not". The "nor" is
a typo. I meant "not in the normal domain".
Has it to do with this 'nor': http://www.merriam-webster.com/dictionary/nor?
"Nor" is indeed a useful word, but it wasn't the word I intended.
[...]
It isn't always None. [...]
you might need to make a unique sentinel value entirely for your
object. The normal way to do this is simply to make a new object:
sentinel = object()
This is a minimal Python object which nobody else is using. Its value
is nothing special (or even meaningful), so you merely want to check
whether what you've got is that particular object, using "is".
Untested example sketch:
class SomethingLikeAQueue:
def __init__(self,..):
self.things = []
# private value to use as a sentinel, unique per instance
self._sentinel = object()
... whatever else ...
def put(self, value):
# public method to put a new value
if value is self._sentinel:
raise ValueError("you may not put the sentinel value")
self._put(value)
def _put(self, value):
# private method accepting any value including the sentinel
# we will use receipt of the sentinel to process the things and
stop # further acceptance of more things
if value is self._sentinel:
things = self.things
self.things = None # will make the .append blow up
... process things here maybe ...
else:
things.append(value) ??? self.things.append(value)
def close(self):
# send the sentinel to indicate no more things
self._put(self._sentinel)
[...]
ps Am I correct that towards the end of your code it should have been
self.things.append(value)
Yes. I probably let myself be lazy because of the earlier "things =
self.things" in the "true" branch of the "if", where I did it to keep the
former value of "things" for processing before scrubbing self.things. Of
course, that way lies buggy code.
I do occasionally pull various object attributes into local variables for
performance and readability; when I do that it really should be right at the
top of the function just after any parameter processing, thus:
class Foo:
def method(self, foo, bar):
things = self.things
for item in foo:
things.append(item) # or whatever
There are usually two reasons I would do that ("things = self.things" at the
top): (a) for readability if I'm going to be saying "things" a lot -
"self.things" may be cumbersome/wordy, making the code very verbose or (b) for
performance.
To the latter: saying "self.things" requires Python to look up the things
attribute in "self" every time you use it; if you put it into a local variable
then Python has direct access to it from the function scope - in CPython his is
very efficient, and likely so in other Python implementations.
Don't forget that because both "self.things" and "things" refer to the same
list object (in the example earlier) there's no need to have any final
"self.things = things" because both are acting on the same list.
Cheers,
Cameron Simpson
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] s.insert(i, x) explanation in docs for Python 3.4 confusing to me
On 17Jan2016 14:50, Alex Kleider wrote:
Again, a personal thank you.
No worries.
More often than not, when answering one thing, you teach me about
other things. The 'thing' thing is only the latest. Of course
I knew that using a name bound to a collection would effect the
contents of the collection but it would never have occurred to me
to use it to advantage as you describe.
Note that it shouldn't be overdone. Pointless optimisation is also a source of
bugs and obfuscation.
As I remarked, I'll do this for readability if needed or for performance where
"things" (or whatever) is going to be heavily accessed (eg in a busy loop),
such that getting-it-from-the-instance might significantly affect the
throughput. For many things you may as well not waste your time with it - the
direct expression of the problem is worth it for maintainability.
It is largely if I'm doing something low level (where interpreted or high level
languages like Python lose because their data abstraction or cost-of-operation
may exceed the teensy teensy task being performed.
As an example, I'm rewriting some code right now that scans bytes objects for
boundaries in the data. It is inherently a sequential "get a byte, update some
numbers and consult some tables, get the next byte ...". It is also heavily
used in the storage phase of this application: all new data is scanned this
way.
In C one might be going:
for (char *cp=buffer; buf_len > 0; cp++, buflen--) {
b = *cp;
... do stuff with "b" ...
}
That will be efficiently compiled to machine code and run flat out at your
CPU's native speed. The Python equivalent, which would look a bit like this:
for offset in range(len(buffer)):
b = bs[offset]
... do stuff with "b" ...
or possibly:
for offset, b in enumerate(buffer):
... do stuff with "b" ...
will inherently be slower because in Python "bs", "b" and "offset" may be any
type, so Python must do all this through object methods, and CPython (for
example) compiles to opcodes for a logial machine, which are themselves
interpreted by C code to decide what to do. You can see that the ratio of
"implement the Python operations" to the core "do something with a byte" stuff
can potentially be very large.
At some point I may be writing a C extension for these very busy parts of the
code but for now I am putting some effort into making the pure Python
implementation as efficient as I can while keeping it correct. This is an
occasion when it is worth expending significant effort on minimising
indirection (using "things" instead of "self.things", etc), because that
indirection has a cost that _in this case_ will be large compared to the core
operations.
Cheers,
Cameron Simpson
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] if request.method == 'GET': NameError: global name 'request' is not defined
On 17Jan2016 20:13, sudipto manna wrote: Here is the code snippet: File#FlaskTest2.py from flask import Flask You need to import "request" from flask as well: from flask import Flask, request For others on this list: the Flask framework presents the current web request as a thread local global called "request"; simplifies writing handlers as the request information is readily available without having to pass it through function calls. Cheers, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Simultaneous read and write on file
On 18Jan2016 16:29, ALAN GAULD wrote:
On 18/01/16 16:01, Anshu Kumar wrote:
I try below code in python 2.7.10 to first create and write into a file and
then read and write that file but what i get is just a file with new
content.
with open('test.txt', 'wb+') as f:
... f.write('this is test file.')
... f.write('ok!!!')
...
with open('test.txt', 'wb+') as f:
... a_str = f.read() + 'read the file'
... f.seek(0)
... f.write(a_str)
...
You called f.seek(0) which puts the cursor right back to the start
of the file. Try writing without using the f.seek().
Agreed.
But since you only want to append, it would be better and safer to use
append mode instead.
The + modes are deceptively appealing but they are full of dangers
for precisely the reasons you have discovered(*). You very rarely
need them and you are better opening/closing the file and
using explicit modes to read/write.
But if he wants to mix the modes, he certainly should be experimenting. Having
a file open for read and write is sometimes useful; I do it myself in certain
circumstances.
Tip for new players: if you do any .write()s, remember to do a .flush() before
doing a seek or a read - unlike the C stdio library where the flush is
automatic in Python the io classes require you to flush written data if you
read or seek. (You don't have to flush before close, close does that for you.)
(*)Another problem, apart from the risks of overwriting your
data, is that the + modes will lock the file even while you
are just reading it, which might cause a problem with shared
files)
Only on Windows. On UNIX everything is fine unless you go out of your way to
make things harder with locking.
I have read in documentation that wb+ mode is for writing and reading. Am i
using wrong mode, should i use rb+ ?
No, you should probably be separately using 'r' to read and 'a'
to append.
Disagree. As far as his question goes, "wb+" is a correct mode for what he is
trying. Whether it is a sensible approach depends very much on what he is doing
with his files.
He _may_ be safer opening the file twice - once for "rb" and once for "ab" -
because he does not have to juggle the modes, but it isn't necessarily what he
wants.
Cheers,
Cameron Simpson
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Simultaneous read and write on file
On 18Jan2016 21:07, ALAN GAULD wrote: On 18/01/16 20:43, Cameron Simpson wrote: The + modes are deceptively appealing but they are full of dangers for precisely the reasons you have discovered(*). You very rarely need them and you are better opening/closing the file and using explicit modes to read/write. But if he wants to mix the modes, he certainly should be experimenting. Having a file open for read and write is sometimes useful; I do it myself in certain circumstances. Yes and so have I. Maybe twice in 30 years of programming. It's sometimes necessary but it's much, much harder to get right and very easy to get wrong, usually with data corruption as a result. I may have done it a little more than that; I agree it is very rare. I may be biased because I was debugging exactly this last week. (Which itself is an argument against mixed rerad/write with one file - it was all my own code and I spent over a day chasing this because I was looking in the wrong spot). So for a beginner I would never encourage it. For an experienced programmer sure' if there is no other option (and especially if you have fixed size records where things get easier). Tip for new players: if you do any .write()s, remember to do a .flush() before doing a seek or a read That's exactly my point. There are so many things you have to do extra when working in mixed mode. Too easy to treat things like normal mode files and get it wrong. Experts can do it and make it work, but mostly it's just not needed. Yes. You're write - for simplicity and reliability two distinct open file instances are much easier. Disagree. As far as his question goes, "wb+" is a correct mode for what he is trying. Whether it is a sensible approach depends very much on what he is doing with his files. I'm not sure we know what he(?) is trying. We only know he successfully overwrote his data and that apparently was not his intention. There are use cases where it makes sense but in most cases you can get by just fine without. Yeah. I think I was more narked by your not answering his "is wb+ correct" literally; it may be only rarely the reasonable course, but for what he was actually _asking_ wb+ is correct. He may not be doing the best design but as you say we don't know his actual use case. For the narkiness I apologise. Cheers, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Simultaneous read and write on file
On 18Jan2016 22:29, Peter Otten <__pete...@web.de> wrote: Anshu Kumar wrote: I have read in documentation that wb+ mode is for writing and reading. Am i using wrong mode, should i use rb+ ? Quoting https://docs.python.org/2.7/library/functions.html#open """ note that 'w+' truncates the file. """ That's why you lose the file's current content, and, yes, "r+b" would avoid that. And I stand corrected; I should have paid more attention. Thanks! Cheers, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Simultaneous read and write on file
On 18Jan2016 20:41, Martin A. Brown wrote: Yes and so have I. Maybe twice in 30 years of programming. [...] I may have done it a little more than that; I agree it is very rare. I may be biased because I was debugging exactly this last week. (Which itself is an argument against mixed rerad/write with one file - it was all my own code and I spent over a day chasing this because I was looking in the wrong spot). Oh yes. Ooof. Today's decisions are tomorrow's albatross. Actually I have good reason to mix these in this instance, and now that it is debugged it is reliable and more efficient to boot. [...] Tip for new players: if you do any .write()s, remember to do a .flush() before doing a seek or a read That's exactly my point. There are so many things you have to do extra when working in mixed mode. Too easy to treat things like normal mode files and get it wrong. Experts can do it and make it work, but mostly it's just not needed. Yes. You're write - for simplicity and reliability two distinct open file instances are much easier. Yes, he's write [sic]. He writes a bunch! ;) Alas, I have a tendency to substitute homophones, or near homophones, when typing in a hurry. You'll see this in a bunch of my messages. More annoyingly, some are only visible when I reread a posted message instead of when I was proofreading prior to send. [Homonyms mess me up when I'm typing, all sew.] Homonyms too. Cheers, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Why do I not get an error when I mistakenly type "humdrum.sigh_strenght" instead of the correct "humdrum.sigh_strength"?
On 20Jan2016 22:34, boB Stepp wrote:
My intent was to deliberately introduce an error into my class definition:
class Hmm(object):
def __init__(self, sigh_type, sigh_strength):
self.sigh_type = sigh_type
self.sigh_strength = sigh_strength
def snort(self):
if self.sigh_strength == 'low':
print("snort")
elif self.sigh_strength == 'med':
print("Snort!")
elif self.sigh_strenght == 'high':
print("SNORT!!!")
else:
print("Hmm...")
def sigh():
if self.sigh_type == 'quiet':
print("pss")
elif self.sigh_type == 'annoying':
print("Whoosh!")
elif self.sigh_type == 'loud':
print("HEAVY SIGH!!!")
else:
print("HMM!!!")
I omitted "self" from the sigh() method to see what would happen plus
some other things.
Well... You've bound a function accepting no arguments to the "sigh" attribute
of the class. Legal. Nonsensical perhaps, but legal.
humdrum = Hmm('quiet', 'low')
humdrum.snort()
snort
humdrum.sigh_strength = 'med'
humdrum.snort()
Snort!
humdrum.sigh_strenght = 'high'
humdrum.snort()
Snort!
At this point I wondered why my output was not "SNORT!!!". Then I
noticed my typo. But now I wonder why I did not get an error from
this typo?
Because your "if" statement matched the "med". So it never tried to look up
"self.sigh_strenght".
humdrum.sigh_strength = 'high'
humdrum.snort()
SNORT!!!
Again, as you expected, yes?
humdrum.sigh()
Traceback (most recent call last):
File "", line 1, in
humdrum.sigh()
TypeError: sigh() takes 0 positional arguments but 1 was given
This was my original point in doing all of this, to see what would
result if I omitted "self". I am pretty sure the error is because the
object instance gets automatically passed to the sigh() method, but by
leaving the "self" parameter out in the method definition, I have a
mismatch between what was defined (0 parameters) and what was passed
to the method (1 argument).
Correct.
humdrum.sigh_strenght
'high'
But what about this? It seems like I can use the humdrum arguments
outside of the Hmm class and merrily define new variables. Why is
this? Is this potentially useful behavior?
"humdrum" is just an object. You can assign attibutes to it at any time.
The code executing inside the class is no different to the code outside the
class; Python is a dynamic language and you can do this stuff at any time.
It isn't _specificly_ useful to assign an attribute long after its normal
initialisation, but it can be quite useful. But consider your initialiser:
def __init__(self, sigh_type, sigh_strength):
self.sigh_type = sigh_type
self.sigh_strength = sigh_strength
By the time __init__ is called, the object already exists with a type/class and
everything. All __init__ is doing is what you find unexpected later; defining
new attribute values not there before. The only thing special about __init__ is
that it is called automatically after an object is created. But that is all.
This is not a static language, and __init__ is not defining what
fields/attributes the object possesses. It is merely setting some of them. It
is executable code, not a static type definition.
Cheers,
Cameron Simpson
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Why is the name "self" optional instead of mandatory?
On 20Jan2016 21:42, boB Stepp wrote: I'm whizzing along in "Python Crash Course" and am in the chapter on classes. Just to satisfy my suspicion that "self" is just a placeholder for creating an object instance, No, it is a placeholder for a _preexiting_ object instance. [...] So I really only have one question: Why not make Python's *traditional* name, "self", mandatory? Why give the programmer this kind of choice? [OK, that was two questions.] Why make it mandatory? What benefit would it bring? Remember, one can write nonsense or impossible to read gibberish in any language; most people don't try to. So to repeat my question: why make it mandatory? Cheers, Cameron Simpson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Why do I not get an error when I mistakenly type "humdrum.sigh_strenght" instead of the correct "humdrum.sigh_strength"?
On 22Jan2016 22:14, boB Stepp wrote:
On Thu, Jan 21, 2016 at 4:57 AM, Steven D'Aprano wrote:
Danny is correct. And it is a useful feature too. For instance, we can
add attributes to functions:
def spam(x, y):
...
spam.extra_info = "whatever"
A new thing that I did not suspect I could do. This bothers me for two reasons:
1) It does not seem right adding attributes to functions outside
of its definition.
Have you tried to do it from inside? Anyway, a function is just another object.
What do you think it should be particularly special?
2) spam.extra_info appears to be global:
spam.extra_info = "whatever"
z = 'WOW!'
def print_global_stuff():
print('z =', z)
print('spam.extra_info =', spam.extra_info)
No, "spam" is global. So you can name it anywhere in that module; therefore you
can name anything inside it. Again, like any other object.
print_global_stuff()
z = WOW!
spam.extra_info = whatever
And I imagine I am being dense about something that is quite obvious
to you: How is this a useful feature to have? What does it give me
that is more useful than just saying something like:
just_another_global variable = "whatever"
spam.extra_info = "blah"
snot = spam
print(snot.extra_info)
Consider if you passed "snot" (or "spam") to a function:
def print_extra_info(obj):
print(obj.extra_info)
This is not something you could do with just_another_global_variable.
Consider: this is information you want assicated with a specific object.
Therefore it really _is_ an arribute of the object so that it can follow it
around.
And what bothered me about my original example that started this
thread is that when my typo
humdrum.sigh_strenght = 'high'
was accepted and did not generate an error, it seemed to mean to me
that I was violating my object's data encapsulation. It just seems to
me that I should not be able to arbitrarily add new attributes from
outside the class definition that created my object. That seems
similar to having a class Dog, to which from outside the class'
definition, I decide to add a new Dog attribute that all dogs in this
class can now have a tail sticking out of their noses. I have no
qualms about subclassing Dog with a MutantDog subclass, but adding new
attributes from outside the class definition? That just does not seem
right to at this point of my understanding. As in your function
example, I am sure that I am missing something quite obvious, but I am
just not seeing it now.
All you're missing is realising that setting an attribute is not a special
operation.
To take an anecdote from elsewhere:
UNIX was not designed to stop you from doing stupid things, because that
would also stop you from doing clever things. - Doug Gwyn
There's no need to prevent you setting "sigh_strenght". There probably is
benefit in linting tools reporting to you about attributes which are set but
apparently never used.
Cheers,
Cameron Simpson
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Why do I not get an error when I mistakenly type "humdrum.sigh_strenght" instead of the correct "humdrum.sigh_strength"?
On 23Jan2016 01:52, boB Stepp wrote:
On Fri, Jan 22, 2016 at 11:04 PM, Cameron Simpson wrote:
On 22Jan2016 22:14, boB Stepp wrote:
All you're missing is realising that setting an attribute is not a special
operation.
I guess no matter how new one is to OOP, one nevertheless brings one's
preconceptions, however malformed, into the learning process. In my
case, one of mine was that once a class is coded, any given instance
of a class is forevermore responsible for managing its *internals*, so
that any change in these would be managed by the object when one of
its (meant to be) publicly available methods is called.
That is the pure OO way; and you can do things that way in Python, though you
can't _prevent_ direct access. However, it isn't the common way with Python;
generally with a Python class there are public attributes with ordinary names;
outsiders can consult them but generally should not change them unless the doco
says that is ok. "Internal" attributes which outsiders should not touch are
general givens names starting with an underscore so that outsiders can know.
In the case
of attributes, I was thinking/expecting something like the following
would have to be done:
class Dog(object):
def __init__(self, name, breed):
self.name = name
self.breed = breed
self.number_legs = 4
self.number_tails = 1
self.number_eyes = 2
self.attributes = {
'name': self.name,
'breed': self.breed,
'number of legs': self.number_legs,
'number of tails': self.number_tails,
'number of eyes': self.number_eyes}
def add_attribute(self, new_attribute_name, new_attribute_value):
self.new_attribute_name = new_attribute_name
self.new_attribute_value = new_attribute_value
self.attributes[self.new_attribute_name] = self.new_attribute_value
def show_attributes(self):
print("The object,", self.name, "has the following attributes:")
for attribute_name, attribute_value in self.attributes.items():
print(attribute_name, "=", attribute_value)
our_dog = Dog('Copper', 'beagle')
our_dog.show_attributes()
The object, Copper has the following attributes:
number of tails = 1
number of legs = 4
breed = beagle
name = Copper
number of eyes = 2
our_dog.add_attribute('unique marking', 'blue right eye')
our_dog.show_attributes()
The object, Copper has the following attributes:
number of legs = 4
breed = beagle
unique marking = blue right eye
number of tails = 1
name = Copper
number of eyes = 2
I suspect there are probably better ways to do what I am trying to
demonstrate [I am trying to use a dict to simulate adding new instance
variables after the fact.],
You're aware that most objects have a .__dict__ attribute containing the
attributes? It is a dict, btw. So:
class O(object):
pass
>>> o=O()
>>> o.x=1
>>> o.__dict__
{'x': 1}
perhaps with built-in functions that I
have not read about yet. But I hope my code illustrates my previous
expectations. I guess I was expecting that access to an object's
internals would be more strictly regulated. But, thinking about it, I
already knew that even if name mangling is done for a method's name,
it can still be directly accessed from outside of that object's class
definition.
Yeah. If you mean .__blah methods and attributes, I used to use them but
generally don't these days. They're more an aid to avoiding collisions when
subclassing to my mind, but they're not bulletproff and the implicit mangling
makes for confusion in my mind.
But if I am understanding everyone's comments correctly, Python allows
me write my classes in such a way, that public access to my object's
internals is controlled, in the sense that Python has coding
conventions that tell all of the quite intelligent, respectful,
consenting adults that might use my code how I intend that code to be
used. And that I, despite my intentions, must respect that users of
my code may have very valid reasons for deviating from my intentions.
Have I captured the Pythonic spirit of what everyone has been trying
to tell me?
Pretty much.
Alan will take you up on doing purer OO practices in Python.
Cheers,
Cameron Simpson
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Why do I not get an error when I mistakenly type "humdrum.sigh_strenght" instead of the correct "humdrum.sigh_strength"?
On 23Jan2016 12:55, boB Stepp wrote:
On Sat, Jan 23, 2016 at 3:30 AM, Cameron Simpson wrote:
On 23Jan2016 01:52, boB Stepp wrote:
I guess no matter how new one is to OOP, one nevertheless brings one's
preconceptions, however malformed, into the learning process. In my
case, one of mine was that once a class is coded, any given instance
of a class is forevermore responsible for managing its *internals*, so
that any change in these would be managed by the object when one of
its (meant to be) publicly available methods is called.
That is the pure OO way; and you can do things that way in Python, though
you can't _prevent_ direct access. However, it isn't the common way with
Python; generally with a Python class there are public attributes with
ordinary names; outsiders can consult them but generally should not change
them unless the doco says that is ok...
I'd like to focus on this last sentence. Are you suggesting that it
is "better" programming practice to code the class so that it has its
own publicly available methods to process its public attributes?
No, I'm suggesting that in a pure OO system, your only access to the internal
state of an object is via .get_thing() methods and the only way to set them is
via .set_thing(value) methods. In a system where outsiders cannot access
internal attributes, that provides a completely opaque layer where the object
mediates these actions to ensure correctness and where the internals are
invisible, allowing a complete change of implementation without breaking the
interface outsiders use.
And
that it is both good practice and Pythonic to allow outsiders to
freely read public attributes as needed?
Generally yes. As the author of the class, you need to decide what should be
visible (meaning "not have a leading underscore"). Of course it is _all_
visible, but when you give something a "public" name you are quietly implying
to outsiders that this is stable, and future implementations will continue to
preserve it.
You can be totally conservative of course and give all the internal state ._*
names. But an object with no attributes is generally not as useful. Note that
there's a grey area here: plenty of objects have methods which return values:
class O(object):
def __init__(self, name):
self._name = name
def name(self):
return self._name
o = O("foo")
print(o.name())
so you can keep the state "private" while presenting useful information via
methods. By having .name be a function, you are free to reimplement the class
using anther mechanism and outsiders will not have to change how they use
things:
class O(object):
def __init__(self, name):
# make a row in a database associating our id with this name
db.set_name(id(self), name)
def name(self):
# fetch the name back out of the database
n = db.get_name_by_id(id(self))
return n
I suspect there are probably better ways to do what I am trying to
demonstrate [I am trying to use a dict to simulate adding new instance
variables after the fact.],
You're aware that most objects have a .__dict__ attribute containing the
attributes? It is a dict, btw. So:
class O(object):
pass
>>> o=O()
>>> o.x=1
>>> o.__dict__
{'x': 1}
The book I am currently working through, "Python Crash Course",
despite its title, is oriented towards beginners to programming. It
does not cover dunder methods and attributes, other than __init__().
I am aware from following this list and reading portions of other
books, that they exist, but I have not gotten into the details of any
of them. So rewriting my class to use .__dict__, I have gotten:
class Dog(object):
def __init__(self, name, breed):
self.name = name
self.breed = breed
self.number_legs = 4
self.number_tails = 1
self.number_eyes = 2
def show_attributes(self):
print("The object,", self.name, "has the following attributes:")
for attribute_name, attribute_value in self.__dict__.items():
print(attribute_name, "=", attribute_value)
our_dog = Dog('Copper', 'beagle')
our_dog.show_attributes()
The object, Copper has the following attributes:
number_tails = 1
number_eyes = 2
breed = beagle
number_legs = 4
name = Copper
our_dog.unique_marking = 'blue right eye' # Adding a new attribute from
outside the class definition.
our_dog.show_attributes()
The object, Copper has the following attributes:
number_eyes = 2
breed = beagle
unique_marking = blue right eye
number_tails = 1
number_legs = 4
name = Copper
I have to say that this seems very simple and direct despite my
initial reluctance to assign new attributes to an object from outside
the class definition of that object.
Yes. No magic here, just expos
Re: [Tutor] Why do I not get an error when I mistakenly type "humdrum.sigh_strenght" instead of the correct "humdrum.sigh_strength"?
On 23Jan2016 16:25, boB Stepp wrote:
On Sat, Jan 23, 2016 at 12:55 PM, boB Stepp wrote:
I still would like to do this via a method, but I am currently stuck
on how to replace ??? with the attribute name I would like to insert:
class Dog(object):
...
def add_attribute(self, attribute_name, attribute_value):
self.??? = attribute_value
If I write something like:
unique_marking = 'blue right eye'
our_dog.add_attribute(unique_marking, 'blue right eye)
I cannot get ??? to be unique_marking. Instead, show_attributes()
will give from .__dict__, 'attribute_name', instead of what I would
like to replace it with, unique_marking. But I have not given up yet!
I think I now have this nuked out. I am only just now realizing how
powerful .__dict__ is:
[... working code ...]
Wow! So much power for so little code!
Sure. But also note that in the real world you will hardly ever work on
.__dict__ directly. If I were collecting a lot of _arbitrary_ features like
.unique_marking I would put them all in some internal attribute:
class Dog(object):
def __init__(self, name):
self.name = name
self.features = {}
def add_feature(self, feature_name, feature_value):
self.features[feature_name] = feature_value
(Note, deliberately avoiding the term "attribute" here.)
Also, Python provideds the functions setattr and getattr to _avoid_ directly
accessing .__dict__:
setattr(self, attribute_name, attribute_value)
because not all objects use .__dict__ to store this stuff. Generally the dunder
names are to be avoided: not that they should never be used, but they tend to
be mechanism which has "public" exposure elsewhere, eg __dict__ vs
getattr/setattr. For exactly the same reason as other OO stuff: because the
internal implementation is not always __dict__.
So you might access __dict__ internally for introspection, but you would more
commonly use getattr (and outsiders should _always_ use getattr!) Or, of
course, if the attribute name is known at coding time, with the better .name
syntax:
v = dog.__dict__['unique_feature']
v = getattr(dog, 'unique_feature')
v = dog.unique_feature
from least desired to most desired. BTW, when the attribute exists these all
return the same thing, but when the attribute does no exist they all raise
different kinds of exception.
Cheers,
Cameron Simpson
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] 2016-02-01 Filter STRINGS in Log File and Pass as VARAIBLE within PYTHON script
On 01Feb2016 15:53, knnleow GOOGLE wrote:
trying out on how to port my unix shell script to python.
get more complicated than i expected.: (
i am not familiar with the modules available in python.
anyone care to share how to better the clumsy approach below.
regards,
kuenn
timestamp02 = time.strftime("%Y-%m-%d-%H%M%S")
banIPaddressesFile = os.popen("cat
/var/log/fail2ban.log| egrep ssh| egrep Ban| egrep " + myDate + "| awk
\'{print $7}\'| sort -n| uniq >/tmp/banIPaddressesFile." +
timestamp02).read()
First up, this is still essentially a shell script. You're constructing a shell
pipeline like this (paraphrased):
cat >/var/log/fail2ban.log
| egrep ssh
| egrep Ban
| egrep myDate
| awk '{print $7}'
| sort -n
| uniq
>/tmp/banIPaddressesFile-timestamp
So really, you're doing almost nothing in Python. You're also writing
intermediate results to a temporary filename, then reading from it. Unless you
really need to keep that file around, you won't need that either.
Before I get into the Python side of things, there are a few small (small)
criticisms of your shell script:
- it has a "useless cat"; this is a very common shell inefficiency there people
put "cat filename | filter1 | filter2 ..." when they could more cleanly just
go "filter1
- you are searching for fixed strings; why are you using egrep? Just say "grep"
(or even "fgrep" if you're old school - you're new to this so I presume not)
- you're using "sort -n | uniq", presumably because uniq requires sorted input;
you are better off using "sort -un" here and skipping uniq. I'd also point
out that since these are IP addresses, "sort -n" doesn't really do what you
want here.
So, to the Python:
You seem to want to read the file /var/log/fail2ban.log and for certain
specific lines, record column 7 which I gather from the rest of the code
(below) is an IP address. I gather you just want one copy of each unique IP
address.
So, to read lines of the file the standard idom goes:
with open('/var/log/fail2ban.log') as fail_log:
for line in fail_log:
... process lines here ...
You seem to be checking for two keywords and a date in the interesting lines.
You can do this with a simple test:
if 'ssh' in line and 'Ban' in line and myDate in line:
If you want the seventh column from the line (per your awk command) you can get
it like this:
words = line.split()
word7 = words[6]
because Python arrays count form 0, therefore index 6 is the seventh word.
You want the unique IP addresses, so I suggest storing them all in a set and
not bothering with a sort until some other time. So make an empty set before
you read the file:
ip_addrs = set()
and add each address to it for the lines you select:
ip_addrs.add(word7)
After you have read the whole file you will have the desired addresses in the
ip_addrs set.
Try to put all that together and come back with working code, or come back with
completed but not working code and specific questions.
Cheers,
Cameron Simpson
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
