Re: [Tutor] Help for Python Beginner with extracting and manipulating data from thousands of ASCII files
Got it, many thanks for your help. On Tue, Oct 2, 2012 at 7:59 PM, Oscar Benjamin wrote: > Hi Cecilia, I'm sending this again as the first message was sent only > to you (I hadn't realised that your own message was sent only to me as > well). If you want to reply please reply-all to this message. > > On 1 October 2012 17:42, Cecilia Chavana-Bryant > wrote: > > On Mon, Oct 1, 2012 at 11:02 AM, Oscar Benjamin < > oscar.j.benja...@gmail.com> > > wrote: > >> > >> On Sep 30, 2012 11:10 PM, "Cecilia Chavana-Bryant" > >> wrote: > >> > > >> > fileDate = data[6][16:26] # location of the creation date on the data > >> > files > >> > >> What format does fileDate have? I guess it's a string of text from the > file. If > >> you can convert it to a datetime (or date) object it will be easy to > >> compare with the dates as required for your calibration file. Can you > >> show us how it looks e.g. > >> > >> '12-Nov-2012' > >> or > >> '12/11/12' > >> or something else? > > > > > > Date is in the following format: dd/mm/ > > The standard way to work with dates is to turn the date strings into > Python datetime objects. You can read about those here: > http://docs.python.org/library/datetime.html > > datetime objects can be create directly: > > >>> from datetime import datetime > >>> start_date = datetime(year=2012, month=11, day=3) > >>> print start_date > 2012-11-03 00:00:00 > > You can also create them from a string: > > >>> datestring = '10/11/2012' > >>> experiment_date = datetime.strptime(datestring, '%d/%m/%Y') > >>> print experiment_date > 2012-11-10 00:00:00 > > Once you have two datetime objects you can compare them directly: > > >>> experiment_date > start_date > True > >>> print experiment_date - start_date > 7 days, 0:00:00 > > Using this you can check whether the date from the data file is in > between the start and end dates for each of the calibration files and > then choose which calibration file to use. > > > Oscar > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] a question about maxint
On Wed, Oct 3, 2012 at 1:28 AM, Katya Stolpovskaya wrote: > > Thank you for you reply, but with "long" I got the same error: > from sys import * long > > Traceback (most recent call last): > File "", line 1, in > long > NameError: name 'long' is not defined I assumed some familiarity with Python 2, which has two integer arithmetic types: int (fixed precision, using the platform's C long type) and long (arbitrary precision). The naming is a bit conflated in that a Python int uses a C long, which has nothing to do with a Python long. In Python 2.2+, arithmetic expressions with Python int types can seamlessly return a Python long. For example, on a system with a 32-bit C long: If the intermittent result never exceeds 2**31 - 1, the expression returns an int: >>> 2**30 - 1 + 2**30 2147483647 However, if the order of operations in this expression are swapped to have an intermittent result of 2**31, it returns a long (the representation ends with an 'L'): >>> 2**30 + 2**30 - 1 2147483647L Python 3, on the other hand, pretty much renamed "long" to "int" (as it similarly renamed unicode to str), so now there's only one integer type named "int". There is no built-in type named "long". But the C API preserves the original type (PyLong_Type) and function names such as PyLong_FromSsize_t: >>> from ctypes import * >>> type( pythonapi.PyLong_FromSsize_t(-1) ) PEP 237 (Unifying Long Integers and Integers) goes into more detail, but it predates the Python 3 implementation: http://www.python.org/dev/peps/pep-0237 ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] a question about maxint
On 03/10/12 12:33, eryksun wrote: On Wed, Oct 3, 2012 at 1:28 AM, Katya Stolpovskaya Thank you for you reply, but with "long" I got the same error: from sys import * long Traceback (most recent call last): File "", line 1, in long NameError: name 'long' is not defined I assumed some familiarity with Python 2, which has two integer arithmetic types: int (fixed precision, using the platform's C long type) and long (arbitrary precision). The naming is a bit conflated in that a Python int uses a C long, which has nothing to do with a Python long. In case Katya didn't follow the technical stuff the bottom line is that 'long' is not a name in Python. You should check maxsize rather than maxint but in practice you probably don't need to, just treat Python integers as being of arbitrary length: >>> bigInt = 25**45 >>> print bigInt 807793566946316088741610050849573099185363389551639556884765625 >>> The only times you really need to worry about maxsize is when interfacing to external non-python code. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] a question about maxint
On Wed, Oct 3, 2012 at 2:53 PM, Alan Gauld wrote: > > The only times you really need to worry about maxsize is when interfacing to > external non-python code. It's not generally a problem, but if you're on a 32-bit platform, for which sys.maxsize is 2**31 - 1, that sets the maximum length of a sequence or string. So, for example, you won't be able to read a 2 GiB file into one string (or Py3k bytes object). However, if you're on a 64-bit platform, there's nothing to worry about since sys.maxsize is 2**63 - 1. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
[Tutor] forcing the None values of a dictionary
I'm wondering if there is something I overlooked for this function I wrote... and also whether it could've been done destrictively instead of returning a new dictionary: def dictNoneValueTo(d, new_value=''): """force None values in a dictionary to a default value""" for k in d: if d[k] is None: d[k] = new_value return d -- This message w/attachments (message) is intended solely for the use of the intended recipient(s) and may contain information that is privileged, confidential or proprietary. If you are not an intended recipient, please notify the sender, and then please delete and destroy all copies and attachments, and be advised that any review or dissemination of, or the taking of any action in reliance on, the information contained in or attached to this message is prohibited. Unless specifically indicated, this message is not an offer to sell or a solicitation of any investment products or other financial product or service, an official confirmation of any transaction, or an official statement of Sender. Subject to applicable law, Sender may intercept, monitor, review and retain e-communications (EC) traveling through its networks/systems and may produce any such EC to regulators, law enforcement, in litigation and as required by law. The laws of the country of each sender/recipient may impact the handling of EC, and EC may be archived, supervised and produced in countries other than the country in which you are located. This message cannot be guaranteed to be secure or free of errors or viruses. References to "Sender" are references to any subsidiary of Bank of America Corporation. Securities and Insurance Products: * Are Not FDIC Insured * Are Not Bank Guaranteed * May Lose Value * Are Not a Bank Deposit * Are Not a Condition to Any Banking Service or Activity * Are Not Insured by Any Federal Government Agency. Attachments that are part of this EC may have additional important disclosures and disclaimers, which you should read. This message is subject to terms available at the following link: http://www.bankofamerica.com/emaildisclaimer. By messaging with Sender you consent to the foregoing. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] forcing the None values of a dictionary
On 10/03/2012 03:59 PM, Brannon, Terrence wrote: > I'm wondering if there is something I overlooked for this function I wrote... > and also whether it could've been done destrictively instead of returning a > new dictionary: > > def dictNoneValueTo(d, new_value=''): > """force None values in a dictionary to a default value""" > for k in d: > if d[k] is None: > d[k] = new_value > return d > Hard to tell what you might have overlooked. Does the function correctly implement the specification you didn't tell us about? There's no English word "destrictively" so I'm not sure what you're asking. The function certainly doesn't return a new dictionary, it returns the same one passed in. In other words, it mutates itself in place. By convention, it should return None (which would happen if you had no return statement). -- DaveA ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] forcing the None values of a dictionary
On 10/3/2012 3:59 PM, Brannon, Terrence wrote: I'm wondering if there is something I overlooked for this function I wrote... and also whether it could've been done destrictively instead of returning a new dictionary: I don't understand your question. Could you clarify? What does "destrictivel" mean? Why do you say it is returning a new dictionary (it simply returns the modified input dictionary)? Since d is mutable, the function changes it "in place". You could drop the return statement and call the function thusly: d = someDictionary dictNoneValueTo(d, 123) *def* dictNoneValueTo(d, new_value=''): *"""force None values in a dictionary to a default value"""* *for* k *in* d: *if* d[k] *is* *None*: d[k] = new_value *return* d For what its worth please inform your lawyers that I do not read or heed any lengthy legalese at the end of emails. I think they do more to irritate recipients than to protect anything. Be warned: this message has no intention to do anything but inform. You may do anything with it you like. So there. -- Bob Gailer 919-636-4239 Chapel Hill NC ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
[Tutor] modifying lists of lists
I'm just learning Python, so I apologize for a newby question. I'm trying to work with lists of lists, the lowest level of which hold one or more tuples. I've tried to condense what I've tried. The code is: #! Python 2.7 import copy list = [] for i in range(8): list.append((i, i+1)) H = [[list[0], list[2]]] I = [[list[0], list[2]]] J = [[list[0], list[2]]] # H.append(tuple) did it three times for example, and because H=I=J won't work for the example. print 'H - list with one list of tuples:', H H.append([list[1], list[3]]) I.append([list[1], list[3]]) J.append([list[1], list[3]]) print 'H - added list of tuples:', H # duplicate first list of tuples -> next in sequence H.insert(1,H[0]) print 'duplicated 1st list in list:', H # works, but can't edit the second list H[1].pop(1) print "First 2 lists can't be independently edited" print H,'\n' # try to split up the list and rebuild J = J[:1] + J[:1] + J[1:] print 'J:', J J[1].pop(1) print "Still can't independently delete tuple:\n", J # try the copy module I = I[:1] + copy.deepcopy(I[:1]) + I[1:] print '\nI:', I I[1].pop(1) print "deepcopy requred for independence:\n", I The third trial works: >>> H - list with one list of tuples: [[(0, 1), (2, 3)]] H - added list of tuples: [[(0, 1), (2, 3)], [(1, 2), (3, 4)]] duplicated 1st list in list: [[(0, 1), (2, 3)], [(0, 1), (2, 3)], [(1, 2), (3, 4)]] First 2 lists can't be independently edited [[(0, 1)], [(0, 1)], [(1, 2), (3, 4)]] J: [[(0, 1), (2, 3)], [(0, 1), (2, 3)], [(1, 2), (3, 4)]] Still can't independently delete tuple: [[(0, 1)], [(0, 1)], [(1, 2), (3, 4)]] I: [[(0, 1), (2, 3)], [(0, 1), (2, 3)], [(1, 2), (3, 4)]] deepcopy requred for independence: [[(0, 1), (2, 3)], [(0, 1)], [(1, 2), (3, 4)]] >>> Is there a better way? I've spent a lot of time looking through the documentation so I anticipated the results of the first attempt. I postulate the split approach didn't work because the assignment was a pointer to the object, in spite of the splitting up of the list. By the way, copy.copy doesn't work. Thanks for your insights. Ed ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
[Tutor] reloading a module
So Ive got code that i import a module to get certain saved variables, where i edit the text file that comprises the module to edit those saved variable. My problem is I cant reload the module to access those modified variables. I was wondering how can i reload or otherwise refresh the module. python3 on linux Thank you. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] reloading a module
On 04/10/12 11:56, Leo Degon wrote: So Ive got code that i import a module to get certain saved variables, where i edit the text file that comprises the module to edit those saved variable. My problem is I cant reload the module to access those modified variables. I was wondering how can i reload or otherwise refresh the module. python3 on linux The reload function is removed from the built-ins from Python 3 because it is a bit of a trap for the careless and the beginner. But it isn't removed completely. Run this when you start up Python: from imp import reload Now you can call reload(module) as in Python 2. -- Steven ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] reloading a module
On 10/3/2012 9:56 PM, Leo Degon wrote: So Ive got code that i import a module to get certain saved variables, where i edit the text file that comprises the module to edit those saved variable. My problem is I cant reload the module to access those modified variables. Why not? What did you try? I was wondering how can i reload or otherwise refresh the module. python3 on linux Answer: >>> import x >>> # make some changes to x.py >>> reload(x) -- Bob Gailer 919-636-4239 Chapel Hill NC ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] reloading a module
On 10/03/2012 09:56 PM, Leo Degon wrote: > So Ive got code that i import a module to get certain saved variables, > where i edit the text file that comprises the module to edit those saved > variable. My problem is I cant reload the module to access those modified > variables. > I was wondering how can i reload or otherwise refresh the module. > python3 on linux > > Python version 3 replaced the reload(xxx) syntax with : import imp imp.reload(xxx) However, there are still lots of caveats, and reasons not to do it. This doesn't initialize the new module, it doesn't follow the imports used there, it doesn't tell other modules of the code about this new version of the module, The feature is there for debugging, as far as I know, and probably shouldn't be used in real code. If your design is to have these variables' default values to come from the file, then I'd suggest making it a data file, and not an import. Then you can literally save and restore those variables flexibly, and without straining the language's limits. Alternatively, just restart the process. -- DaveA ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] modifying lists of lists
On 04/10/12 11:46, Ed Owens wrote: I'm just learning Python, so I apologize for a newby question. I'm trying to work with lists of lists, the lowest level of which hold one or more tuples. I've tried to condense what I've tried. Hi Ed, and welcome! The code is: I'm afraid I can't make heads nor tails of *why* you are trying this. I can guess you are testing something about copying lists, but comments like this: # H.append(tuple) did it three times for example, and because H=I=J won't work for the example. leave me mystified. "did it three times for example"? Did what? What counts as working? You pepper your comments with things like "doesn't work", "can't do this", "can't be edited" etc., but without knowing what you expect to happen it's hard to understand what you mean. How about if you start off with a simple example, and you show what happens when you run the code, AND what you expected to happen? To make it easy, here is my *guess* as to the sort of thing that is confusing you. py> H = [[1, 2]] py> J = [H[0]] py> print H [[1, 2]] py> print J [[1, 2]] py> H[0][0] = 99 py> print H # expected, and got, [[99, 2]] [[99, 2]] py> print J # expected [[1, 2]] [[99, 2]] Am I close? If I am, I (or one of the other tutors) will explain what is going on. Otherwise, you will have to explain what you are trying to do in more detail. -- Steven ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Why difference between printing string & typing its object reference at the prompt?
Thanks to all who responded. There was much more going on here than I ever would have suspected. I am glad I asked the questions I did. This has been very informative. On Tue, Oct 2, 2012 at 11:53 PM, Dave Angel wrote: > > There are two operations supported by (most) objects that produce a > string. One is exemplified by the str() function, which converts an > object to a string. That's the one called implicitly by print(). This > form just represents the data, in the form most likely to be needed by > the end user. What happens if str() or repr() is not supported by a particular object? Is an exception thrown, an empty string returned or something else I am not imagining? > > The other operation is repr(), which attempts to produce a string that > could be used in a program to reproduce the actual object. So a repr() > will have quote marks artificially added, or brackets, or commas, or > whatever seems appropriate for the particular object. This is intended > for the programmer's use, not for the end user. What larger phrase does "repr" stand for? My text mentions "representational form" later in the book, which sounds similar in concept to what you are discussing. [...] > Your question was about string objects, but I tried to make the > explanation as generic as possible. Those two functions, str() and > repr(), are used, or implied in many places. For example, if you print > a list, it'll call str() on the whole list. But the list object's logic > will in turn call repr() on each of its elements, and put the whole > thing together with braces and commas. > As I go along in my study of Python will it become clear to me when and how repr() and str() are being "...used, or implied in many places"? Thanks! boB ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Why difference between printing string & typing its object reference at the prompt?
On Wed, Oct 3, 2012 at 1:38 AM, Steven D'Aprano wrote:
> The long answer is a bit more subtle, and rather long.
I had initial suspicions this would be the case Thanks for yours and
Dave's detailed exposition!
[...]
> Python is no different: words, text if you will, that are part of the
> code are written as normal:
>
> # source code
> class Test:
> pass
>
> x = Test # Test here refers to the variable Test, a class
>
> But to create a string object, you use quotation marks to tell Python
> that this is data, not code, please create a string object:
>
> x = "Test" # Test here refers to a string, which is data
>
> Notice that the quotation marks are *delimiters*, they mark the start
> and end of the string, but aren't part of the string in any way. Python
> knows that the object is a string because you put it in string
> delimiters, but the delimiters are not part of the string.
I was not sure if the quotes were considered part of the string or
not. Thanks for the clarification.
> Now, take a step back and consider objects in general. There are two
> things we might like to do to an arbitrary object:
>
> * display the object, which implicitly means turning it into a
> string, or at least getting some representation of that object
> as a string;
>
> * convert the object into a string.
>
> Python has two built-in functions for that:
>
> * repr, which takes any object and returns a string that represents
> that object;
>
> * str, which tries to convert an object into a string, if that makes
> sense.
>
> Often those will do the same thing. For example:
>
> py> str(42) == repr(42) == "42"
> True
>
> But not always. For example:
>
> py> from decimal import Decimal as D
> py> x = D("1.23")
> py> print(str(x))
> 1.23
> py> print(repr(x))
> Decimal('1.23')
These contrasting examples are very illuminating. So in print(str(x))
the object, D("1.23"), is being converted into a readable string,
which makes the most sense as 1.23. But print(repr(x)) is giving a
string representation of the object as code, which is more than just
1.23, the Decimal('1.23'). Am I understanding this correctly?
> Unfortunately, the difference between str() and repr() is kind of
> arbitrary and depends on the object. str() is supposed to return a
> "human-readable" version of the object, for display, while repr() is
> supposed to return a string which would work as code, but those are more
> guidelines than hard rules.
Will these fine distinctions be easy for me to pick up on as I
progress in my Python studies? I suspect that I am going to have to
experiment with str() and repr() in each new situation to see what
results.
> So we have two different ways of converting an object to a string. But
> strings themselves are objects too. What happens there?
>
> py> s = "Hello world" # remember the quotes are delimiters, not part of the
> string
> py> print(str(s))
> Hello world
> py> print(repr(s))
> 'Hello world'
>
> str() of a string is unchanged (and why shouldn't it be? it's already a
> string, there's nothing to convert).
>
> But repr() of a string creates a new string showing the representation
> of the original string, that is, what you would need to type in source
> code to make that string. That means:
>
> 1) wrap the whole thing in delimiters (quotation marks)
> 2) escaping special characters like tabs, newlines, and binary
>characters.
As to point 2), will repr() insert "\" (I am assuming Python uses a
backslash like other languages to escape. I have not read about this
in Python yet.) for these special characters? Will str() do the same?
> Notice that the string returned by repr() includes quote marks as part
> of the new string. Given the s above:
>
> py> t = repr(s)
> py> print(t)
> 'Hello world'
> py> t
> "'Hello world'"
>
> This tells us that the new string t includes single quote marks as the
> first and last character, so when you print it, the single quote marks
> are included in the output. But when you just display t interactively
> (see below), the delimiters are shown.
Another great example. I probably would have overlooked this.
> Now, at the interactive interpreter, evaluating an object on its own
> without saving the result anywhere displays the repr() to the screen.
> Why repr()? Well, why not? The decision was somewhat arbitrary.
So the designers of Python made this decision. I guess it had to be
one way or the other.
> print, on the other hand, displays the str() of the object directly to
> the screen. For strings, that means the delimiters are not shown,
> because they are not part of the string itself. Why str() rather than
> repr()? Because that's what people mostly want, and if you want the
> other, you can just say print(repr(obj)).
So in the end it is a simple choice to give the users what they want
and are already used to.
>
> Does this help, or are you more confused than ever?
>
This has been incredibly useful! Many thanks!!
boB
___
Re: [Tutor] modifying lists of lists
You are fundamentally correct about my confusion, though I'm trying to work with tuples as the lowest level, which may not be relevant here. -Original Message- . py> H = [[1, 2]] py> J = [H[0]] py> print H [[1, 2]] py> print J [[1, 2]] py> H[0][0] = 99 py> print H # expected, and got, [[99, 2]] [[99, 2]] py> print J # expected [[1, 2]] [[99, 2]] -- Using your example: Py> Import copy py> H = [[1, 2]] py> J = [H[0]] py> print H [[1, 2]] py> print J [[1, 2]] py> H[0][1] = copy.deepcopy(H[0][0]) py> print H # expected, and got, [[1,1]] [[1,1]] py> print J # expected [[1,2]] [[1, 1]] How do I decouple these references? ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] modifying lists of lists
On Wed, Oct 3, 2012 at 11:52 PM, Ed Owens wrote: > > py> H = [[1, 2]] > py> J = [H[0]] > py> H[0][1] = copy.deepcopy(H[0][0]) > > How do I decouple these references? You can use the slice H[0][:] to get a shallow copy of the H[0] list. By "shallow copy" I mean you get a new list that contains the items of the source list. You can mutate this new list without changing the source list. Returning to your first example: >>> H = [[(0, 1), (2, 3)], [(1, 2), (3, 4)]] Insert a shallow copy of H[0] at index 1: >>> H.insert(1, H[0][:]) >>> H [[(0, 1), (2, 3)], [(0, 1), (2, 3)], [(1, 2), (3, 4)]] The new list at H[1] is independent of H[0]: >>> H[1].pop(1) (2, 3) >>> H [[(0, 1), (2, 3)], [(0, 1)], [(1, 2), (3, 4)]] ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
