Re: [Tutor] WRITING XLS FROM OS.WALK()
"Steven D'Aprano" wrote Only if glob now descends into the file system... which is why you'd choose os.walk instead. Do you mean you want to point at a single directory and have it search any and all subdirectories, no matter how deeply nested? Well, yes, that would be a good use-case for os.walk, and yes, I completely missed it, so I withdraw my comment that Susana had re-invented the wheel. I think the solution should use both. os.walk to traverse the directories and glob to build the file list at each level. Each tool has its role. I didn't see anything untoward in recommending glob to replace the use of the string based list comp for selection of the files, I just assemed you meant in conjunction with os.walk!. -- Alan Gauld Author of the Learn to Program web site http://www.alan-g.me.uk/ ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] OpenMP
On 10/9/2010 9:45 PM Ahmed AL-Masri said... Thanks for fast responding. I will try to use the threads and see how the performance would be.. actually I am using that for my artificial neural network and the problem is regarding to the ANN limitation when I used a big no of inputs. so one way to overcome this problem is by distributing and now I have like 3 networks in my system with slow processing. May be parallel could have little effort. You may also find stackless helpful at http://www.stackless.com which forms the backend of EVE Online and claims "large scale shared state simulation across tens of thousands of CPUs" Emile ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] OpenMP
Thanks a lots. it's really helpful,, I am reading it -- From: "Emile van Sebille" Sent: Sunday, October 10, 2010 11:51 PM To: Subject: Re: [Tutor] OpenMP On 10/9/2010 9:45 PM Ahmed AL-Masri said... Thanks for fast responding. I will try to use the threads and see how the performance would be.. actually I am using that for my artificial neural network and the problem is regarding to the ANN limitation when I used a big no of inputs. so one way to overcome this problem is by distributing and now I have like 3 networks in my system with slow processing. May be parallel could have little effort. You may also find stackless helpful at http://www.stackless.com which forms the backend of EVE Online and claims "large scale shared state simulation across tens of thousands of CPUs" Emile ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] list of dict question
On 09/10/2010 10.25, Alan Gauld wrote: "Francesco Loffredo" wrote > On the next iteration you overwrite those two dictionaries > with new values then append them to the list again. > So you wind up with 2 copies of the updated dictionaries. > ... This is difficult for me too: why does this happen? Or, more correctly, why should this happen? It happens because you can't store two objects in one. There is only one dictionary. If you change its value you change its value. How can you save the current contents of a dictionary in a list, making sure that the saved values won't change if you update the dict? You need to save a copy of the dictionary. ie Create a new dictionary. If you put a box of white eggs in your shopping basket you cannot put brown eggs into that box and expect to still have the white ones as well. You need to get two boxes! I obviously haven't been clear in my question, sure I know that if you change one object you lose the previous contents, but (hope this makes my question clear) why should the append method of the list store a pointer to the dictionary, rather then a copy of that dictionary? If it did, Roelof's code would work perfectly, and you could store in a list all the subsequent changes of a dictionary without calling them with different names. For example (and this should make my previous example a bit fuller), if you had a dictionary containing the current status of a system, and you wanted to keep an history of that status (well, I'd use a file, but let's imagine you had to use a list), you could simply add the (newer version of the) dictionary to the list: myDictList.append(UpdateIt(myDict)) This would be much more elegant and readable than creating a different dictionary (meaning a different name) for every state of the said system, and making sure you never repeat a name twice, or you'll lose the state you saved some time ago... I understand that if .append() stored a copy of the dict in the list, you will end up with lots of copies and a huge amount of memory used by your list, but that's exactly what will happen if you make those copies yourself. But you wouldn't have to devise a way to generate a new name for the dictionary every time you need to update it. Thank you for your help Francesco Nessun virus nel messaggio in uscita. Controllato da AVG - www.avg.com Versione: 9.0.862 / Database dei virus: 271.1.1/3186 - Data di rilascio: 10/09/10 08:34:00 ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] how to extract data only after a certain condition is met
Hi, First let me apologize for taking so long to acknowledge your answers and to thank you (Eduardo, Peter, Greg, Emile, Joel and Alan, sorry if I left anyone) for your help and your time. One of the reasons I took so long in responding (besides having gotten busy with some urgent matters related to my work) is that I was a bit embarrassed at realizing how poorly I had defined my problem. As Alan said, I should at least have told you which operations were giving me a headache. So I went back to my Python reference books to try to write some code and thus be able to define my problems more precisely. Only after I did that, I said to myself, I would come back to the list with more specific questions. The only problem is that doing this made me painfully aware of how little Python I know. Well, actually my problem is not so much that I don't know Python as that I have very little experience programming in general. Some years ago I learned a little Perl and basically I used it to do some text manipulation using regular expressions but that's all my experience. In order to learn Python, I read a book called "Beginning Python: From Novice to Professional" and I was hoping that just by starting to use the knowledge I had supposedly acquired by reading that book to solve real problems related to my project I would learn. But this turned out to be much more difficult than I had expected. Perhaps if I had worked through the excellent book/tutorial Alan has written (of which I was not aware when I started), I would be better prepared to confront this problem. Anyway (sorry for the long intro), since Emile laid out the problem very clearly, I will use his outline to point out the problems I'm having: Emile says: -- Conceptually, you'll need to: -a- get the list of file names to change then for each -b- determine the new name -c- rename the file For -a- you'll need glob. For -c- use os.rename. -b- is a bit more involved. To break -b- down: -b1- break out the x-xx portion of the file name -b2- look up the corresponding year in the other file -b3- convert the year to the century-half structure -b4- put the pieces together to form the new file name For -b2- I'd suggest building a dictionary from your second files contents as a first step to facilitate the subsequent lookups. - OK. Let's start with -b- . My first problem is that I don't really know how to go about building a dictionary from the file with the comma separated values. I've discovered that if I use a file method called 'readlines' I can create a list whose elements would be each of the lines contained in the document with all the codes followed by comma followed by the year. Thus if I do: fileNameCentury = open(r '/Volumes/DATA/Documents/workspace/GCA/CORPUS_TEXT_LATIN_1/FileNamesYears.txt' ).readlines() Where 'FileNamesYears.txt' is the document with the following info: A-01, 1278 A-02, 1501 ... N-09, 1384 I get a list of the form ['A-01,1374\rA-02,1499\rA-05,1449\rA-06,1374\rA-09, ...] Would this be a good first step to creating a dictionary? It seems to me that I should be able to iterate over this list in some way and make the substring before the comma the key and the substring after the comma its value. The problem is that I don't know how. Reading the book I read has not prepared me for this. I have the feeling that all the pieces of knowledge I need to solve the problem where there, but I don't know how to put them together. Greg mentioned the csv module. I checked the references but I could not see any way in which I could create a dictionary using that module, either. Once I have the dictionary built, what I would have to do is use the os module (or would it be the glob module?) to get a list of the file names I want to change and build another loop that would iterate over those file names and, if the first part of the name (possibly represented by a regular expression of the form r'[A-Z]-[0-9]+') matches one of the keys in the dictionary, then a) it would get the value for that key, b) would do the numerical calculation to determine whether it is the first part of the century or the second part and c) would insert the string representing this result right before the extension .txt. In the abstract it sounds easy, but I don't even know how to start. Doing some testing with glob I see that it returns a list of strings representing the whole paths to all the files whose names I want to manipulate. But in the reference documents that I have consulted, I see no way to change those names. How do I go about inserting the information about the century right before the substring '.txt'? As you see, I am very green. My embarrassment at realizing how basic my problems were made me delay writing another message but I decided that if I don't do it, I will never learn. Again, thanks so much for all your help. Josep M. > Message: 2 > Date: Sat, 2 Oct 2010 17:56:53 +0200 > From: "Josep M. Fon
Re: [Tutor] how to extract data only after a certain condition is met
On 10/10/2010 12:35 PM Josep M. Fontana said...
OK. Let's start with -b- . My first problem is that I don't really know how
to go about building a dictionary from the file with the comma separated
values. I've discovered that if I use a file method called 'readlines' I can
create a list whose elements would be each of the lines contained in the
document with all the codes followed by comma followed by the year. Thus if
I do:
fileNameCentury = open(r
'/Volumes/DATA/Documents/workspace/GCA/CORPUS_TEXT_LATIN_1/FileNamesYears.txt'
).readlines()
Where 'FileNamesYears.txt' is the document with the following info:
A-01, 1278
A-02, 1501
...
N-09, 1384
I get a list of the form ['A-01,1374\rA-02,1499\rA-05,1449\rA-06,1374\rA-09,
...]
Would this be a good first step to creating a dictionary?
Hmmm... It looks like you got a single string -- is that the output from
read and not readlines? I also see you're just getting \r which is the
Mac line terminator. Are you on a Mac, or was 'FileNamesYears.txt'
created on a Mac?. Python's readlines tries to be smart about which
line terminator to expect, so if there's a mismatch you could have
issues related to that. I would have expected you'd get something more
like: ['A-01,1374\r','A-02,1499\r','A-05,1449\r','A-06,1374\r','A-09, ...]
In any case, as you're getting a single string, you can split a string
into pieces, for example, print "1\r2\r3\r4\r5".split("\r"). That way
you can force creation of a list of strings following the format
"X-NN," each of which can be further split with xxx.split(",").
Note as well that you can assign the results of split to variable names.
For example, ky,val = "A-01, 1278".split(",") sets ky to A-01 and val
to 1278. So, you should be able to create an empty dict, and for each
line in your file set the dict entry for that line.
Why don't you start there and show us what you get.
HTH,
Emile
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] list of dict question
"Francesco Loffredo" wrote
did, Roelof's code would work perfectly, and you could store in a
list
all the subsequent changes of a dictionary without calling them with
different names.
You don;'t need dfifferent names. Provided the name creates a
new object inside the loop you can reuse the same name.
Roeloff's problem was that he only created one object, outside
his loop.
lst = []
for n in range(3):
obj = {}
obj[n] = str(n)
lst.append(obj)
Creats a list of 3 distinct dictionaries but only uses one name - obj.
I understand that if .append() stored a copy of the dict in the
list,
you will end up with lots of copies and a huge amount of memory used
by
your list, but that's exactly what will happen if you make those
copies
yourself. But you wouldn't have to devise a way to generate a new
name
for the dictionary every time you need to update it.
Python uses references throughout, what you are suggesting would
be a change to the normal way that Python uses names.
HTH,
--
Alan Gauld
Author of the Learn to Program web site
http://www.alan-g.me.uk/
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] how to extract data only after a certain condition is met
Emile beat me to it, but here goes anyway...
On 10/10/2010 3:35 PM, Josep M. Fontana wrote:
Hi,
First let me apologize for taking so long to acknowledge your answers
and to thank you (Eduardo, Peter, Greg, Emile, Joel and Alan, sorry if
I left anyone) for your help and your time.
One of the reasons I took so long in responding (besides having gotten
busy with some urgent matters related to my work) is that I was a bit
embarrassed at realizing how poorly I had defined my problem.
As Alan said, I should at least have told you which operations were
giving me a headache. So I went back to my Python reference books to
try to write some code and thus be able to define my problems more
precisely. Only after I did that, I said to myself, I would come back
to the list with more specific questions.
The only problem is that doing this made me painfully aware of how
little Python I know. Well, actually my problem is not so much that I
don't know Python as that I have very little experience programming in
general. Some years ago I learned a little Perl and basically I used
it to do some text manipulation using regular expressions but that's
all my experience. In order to learn Python, I read a book called
"Beginning Python: From Novice to Professional" and I was hoping that
just by starting to use the knowledge I had supposedly acquired by
reading that book to solve real problems related to my project I would
learn. But this turned out to be much more difficult than I had
expected. Perhaps if I had worked through the excellent book/tutorial
Alan has written (of which I was not aware when I started), I would be
better prepared to confront this problem.
Anyway (sorry for the long intro), since Emile laid out the problem
very clearly, I will use his outline to point out the problems I'm having:
Emile says:
--
Conceptually, you'll need to:
-a- get the list of file names to change then for each
-b- determine the new name
-c- rename the file
For -a- you'll need glob. For -c- use os.rename. -b- is a bit more
involved. To break -b- down:
-b1- break out the x-xx portion of the file name
-b2- look up the corresponding year in the other file
-b3- convert the year to the century-half structure
-b4- put the pieces together to form the new file name
For -b2- I'd suggest building a dictionary from your second files
contents as a first step to facilitate the subsequent lookups.
-
OK. Let's start with -b- . My first problem is that I don't really
know how to go about building a dictionary from the file with the
comma separated values. I've discovered that if I use a file method
called 'readlines' I can create a list whose elements would be each of
the lines contained in the document with all the codes followed by
comma followed by the year. Thus if I do:
fileNameCentury =
open(r'/Volumes/DATA/Documents/workspace/GCA/CORPUS_TEXT_LATIN_1/FileNamesYears.txt').readlines()
Where 'FileNamesYears.txt' is the document with the following info:
A-01, 1278
A-02, 1501
...
N-09, 1384
I get a list of the
form ['A-01,1374\rA-02,1499\rA-05,1449\rA-06,1374\rA-09, ...]
I'm guessing that you are running on a Linux system and that the file
came from a Mac. This is based on the fact that \r appears in the string
instead of acting as a line separator.
Regardless -
dct = {}
fileNameCentury = fileNameCentury.split('\r') # gives you ['A-01,1374',
'A-02,1499', 'A-05,1449', 'A-06,1374', 'A-09, ...]
for pair in fileNameCentury:
key,value = pair.split(',')
dct[key] = value
Greg mentioned the csv module. I checked the references but I could
not see any way in which I could create a dictionary using that module.
True - the csv reader is just another way to get the list of pairs.
Once I have the dictionary built, what I would have to do is use the
os module (or would it be the glob module?) to get a list of the file
names I want to change and build another loop that would iterate over
those file names and, if the first part of the name (possibly
represented by a regular expression of the form r'[A-Z]-[0-9]+')
matches one of the keys in the dictionary, then a) it would get the
value for that key, b) would do the numerical calculation to determine
whether it is the first part of the century or the second part and c)
would insert the string representing this result right before the
extension .txt.
In the abstract it sounds easy, but I don't even know how to start.
Doing some testing with glob I see that it returns a list of strings
representing the whole paths to all the files whose names I want to
manipulate. But in the reference documents that I have consulted, I
see no way to change those names. How do I go about inserting the
information about the century right before the substring '.txt'?
Suppose fn = "blah.txt"
fn2 = f
As you see, I am very green. My embarrassment at realizing how basic
my problems were made me delay w
Re: [Tutor] list of dict question
On 2:59 PM, Alan Gauld wrote:
"Francesco Loffredo" wrote
did, Roelof's code would work perfectly, and you could store in a list
all the subsequent changes of a dictionary without calling them with
different names.
You don;'t need dfifferent names. Provided the name creates a
new object inside the loop you can reuse the same name.
Roeloff's problem was that he only created one object, outside
his loop.
lst = []
for n in range(3):
obj = {}
obj[n] = str(n)
lst.append(obj)
Creats a list of 3 distinct dictionaries but only uses one name - obj.
I understand that if .append() stored a copy of the dict in the list,
you will end up with lots of copies and a huge amount of memory used by
your list, but that's exactly what will happen if you make those copies
yourself. But you wouldn't have to devise a way to generate a new name
for the dictionary every time you need to update it.
Python uses references throughout, what you are suggesting would
be a change to the normal way that Python uses names.
HTH,
Probably more importantly, if a language only implemented copies, you
couldn't have references without some different syntax. On the other
hand, with Python, everything's a reference, and if you want a copy, you
make one when you need it. You never do for immutables, and only you
know when you need it for mutable objects.
DaveA
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
[Tutor] Ide To use? Other extras to install?
Hello all. I am new to Python. I have been reading and studying most of the information I have found on the site. I really like what I have seen until now. I was wondering if you can help on this. I am in the process of leaving Windows as my environment and moving to Ubuntu or a Mac. For a few months I will have to continue working under windows until I finish my moving process. Anyway, I would like to start using and IDE that I can install at least in Windows and Linux . What would be the IDE you recommend me to install that would be almost transparent to be using in both platforms? What extras do you recommend me to install? (plug ins, libraries, etc) My needs are very simple now. My learning project is to have one of my websites running under the web, working with a database behind, nothing too complicated to start. Thanks in advance for all your help. Jorge Biquez ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
[Tutor] Hiding Superclass Methods
Hi Everyone, I have a basic python question. I am writing an n dimensional vector class by inheriting from the builtin python list object. I want to be able to hide the parent object's methods in the derived class instances. I know I can overload the method in the derived class and raise some sort of an implementation error but that is not what I had in mind. I am also not looking to use numpy. This is strictly for learning purposes. Is there a way to hide superclass methods in python? Thanks to all, Denis ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Hiding Superclass Methods
Am 11.10.2010 06:24, schrieb Denis Gomes: Hi Everyone, I have a basic python question. I am writing an n dimensional vector class by inheriting from the builtin python list object. I want to be able to hide the parent object's methods in the derived class instances. Why inheriting then? Another approach to reusing exisiting methods is to wrap them into your newly defined methods. Here you would not inherit from a list, but create a list in your class (composition). E.g. class MyVector(object): def __init__(self): self.data = [] def append_data(self, new_data): self.data.append(new_data) Actually I use this all the time. And I used this before I knew about inheritance. Inheritance makes sence, when you want to reuse (almost) all methods of the superclass, like in GUI toolkits, where you typically have a base widget as superclass of a all other widgets. HTH, Jan I know I can overload the method in the derived class and raise some sort of an implementation error but that is not what I had in mind. I am also not looking to use numpy. This is strictly for learning purposes. Is there a way to hide superclass methods in python? Thanks to all, Denis ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
