Re: [Tutor] UnicodeDecodeError while parsing a .csv file.

2013-10-29 Thread eryksun
On Tue, Oct 29, 2013 at 6:33 AM, Albert-Jan Roskam wrote: > Why is do_setlocale=False here? Actually, what does this parameter do? > It seems strange that a getter function has a 'set' argument. On Windows, getpreferredencoding doesn't use setlocale. It calls WinAPI GetACP to fetch the ANSI codep

Re: [Tutor] UnicodeDecodeError while parsing a .csv file.

2013-10-29 Thread Albert-Jan Roskam
--- On Tue, 10/29/13, eryksun wrote: Subject: Re: [Tutor] UnicodeDecodeError while parsing a .csv file. To: "Steven D'Aprano" Cc: tutor@python.org Date: Tuesday, October 29, 2013, 3:24 AM On Mon, Oct 28, 2013 at 7:49 PM, Steven D

Re: [Tutor] UnicodeDecodeError while parsing a .csv file.

2013-10-28 Thread eryksun
On Mon, Oct 28, 2013 at 7:49 PM, Steven D'Aprano wrote: > > By default Python 3 uses UTF-8 when reading files. As the error below > shows, your file actually isn't UTF-8. Modules default to UTF-8, but io.TextIOWrapper defaults to the locale preferred encoding. To handle terminals, it first tries

Re: [Tutor] UnicodeDecodeError while parsing a .csv file.

2013-10-28 Thread SM
Thanks to all, for so much information. I just checked that the encoding is Latin-1 and it works when I use it in the file open call. (instead of ignoring it). On Mon, Oct 28, 2013 at 7:54 PM, Danny Yoo wrote: > Hi Sm, > > Note: if possible, I would strongly suggest reusing csv.reader rather th

Re: [Tutor] UnicodeDecodeError while parsing a .csv file.

2013-10-28 Thread SM
Hi Steven, Thanks, very much, for the very detailed reply. It was very useful. This is just a utility script to read some sentiment analysis data to manipulate the positive and negative sentiments of multiple people into a single sentiment per line. So the data I got was from some public domain whi

Re: [Tutor] UnicodeDecodeError while parsing a .csv file.

2013-10-28 Thread SM
Hi Bob, Thanks, very much, for your quick and detailed reply. This is just a utility script to read some sentiment analysis data to manipulate the positive and negative sentiments of multiple people into a single sentiment per line. The data I got was from some public domain which I have no control

Re: [Tutor] UnicodeDecodeError while parsing a .csv file.

2013-10-28 Thread Danny Yoo
Hi Sm, Note: if possible, I would strongly suggest reusing csv.reader rather than reinvent this. It comes in Python's Standard Library: http://docs.python.org/3/library/csv.html#module-csv ​ --- Anyway, to your question. Do you know what is the encoding of your input file? (And if you don

Re: [Tutor] UnicodeDecodeError while parsing a .csv file.

2013-10-28 Thread Mark Lawrence
On 28/10/2013 22:13, SM wrote: Hello, I have an extremely simple piece of code which reads a .csv file Could have fooled me, why not use the stdlib csv module? For the UnicodeDecodeError Bob Gailer has already pointed you in the right direction. -- Python is the second best programming lang

Re: [Tutor] UnicodeDecodeError while parsing a .csv file.

2013-10-28 Thread Steven D'Aprano
On Mon, Oct 28, 2013 at 06:13:59PM -0400, SM wrote: > Hello, > I have an extremely simple piece of code which reads a .csv file, which has > 1000 lines of fixed fields, one line at a time, and tries to print some > values. > > 1 #!/usr/bin/python3 > 2 # > 3 import sys, time, re, os > 4 >

Re: [Tutor] UnicodeDecodeError while parsing a .csv file.

2013-10-28 Thread bob gailer
On 10/28/2013 6:13 PM, SM wrote: > Hello, Hi welcome to the Tutor list > I have an extremely simple piece of code which could be even simpler - see my comments below > which reads a .csv file, which has 1000 lines of fixed fields, one line at a time, and tries to print some values. > > 1 #!

[Tutor] UnicodeDecodeError while parsing a .csv file.

2013-10-28 Thread SM
Hello, I have an extremely simple piece of code which reads a .csv file, which has 1000 lines of fixed fields, one line at a time, and tries to print some values. 1 #!/usr/bin/python3 2 # 3 import sys, time, re, os 4 5 if __name__=="__main__": 6 7 ifd = open("infile.csv", 'r')