Re: [Tutor] UnicodeDecodeError while parsing a .csv file.

2013-10-29 Thread eryksun
On Tue, Oct 29, 2013 at 6:33 AM, Albert-Jan Roskam wrote: > Why is do_setlocale=False here? Actually, what does this parameter do? > It seems strange that a getter function has a 'set' argument. On Windows, getpreferredencoding doesn't use setlocale. It calls WinAPI GetACP to fetch the ANSI codep

Re: [Tutor] UnicodeDecodeError while parsing a .csv file.

2013-10-29 Thread Albert-Jan Roskam
--- On Tue, 10/29/13, eryksun wrote: Subject: Re: [Tutor] UnicodeDecodeError while parsing a .csv file. To: "Steven D'Aprano" Cc: tutor@python.org Date: Tuesday, October 29, 2013, 3:24 AM On Mon, Oct 28, 2013 at 7:49 PM, Steven D

Re: [Tutor] UnicodeDecodeError while parsing a .csv file.

2013-10-28 Thread eryksun
On Mon, Oct 28, 2013 at 7:49 PM, Steven D'Aprano wrote: > > By default Python 3 uses UTF-8 when reading files. As the error below > shows, your file actually isn't UTF-8. Modules default to UTF-8, but io.TextIOWrapper defaults to the locale preferred encoding. To handle terminals, it first tries

Re: [Tutor] UnicodeDecodeError while parsing a .csv file.

2013-10-28 Thread SM
Thanks to all, for so much information. I just checked that the encoding is Latin-1 and it works when I use it in the file open call. (instead of ignoring it). On Mon, Oct 28, 2013 at 7:54 PM, Danny Yoo wrote: > Hi Sm, > > Note: if possible, I would strongly suggest reusing csv.reader rather th

Re: [Tutor] UnicodeDecodeError while parsing a .csv file.

2013-10-28 Thread SM
Hi Steven, Thanks, very much, for the very detailed reply. It was very useful. This is just a utility script to read some sentiment analysis data to manipulate the positive and negative sentiments of multiple people into a single sentiment per line. So the data I got was from some public domain whi

Re: [Tutor] UnicodeDecodeError while parsing a .csv file.

2013-10-28 Thread SM
Hi Bob, Thanks, very much, for your quick and detailed reply. This is just a utility script to read some sentiment analysis data to manipulate the positive and negative sentiments of multiple people into a single sentiment per line. The data I got was from some public domain which I have no control

Re: [Tutor] UnicodeDecodeError while parsing a .csv file.

2013-10-28 Thread Danny Yoo
Hi Sm, Note: if possible, I would strongly suggest reusing csv.reader rather than reinvent this. It comes in Python's Standard Library: http://docs.python.org/3/library/csv.html#module-csv ​ --- Anyway, to your question. Do you know what is the encoding of your input file? (And if you don

Re: [Tutor] UnicodeDecodeError while parsing a .csv file.

2013-10-28 Thread Mark Lawrence
On 28/10/2013 22:13, SM wrote: Hello, I have an extremely simple piece of code which reads a .csv file Could have fooled me, why not use the stdlib csv module? For the UnicodeDecodeError Bob Gailer has already pointed you in the right direction. -- Python is the second best programming lang

Re: [Tutor] UnicodeDecodeError while parsing a .csv file.

2013-10-28 Thread Steven D'Aprano
On Mon, Oct 28, 2013 at 06:13:59PM -0400, SM wrote: > Hello, > I have an extremely simple piece of code which reads a .csv file, which has > 1000 lines of fixed fields, one line at a time, and tries to print some > values. > > 1 #!/usr/bin/python3 > 2 # > 3 import sys, time, re, os > 4 >

Re: [Tutor] UnicodeDecodeError while parsing a .csv file.

2013-10-28 Thread bob gailer
On 10/28/2013 6:13 PM, SM wrote: > Hello, Hi welcome to the Tutor list > I have an extremely simple piece of code which could be even simpler - see my comments below > which reads a .csv file, which has 1000 lines of fixed fields, one line at a time, and tries to print some values. > > 1 #!

[Tutor] UnicodeDecodeError while parsing a .csv file.

2013-10-28 Thread SM
Hello, I have an extremely simple piece of code which reads a .csv file, which has 1000 lines of fixed fields, one line at a time, and tries to print some values. 1 #!/usr/bin/python3 2 # 3 import sys, time, re, os 4 5 if __name__=="__main__": 6 7 ifd = open("infile.csv", 'r')

[Tutor] UnicodeDecodeError in kinterbasdb

2007-04-05 Thread Andy Koch
Hello, I've installed Python 25 on an XP machine, installed kinterbasdb (and eginix-mx-base). Python works fine. However, when I try to load the firebird module in IDLE I get ... Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "cr

Re: [Tutor] UnicodeDecodeError when copying files with Umlauten

2005-07-10 Thread Severin Kacianka
Hello, I finally could solve the problem on my own. These two articles helped me a lot: http://www.onlamp.com/pub/a/python/excerpt/pythonckbk_chap1/ http://www.reportlab.com/i18n/python_unicode_tutorial.html I simply had to ensure that all the file names I got from the m3u file were in unicode

[Tutor] UnicodeDecodeError when copying files with Umlauten

2005-07-09 Thread Severin Kacianka
Hello, I am very new to python programming and totally new to GUI-programming. Now I try to write a GUI for a little script I wrote some time ago, and get a Error I cannot explain. As it may be helpful for you to see the whole scripts I put them both online: The CLI-application (and library):

Re: [Tutor] UnicodeDecodeError

2005-02-23 Thread Kent Johnson
Michael Lange wrote: now it looks like the total confusion seems to clear up (at least partially). After some googling it seems to me that the best bet is to use unicode strings exclusively. I think that is a good plan. When I set the unicode flag in gettext.install() to 1 the gettext strings are

Re: [Tutor] UnicodeDecodeError

2005-02-23 Thread Michael Lange
On Wed, 23 Feb 2005 07:21:40 -0500 Kent Johnson <[EMAIL PROTECTED]> wrote: > > This is a part of Python that still confuses me. I think what is happening is > - self.nextfile is a Unicode string sometimes (when it includes special > characters) > - the gettext string is a byte string > - to comp

Re: [Tutor] UnicodeDecodeError

2005-02-23 Thread Kent Johnson
Michael Lange wrote: Hello list, I've encountered an (at least for me) weird error in the project I'm working on (see the traceback below). Unfortunately there are several of my application's modules involved, so I cannot post all of my code here. I hope I can explain in plain words what I'm doin

Re: [Tutor] UnicodeDecodeError

2005-02-23 Thread Michael Lange
On Tue, 22 Feb 2005 19:17:40 -0500 "Isr Gish" <[EMAIL PROTECTED]> wrote: > This part of the error is saying what the problem is. > >>UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 22: > ordinal not in range(128) > > Thatthe ascii codec that's being used can't decod

RE: [Tutor] UnicodeDecodeError

2005-02-22 Thread Isr Gish
Forgot something in previOuse post. You can set the defualt encoding to something other then 'ascii'. This is done with the function Qsys.setdefualtencoding('mbcs')". I think that this has to be done in the module sitecostomize, because the function is deleted after the module site is run. And t

RE: [Tutor] UnicodeDecodeError

2005-02-22 Thread Isr Gish
Michael Lange"<[EMAIL PROTECTED]> wrote: >Hello list, > [Snip] This part of the error is saying what the problem is. >UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 22: ordinal not in range(128) Thatthe ascii codec that's being used can't decode any ascii code

[Tutor] UnicodeDecodeError

2005-02-22 Thread Michael Lange
Hello list, I've encountered an (at least for me) weird error in the project I'm working on (see the traceback below). Unfortunately there are several of my application's modules involved, so I cannot post all of my code here. I hope I can explain in plain words what I'm doing. The line in the