How do I automate the removal of all non-ascii characters from my code?

[Alec Taylor]

Good evening,

I have converted ODT to HTML using LibreOffice Writer, because I want
to convert from HTML to Creole using python-creole. Unfortunately I
get this error: "File "Convert to Creole.py", line 17
SyntaxError: Non-ASCII character '\xe2' in file Convert to Creole.py
on line 18, but no encoding declared; see
http://www.python.org/peps/pep-0263.html for details".

Unfortunately I can't post my document yet (it's a research paper I'm
working on), but I'm sure you'll get the same result if you write up a
document in LibreOffice Writer and add some End Notes.

How do I automate the removal of all non-ascii characters from my code?

Thanks for all suggestions,

Alec Taylor
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How do I automate the removal of all non-ascii characters from my code?

[Gary Herron]

On 09/12/2011 12:49 AM, Alec Taylor wrote:

Good evening,

I have converted ODT to HTML using LibreOffice Writer, because I want
to convert from HTML to Creole using python-creole. Unfortunately I
get this error: "File "Convert to Creole.py", line 17
SyntaxError: Non-ASCII character '\xe2' in file Convert to Creole.py
on line 18, but no encoding declared; see
http://www.python.org/peps/pep-0263.html for details".

Unfortunately I can't post my document yet (it's a research paper I'm
working on), but I'm sure you'll get the same result if you write up a
document in LibreOffice Writer and add some End Notes.

How do I automate the removal of all non-ascii characters from my code?

Thanks for all suggestions,

Alec Taylor




This question does not quite make sense.  The error message is 
complaining about a python file.  What does that file have to do with 
ODT to HTML conversion and LibreOffice?


The error message means the python file (wherever it came from) has a 
non-ascii character (as you noted), and so it needs something to tell it 
what such a character means.  (That what the encoding is.)


A comment like this in line 1 or 2 will specify an encoding:
  # -*- coding:  -*-
but, we'll have to know more about the file "Convert to Creole.py" to 
guess what encoding name should be specified there.


You might try utf-8 or latin-1.


--
http://mail.python.org/mailman/listinfo/python-list

Re: How do I automate the removal of all non-ascii characters from my code?

[Alec Taylor]

from creole import html2creole

from BeautifulSoup import BeautifulSoup

VALID_TAGS = ['strong', 'em', 'p', 'ul', 'li', 'br', 'b', 'i', 'a', 'h1', 'h2']

def sanitize_html(value):

   soup = BeautifulSoup(value)

   for tag in soup.findAll(True):
       if tag.name not in VALID_TAGS:
           tag.hidden = True

   return soup.renderContents()
html2creole(u(sanitize_html('''Abstract
   
[more stuff here]
"""))

On Mon, Sep 12, 2011 at 6:17 PM, Gary Herron  wrote:
> On 09/12/2011 12:49 AM, Alec Taylor wrote:
>>
>> Good evening,
>>
>> I have converted ODT to HTML using LibreOffice Writer, because I want
>> to convert from HTML to Creole using python-creole. Unfortunately I
>> get this error: "File "Convert to Creole.py", line 17
>> SyntaxError: Non-ASCII character '\xe2' in file Convert to Creole.py
>> on line 18, but no encoding declared; see
>> http://www.python.org/peps/pep-0263.html for details".
>>
>> Unfortunately I can't post my document yet (it's a research paper I'm
>> working on), but I'm sure you'll get the same result if you write up a
>> document in LibreOffice Writer and add some End Notes.
>>
>> How do I automate the removal of all non-ascii characters from my code?
>>
>> Thanks for all suggestions,
>>
>> Alec Taylor
>
>
>
> This question does not quite make sense.  The error message is complaining
> about a python file.  What does that file have to do with ODT to HTML
> conversion and LibreOffice?
>
> The error message means the python file (wherever it came from) has a
> non-ascii character (as you noted), and so it needs something to tell it
> what such a character means.  (That what the encoding is.)
>
> A comment like this in line 1 or 2 will specify an encoding:
>  # -*- coding:  -*-
> but, we'll have to know more about the file "Convert to Creole.py" to guess
> what encoding name should be specified there.
>
> You might try utf-8 or latin-1.
>
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How do I automate the removal of all non-ascii characters from my code?

[John-John Tedro]

On Mon, Sep 12, 2011 at 8:17 AM, Gary Herron wrote:

> On 09/12/2011 12:49 AM, Alec Taylor wrote:
>
>> Good evening,
>>
>> I have converted ODT to HTML using LibreOffice Writer, because I want
>> to convert from HTML to Creole using python-creole. Unfortunately I
>> get this error: "File "Convert to Creole.py", line 17
>> SyntaxError: Non-ASCII character '\xe2' in file Convert to Creole.py
>> on line 18, but no encoding declared; see
>> http://www.python.org/peps/**pep-0263.htmlfor
>>  details".
>>
>> Unfortunately I can't post my document yet (it's a research paper I'm
>> working on), but I'm sure you'll get the same result if you write up a
>> document in LibreOffice Writer and add some End Notes.
>>
>> How do I automate the removal of all non-ascii characters from my code?
>>
>> Thanks for all suggestions,
>>
>> Alec Taylor
>>
>
>
>
> This question does not quite make sense.  The error message is complaining
> about a python file.  What does that file have to do with ODT to HTML
> conversion and LibreOffice?
>
> The error message means the python file (wherever it came from) has a
> non-ascii character (as you noted), and so it needs something to tell it
> what such a character means.  (That what the encoding is.)
>
> A comment like this in line 1 or 2 will specify an encoding:
>  # -*- coding:  -*-
> but, we'll have to know more about the file "Convert to Creole.py" to guess
> what encoding name should be specified there.
>
> You might try utf-8 or latin-1.
>
>
>
> --
> http://mail.python.org/**mailman/listinfo/python-list
>

If you are having trouble figuring out which encoding your file has, the
"file" util is often a quick and dirty solution.

#> echo "åäö" > test.txt
#> file test.txt
test.txt: UTF-8 Unicode text
#> iconv test.txt -f utf-8 -t latin1 > test.l1.txt
#> file test.l1.txt
test.l1.txt: ISO-8859 text

Note: I use latin1 (iso-8859-1) because it can describe the characters 'å',
'ä', 'ö'. Your encoding might be different depending on what system you are
using.

The gist is that if you specify the correct encoding as mentioned above with
the "coding"-comment, your program will probably (ish) run as intended.

-- John-John Tedro
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How do I automate the removal of all non-ascii characters from my code?

[Stefan Behnel]

Alec Taylor, 12.09.2011 10:33:

from creole import html2creole

from BeautifulSoup import BeautifulSoup

VALID_TAGS = ['strong', 'em', 'p', 'ul', 'li', 'br', 'b', 'i', 'a', 'h1', 'h2']

def sanitize_html(value):

soup = BeautifulSoup(value)

for tag in soup.findAll(True):
if tag.name not in VALID_TAGS:
tag.hidden = True

return soup.renderContents()
html2creole(u(sanitize_html('''Abstract

[more stuff here]
"""))


Hi,

I'm not sure what you are trying to say with the above code, but if it's 
the code that fails for you with the exception you posted, I would guess 
that the problem is in the "[more stuff here]" part, which likely contains 
a non-ASCII character. Note that you didn't declare the source file 
encoding above. Do as Gary told you.


Stefan

--
http://mail.python.org/mailman/listinfo/python-list

Re: PC locks up with list operations

[Steven D'Aprano]

On Wed, 31 Aug 2011 10:33 pm Steven D'Aprano wrote:

> Twice in a couple of weeks, I have locked up my PC by running a Python 2.5
> script that tries to create a list that is insanely too big.
> 
> In the first case, I (stupidly) did something like:
> 
> mylist = [0]*12345678901234
[...]
> Apart from "Then don't do that!", is there anything I can do to prevent
> this sort of thing in the future? Like instruct Python not to request more
> memory than my PC has?


For anyone who may care, I can report that ulimit under Linux will help with
this situation.

[steve@wow-wow ~]$ ulimit -v 20
[steve@wow-wow ~]$ python
Python 2.5 (r25:51908, Nov  6 2007, 16:54:01)
[GCC 4.1.2 20070925 (Red Hat 4.1.2-27)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> L = range(1)
Traceback (most recent call last):
  File "", line 1, in 
MemoryError

(Of course, I would have eventually got a MemoryError even without the
ulimit. *Eventually.* After much thrashing and processes crashing and
pain.)


Does anyone else think it would be useful for Python's memory manager to
enforce user-settable limits? Even just a simple thing like "never try to
allocate more than N bytes at once" would probably go a long way to prevent
a certain class of accidental (or deliberate) DOSes.


-- 
Steven

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How do I automate the removal of all non-ascii characters from my code?

[Steven D'Aprano]

On Mon, 12 Sep 2011 06:43 pm Stefan Behnel wrote:

> I'm not sure what you are trying to say with the above code, but if it's
> the code that fails for you with the exception you posted, I would guess
> that the problem is in the "[more stuff here]" part, which likely contains
> a non-ASCII character. Note that you didn't declare the source file
> encoding above. Do as Gary told you.

Even with a source code encoding, you will probably have problems with
source files including \xe2 and other "bad" chars. Unless they happen to
fall inside a quoted string literal, I would expect to get a SyntaxError.

I have come across this myself. While I haven't really investigated in great
detail, it appears to happen when copying and pasting code from a document
(usually HTML) which uses non-breaking spaces instead of \x20 space
characters. All it takes is just one to screw things up.



-- 
Steven

-- 
http://mail.python.org/mailman/listinfo/python-list

Terminating an embedded interpreter

[Ulrich Eckhardt]

Hi!

I'm trying to provide some scripting capabilities to a program. For that, 
I'm embedding a Python interpreter, running a script in a separate thread to 
decouple it from the UI.

Now, how should I handle rogue scripts? For example, when a script hangs in 
an endless loop, how do I signal the Python interpreter to shut down? In 
other words, I want to trigger something like what Control-C does in a 
"normal" environment.

In case it matters, the OS here is MS Windows.

Thanks!

Uli

-- 
Domino Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

-- 
http://mail.python.org/mailman/listinfo/python-list

working with a large file

[Rita]

I have a large file, 18gb uncompressed, and I would like to know what is the
preferred method to read this file for random access. I have several
processes reading the file which different calculate arguments. My server
has 64gb of memory. Not sure what is the preferred way to do this?




-- 
--- Get your facts first, then you can distort them as you please.--
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Terminating an embedded interpreter

[Chris Angelico]

On Mon, Sep 12, 2011 at 8:00 PM, Ulrich Eckhardt
 wrote:
> Now, how should I handle rogue scripts? For example, when a script hangs in
> an endless loop, how do I signal the Python interpreter to shut down? In
> other words, I want to trigger something like what Control-C does in a
> "normal" environment.
>

You can use PyErr_SetInterrupt to raise KeyboardInterrupt, but it can
be caught by the script. There's no guaranteed way, short of killing
the process.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: working with a large file

[Wolfgang Rohdewald]

On Montag 12 September 2011, Rita wrote:
> I have a large file, 18gb uncompressed, and I would like to
> know what is the preferred method to read this file for
> random access. I have several processes reading the file
> which different calculate arguments. My server has 64gb of
> memory. Not sure what is the preferred way to do this?

if the data is structured, you could store it in 
something like sqlite

-- 
Wolfgang
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Terminating an embedded interpreter

[Steven D'Aprano]

Ulrich Eckhardt wrote:

> Hi!
> 
> I'm trying to provide some scripting capabilities to a program. For that,
> I'm embedding a Python interpreter, running a script in a separate thread
> to decouple it from the UI.
> 
> Now, how should I handle rogue scripts? For example, when a script hangs
> in an endless loop, how do I signal the Python interpreter to shut down?

If you are using threads, they all run within the same Python process. You
can ask a thread to shut down, but you can't force it to from the outside.
If the thread runs a script that goes rogue, the script may never return
control to the thread long enough for it to respond to your request.

The main UI loop can still kill itself, and take all the threads with it, by
calling sys.exit. Or if you really need to, os.abort or os._exit can be
used for immediate "terminate yourself NOW!!!" commands. (But read the docs
for them first.)

A better way may be to run the script in a separate process. You can kill
the process the same way you would any other rogue process: by sending it a
signal. See the os and subprocess modules.

But Python's sandboxing abilities are not great. If you really fear rogue,
or malicious, scripts, perhaps Python is not the right language for this
task. Otherwise, just trust the script to be sane.



-- 
Steven

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PC locks up with list operations

[Roy Smith]

In article <4e6dc66e$0$29986$c3e8da3$54964...@news.astraweb.com>,
 Steven D'Aprano  wrote:

> > mylist = [0]*12345678901234
> [...]
> > Apart from "Then don't do that!", is there anything I can do to prevent
> > this sort of thing in the future? Like instruct Python not to request more
> > memory than my PC has?
> 
> 
> For anyone who may care, I can report that ulimit under Linux will help with
> this situation.
> [...]
> Does anyone else think it would be useful for Python's memory manager to
> enforce user-settable limits?

Not me.  You've already discovered that ulimit does exactly what you 
want.  Why would be gained by having Python duplicate this functionality?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How do I automate the removal of all non-ascii characters from my code?

[Dave Angel]

On 01/-10/-28163 02:59 PM, Steven D'Aprano wrote:

On Mon, 12 Sep 2011 06:43 pm Stefan Behnel wrote:


I'm not sure what you are trying to say with the above code, but if it's
the code that fails for you with the exception you posted, I would guess
that the problem is in the "[more stuff here]" part, which likely contains
a non-ASCII character. Note that you didn't declare the source file
encoding above. Do as Gary told you.

Even with a source code encoding, you will probably have problems with
source files including \xe2 and other "bad" chars. Unless they happen to
fall inside a quoted string literal, I would expect to get a SyntaxError.

I have come across this myself. While I haven't really investigated in great
detail, it appears to happen when copying and pasting code from a document
(usually HTML) which uses non-breaking spaces instead of \x20 space
characters. All it takes is just one to screw things up.




For me, more common than non-breaking space is the "smart quotes" 
characters.  In that case, one probably doesn't want to delete them, but 
instead convert them into standard quotes.


DaveA
--
http://mail.python.org/mailman/listinfo/python-list

Re: Terminating an embedded interpreter

[Ulrich Eckhardt]

Steven D'Aprano wrote:
> Ulrich Eckhardt wrote:
>> I'm trying to provide some scripting capabilities to a program. For that,
>> I'm embedding a Python interpreter, running a script in a separate thread
>> to decouple it from the UI.
>> 
>> Now, how should I handle rogue scripts? For example, when a script hangs
>> in an endless loop, how do I signal the Python interpreter to shut down?
> 
> If you are using threads, they all run within the same Python process. You
> can ask a thread to shut down, but you can't force it to from the outside.
> If the thread runs a script that goes rogue, the script may never return
> control to the thread long enough for it to respond to your request.

Sorry, I described badly what I was doing. The program itself is written in 
C++ and I'm running the Python interpreter in a thread separate to the UI, 
just in order to not hang the UI if anything in the interpreter blocks for 
extended amounts of time. I know that a Python thread is not a "system" 
thread but handled and scheduled internally by Python.


> The main UI loop can still kill itself, and take all the threads with it,
> by calling sys.exit. Or if you really need to, os.abort or os._exit can be
> used for immediate "terminate yourself NOW!!!" commands. (But read the
> docs for them first.)

> A better way may be to run the script in a separate process. You can kill
> the process the same way you would any other rogue process: by sending it
> a signal. See the os and subprocess modules.

Yes, a separate process would be much cleaner, but it would complicate 
communication back to the parent process. The Python code is supposed to 
call a few functions I exported. I'd have to tunnel these calls through 
stdin/stdout/stderr and that is more than what I'm willing to do at the 
moment. It does sound intriguing though, since that would allow me to embed 
any scripting language, not just Python.


> But Python's sandboxing abilities are not great. If you really fear rogue,
> or malicious, scripts, perhaps Python is not the right language for this
> task. Otherwise, just trust the script to be sane.

I'm not fearing malicious scripts. I'm also not concerned about rare hangs 
since this in an internal tool. In this case, I'd take a simple 99% solution 
over a complex 100% solution.


Thank you for your thoughts, Steven!

Uli

-- 
Domino Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Terminating an embedded interpreter

[Ulrich Eckhardt]

Chris Angelico wrote:
> You can use PyErr_SetInterrupt to raise KeyboardInterrupt

This sounds useful. Just to make sure, this would be called from a different 
thread than the one running the Python script, is that still OK?

> , but it can be caught by the script. There's no guaranteed way,
> short of killing the process.

This will do for my plans, I'm not trying to defend against anything 
malicious.


Thanks you!

Uli


-- 
Domino Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PC locks up with list operations

[Nobody]

On Wed, 31 Aug 2011 22:47:59 +1000, Steven D'Aprano wrote:

>> Linux seems to fair badly when programs use more memory than physically
>> available. Perhaps there's some per-process thing that can be used to
>> limit things on Linux?
> 
> As far as I know, ulimit ("user limit") won't help. It can limit the amount
> of RAM available to a process, but that just makes the process start using
> virtual memory more quickly. It can also limit the amount of virtual memory
> used by the shell, but not of other processes. In other words, Linux will
> try really, really, really hard to give you the 84 gigabytes you've asked
> for on a 2 GB system, even if it means DOSing your system for a month.
> 
> Of course, I would be very happy to learn I'm wrong.

Resource limits set by ulimit are inherited by any processes spawned from
the shell. They also affect the shell itself, but a shell process
shouldn't require a lot of resources. You can use a subshell if you want
to impose limits on a specific process.

For Python, setting the limit on virtual memory (RAM + swap) to no more
than the amount of physical RAM is probably a wise move. Some processes
can use swap effectively, but the typical Python program probably can't.
There are exceptions, e.g. if most of the memory is accounted for by large
NumPy arrays and you're careful about the operations which are performed
upon them. But using large amounts of memory for many small objects is
almost bound to result in swap-thrashing.

One problem with doing this automatically (e.g. in .pythonrc) is the
inheritance issue; any processes spawned from the interpreter will also be
resource limited. Similarly, any binary libraries loaded into the
interpreter will be subject to the process' resource limits. Consequently,
there would be some advantages to the Python interpreter having its own
resource-limiting mechanism.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Portable locale usage

[Siniša Šegvić]

> From: "Laszlo Nagy" 
> To: "Siniša Šegvić" , python-list@python.org
> Sent: Friday, September 9, 2011 11:39:52 AM
> Subject: Re: Portable locale usage
> 
> Looks like you have found a bug! :-) Why don't you post a bug report?

I just did:

http://bugs.python.org/issue12964

Thanks everyone for helping me to sort this out!

Sinisa
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Terminating an embedded interpreter

[Chris Angelico]

On Mon, Sep 12, 2011 at 10:05 PM, Ulrich Eckhardt
 wrote:
> Chris Angelico wrote:
>> You can use PyErr_SetInterrupt to raise KeyboardInterrupt
>
> This sounds useful. Just to make sure, this would be called from a different
> thread than the one running the Python script, is that still OK?
>
>> , but it can be caught by the script. There's no guaranteed way,
>> short of killing the process.
>
> This will do for my plans, I'm not trying to defend against anything
> malicious.

Yes, that would be what you want then. The main thing to take care of
is a blanket 'except' that doesn't specify what it's accepting - it'll
snarf the KeyboardInterrupt and carry on its merry way.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Idioms combining 'next(items)' and 'for item in items:'

[Duncan Booth]

Terry Reedy  wrote:

> The statement containing the explicit next(items) call can optionally be 
> wrapped to explicitly handle the case of an empty iterable in whatever 
> manner is desired.
> 
> try:
>  
> except StopIteration:
>  raise ValueError("iterable cannot be empty")
> 
> 
Alternatively, if all you want is for an empty iterable to do nothing, you 
could write it like this:

items = iter(iterable)
for first in items:

break
for item in items:


However, the issue I have with any of this pulling the first element out of 
the loop is that if you want special processing for the first element you 
are likely to also want it for the last, and if it is a single item you need 
to process that item with both bits of special code. I don't see how that works 
unless you have all elements within the single loop and test for first/last.

> 2. Process the last item of an iterable differently. As far as I know, 
> this cannot be done for a generic iterable with a flag. It requires a 
> look ahead.

I think that must be correct, e.g. if reading from an interactive prompt you 
cannot detect end of input until you fail to read any more.

See my answer to 
http://stackoverflow.com/questions/7365372/is-there-a-pythonic-way-of-knowing-when-the-first-and-last-loop-in-a-for-is-being/7365552#7365552
for a generator that wraps the lookahead.

-- 
Duncan Booth http://kupuguy.blogspot.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How do I automate the removal of all non-ascii characters from my code?

[Miki Tebeka]

You can add "# coding=UTF8" to the top of your file (see 
http://www.python.org/dev/peps/pep-0263/).

Of you want to remove unicode, there are several options, one of them is 
passing the file through "iconv --to ascii".
-- 
http://mail.python.org/mailman/listinfo/python-list

Easiest framework to develop simple interactive web site in python?

[John Reid]

Hi,

I need to write a web interface for some computational biology software 
I've written:

http://sysbio.mrc-bsu.cam.ac.uk/johns/STEME/rst/_build/html/index.html

I don't have much experience writing web sites or applications. Can 
anyone recommend a python framework that will allow me to easily write a 
few pages? I need to accept some user input in the form of some options 
and download a potentially large file from the user's computer. The job 
can take some time to run so I'll need to notify them via email when it 
has finished.


I should say our group already uses an Apache web server so I'd like to 
host the pages from that server.


I know there are various python web frameworks but I don't want to learn 
a complicated one, I really just want the simplest tool for the job. Are 
any of the following suitable?


Zope, Django, Pylons, Grok, Pyramid

I see quite a few are listed here:
http://wiki.python.org/moin/WebFrameworks

Thanks in advance for any help,
John.

--
http://mail.python.org/mailman/listinfo/python-list

Re: working with a large file

[Stefan Behnel]

Rita, 12.09.2011 12:34:

I have a large file, 18gb uncompressed, and I would like to know what is the
preferred method to read this file for random access. I have several
processes reading the file which different calculate arguments. My server
has 64gb of memory. Not sure what is the preferred way to do this?


It depends on the content (and likely also the operating system), but you 
might want to take a look at the mmap module.


Stefan

--
http://mail.python.org/mailman/listinfo/python-list

Re: How do I automate the removal of all non-ascii characters from my code?

[jmfauth]

On 12 sep, 10:17, Gary Herron  wrote:
> On 09/12/2011 12:49 AM, Alec Taylor wrote:
>
>
>
> > Good evening,
>
> > I have converted ODT to HTML using LibreOffice Writer, because I want
> > to convert from HTML to Creole using python-creole. Unfortunately I
> > get this error: "File "Convert to Creole.py", line 17
> > SyntaxError: Non-ASCII character '\xe2' in file Convert to Creole.py
> > on line 18, but no encoding declared; see
> >http://www.python.org/peps/pep-0263.htmlfor details".
>
> > Unfortunately I can't post my document yet (it's a research paper I'm
> > working on), but I'm sure you'll get the same result if you write up a
> > document in LibreOffice Writer and add some End Notes.
>
> > How do I automate the removal of all non-ascii characters from my code?
>
> > Thanks for all suggestions,
>
> > Alec Taylor
>

The coding of the characters is a domain per se.
It is independent from any OS's or applications.

When working with (plain) text files, you should
always be aware about the coding of the text you
are working on. If you are using coding directives,
you must ensure your coding directive matches
the real coding of the text files. A coding
directive is only informative, it does not set
the coding.

I'm pretty sure, you problem comes from this. There
is a mismatch somewhere, you are not aware of.
Removing ascii chars is certainly not a valuable
solution. It must work. If your are working
properly, it can not, not work.

Frome a linguistic point of view, the web has informed
me Creole (*all the Creoles*) can be composed with
the iso-8859-1 coding. That means, iso-8859-1, cp1252 and
all Unicode coding variants are possible coding directives.

jmf

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How do I automate the removal of all non-ascii characters from my code?

[Eric Snow]

On Mon, Sep 12, 2011 at 1:49 AM, Alec Taylor  wrote:
> Good evening,
>
> I have converted ODT to HTML using LibreOffice Writer, because I want
> to convert from HTML to Creole using python-creole. Unfortunately I
> get this error: "File "Convert to Creole.py", line 17
> SyntaxError: Non-ASCII character '\xe2' in file Convert to Creole.py
> on line 18, but no encoding declared; see
> http://www.python.org/peps/pep-0263.html for details".
>
> Unfortunately I can't post my document yet (it's a research paper I'm
> working on), but I'm sure you'll get the same result if you write up a
> document in LibreOffice Writer and add some End Notes.
>
> How do I automate the removal of all non-ascii characters from my code?

Perhaps try "The Unicode Hammer".

http://code.activestate.com/recipes/251871/

-eric

>
> Thanks for all suggestions,
>
> Alec Taylor
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How do I automate the removal of all non-ascii characters from my code?

[jmfauth]

On 12 sep, 10:49, Steven D'Aprano  wrote:
>
> Even with a source code encoding, you will probably have problems with
> source files including \xe2 and other "bad" chars. Unless they happen to
> fall inside a quoted string literal, I would expect to get a SyntaxError.
>

This is absurd and a complete non sense. The purpose
of a coding directive is to inform the engine, which
is processing a text file, about the "language" it
has to speak. Can be a html, py or tex file.
If you have problem, it's probably a mismatch between
your coding directive and the real coding of the
file. Typical case: ascii/utf-8 without signature.

jmf
-- 
http://mail.python.org/mailman/listinfo/python-list

Could you please give me some advise on this piece of code?

[G.]

Dear all,
I am a python newbie, but I intend to program a python script that
communicates with Labview via a UDP socket.
I managed to send LabView strings, but the other way around is not
working.
The code seems to stop working while in the first while loop. I can
see the word "test" being print out.

When I start LabView UDP_sender.vi it is supposed to send a string
back to python, but sometimes it ends with an error saying the port
and the ip is already in usage. Do I have to start first LabView or
the python scrip when listening to LabView, please?

It would be great if you could have a look on my code and could maybe
see why it does nothing after "print test", please.

Thank you very much.

Kind regards,
G.



# To change this template, choose Tools | Templates
# and open the template in the editor.

__author__="User"
__date__ ="$11.09.2011 19:34:03$"

if __name__ == "__main__":
print "Das ist ein Test"

import socket
import time

#Define global variables for UDP_IP and UDP_Port, needs to be changed
for PETRA-3
#global UDP_IP
UDP_IP="127.0.0.1"
print "UDP_IP is", UDP_IP

#global UDP_PORT
UDP_PORT=21312
print "UDP_PORT is", UDP_PORT





def openUDPSocket(UDP_IP,UDP_PORT):
#UDP_IP="127.0.0.1"
#UDP_PORT=5005
print "Creating socket ..."
sock=socket.socket( socket.AF_INET, # Internet
socket.SOCK_DGRAM ) # UDP
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
#sock.setblocking(0)
#sock.bind((UDP_IP,UDP_PORT))
#sock.listen(1)
print "Done."
return sock

def fastPump(sock):
# Start pumping and return
MESSAGE="pumpfast"

   # print "UDP target IP:", UDP_IP
   # print "UDP target port:", UDP_PORT
   # print "message:", MESSAGE

sock.sendto( MESSAGE, (UDP_IP, UDP_PORT) )

def slowPump(sock):
MESSAGE="pumpslow"
sock.sendto( MESSAGE, (UDP_IP, UDP_PORT) )


pumpsocketUDP=openUDPSocket(UDP_IP,UDP_PORT)


# Receive messages
counter_while_loop=0
print "test"
while counter_while_loop < 3:
data,addr = pumpsocketUDP.recvfrom(1024)
print "test"
if not data:
print "Client has exited!"
break
else:
print "\nReceived message '", data,"'"
counter_while_loop=counter_while_loop+1


counter_while_loop=0

while counter_while_loop < 3:
fastPump(pumpsocketUDP)
time.sleep(5)
slowPump(pumpsocketUDP)
time.sleep(3)

counter_while_loop=counter_while_loop+1
print "Counter_while_loop", counter_while_loop


# Close socket
pumpsocketUDP.close()
-- 
http://mail.python.org/mailman/listinfo/python-list

ipython -wthread vs. ipython -pylab

[Wolfram Brenig]

Hi,

I have a problem with the ipython shell: frequently, within one session 
I would like to do 2D plotting, using  matplotlib, as well as 3D 
visualization using mayavi


The man page for ipython, matplotlib, and mayavi tell me, that I must 
invoke ipython with

ipython -wthread for mayavi and
ipython -pylab for matplotlib
in order to avoid blocking (which indeed happens if I don't
use those options).
However, ipython's manpage also states, that only one of -wthread or 
-pylab can be given?


Does that imply I have to get out of the shell each time I want to 
switch from matplotlib to mayavi usage? Most likely no. But what should 
I do?


Thanks for your help

Wolfram
--
http://mail.python.org/mailman/listinfo/python-list

Re: Easiest framework to develop simple interactive web site in python?

[Matthias Huening]

Am 12.09.2011 16:03, schrieb John Reid:


I don't have much experience writing web sites or applications. Can
anyone recommend a python framework that will allow me to easily write a
few pages?


You want a simple framework? Try Bottle:
http://bottlepy.org/

Matthias
--
http://mail.python.org/mailman/listinfo/python-list

Re: recursive algorithm for balls in numbered boxes

[Dr. Phillip M. Feldman]

Mark Dickinson-2 wrote:
> 
> 
> This is a well-known trick:  to divide 5 (unlabeled) balls amongst 3
> (labeled) boxes, you write down sequences of 5 o's and 2 x's, where
> the o's represent the 5 balls and the 'x's represent dividers:
> 
> ooxooxo  -> [2, 2, 1]
> xoooxoo  -> [0, 3, 2]
> 
> And 'combinations(7, 2)' yields successively all the possible
> different placements for the 2 dividers in the 7 symbols.
> 
> 
> This question seems to come up often enough (without the box size
> limit twist) that maybe it would be useful to include something like
> this recipe in the itertool documentation.
> 
> 
> For getting this into itertools, I'd suggest opening a feature request
> on bugs.python.org and assigning it to Raymond Hettinger.
> 
> --
> Mark
> -- 
> http://mail.python.org/mailman/listinfo/python-list
> 
> 

You are correct--the case without capacity limits can be handled using the
existing machinery in `itertools`.  BTW--That trick with the dividers is
discussed on page 38 of William Feller's classic text, "An Introduction to
Probability Theory and Its Applications".

As per your suggestion, I have opened a feature request and assigned it to
Raymond.  Thanks!
-- 
View this message in context: 
http://old.nabble.com/recursive-algorithm-for-balls-in-numbered-boxes-tp32440187p32449079.html
Sent from the Python - python-list mailing list archive at Nabble.com.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Easiest framework to develop simple interactive web site in python?

[Miki Tebeka]

I personally like CherryPy. But it all depends on your needs and style. I 
suggest you play with some of the packages and select one that you feel best 
with.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Idioms combining 'next(items)' and 'for item in items:'

[Ian Kelly]

On Sun, Sep 11, 2011 at 6:45 PM, Terry Reedy  wrote:
> whereas, you are right, it breaks it noisily in the body. So Ian's claim
> that StopIteration must be caught to avoid silent termination is not true.
> Thanks for pointing out what I saw but did not cognize the full implication
> of before. A better exception and an error message with an explaination
> might still be a good idea, though.

But you can't write the function under the assumption that it will
only be called from the function body.  The following is a slight
reorganization of your example that does exhibit the problem:

for title in map(fix_title, ['amazinG', 'a helL of a fiGHT', '', 'igNordEd']):
print(title)

Output:
amazing
a Hell of a Fight

Note that at first glance, my example would appear to be functionally
equivalent to yours -- I've merely pulled the fix_title call out of
the loop body and into the iterator.  But actually they produce
different results because fix_title misbehaves by not catching the
StopIteration.

Cheers,
Ian
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Idioms combining 'next(items)' and 'for item in items:'

[Ian Kelly]

On Mon, Sep 12, 2011 at 10:55 AM, Ian Kelly  wrote:
> But you can't write the function under the assumption that it will
> only be called from the function body.  The following is a slight
> reorganization of your example that does exhibit the problem:

s/function body/for-loop body/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Could you please give me some advise on this piece of code?

[Ian Kelly]

On Mon, Sep 12, 2011 at 8:51 AM, G.  wrote:
> When I start LabView UDP_sender.vi it is supposed to send a string
> back to python, but sometimes it ends with an error saying the port
> and the ip is already in usage. Do I have to start first LabView or
> the python scrip when listening to LabView, please?

If LabView is sending, then the Python script should already be
receiving, so start Python first.

> def openUDPSocket(UDP_IP,UDP_PORT):
>    #UDP_IP="127.0.0.1"
>    #UDP_PORT=5005
>    print "Creating socket ..."
>    sock=socket.socket( socket.AF_INET, # Internet
>                        socket.SOCK_DGRAM ) # UDP

I don't think this is sufficient to create a UDP socket.  To be safe,
you should specify the protocol as well:

sock = socket.socket(socket.AF_INET,
socket.SOCK_DGRAM, socket.IPPROTO_UDP)

>    #sock.bind((UDP_IP,UDP_PORT))

Your bind call is commented out, which means that your socket will be
bound to an arbitrary port selected by the OS.  This will make it hard
for LabView to send it messages, since it won't know what port to use.

Also, you appear to be using the same address and port for both
endpoints.  The UDP socket in Python and the UDP socket in LabView
should be bound to two separate ports.  This is probably why you were
getting the "port already in use" error, before you commented this
out.

>    #sock.listen(1)

You don't need this at all.  listen() is for TCP sockets.

> def fastPump(sock):
>    # Start pumping and return
>    MESSAGE="pumpfast"
>
>   # print "UDP target IP:", UDP_IP
>   # print "UDP target port:", UDP_PORT
>   # print "message:", MESSAGE
>
>    sock.sendto( MESSAGE, (UDP_IP, UDP_PORT) )

The IP and port here should be that of the destination, i.e. the
address of the LabView socket.

> counter_while_loop=0
> print "test"
> while counter_while_loop < 3:
>        data,addr = pumpsocketUDP.recvfrom(1024)
>        print "test"
>        if not data:
>                print "Client has exited!"
>                break
>        else:
>                print "\nReceived message '", data,"'"
>        counter_while_loop=counter_while_loop+1

You should really use a for loop here:

for counter in range(3):
data, addr = pumpsocketUPD.recvfrom(1024)
print "Received message %r" % data

Note that if data is empty, it means that it received an empty
message, not that the client has exited.  It may be that you're having
the LabView client send an empty message when it exits, but this does
not happen automatically.

HTH,
Ian
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: convert time

[nn]

On Sep 11, 1:00 am, Steven D'Aprano  wrote:
> 守株待兔 wrote:
> > how can i convert "Dec 11" into  2011-12?
>
> if my_str == "Dec 11":
>     return 1999  # 2011 - 12
>
> Does that help?
>
> But seriously... 2011-12 is not a proper date, so the simplest way is
> probably something like this:
>
> def convert(date_str):
>     month, short_year = date_str.split()
>     if len(short_year) == 4:
>         year = short_year
>     else:
>         year = '20' + short_year
>     months = 'Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec'.split()
>     month = months.index(month) + 1
>     return year + '-' + str(month)
>
> Otherwise see the time and datetime modules:
>
> http://docs.python.org/library/time.htmlhttp://docs.python.org/library/datetime.html
>
> --
> Steven

Just a small comment that you can get "months" by doing:

from calendar import month_abbr
months = list(month_abbr)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Easiest framework to develop simple interactive web site in python?

[Stefaan Himpe]

The simplest one to learn is web2py http://www.web2py.com
No configuration needed, just unpack and get started.
It also has very good documentation and tons of little examples to get 
things done.


The other options you mentioned are good too :)

--
http://mail.python.org/mailman/listinfo/python-list

Re: Idioms combining 'next(items)' and 'for item in items:'

[Terry Reedy]

On 9/12/2011 9:06 AM, Duncan Booth wrote:

Terry Reedy  wrote:


The statement containing the explicit next(items) call can optionally be
wrapped to explicitly handle the case of an empty iterable in whatever
manner is desired.

try:
  
except StopIteration:
  raise ValueError("iterable cannot be empty")



Alternatively, if all you want is for an empty iterable to do nothing,


To do nothing, just pass above. If the function does nothing, it returns 
None. In the fix_title function, it should return '', not None.



you could write it like this:

 items = iter(iterable)
 for first in items:
 
 break


I could, but I doubt I would ;-). Try...except StopIteration: pass is 
more explicit and less roundabout.



 for item in items:
 

However, the issue I have with any of this pulling the first element out of
the loop is that if you want special processing for the first element you
are likely to also want it for the last,


Likely? I would say occasionally. Sentences have first words; file have 
headers. Special processing for last items is only an issue if it 
*replaces* the normal processing, rather than following the normal 
processing.


> and if it is a single item you need

to process that item with both bits of special code. I don't see how that works
unless you have all elements within the single loop and test for first/last.


Like so, with tests:

def first_last_special(iterable):
print("\nIterable is",repr(iterable))
items = iter(iterable)
try:
first = next(items)
except StopIteration:
print('Nothing'); return
print(first, 'is the first item')
try:
current = next(items)
except StopIteration:
current = first
else:
for item in items:
print(current, 'is a middle item')
current = item
print(current, 'is the last item')

first_last_special('')
first_last_special('1')
first_last_special('12')
first_last_special('123')
first_last_special('12345')

--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: Python: Deleting specific words from a file.

[gry]

On Sep 9, 2:04 am, Terry Reedy  wrote:
> On 9/8/2011 9:09 PM, papu wrote:
>
>
>
> > Hello, I have a data file (un-structed messy file) from which I have
> > to scrub specific list of words (delete words).
>
> > Here is what I am doing but with no result:
>
> > infile = "messy_data_file.txt"
> > outfile = "cleaned_file.txt"
>
> > delete_list = ["word_1","word_2","word_n"]
> > new_file = []
> > fin=open(infile,"")
> > fout = open(outfile,"w+")
> > for line in fin:
> >      for word in delete_list:
> >          line.replace(word, "")
> >      fout.write(line)
> > fin.close()
> > fout.close()
>
> If you have very many words (and you will need all possible forms of
> each word if you do exact matches), The following (untested and
> incomplete) should run faster.
>
> delete_set = {"word_1","word_2","word_n"}
> ...
> for line in fin:
>      for word in line.split()
>          if word not in delete_set:
>              fout.write(word) # also write space and nl.
>
> Depending on what your file is like, you might be better with
> re.split('(\W+)', line). An example from the manual:
>  >>> re.split('(\W+)', '...words, words...')
> ['', '...', 'words', ', ', 'words', '...', '']
>
> so all non-word separator sequences are preserved and written back out
> (as they will not match delete set).
>
> --
> Terry Jan Reedy

re.sub is handy too:
import re
delete_list=('the','rain','in','spain')
regex =  re.compile('\W' + '|'.join(delete_list) + '\W')
infile='messy'
with open(infile, 'r') as f:
for l in f:
print regex.sub('', l)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python: Deleting specific words from a file.

[MRAB]

On 12/09/2011 20:49, gry wrote:

On Sep 9, 2:04 am, Terry Reedy  wrote:

On 9/8/2011 9:09 PM, papu wrote:




Hello, I have a data file (un-structed messy file) from which I have
to scrub specific list of words (delete words).



Here is what I am doing but with no result:



infile = "messy_data_file.txt"
outfile = "cleaned_file.txt"



delete_list = ["word_1","word_2","word_n"]
new_file = []
fin=open(infile,"")
fout = open(outfile,"w+")
for line in fin:
  for word in delete_list:
  line.replace(word, "")
  fout.write(line)
fin.close()
fout.close()


If you have very many words (and you will need all possible forms of
each word if you do exact matches), The following (untested and
incomplete) should run faster.

delete_set = {"word_1","word_2","word_n"}
...
for line in fin:
  for word in line.split()
  if word not in delete_set:
  fout.write(word) # also write space and nl.

Depending on what your file is like, you might be better with
re.split('(\W+)', line). An example from the manual:
  >>>  re.split('(\W+)', '...words, words...')
['', '...', 'words', ', ', 'words', '...', '']

so all non-word separator sequences are preserved and written back out
(as they will not match delete set).

--
Terry Jan Reedy


re.sub is handy too:
import re
delete_list=('the','rain','in','spain')
regex =  re.compile('\W' + '|'.join(delete_list) + '\W')


You need parentheses around the words (I'm using non-capturing
parentheses):

regex =  re.compile(r'\W(?:' + '|'.join(delete_list) + r')\W')

otherwise you'd get: '\Wthe|rain|in|spain\W'.

Even better is the word-boundary, in case there's no previous or next
character:

regex =  re.compile(r'\b(?:' + '|'.join(delete_list) + r')\b')

Raw string literals are recommended for regexes.


infile='messy'
with open(infile, 'r') as f:
 for l in f:
 print regex.sub('', l)

--
http://mail.python.org/mailman/listinfo/python-list

Re: Idioms combining 'next(items)' and 'for item in items:'

[Terry Reedy]

On 9/12/2011 12:55 PM, Ian Kelly wrote:

On Sun, Sep 11, 2011 at 6:45 PM, Terry Reedy  wrote:

whereas, you are right, it breaks it noisily in the body. So Ian's claim
that StopIteration must be caught to avoid silent termination is not true.
Thanks for pointing out what I saw but did not cognize the full implication
of before. A better exception and an error message with an explaination
might still be a good idea, though.


But you can't write the function under the assumption that it will
only be called from the function body.


Sigh. You are right.


 The following is a slight
reorganization of your example that does exhibit the problem:

for title in map(fix_title, ['amazinG', 'a helL of a fiGHT', '', 'igNordEd']):
 print(title)

Output:
amazing
a Hell of a Fight

Note that at first glance, my example would appear to be functionally
equivalent to yours -- I've merely pulled the fix_title call out of
the loop body and into the iterator.  But actually they produce
different results because fix_title misbehaves by not catching the
StopIteration.


You are right, non-iterators should not raise or pass on StopIteration. 
There are actually several reasons.


1. The manual defined StopIteration as meaning '[I have] no more values 
[to give you]'. This is only meaningful coming from an iterator.


2. Your particular point is that StopIteration is (almost) unique in 
being sometimes, but only sometimes, caught by the interpreter, rather 
than just by user except clauses. AttributeError is another, which has 
occasionally caused its own problems. But we cannot stop raising 
AttributeError while we can always catch StopIteration for explicit 
next() (and should outside of iterators).


3. In the case of grabbing the first item from an iterator, no first 
item is a boundary case for the expected, legal type of input. I believe 
boundary cases should be included in function specifications. While 
there may be a couple of choices as to response, that is much less than 
infinity. For fix_title, the choices are ValueError or ''. Any other 
return would be an error unless explicitly given in the specs. So the 
boundary case should be included in the test suite to exclude any other 
random response.


4. StopIteration is an artifact of the choice of implementation. Pulling 
the first item out before the loop is an alternative to a flag and 
testing within the loop. Such an implementation detail should not leak 
into the user's view of the function as an abstraction.


If fix_title were a generator function whose instances yielded fixed 
title words one at a time, then the bare next() would be correct (as you 
noted). But it is not, and the difference is important, more important 
than having 'minimal clean code'. Thank you for persisting until I saw 
that in this context.


--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: PC locks up with list operations

[Terry Reedy]

On 9/12/2011 7:40 AM, Roy Smith wrote:

In article<4e6dc66e$0$29986$c3e8da3$54964...@news.astraweb.com>,
  Steven D'Aprano  wrote:


mylist = [0]*12345678901234

[...]

Apart from "Then don't do that!", is there anything I can do to prevent
this sort of thing in the future? Like instruct Python not to request more
memory than my PC has?



For anyone who may care, I can report that ulimit under Linux will help with
this situation.
[...]
Does anyone else think it would be useful for Python's memory manager to
enforce user-settable limits?


Not me.  You've already discovered that ulimit does exactly what you
want.  Why would be gained by having Python duplicate this functionality?


Having it on Windows.

--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: What do you guys think about adding a method "to_json"

[Terry Reedy]

On 9/12/2011 12:34 AM, Juan Pablo Romero Méndez wrote:

Hello,

What do you guys think about adding a method "to_json" to dictionaries
and sequence types? Perhaps through a module import?


Negative. If this were added, why not to_yaml, to_marshal, to_pickle, 
to_zip, and so on. Better to have each storage or transfer class handle 
its own instance creation. The one to_x method that every class should 
have is to_string, which is spelled __str__ and inherited from object.


--
Terry Jan Reedy


--
http://mail.python.org/mailman/listinfo/python-list

Re: How do I automate the removal of all non-ascii characters from my code?

[Rhodri James]

On Mon, 12 Sep 2011 15:47:00 +0100, jmfauth  wrote:


On 12 sep, 10:49, Steven D'Aprano  wrote:


Even with a source code encoding, you will probably have problems with
source files including \xe2 and other "bad" chars. Unless they happen to
fall inside a quoted string literal, I would expect to get a  
SyntaxError.




This is absurd and a complete non sense. The purpose
of a coding directive is to inform the engine, which
is processing a text file, about the "language" it
has to speak. Can be a html, py or tex file.
If you have problem, it's probably a mismatch between
your coding directive and the real coding of the
file. Typical case: ascii/utf-8 without signature.


Now read what Steven wrote again.  The issue is that the program contains  
characters that are syntactically illegal.  The "engine" can be perfectly  
correctly translating a character as a smart quote or a non breaking space  
or an e-umlaut or whatever, but that doesn't make the character legal!


--
Rhodri James *-* Wildebeest Herder to the Masses
--
http://mail.python.org/mailman/listinfo/python-list

PyWart: Itertools module needs attention

[rantingrick]

#  Quote   #

# The itertools module is great HOWEVER i believe most #
# people are recreating the functionalities due to the #
# insanely cryptic and/or missing examples from each   #
# method   #


py> print itertools.chain.__doc__
chain(*iterables) --> chain object
Return a chain object whose .next() method returns elements from the
first iterable until it is exhausted, then elements from the next
iterable, until all of the iterables are exhausted.


#  Quote   #

# Okay not TOO bad however this simple example would   #
# suffice: #


py> list(itertools.chain([1,2], [3,[4,5],6]))
[1, 2, 3, [4, 5], 6]


#  Quote   #

# Same for these...#


py> ''.join(list(itertools.dropwhile(lambda x:x==" ", "hello
word")))
'hello word'
py> ''.join(list(itertools.takewhile(lambda x:x==" ", "hello
word")))
''
py> print itertools.compress.__doc__
compress(data, selectors) --> iterator over selected data
Return data elements corresponding to true selector elements.
Forms a shorter iterator from selected data elements using the
selectors to choose the data elements.


#  Quote   #

# WTF! Would you like to define a Python "selector". Could #
# it be that we should be using "selector function" or #
# "predicate function" instead?#

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PyWart: Itertools module needs attention

[Nick Stinemates]

On Mon, Sep 12, 2011 at 10:04 PM, rantingrick  wrote:

>
> 
> #  Quote   #
> 
> # The itertools module is great HOWEVER i believe most #
> # people are recreating the functionalities due to the #
> # insanely cryptic and/or missing examples from each   #
> # method   #
> 
>
> py> print itertools.chain.__doc__
> chain(*iterables) --> chain object
> Return a chain object whose .next() method returns elements from the
> first iterable until it is exhausted, then elements from the next
> iterable, until all of the iterables are exhausted.
>
> 
> #  Quote   #
> 
> # Okay not TOO bad however this simple example would   #
> # suffice: #
> 
>
> py> list(itertools.chain([1,2], [3,[4,5],6]))
> [1, 2, 3, [4, 5], 6]
>
> 
> #  Quote   #
> 
> # Same for these...#
> 
>
> py> ''.join(list(itertools.dropwhile(lambda x:x==" ", "hello
> word")))
> 'hello word'
> py> ''.join(list(itertools.takewhile(lambda x:x==" ", "hello
> word")))
> ''
> py> print itertools.compress.__doc__
> compress(data, selectors) --> iterator over selected data
> Return data elements corresponding to true selector elements.
> Forms a shorter iterator from selected data elements using the
> selectors to choose the data elements.
>
> 
> #  Quote   #
> 
> # WTF! Would you like to define a Python "selector". Could #
> # it be that we should be using "selector function" or #
> # "predicate function" instead?#
> 
> --
> http://mail.python.org/mailman/listinfo/python-list


I'm honestly missing the point of this mail.

Can you elaborate?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How do I automate the removal of all non-ascii characters from my code?

[Vlastimil Brom]

2011/9/12 Alec Taylor :
> Good evening,
>
> I have converted ODT to HTML using LibreOffice Writer, because I want
> to convert from HTML to Creole using python-creole. Unfortunately I
> get this error: "File "Convert to Creole.py", line 17
> SyntaxError: Non-ASCII character '\xe2' in file Convert to Creole.py
> on line 18, but no encoding declared; see
> http://www.python.org/peps/pep-0263.html for details".
>
> Unfortunately I can't post my document yet (it's a research paper I'm
> working on), but I'm sure you'll get the same result if you write up a
> document in LibreOffice Writer and add some End Notes.
>
> How do I automate the removal of all non-ascii characters from my code?
>
> Thanks for all suggestions,
>
> Alec Taylor
> --
> http://mail.python.org/mailman/listinfo/python-list
>

It would obviously help to see the content of the line mentioned in
the traceback (and probably its context);
however, that value seems to correspond with â in some European
encodings, in which case it would probably be part of some quoted
unicode/string literal. (at least in python 2, in python3 it could
also be a name of an object in python code, the traceback seems to be
the same for both cases.)
cf.
>>> print '\xe2'.decode("iso-8859-1")
â
# and likewise for iso-8859-... 1,2,3,4; 9, 10, 14, 15, 16, some
windows- encodings etc.

Possibly (as previouslz suggested) adding the encoding information
like iso-8859-1 or windows-1252 or others depending on other data etc.
at the top of the source file might fix the error. Which would be
certainly preferable to throwing all non ascii data away.
Zou would add e.g.
# -*- coding: iso-8859-1 -*-
as the first or second line of the file.

hth,
 vbr
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PyWart: Itertools module needs attention

[Ethan Furman]

Nick Stinemates wrote:

I'm honestly missing the point of this mail.



rantingrick is a well-known troll, and doesn't need to have a point.

Please do not feed the troll.

~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list

回复： strang thing:

[守株待兔]

i change  open into open1,it's ok now
import os
import  csv

for  name  in   os.listdir('/tmp/quote/'):
filename='/tmp/quote/'+name
file = open(filename,'r')
file.readline() 
for row in csv.reader(file):
(date,open1,high,low,close,vol,adjclose) = (row[0], row[1], row[2], 
row[3],row[4], row[5], row[6])
print   row[0], row[1], row[2], row[3],row[4], row[5], row[6]


but i want "open"  to be my data field,not  open1  to be my field ,
how can i do?
 
 
-- 原始邮件 --
发件人: "Chris Angelico";
发送时间: 2011年9月6日(星期二) 下午4:22
收件人: "python-list"; 

主题: Re: strang thing:

 
2011/9/6 守株待兔 <1248283...@qq.com>:
> file = open(filename,'r')

> when i  add(date,open,high,low,close,vol,adjclose) = (row[0], row[1],

You're assigning to the name "open", which is shadowing the built-in
of the same name. The second time through the loop, you're not calling
the usual open() function, you're trying to call your string. That's
what your error is telling you.

Hope that helps!

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list-- 
http://mail.python.org/mailman/listinfo/python-list

回复： strang thing:

[守株待兔]

i change  open into open1,it's ok now
import os
import  csv

for  name  in   os.listdir('/tmp/quote/'):
filename='/tmp/quote/'+name
file = open(filename,'r')
file.readline() 
for row in csv.reader(file):
(date,open1,high,low,close,vol,adjclose) = (row[0], row[1], row[2], 
row[3],row[4], row[5], row[6])
print   row[0], row[1], row[2], row[3],row[4], row[5], row[6]


but i want "open"  to be my data field,not  open1  to be my field ,
how can i do?
 
 
-- 原始邮件 --
发件人: "Chris Angelico";
发送时间: 2011年9月6日(星期二) 下午4:22
收件人: "python-list"; 

主题: Re: strang thing:

 
2011/9/6 守株待兔 <1248283...@qq.com>:
> file = open(filename,'r')

> when i  add(date,open,high,low,close,vol,adjclose) = (row[0], row[1],

You're assigning to the name "open", which is shadowing the built-in
of the same name. The second time through the loop, you're not calling
the usual open() function, you're trying to call your string. That's
what your error is telling you.

Hope that helps!

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list-- 
http://mail.python.org/mailman/listinfo/python-list

Re: strang thing:

[Steven D'Aprano]

(I sent this reply a week ago, but it seems to have disappeared. So trying
again.)


On Tue, 6 Sep 2011 06:18 pm 守株待兔 wrote:

> when i  add    (date,open,high,low,close,vol,adjclose) = (row[0], row[1],
> row[2], row[3],row[4], row[5], row[6]) change the code into

Here you define a new variable called "open", which has the value of row[1].
This shadows (hides) the built-in function also called "open", so later
when you do this:

>     file = open(filename,'r')
> TypeError: 'str' object is not callable

Python uses your string variable "open" instead of the built-in function.

The best solution is to avoid using the name "open", instead call it "open_"
(underscore at the end is the usual convention to avoid shadowing
built-ins). Or "open_value" or any other appropriate name.

Another solution is to save a reference to the open built-in first:

my_open = open
open = "something"
file = my_open("filename", "r")




-- 
Steven

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: strang thing:

[Chris Angelico]

On Tue, Sep 13, 2011 at 1:28 PM, Steven D'Aprano
 wrote:
> The best solution is to avoid using the name "open", instead call it "open_"
> (underscore at the end is the usual convention to avoid shadowing
> built-ins). Or "open_value" or any other appropriate name.
>

This is why every good programmer keeps a thesaurus handy. I just now
had a "concept collision" on the word 'cancel', and did a quick search
to come up with 'annul' as an alternative. Although in this case I
didn't search my thesaurus, I actually looked for Magic: The Gathering
cards... yeah, I'm a nerd, aren't you? :)

There's lots of synonyms for open, and it's entirely possible that one
will work. Otherwise, Steven's other solution works just fine too, and
you can go humorous with that too:

sesame = open
open = "something"
file = sesame("filename", "r")

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Should a beginner do some coding excises? How can I find the sources？

[sillyou su]

I'm reading "Learning Python"（ Chinese version）. Before I go through
the whole book, I want to do some excises matching each charter.
Any tips? Any better advice？

-- 
http://mail.python.org/mailman/listinfo/python-list

Default Admin user on weblogic domain deleted after executing create command using WLST

[Amit Jain]

Hello All,

we observed that user exactly deleted when 'create' cmd is executed:
filestore = create("wlstProperties.getProperty(fileStoreName)",
"FileStore")
jmsServer = create("AUAJMSServer", "JMSServer")
...
..
etc.

http://groups.google.com/group/comp.lang.java.programmer/browse_thread/thread/1927322accdb6e6e/223e1c7cd208a86c

Please suggestion...

regards, Amit J.
-- 
http://mail.python.org/mailman/listinfo/python-list

update all python packages on Debian/Ubuntu

[Jabba Laci]

Hi,

I use Ubuntu and the Python packages on my system were either
installed with (1) apt-get, or with (2) pip. Since the number of
python packages in the Ubuntu repositories is limited, I had to
install several packages with pip.

Now I want to upgrade the packages that were installed with pip (and
only those packages).

I can collect the list of python packages:
(pip freeze | cut -d = -f 1 | grep -v FIXME | xargs echo | tr ' ' '\n'
>list.txt) 2>/dev/null

However, if I update every package in list.txt with "pip install -U",
it will also update the ones that were installed with apt-get. Since
apt-get builds a database with the installed files, I'm sure it won't
like that pip replaces those files. I didn't try it yet but I'm afraid
it would mess up the system.

So, how to upgrade the python packages correctly?

Thanks,

Laszlo
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Should a beginner do some coding excises? How can I find the sources？

[memilanuk]

On 09/12/2011 09:20 PM, sillyou su wrote:

I'm reading "Learning Python"（ Chinese version）. Before I go through
the whole book, I want to do some excises matching each charter.
Any tips? Any better advice？




For the code examples, have you tried looking up the home page for the 
book?  Google for 'oreilly learning python' and find the correct edition 
that you have.


If its the 4th ed (current), you should end up on a page like this:

http://shop.oreilly.com/product/9780596158071.do

Down in the right hand side-bar, there should be a menu 'Essential 
Links' and one of the options is 'Download code' or something along 
those lines.  The link should take you to a zip file with all the code 
examples in the book.


As far as practice exercises... maybe something like 
codingbat.com/python would be helpful.  Its not related to the book at 
all, and doesn't go nearly as in depth... but its kind of neat to play 
with and see how your code works when someone else is grading it! (at 
least for me).


HTH,

Monte

--
http://mail.python.org/mailman/listinfo/python-list

Get careers in Management work.

[gaurav]

Wide ranges of careers opportunity. Management careers.
http://rojgars1.webs.com/hrm.htm http://topcareer.webs.com/qualitymanagement.htm

HRM, PM, marketing manager jobs and accounts jobs move to your next
career start earning with manager level.
http://todayvacancy.blogspot.com/2011/07/account-assistant.html
http://freshersemployment.blogspot.com/2011/07/finance-management.html

High-quality careers. All types of Management careers.
http://sites.google.com/site/jobinmanagement/operation-management-jobs

-- 
http://mail.python.org/mailman/listinfo/python-list