Re: question about speed of sequential string replacement vs regex or
Xah Lee wrote: ) the question originally came from when i was coding elisp of a ) function that changes html entities to unicode char literal. The ) problem is slightly complicated, involving a few questions about speed ) in emacs. e.g. string vs buffer, and much more... i spent several ) hours on this but it's probably too boring to detail (but i'll do so ) if anyone wishes). But anyway, while digging these questions that's ) not clear in my mind, i thought of why not generate a regex or ) construct and do it in one shot, and wondered if that'd be faster. But ) afterwards, i realized this wouldn't be applicable to my problem ) because for my problem each string needs to be changed to a unique ) string, not all to the same string. In Perl, it would be applicable. You see, in Perl, you can call a function in the replacement of the regex substitution, which can then look up the html entity and return the wanted unicode literal. I think you can do that in some other languages as well. SaSW, Willem -- Disclaimer: I am in no way responsible for any of the statements made in the above text. For all I know I might be drugged or something.. No I'm not paranoid. You all think I'm paranoid, don't you ! #EOT -- http://mail.python.org/mailman/listinfo/python-list
Re: question about speed of sequential string replacement vs regex or
Eli the Bearded wrote: ) In comp.lang.perl.misc, Willem wrote: )> In Perl, it would be applicable. You see, in Perl, you can call a function )> in the replacement of the regex substitution, which can then look up the )> html entity and return the wanted unicode literal. ) ) A function? I'd use a hash. A function can return a sensible value for unknown substitutions. In the case where you prebuild a giant regex or-list, that is not an issue, but I would match html entities generically. SaSW, Willem -- Disclaimer: I am in no way responsible for any of the statements made in the above text. For all I know I might be drugged or something.. No I'm not paranoid. You all think I'm paranoid, don't you ! #EOT -- http://mail.python.org/mailman/listinfo/python-list
Re: Game design : Making computer play
v4vijayakumar wrote: ) On Apr 14, 12:35 pm, Richard Heathfield <[EMAIL PROTECTED]> wrote: )> v4vijayakumar said: )> > In computer based, two player, board games, how to make computer play? )> )> Write some code that works out what the computer player should do. If you )> want a better answer, ask a better question. ) ) I am just implementing a game played in my village (Tamilnadu / ) India), called "aadupuli" (goats and tigers). There are only 23 ) positions and 18 characters (15 goats and 3 tigers). Some of you might ) be aware of this. Odd, I thought there were 25 positions, 20 goats and 4 tigers. *googles* Oh wait, that is Bagh Chal. Roughly the same rules, different board layout and number of tigers. ) I can post initial version of the game (implemented using html/ ) javascript) in couple of hours here. ) ) Welcome any help in making computer to play one of these player. If the board is that small then an exhaustive search might work, but then the computer would be unbeatable. Minmax would be best I guess. SaSW, Willem -- Disclaimer: I am in no way responsible for any of the statements made in the above text. For all I know I might be drugged or something.. No I'm not paranoid. You all think I'm paranoid, don't you ! #EOT -- http://mail.python.org/mailman/listinfo/python-list
Re: Game design : Making computer play
Richard wrote: ) Here's the board (which bears only a slight resemblance to one I'd seen on ) the Web): ) ) +---+ ) | HORN $ | ) +---+---+---+---+---+---+ ) |L W| | $ | $ | |R W| ) +E-I+--CHEST+---+---+I-I+ ) |F N| | | | |G N| ) +T-G+---+---+---+---+H-G+ ) | | | | | |T | ) +---+---+---+---+---+---+ ) | LEGS| | | ) +---+---+---+---+ ) ) There are three tigers and fifteen goats. ) The tigers' goal is to eat all the goats and remain mobile. ) It seems that the initial tiger positions are: one on the horn, and one ) each on CHEST-2 and CHEST-3 (see $ marks, above). ) The goats' goal is to block the tigers from moving. ) The goats are placed one by one. ) Tigers appear only to be able to move orthogonally (up/down/left/right) - ) although they can use the horn to whizz across the chest (e.g. CHEST-1 to ) HORN, HORN to CHEST-4, in two moves). ) The rest of the rules are beyond me, I'm afraid. It's not clear how tigers ) eat goats or how goats block tigers. If it's similar to the 'other' goats and tigers game, a tiger eats a goat by jumping over it, for which the square behind it needs to be empty, obviously. SaSW, Willem -- Disclaimer: I am in no way responsible for any of the statements made in the above text. For all I know I might be drugged or something.. No I'm not paranoid. You all think I'm paranoid, don't you ! #EOT -- http://mail.python.org/mailman/listinfo/python-list
Re: Create a variable "on the fly"
Steve M: > >>> locals()['OSCAR'] = 'the grouch' > >>> OSCAR > 'the grouch' > >>> Use "globals", not "locals": globals()['OSCAR'] = 'the grouch' because <http://www.python.org/doc/current/lib/built-in-funcs.html> states: locals() Update and return a dictionary representing the current local symbol table. Warning: The contents of this dictionary should not be modified; changes may not affect the values of local variables used by the interpreter. Function globals() is not subject to this restriction. - Willem -- http://mail.python.org/mailman/listinfo/python-list
XML parsing per record
I want to parse a very large (2.4 gig) XML file (bioinformatics ofcourse :)) But I have no clue how to do that. Most things I see read the entire xml file at once. That isn't going to work here ofcourse. So I would like to parse a XML file one record at a time and then be able to store the information in another object. How should I do that? Thanks in advance, Willem Ligtenberg A total newbie to python by the way. -- http://mail.python.org/mailman/listinfo/python-list
Re: XML parsing per record
On Sun, 17 Apr 2005 02:16:04 +, William Park wrote: > Willem Ligtenberg <[EMAIL PROTECTED]> wrote: >> I want to parse a very large (2.4 gig) XML file (bioinformatics >> ofcourse :)) But I have no clue how to do that. Most things I see read >> the entire xml file at once. That isn't going to work here ofcourse. >> >> So I would like to parse a XML file one record at a time and then be >> able to store the information in another object. How should I do >> that? >> >> Thanks in advance, >> >> Willem Ligtenberg A total newbie to python by the way. > > You may want to try Expat (www.libexpat.org) or Python wrapper to it. > You can feed small piece at a time, say by lines or whatever. Of > course, it all depends on what kind of parsing you have in mind. :-) > > Care to post more details? The XML file I need to parse contains information about genes. So the first element is a gene and then there are a lot sub-elements with sub-elements. I only need some of the informtion and want to store it in my an object called gene. Lateron this information will be printed into a file, which in it's turn will be fed into some other program. This is an example of the XML 9996 1 LocusID 320632 GeneID 320632 2003 8 28 21 39 0 2005 2 17 12 54 0 6 1 1 Mus musculus house mouse taxon 10090 mouse Mus musculus Eukaryota; Metazoa; Chordata; Craniata; Vertebrata; Euteleostomi; Mammalia; Eutheria; Euarchontoglires; Glires; Rodentia; Sciurognathi; Muridae; Murinae; Mus 1 2 ROD LocusLink 9996 9996 1 0 LocusID 9996 LOC320632 -- http://mail.python.org/mailman/listinfo/python-list
Re: XML parsing per record
I'll first try it using SAX, because I want to have as little dependancies
as possible. I already have BioPython as a dependancy. And I personally
don't like to install lot's of packages for a program to work. So I don't
want to impose that on other people.
But thanks anyway and I might go for the cElementTree later on, if the
ordinary SAX proves to slow...
On Wed, 20 Apr 2005 08:03:00 -0400,
Kent Johnson wrote:
> Willem Ligtenberg wrote:
>>>Willem Ligtenberg <[EMAIL PROTECTED]> wrote:
>>>
>>>>I want to parse a very large (2.4 gig) XML file (bioinformatics
>>>>ofcourse :)) But I have no clue how to do that. Most things I see read
>>>>the entire xml file at once. That isn't going to work here ofcourse.
>>>>
>>>>So I would like to parse a XML file one record at a time and then be
>>>>able to store the information in another object. How should I do
>>>>that?
>>
>> The XML file I need to parse contains information about genes.
>> So the first element is a gene and then there are a lot sub-elements with
>> sub-elements. I only need some of the informtion and want to store it in
>> my an object called gene. Lateron this information will be printed into a
>> file, which in it's turn will be fed into some other program.
>> This is an example of the XML
>>
>> > "NCBI_Entrezgene.dtd">
>>
>>
>>
>>
>>
>
> This should get you started with cElementTree:
>
> import cElementTree as ElementTree
>
> source = 'Entrezgene.xml'
>
> for event, elem in ElementTree.iterparse(source):
> if elem.tag == 'Entrezgene':
> # Process the Entrezgene element
> geneid =
> elem.findtext('Entrezgene_track-info/Gene-track/Gene-track_geneid')
> print 'Gene id', geneid
>
> # Throw away the element, we're done with it
> elem.clear()
>
> Kent
--
http://mail.python.org/mailman/listinfo/python-list
Re: XML parsing per record
Sorry I just decided that I want to use your solution, but I am wondering
is cElemenTree in expat or is that something different?
On Wed, 20 Apr 2005
08:03:00 -0400, Kent Johnson wrote:
> Willem Ligtenberg wrote:
>>>Willem Ligtenberg <[EMAIL PROTECTED]> wrote:
>>>
>>>>I want to parse a very large (2.4 gig) XML file (bioinformatics
>>>>ofcourse :)) But I have no clue how to do that. Most things I see read
>>>>the entire xml file at once. That isn't going to work here ofcourse.
>>>>
>>>>So I would like to parse a XML file one record at a time and then be
>>>>able to store the information in another object. How should I do
>>>>that?
>>
>> The XML file I need to parse contains information about genes.
>> So the first element is a gene and then there are a lot sub-elements with
>> sub-elements. I only need some of the informtion and want to store it in
>> my an object called gene. Lateron this information will be printed into a
>> file, which in it's turn will be fed into some other program.
>> This is an example of the XML
>>
>> > "NCBI_Entrezgene.dtd">
>>
>>
>>
>>
>>
>
> This should get you started with cElementTree:
>
> import cElementTree as ElementTree
>
> source = 'Entrezgene.xml'
>
> for event, elem in ElementTree.iterparse(source):
> if elem.tag == 'Entrezgene':
> # Process the Entrezgene element
> geneid =
> elem.findtext('Entrezgene_track-info/Gene-track/Gene-track_geneid')
> print 'Gene id', geneid
>
> # Throw away the element, we're done with it
> elem.clear()
>
> Kent
--
http://mail.python.org/mailman/listinfo/python-list
Re: XML parsing per record
This is all the info I need from the xml file: ID --> 320632 Name --> Pzp Startbase --> 126957426 126989473 51860766 Endbase Function --> U5 snRNP-specific protein, 200 kDa U5 snRNP-specific protein, 200 kDa (DEXH RNA helicase family) DBLink --> MGI:201 GO 5524 ATP binding evidence: ISS Product-type --> 6 gene-comment --> activating signal cointegrator 1 complex subunit 3-like 1 synonym --> HELIC2 KIAA0788 U5-200KD U5-200-KD A330064G03Rik EC --> 1.5.1.5 3.5.4.9 Chromosome: 1 6 Some can happen more than once in a record. On Fri, 22 Apr 2005 02:41:46 -0400, William Park wrote: > Willem Ligtenberg <[EMAIL PROTECTED]> wrote: >> On Sun, 17 Apr 2005 02:16:04 +, William Park wrote: >> > Care to post more details? >> >> The XML file I need to parse contains information about genes. >> So the first element is a gene and then there are a lot sub-elements with >> sub-elements. I only need some of the informtion and want to store it in >> my an object called gene. Lateron this information will be printed into a >> file, which in it's turn will be fed into some other program. > > You have to help us a little more here. Which info do you want to > extract from below example? > >> >> ... >> -- http://mail.python.org/mailman/listinfo/python-list
Re: XML parsing per record
As I'm trying to write the code using cElementTree. I stumble across one problem. Sometimes there are multiple values to retrieve from one record for the same element. Like this: ATP-binding cassette, subfamily G, member 1 ATP-binding cassette 8 How do you get not only the first, but the rest as well, so that I can store it in a list. Thanks in advance, Willem Ligtenberg On Fri, 22 Apr 2005 13:48:15 +0200, Willem Ligtenberg wrote: > This is all the info I need from the xml file: > ID -->320632 > > Name --> > Pzp > > Startbase --> > > > > 126957426 > 126989473 > > > > > > 51860766 > > > > > > > Endbase > > Function --> > U5 snRNP-specific protein, 200 kDa > U5 snRNP-specific protein, 200 kDa (DEXH RNA helicase > family) > > > DBLink --> MGI:201 > > > > > GO > > > 5524 > > > > > ATP binding > evidence: > ISS > > > > Product-type --> 6 > > gene-comment --> activating signal cointegrator 1 complex > subunit 3-like > 1 > > synonym --> > HELIC2 > KIAA0788 > U5-200KD > U5-200-KD > A330064G03Rik > > > EC --> > 1.5.1.5 > 3.5.4.9 > > > Chromosome: > 1 > 6 > > > Some can happen more than once in a record. > > > On Fri, 22 Apr 2005 02:41:46 -0400, William Park wrote: > >> Willem Ligtenberg <[EMAIL PROTECTED]> wrote: >>> On Sun, 17 Apr 2005 02:16:04 +, William Park wrote: >>> > Care to post more details? >>> >>> The XML file I need to parse contains information about genes. >>> So the first element is a gene and then there are a lot sub-elements with >>> sub-elements. I only need some of the informtion and want to store it in >>> my an object called gene. Lateron this information will be printed into a >>> file, which in it's turn will be fed into some other program. >> >> You have to help us a little more here. Which info do you want to >> extract from below example? >> >>> >>> ... >>> -- http://mail.python.org/mailman/listinfo/python-list
Re: XML parsing per record
By the way, I know about findall, but when I iterate thruogh it like: for x in function: print 'function', x I get: function function But ofcourse I want the information in there... On Fri, 22 Apr 2005 15:22:17 +0200, Willem Ligtenberg wrote: > As I'm trying to write the code using cElementTree. > I stumble across one problem. Sometimes there are multiple values to > retrieve from one record for the same element. Like this: > ATP-binding cassette, subfamily G, member 1 > ATP-binding cassette 8 > > How do you get not only the first, but the rest as well, so that I can > store it in a list. > > Thanks in advance, > > Willem Ligtenberg > -- http://mail.python.org/mailman/listinfo/python-list
Re: XML parsing per record
As you can read in the other post of mine, my problem was with the
iterating through the list. didn't know that you should do. e.text. I did
only print e, not print e.text
Did read documentation, but must admit not everything.
Anyway, thank you very much!
On Fri, 22 Apr 2005 15:47:08 +0200, Fredrik Lundh wrote:
> Willem Ligtenberg wrote:
>
>> As I'm trying to write the code using cElementTree.
>> I stumble across one problem. Sometimes there are multiple values to
>> retrieve from one record for the same element. Like this:
>> ATP-binding cassette, subfamily G, member
>> 1
>> ATP-binding cassette 8
>>
>> How do you get not only the first, but the rest as well, so that I can
>> store it in a list.
>
> findall returns a list of matching elements. if "elem" is the paretnt
> element,
> this gives you a list of the text inside all Prot-ref_name_E child elements:
>
> [e.text for e in elem.findall("Prot-ref_name_E")]
>
> (you have read the elementtree documentation, I hope?)
>
>
--
http://mail.python.org/mailman/listinfo/python-list
XML file parsing with SAX
I decided to use SAX to parse my xml file.
But the parser crashes on:
File "/usr/lib/python2.3/site-packages/_xmlplus/sax/handler.py", line 38, in
fatalError
raise exception
xml.sax._exceptions.SAXParseException: NCBI_Entrezgene.dtd:8:0: error in
processing external entity reference
This is caused by:
If I remove it, it parses normally.
I've created my parser like this:
import sys
from xml.sax import make_parser
from handler import EntrezGeneHandler
fopen = open("mouse2.xml", "r")
ch = EntrezGeneHandler()
saxparser = make_parser()
saxparser.setContentHandler(ch)
saxparser.parse(fopen)
And the handler is:
from xml.sax import ContentHandler
class EntrezGeneHandler(ContentHandler):
"""
A handler to deal with EntrezGene in XML
"""
def startElement(self, name, attrs):
print "Start element:", name
So it doesn't do much yet. And still it crashes...
How can I tell the parser not to look at the DOCTYPE declaration.
On a website:
http://www.devarticles.com/c/a/XML/Parsing-XML-with-SAX-and-Python/1/
it states that the SAX parsers are not validating, so this error shouldn't
even occur?
Cheers,
Willem
--
http://mail.python.org/mailman/listinfo/python-list
Re: XML file parsing with SAX
I didn't make the XML file. And I don't like messing with other peoples
data. So I just want my SAX parser to ignore it. I can't help if other
people make it hard for me to read their xml file...
On Sat, 23 Apr 2005 13:48:49 -0600, Uche Ogbuji wrote:
> On Sat, 2005-04-23 at 15:20 +0200, Willem Ligtenberg wrote:
>> I decided to use SAX to parse my xml file.
>> But the parser crashes on:
>> File "/usr/lib/python2.3/site-packages/_xmlplus/sax/handler.py", line 38,
>> in fatalError
>> raise exception
>> xml.sax._exceptions.SAXParseException: NCBI_Entrezgene.dtd:8:0: error in
>> processing external entity reference
>>
>> This is caused by:
>> > "NCBI_Entrezgene.dtd">
>>
>> If I remove it, it parses normally.
>> I've created my parser like this:
>> import sys
>> from xml.sax import make_parser
>> from handler import EntrezGeneHandler
>>
>> fopen = open("mouse2.xml", "r")
>> ch = EntrezGeneHandler()
>> saxparser = make_parser()
>> saxparser.setContentHandler(ch)
>> saxparser.parse(fopen)
>>
>> And the handler is:
>> from xml.sax import ContentHandler
>>
>> class EntrezGeneHandler(ContentHandler):
>> """
>> A handler to deal with EntrezGene in XML
>> """
>>
>> def startElement(self, name, attrs):
>> print "Start element:", name
>>
>> So it doesn't do much yet. And still it crashes...
>> How can I tell the parser not to look at the DOCTYPE declaration.
>> On a website:
>> http://www.devarticles.com/c/a/XML/Parsing-XML-with-SAX-and-Python/1/
>> it states that the SAX parsers are not validating, so this error shouldn't
>> even occur?
>
> Just because it's not validating doesn't mean that the parser won't try
> to read the external entity.
>
> Maybe you're looking for
>
> """
> feature_external_ges
> Value: "http://xml.org/sax/features/external-general-entities";
> true: Include all external general (text) entities.
> false: Do not include external general entities.
> access: (parsing) read-only; (not parsing) read/write
> """
>
> Quote from:
>
> http://docs.python.org/lib/module-xml.sax.handler.html
>
> But you're on pretty shaky ground in any XML 1.x toolkit using a bogus
> DTDecl in this way. Why go through the hassle? Why not use a catalog,
> or remove the DTDecl?
--
http://mail.python.org/mailman/listinfo/python-list
[Newbie] Plone/workflow change ownership
Hi all, I would like to change the ownership of an object when changing its state. However when I call : obj=state_change print obj.getOwnerTuple() I get (['MyPortal', 'portal_workflow', 'TC_Workflow'], 'scripts') with scripts as owner. The method changeOwnership do nothing. Are there any tips to change the ownership during a state change? Thanks in advance and have a pretty new year. -- Frédéric Willem -- http://mail.python.org/mailman/listinfo/python-list
Re: merits of Lisp vs Python
Paul Rubin wrote: > Does this count as a "children of a lesser Python"? This sounds like a quite derogatory first question. CLPython is not a dead and abandoned project, nor is execution speed its main goal, nor are Python semantics bended anywhere (it can run the Pie-thon benchmark). Sure, some recently introduced language features are missing, but with just a little effort that's solved... Moreover, in Common Lisp source code analysis and manipulation can be expressed easily. CLPython thus provides ample opportunities to analyze type inference or caching schemes. Most of that is unexplored territory, I think. I like the journey so far. > How does clpython implement Python's immutable strings, for example? Normal Python strings are represented by normal Lisp strings. Instances of subclasses of 'str' are represented by CLOS instances. That's for performance reasons. This dual-representation aspect is nicely hidden behind macros, so that even in the code of CLPython itself there's no need to worry, or even know, about it. Why not take a look in the code -- I'll be happy to explain things. - Willem -- http://mail.python.org/mailman/listinfo/python-list
Re: CLPython (was Re: merits of Lisp vs Python)
Paul Boddie wrote: > What would it take to get Python people more interested in it? I've > been monitoring the site [1] and the mailing list [2] for some time, > but nothing particularly visible seems to be happening. Well, judging from the reactions on blogs to the initial announcement here, quite some people have heard about it and seem interested. But that didn't result in more people hacking on it, which is unfortunate. I guess in part it's because there are not that many people really into both Python and Lisp, and those who are might not find this an interesting project because there is nothing "wow" to show, yet. So you are right about the project silence. Right now it's a one-person pet project, and progress is slow. But I'm already quite happy about the current state: stable, usable, and fairly complete. As for more exposure: I'd prefer to wait with big announcements until there are really interesting things achieved. Thanks a lot for your interest and comments! - Willem -- http://mail.python.org/mailman/listinfo/python-list
Re: python simply not scaleable enough for google?
larations and compiler-based type inferencing. It has been said that CLPython is a very good counterargument for "just write a Python to Lisp compiler to make things fast", and even I as its developer agree. Lisp offers lots of readily available optimization opportunities, but Python simply doesn't. I remember reading some years ago about a Smalltalk compiler guru who said he would come up with a ridiculously fast Python implementation based on all the message sending optimizations he knew. It does not surprise me that we've never heard from him yet. - Willem -- http://mail.python.org/mailman/listinfo/python-list
pylab without X11
I want to use pylab (matplotlib) on a machine without X11. I'm trying to generate onthefly graphics for an apache2 web service, so they do not have to be displayed on this machine ! When i do pylab.figure() I get the error TclError: couldn't connect to display ":0.0" I tried setting the DISPLAY environment variable to "", ":0.0" but I got the same error message. Any help is very much appreciated. -- http://mail.python.org/mailman/listinfo/python-list
Re: Re: pylab without X11
Thanks that worked ! I changed the backend to Agg. Allthough a problem there is that with the backend Agg you only can write the resulting image to a file or a file like object, it does not accept the (apache) request handler to write the file to. And I do not want to save the file to disk first before sending to the client. WJV marc wrote: This may help you ... or not You may have to change your backend : p13 of the matplotlib user guide: backends p17 how to change the backend in the matplotlibrc An example of matplotlibrc ( backend choice is at the beginning ): http://matplotlib.sourceforge.net/matplotlibrc You may choose PS or SVG or pdf, there might be other possibilities, I do not know To save the file : savefig Marc Willem-Jan Vriend a écrit : I want to use pylab (matplotlib) on a machine without X11. I'm trying to generate onthefly graphics for an apache2 web service, so they do not have to be displayed on this machine ! When i do pylab.figure() I get the error TclError: couldn't connect to display ":0.0" I tried setting the DISPLAY environment variable to "", ":0.0" but I got the same error message. Any help is very much appreciated. -- http://mail.python.org/mailman/listinfo/python-list
