Python good for data mining?
I'm starting a project in data mining, and I'm considering Python and Java as possible platforms. I'm conserned by performance. Most benchmarks report that Java is about 10-15 times faster than Python, and my own experiments confirms this. I could imagine this to become a problem for very large datasets. How good is the integration with MySQL in Python? What about user interfaces? How easy is it to use Tkinter for developing a user interface without an IDE? And with an IDE? (which IDE?) What if I were to use my Python libraries with a web site written in PHP, Perl or Java - how do I intergrate with Python? I really like Python for a number of reasons, and would like to avoid Java. Sorry - lot of questions here - but I look forward to your replies! -- http://mail.python.org/mailman/listinfo/python-list
Re: Python good for data mining?
> > What if I were to use my Python libraries with a web site written in > > PHP, Perl or Java - how do I integrate with Python? > > Possibly the simplest way would be python .cgi files. The cgi and cgitb > modules allow form data to be read fairly easily. Cookies are also > fairly simple. For a more complicated but more customisable approach, > you could look in to the BaseHTTPServer module or a socket listener of > some sort, running that alongside the webserver publicly or privately. > Publicly you'd have links from the rest of your php/whatever pages to > the python server. Privately the php/perl/java backend would request > data from the local python server before feeding the results back > through the main server (apache?) to the client. Thanks a lot! I'm not sure I completely understand your description of how to integrate Python with, say PHP. Could you please give a small example? I have no experience with Python web development using CGI. How easy is it compared to web development in PHP? I still havent't made my mind up about the choice of programming language for my data mining project. I think it's a difficult decision. My heart tells me "Python" and my head tells me "Java" :-) -- http://mail.python.org/mailman/listinfo/python-list
Re: Python good for data mining?
On 5 Nov., 04:42, "D.Hering" <[EMAIL PROTECTED]> wrote: > On Nov 3, 9:02 pm, Jens <[EMAIL PROTECTED]> wrote: > > > > > I'm starting a project indatamining, and I'm considering Python and > > Java as possible platforms. > > > I'm conserned by performance. Most benchmarks report that Java is > > about 10-15 times faster than Python, and my own experiments confirms > > this. I could imagine this to become a problem for very large > > datasets. > > > How good is the integration with MySQL in Python? > > > What about user interfaces? How easy is it to use Tkinter for > > developing a user interface without an IDE? And with an IDE? (which > > IDE?) > > > What if I were to use my Python libraries with a web site written in > > PHP, Perl or Java - how do I intergrate with Python? > > > I really like Python for a number of reasons, and would like to avoid > > Java. > > > Sorry - lot of questions here - but I look forward to your replies! > > All of my programming is data centric. Data mining is foundational > there in. I started learning computer science via Python in 2003. I > too was concerned about it's performance, especially considering my > need for literally trillions of iterations of financial data tables > with mathematical algorithms. > > I then leaned C and then C++. I am now coming home to Python realizing > after my self-eduction, that programming in Python is truly a pleasure > and the performance is not the concern I first considered to be. > Here's why: > > Python is very easily extended to near C speed. The Idea that FINALLY > sunk in, was that I should first program my ideas in Python WITHOUT > CONCERN FOR PERFOMANCE. Then, profile the application to find the > "bottlenecks" and extend those blocks of code to C or C++. Cython/ > Pyrex/Sip are my preferences for python extension frameworks. > > Numpy/Scipy are excellent libraries for optimized mathematical > operations. Pytables is my preferential python database because of > it's excellent API to the acclaimed HDF5 database (used by very many > scientists and government organizations). > > As for GUI framework, I have studied Qt intensely and would therefore, > very highly recommend PyQt. > > After four years of intense study, I can say that with out a doubt, > Python is most certainly the way to go. I personally don't understand > why, generally, there is any attraction to Java, though I have yet to > study it further. Thanks a lot! I agree, Python is a pleasure to program in. So what you're saying is, don't worry about performance when you start coding, but use profiling and optimization in C/C++. Sounds reasonable. It's been 10 years ago since I've done any programming in C ++, so I have to pick up on that soon I guess. I've used NumPy for improving my K-Means algorithm, and it now runs 33% faster than "pure" Python. I guess it could be improved upon further. I will have a look at PyQt! -- http://mail.python.org/mailman/listinfo/python-list
Re: Python good for data mining?
On 5 Nov., 16:29, Maarten <[EMAIL PROTECTED]> wrote: > On Nov 5, 1:51 pm, Jens <[EMAIL PROTECTED]> wrote: > "Premature optimization is the root of all evil", to quote a famous > person. And he's right, as most people working larger codes will > confirm. I guess I'll have to agree with that. Still, I would like to get some kind of indication of if it's a good idea to use NumPy from the start of the project - for example. It depends on the situation of course. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python good for data mining?
On 5 Nov., 16:29, Maarten <[EMAIL PROTECTED]> wrote: > As for pytables: it is the most elegant programming interface for HDF > on any platform that I've encountered so far. Most other platforms > stay close the HDF5 library C-interface, which is low-level, and quite > complex. PyTables was written with the end-user in mind, and it shows. > One correction though: PyTables is not a database: it is a storage for > (large) arrays, datablocks that you don't want in a database. Use a > database for the metadata to find the right file and field within that > file. Keep in mind though that I mostly work with externally created > HDF-5 files, not with files created in pytables. PyTables Pro has an > indexing feature which may be helpful for datamining (if you write the > hdf-5 files from python). > PyTables? Wow! Looks amazing - I'll have to try that out soon. Thanks! -- http://mail.python.org/mailman/listinfo/python-list
Re: Python good for data mining?
> > Jens, > > You might be interested in this > bookhttp://www.oreilly.com/catalog/9780596529321/index.html > which is new, I just ordered my copy. From the contents shown online, > it has lot of applicability to data mining, using Python, although it > its primary topic is data mining the web, it also covers analyzing the > data etc. > > Ron Stephens > I'm a big fan of O'Reilly's books. This one looks very promising, and I have just added it to my wishlist. Thanks for the tip Ron! As for databases - I think its a good idea to support a good selection of data sources. Just now, one of my main concerns, is to make a structure that is flexible enough, as well as being easy to use. Flexibility and usability! -- http://mail.python.org/mailman/listinfo/python-list
Re: Looking for a good Python environment
Just wrote a mini "review" of three Python code editors on my blog... http://pyminer.blogspot.com/2007/11/python-code-editors.html I use PSPad or Notepad++ for quick editing, and Komodo Edit 4.2 for longer sessions. Komodo Edit is the only one with code completion - a very nice feature. You can pay $299 and get Komodo Edit, which has a debugger. I've only used Eclipse for Java programming, but there's a Python plug- in called PyDev - just haven't figured out how to install it :-( -- http://mail.python.org/mailman/listinfo/python-list
pydoc - how to generate documentation for an entire package?
I have a project/package for which I want to generate documentation using pydoc. My problem is that when I type "pydoc.py -w MyPackage" it only generates documentation for the package - no modules, classes or methods or sub-packages. Just a single HTML file called "MyPackage.html" That's strange - is there something here I'm missing. How do you generate documentation for a whole package? -- http://mail.python.org/mailman/listinfo/python-list
Re: Using python as primary language
On 8 Nov., 08:52, Michel Albert <[EMAIL PROTECTED]> wrote: > In our company we are looking for one language to be used as default > language. So far Python looks like a good choice (slacking behind > Java). A few requirements that the language should be able cope with > are: > > * Database access to Sybase. > This seems to be available for python, but the windows-binaries for > the library > are not available. Self-Compiling them proved to be non-trivial (As > always > on windows). > * Easy GUI creation. > Solved using PyQt. > * Cross Platform (Linux + Windows). > Again, PyQt, solves this > * Easy deployment. > Solved using py2exe + innosetup > * Charting (Histograms, Line charts, bar charts, pie charts, ...) > I am currently looking into PyQwt, which looks promising. > * Report generation with GUI support > reportlab + rml? > > So far, nearly all point seems to be manageable. But I was not yet > able to find a solution for the report generation. What we would like > to have is a sort of GUI interface to prepare the reports without > having to "code" the positioning. I found reportlab, and it looks like > it can do all that is needed in terms of output. But you still have to > code the report. And this is a no go. In context, I found RML and saw > links to graphical RML editors. But I have not yet found a concrete > example code, or documentation. What am I missing? Is RML a reportlab > creation or is it a recognised open standard? If not, is there an open > standard, that is easily to process with python? > > Any pointers? I would prefer coding Python to coding Java or > worse. VB ;) which is another contender in our roundup. I've been using Java since 1998, and Python for just about a year. I've tried to do an objective analysis of the two languages. Both languages have good and weak points, but personally I'd choose Python. In the end, its a matter of personal taste and preferences. They're both good languages. Try them both and see which one you like. Also, try to write a wishlist for what you'd like in a language, and then compare the wishlist to Java and Python. About graphing - you should really try the NumPy/Matplotlib combination. Works similar to MatLab. About VB - I've coded a lot of VB. It is a very easy language to start out with and it's based on .NET. Again, it depends on your needs, but make that wishlist and compare with Java and Python. Good luck! -- http://mail.python.org/mailman/listinfo/python-list
Re: pydoc - how to generate documentation for an entire package?
On 8 Nov., 02:46, Jens <[EMAIL PROTECTED]> wrote: > I have a project/package for which I want to generate documentation > usingpydoc. > > My problem is that when I type "pydoc.py -w MyPackage" it only > generates documentation for the package - no modules, classes or > methods or sub-packages. Just a single HTML file called > "MyPackage.html" > > That's strange - is there something here I'm missing. How do you > generate documentation for a whole package? No suggestions? All I can think of is to make a *.bat file on Windows to call pydoc.py for each of my modules. Seems like a silly solution. Also, when I have a module that imports from math (for example), pydoc.py generates a broken link to the math module. This just seems very silly. Generating documentation form code is a nice thing, but this pydoc.py is driving me insane - isn't there are better way? -- http://mail.python.org/mailman/listinfo/python-list
Re: display messages in python shell
On 19 Nov., 19:48, [EMAIL PROTECTED] wrote: > Is it possible to display messages in the python shell? I want to > display error messages based on parameters in my scripts to the > users. Is there another way to display messages other than log > files? Thanks. > > Kou What about using print? For example: print "This is a message." -- http://mail.python.org/mailman/listinfo/python-list
Re: pydoc - how to generate documentation for an entire package?
On 20 Nov., 08:20, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote: > On Nov 19, 12:50 pm, Jens <[EMAIL PROTECTED]> wrote: > > > > > On 8 Nov., 02:46, Jens <[EMAIL PROTECTED]> wrote: > > > > I have a project/package for which I want to generate documentation > > > usingpydoc. > > > > My problem is that when I type "pydoc.py -w MyPackage" it only > > > generates documentation for the package - no modules, classes or > > > methods or sub-packages. Just a single HTML file called > > > "MyPackage.html" > > > > That's strange - is there something here I'm missing. How do you > > > generate documentation for a whole package? > > > No suggestions? All I can think of is to make a *.bat file on Windows > > to call pydoc.py for each of my modules. Seems like a silly solution. > > > Also, when I have a module that imports from math (for example), > > pydoc.py generates a broken link to the math module. This just seems > > very silly. > > > Generating documentation form code is a nice thing, but this pydoc.py > > is driving me insane - isn't there are better way? > > pydoc -h > > [...] > > pydoc -w ... > Write out the HTML documentation for a module to a file in the > current > directory. If contains a '/', it is treated as a filename; > if > it names a directory, documentation is written for all the > contents. > > Have you tried pydoc -w ? > > --Nathan Davis Yes, I have! It just generates a single html file. It just contains a lot of broken links to non-existing module files. Strange! -- http://mail.python.org/mailman/listinfo/python-list
Re: pydoc - how to generate documentation for an entire package?
On 20 Nov., 08:19, Marc 'BlackJack' Rintsch <[EMAIL PROTECTED]> wrote: > On Mon, 19 Nov 2007 10:50:28 -0800, Jens wrote: > > Generating documentation form code is a nice thing, but this pydoc.py > > is driving me insane - isn't there are better way? > > Epydoc!? > > Ciao, > Marc 'BlackJack' Rintsch Thanks! Epydoc looks promising - shame about the user interface though :-( -- http://mail.python.org/mailman/listinfo/python-list
Code Management
Dear Reader, I'm writing some modules in Python, and I'm also using unittests. I'm wondering about some things: 1) Should I put my unittests in a subdirectory? Does the subdirectory have to be a package? 2) Does the main folder /myproject have to be a package? Should I put my modules directly under /myproject, or should I create a subfolder, for example /myproject/modules Does anyone have any "best practices" as to how to manage your code? Thanks! -- http://mail.python.org/mailman/listinfo/python-list
Re: Code Management
On 21 Nov., 01:46, [EMAIL PROTECTED] wrote: > On Nov 20, 4:09 pm, Jens <[EMAIL PROTECTED]> wrote: > > > Dear Reader, > > > I'm writing some modules in Python, and I'm also using unittests. I'm > > wondering about some things: > > I'd love to hear how others manage this sort of thing as well. I'll > describe what I've found works best for me, and if others do the same, > maybe we can all learn from each other. > > > 1) Should I put my unittests in a subdirectory? Does the subdirectory > > have to be a package? > > I put them in a subdirectory and make it a package. The biggest > advantage I see of using a subdirectory is that I can have lots of > test scripts and it's easy to keep them organized separately from > production code. The biggest disadvantage is that if I run the test > scripts from inside that subdirectory, they need to import modules > from their parent directory, and I'm using Python 2.4 which doesn't > have relative imports. So I put a .pth file in my site-packages > directory that adds the top-level package of my project to the > pythonpath. Then the test modules can import production code using > the fully qualified package.subpackage.module name for each production > module being tested. > > > 2) Does the main folder /myproject have to be a package? Should I put > > my modules directly under /myproject, or should I create a subfolder, > > for example /myproject/modules > > I make all non-trivial projects into packages. It allows me to do the > trick with a .pth file I described above. It makes it easier to reuse > all or part of my project as a component of a second project. And it > makes creating documentation with epydoc easier. > > I typically lay out a project directory something like this: > > projectname/ > setup.py > dev_install.py # automatically creates .pth file in site-packages > alltests.py # runs all unit tests > main_package_name/ > __init__.py > module1.py > module2.py > doc/ > img/ > subpackage1/ > __init__.py > module3.py > module4.py > tests/ > __init__.py > test1.py > test2.py > subpackage2/ > etc > > With this setup, tests are organized by subpackage, and each test > script can be run by itself, or all together from alltests.py. If a > module in subpackage2 needs to import a module from subpackage1 > (generally a sign of poor design, but it happens), it would need to > use the fully qualified "import > main_package_name.subpackage1.module1". Each time I check a new copy > of the project out from version control to a new location, I have to > make that the "active" version by running dev_install.py, which puts > a .pth file in site-packages that adds the newly checked out path to > the pythonpath. As long as I remember that step, this approach works > well. > > > Does anyone have any "best practices" as to how to manage your code? > > > Thanks! > > If anyone does, I'd love to hear about them. > > -Casey Raymondson Thank you very much for your ideas! I'm trying to make it work, and have created a small example: dummy/ dummy_package/ __init__.py moduleA.py tests/ __init__.py test.py I'm using Python 2.5.1. When I'm trying to call a function in 'moduleA' from 'test' it won't work unless I make the 'dummy' folder a package as well. That's pretty weird. Does 'dummy_package' have to be in my pythonpath or something? How do I reference moduleA from test? I would like to avoid making 'dummy' into a package as well. -- http://mail.python.org/mailman/listinfo/python-list
Re: Code Management
On 21 Nov., 04:16, Jens <[EMAIL PROTECTED]> wrote: > On 21 Nov., 01:46, [EMAIL PROTECTED] wrote: > > > > > On Nov 20, 4:09 pm, Jens <[EMAIL PROTECTED]> wrote: > > > > Dear Reader, > > > > I'm writing some modules in Python, and I'm also using unittests. I'm > > > wondering about some things: > > > I'd love to hear how others manage this sort of thing as well. I'll > > describe what I've found works best for me, and if others do the same, > > maybe we can all learn from each other. > > > > 1) Should I put my unittests in a subdirectory? Does the subdirectory > > > have to be a package? > > > I put them in a subdirectory and make it a package. The biggest > > advantage I see of using a subdirectory is that I can have lots of > > test scripts and it's easy to keep them organized separately from > > production code. The biggest disadvantage is that if I run the test > > scripts from inside that subdirectory, they need to import modules > > from their parent directory, and I'm using Python 2.4 which doesn't > > have relative imports. So I put a .pth file in my site-packages > > directory that adds the top-level package of my project to the > > pythonpath. Then the test modules can import production code using > > the fully qualified package.subpackage.module name for each production > > module being tested. > > > > 2) Does the main folder /myproject have to be a package? Should I put > > > my modules directly under /myproject, or should I create a subfolder, > > > for example /myproject/modules > > > I make all non-trivial projects into packages. It allows me to do the > > trick with a .pth file I described above. It makes it easier to reuse > > all or part of my project as a component of a second project. And it > > makes creating documentation with epydoc easier. > > > I typically lay out a project directory something like this: > > > projectname/ > > setup.py > > dev_install.py # automatically creates .pth file in site-packages > > alltests.py # runs all unit tests > > main_package_name/ > > __init__.py > > module1.py > > module2.py > > doc/ > > img/ > > subpackage1/ > > __init__.py > > module3.py > > module4.py > > tests/ > > __init__.py > > test1.py > > test2.py > > subpackage2/ > > etc > > > With this setup, tests are organized by subpackage, and each test > > script can be run by itself, or all together from alltests.py. If a > > module in subpackage2 needs to import a module from subpackage1 > > (generally a sign of poor design, but it happens), it would need to > > use the fully qualified "import > > main_package_name.subpackage1.module1". Each time I check a new copy > > of the project out from version control to a new location, I have to > > make that the "active" version by running dev_install.py, which puts > > a .pth file in site-packages that adds the newly checked out path to > > the pythonpath. As long as I remember that step, this approach works > > well. > > > > Does anyone have any "best practices" as to how to manage your code? > > > > Thanks! > > > If anyone does, I'd love to hear about them. > > > -Casey Raymondson > > Thank you very much for your ideas! I'm trying to make it work, and > have created a small example: > > dummy/ > dummy_package/ > __init__.py > moduleA.py > tests/ >__init__.py >test.py > > I'm using Python 2.5.1. When I'm trying to call a function in > 'moduleA' from 'test' it won't work unless I make the 'dummy' folder a > package as well. That's pretty weird. Does 'dummy_package' have to be > in my pythonpath or something? How do I reference moduleA from test? > > I would like to avoid making 'dummy' into a package as well. Problem solved. I added 'dummy' to the PYTHONPATH. (Do I really have to do that for every project I create?) Anyway, it works the way I'd like it to now. -- http://mail.python.org/mailman/listinfo/python-list
Re: Code Management
On 21 Nov., 05:34, Ben Finney <[EMAIL PROTECTED]> wrote: > > I hope the above makes it clearer what I prefer for this situation. > It does. You've been most helpful - thanks a lot! I'll bookmark this thread and keep it under my pillow :-) -- http://mail.python.org/mailman/listinfo/python-list
eval and exec in an own namespace
Hi,
has anyone an idea why the following code does not work.
s = """
def a(n):
return n*n
def b(t):
return a(t)
"""
ns = {}
exec(s, {}, ns)
eval("b(2)", ns, {})
executing this script raises an exception (NameError: global name 'a'
is not defined) in the last line.
Hope for your help.
--
http://mail.python.org/mailman/listinfo/python-list
problem with namespaces using eval and exec
Hi,
has anyone an idea why the following code does not work.
s = """
def a(n):
return n*n
def b(t):
return a(t)
"""
ns = {}
exec(s, {}, ns)
eval("b(2)", ns, {})
executing this script raises an exception (NameError: global name 'a'
is not defined) in the last line.
Hope for your help.
--
http://mail.python.org/mailman/listinfo/python-list
Re: problem with namespaces using eval and exec
Thanks a lot, that works for me. -- http://mail.python.org/mailman/listinfo/python-list
Terse Syntax through External Methods
Hello Everyone I'm newbie to Zope and i have a few questions regarding external methods. What i wan't to do is provide a terse syntax for converting urls to special tracking urls: http://myurl/')"> turns the provided url into something like http://host/tracking?url=http%3A%2F%2Fmyurl%2F in the output. i've been trying to use a external procedure like this. ## Script (Python) "track_link" ##bind container=container ##bind context=context ##bind namespace=_ ##bind script=script ##bind subpath=traverse_subpath ##parameters=self,url ##title=track link ## return "%s%s" % (self.tracking_prefix, url_quote(url)) This doesn't work because because the method doesn't have access to the environment. Obviously I don't wan't to pass everything explicitly into the function as this would defeat the purpose of the exercise, namely to provide a terse syntax. I have a background in other languages so I might be missing something conceptually with regard Zope and DTML.. Is there a better was of doing this, perhaps without using external methods? Currently im doing the following which isn't very elegant: in content document http://www.mylink.com/";>">link ... tracking: Appreciate any input you might have on this- -- http://mail.python.org/mailman/listinfo/python-list
Re: Terse Syntax through External Methods
On Jan 25, 3:19 pm, "Diez B. Roggisch" <[EMAIL PROTECTED]> wrote: > Jens schrieb: > > > > > Hello Everyone > > > I'm newbie to Zope and i have a few questions regarding external > > methods. What i wan't to do > > is provide a terse syntax for converting urls to special tracking > > urls: > > > http://myurl/')"> > > > turns the provided url into something like > > >http://host/tracking?url=http%3A%2F%2Fmyurl%2F > > > in the output. > > > i've been trying to use a external procedure like this. > > > ## Script (Python) "track_link" > > ##bind container=container > > ##bind context=context > > ##bind namespace=_ > > ##bind script=script > > ##bind subpath=traverse_subpath > > ##parameters=self,url > > ##title=track link > > ## > > return "%s%s" % (self.tracking_prefix, url_quote(url)) > > > This doesn't work because because the method doesn't have access to > > the environment. Obviously I don't wan't to pass everything explicitly > > into the function as this would defeat the purpose of the exercise, > > namely to provide a terse syntax. > > > I have a background in other languages so I might be missing something > > conceptually with regard Zope and DTML.. Is there a better was of > > doing this, perhaps without using external methods? Currently im doing > > the following which isn't very elegant: > > > in content document > > http://www.mylink.com/";> > tracking>">link > > ... > > tracking: > > > > > Appreciate any input you might have on this- > > Is it really needed to use an external method for this, or isn't a > "normal" python script enough already? > > If it has to be an External method, you can't access such a context > AFAIK. But then you can create a python script that _has_ this context, > and passese it to the external method. Not the nicest solution, but > should work. > > Diez Like I said i'm a newbie. I though the deal with Zope was that i couldn't really do inline scripting (for security reasons) like in php but had to use these external methods. how does one go about creating a "normal" python script exactly and how do I invoke it's functionality? -- http://mail.python.org/mailman/listinfo/python-list
Zope/DTML Infuriating...
Hello Everyone. I am relatively new to Zope(using it for a work project) and I was wondering if someone here could help me out or at least refer me to a decent documentationg for Zope/DTML/Python (at the detail level of php.net or Java API reference). http://www.zope.org/Documentation/Books/ZopeBook/2_6Edition/ isn't really detailed enough for my taste. if it doesn't contain a exhautive description of all available base classes it's simply no good as a reference resource. Anyway I have the following problem. I'm using zope 2.9 and I would expect the following dtml code to yield a list containing main1, main2 etc. But it doesn't work(and yes i know that i could simply do "... ...", but that's not what i need). I've the checked that i'm referring to the variables correctly, so the only explanation i can come up with, is that '+' doesn't result in a string concatenation (with implicit typecast to string of the integer variable(this is a interpreted language after all)). It apparently works in other cases but for some reason not here. I get the following cryptical error message which makes me none the wiser. An error was encountered while publishing this resource. Error Type: AttributeError Error Value: 'NoneType' object has no attribute 'group' I would appreciate any feedback you might have regarding this. Thanks in Advance. -- http://mail.python.org/mailman/listinfo/python-list
Re: Zope/DTML Infuriating...
On Apr 29, 1:59 pm, Marco Mariani <[EMAIL PROTECTED]> wrote: > Jens wrote: > > I've the checked that i'm referring to the variables correctly, so the > > only explanation i can come up with, is that '+' doesn't result in a > > string concatenation (with implicit typecast to string of the integer > > variable(this is a interpreted language after all)). > > No, sorry. You really need to read the python tutorial at the very least. > You might have wrong assumptions from previous PHP experiences. > > >>> 'x'+4 > Traceback (most recent call last): > File "", line 1, in > TypeError: cannot concatenate 'str' and 'int' objects > > ... and the non snobby answer would have been: ... -- http://mail.python.org/mailman/listinfo/python-list
Re: Zope/DTML Infuriating...
On Apr 29, 1:59 pm, Marco Mariani <[EMAIL PROTECTED]> wrote: > Jens wrote: > > I've the checked that i'm referring to the variables correctly, so the > > only explanation i can come up with, is that '+' doesn't result in a > > string concatenation (with implicit typecast to string of the integer > > variable(this is a interpreted language after all)). > > No, sorry. You really need to read the python tutorial at the very least. > You might have wrong assumptions from previous PHP experiences. > > >>> 'x'+4 > Traceback (most recent call last): > File "", line 1, in > TypeError: cannot concatenate 'str' and 'int' objects > > ... and the non snobby answer would have been: ... -- http://mail.python.org/mailman/listinfo/python-list
Re: Zope/DTML Infuriating...
On Apr 29, 2:45 pm, Marco Mariani <[EMAIL PROTECTED]> wrote: > Jens wrote: > >> You might have wrong assumptions from previous PHP experiences. > > >> >>> 'x'+4 > >> Traceback (most recent call last): > >> File "", line 1, in > >> TypeError: cannot concatenate 'str' and 'int' objects > > > ... and the non snobby answer would have been: > > > ... > > Sorry. Not trying to be snobbish, only in a hurry. > > That answer would have been less useful, because there are TONS of > details in the python tutorial, that set the language apart from its > "scripting cousins". > Reading the syntax and thinking "yeah, got it, boring, next chapter" is > a common mistake I've also made sometime, especially with python when > I've been deceived by its apparent simplicity. > Then, later, the same happened with Javascript, of course. > And it's bound to happen again, as much as I'll try to be careful :-( Hey no worriest. Is this the tutorial you're referring to: http://docs.python.org/lib/typesmapping.html Is there anything better? I miss the discussion and examples that accompany most entries in php.net. -- http://mail.python.org/mailman/listinfo/python-list
Re: Zope/DTML Infuriating...
On Apr 29, 2:45 pm, Marco Mariani <[EMAIL PROTECTED]> wrote: > Jens wrote: > >> You might have wrong assumptions from previous PHP experiences. > > >> >>> 'x'+4 > >> Traceback (most recent call last): > >> File "", line 1, in > >> TypeError: cannot concatenate 'str' and 'int' objects > > > ... and the non snobby answer would have been: > > > ... > > Sorry. Not trying to be snobbish, only in a hurry. > > That answer would have been less useful, because there are TONS of > details in the python tutorial, that set the language apart from its > "scripting cousins". > Reading the syntax and thinking "yeah, got it, boring, next chapter" is > a common mistake I've also made sometime, especially with python when > I've been deceived by its apparent simplicity. > Then, later, the same happened with Javascript, of course. > And it's bound to happen again, as much as I'll try to be careful :-( Hey no worriest. Is this the tutorial you're referring to: http://docs.python.org/lib/typesmapping.html Is there anything better? I miss the discussion and examples that accompany most entries in php.net. -- http://mail.python.org/mailman/listinfo/python-list
Re: Zope/DTML Infuriating...
On Apr 29, 2:45 pm, Marco Mariani <[EMAIL PROTECTED]> wrote: > Jens wrote: > >> You might have wrong assumptions from previous PHP experiences. > > >> >>> 'x'+4 > >> Traceback (most recent call last): > >> File "", line 1, in > >> TypeError: cannot concatenate 'str' and 'int' objects > > > ... and the non snobby answer would have been: > > > ... > > Sorry. Not trying to be snobbish, only in a hurry. > > That answer would have been less useful, because there are TONS of > details in the python tutorial, that set the language apart from its > "scripting cousins". > Reading the syntax and thinking "yeah, got it, boring, next chapter" is > a common mistake I've also made sometime, especially with python when > I've been deceived by its apparent simplicity. > Then, later, the same happened with Javascript, of course. > And it's bound to happen again, as much as I'll try to be careful :-( Hey no worriest. Is this the tutorial you're referring to: http://docs.python.org/lib/typesmapping.html Is there anything better? I miss the discussion and examples that accompany most entries in php.net. -- http://mail.python.org/mailman/listinfo/python-list
Re: Zope/DTML Infuriating...
On Apr 29, 3:16 pm, Christian Heimes <[EMAIL PROTECTED]> wrote:
> Jens schrieb:
>
> > Hello Everyone.
>
> > I am relatively new to Zope(using it for a work project) and I was
> > wondering if someone here could help me out or at least refer me to a
> > decent documentationg for Zope/DTML/Python (at the detail level of
> > php.net or Java API reference).
> > http://www.zope.org/Documentation/Books/ZopeBook/2_6Edition/
> > isn't really detailed enough for my taste. if it doesn't contain a
> > exhautive description of all available base classes it's simply no
> > good as a reference resource.
>
> Are you forced to use DTML for the job? ZPT are far superior and easier
> to work with, if you have to output HTML.
>
> Christian
For now, I have to use DTML as I am building on an existing solution.
I might look into ZPT's but I have limited time and theres no to time
at this point to redo everything as ZPT. In the long run we're
probably
going in a completely different direction, but that a another story.
I like some aspects of zope but I find that it ends up sitting between
two chairs. On one hand it want's to be a templating language which is
easy to get into for novices, but on the other hand you really have be
familiar with python to be able to so stuff thats only slighty more
complicated
then standard SSI. And don't get me started on the whole security
philosophy.
@Marco:
Thanks for the links :-) Python may be one of those really elegant
languages, but the reference is really sub standard. Checkout the
layout
of php.net for comparison. Think what you will about php, but the
reference
is excellent. For that matter check out msdn section on old-school
asp, or
even the common-lisp documentation(http://www.lispworks.com/
documentation/HyperSpec/Front/Contents.htm)
It's accessibility like that i'm missing. It shouldn't take 10 min
and a usenet
post to figure to how to basic stuff like string concatenation. And
theres
still plenty of unanswered questions after checking the reference:
- What is the exact definition of the operator e.g. op + (,
) -> , op + (, ) : , op + ( ...
- What is the exact operator precedence
- Why is it, when primitive data types seem to be objects (similar to
javascript), that type casting is done through build-in functions
rather than methods, e.g. String.toInt('5') or '5'.toInt() or x =
Integer.fromString('5').
--
http://mail.python.org/mailman/listinfo/python-list
Re: Zope/DTML Infuriating...
On Apr 29, 3:16 pm, Christian Heimes <[EMAIL PROTECTED]> wrote:
> Jens schrieb:
>
> > Hello Everyone.
>
> > I am relatively new to Zope(using it for a work project) and I was
> > wondering if someone here could help me out or at least refer me to a
> > decent documentationg for Zope/DTML/Python (at the detail level of
> > php.net or Java API reference).
> > http://www.zope.org/Documentation/Books/ZopeBook/2_6Edition/
> > isn't really detailed enough for my taste. if it doesn't contain a
> > exhautive description of all available base classes it's simply no
> > good as a reference resource.
>
> Are you forced to use DTML for the job? ZPT are far superior and easier
> to work with, if you have to output HTML.
>
> Christian
For now, I have to use DTML as I am building on an existing solution.
I might look into ZPT's but I have limited time and theres no to time
at this point to redo everything as ZPT. In the long run we're
probably
going in a completely different direction, but that a another story.
I like some aspects of zope but I find that it ends up sitting between
two chairs. On one hand it want's to be a templating language which is
easy to get into for novices, but on the other hand you really have be
familiar with python to be able to so stuff thats only slighty more
complicated
then standard SSI. And don't get me started on the whole security
philosophy.
@Marco:
Thanks for the links :-) Python may be one of those really elegant
languages, but the reference is really sub standard. Checkout the
layout
of php.net for comparison. Think what you will about php, but the
reference
is excellent. For that matter check out msdn section on old-school
asp, or
even the common-lisp documentation(http://www.lispworks.com/
documentation/HyperSpec/Front/Contents.htm)
It's accessibility like that i'm missing. It shouldn't take 10 min
and a usenet
post to figure to how to basic stuff like string concatenation. And
theres
still plenty of unanswered questions after checking the reference:
- What is the exact definition of the operator e.g. op + (,
) -> , op + (, ) : , op + ( ...
- What is the exact operator precedence
- Why is it, when primitive data types seem to be objects (similar to
javascript), that type casting is done through build-in functions
rather than methods, e.g. String.toInt('5') or '5'.toInt() or x =
Integer.fromString('5').
--
http://mail.python.org/mailman/listinfo/python-list
Regular Expression for words (with umlauts, without numbers)
Dear experts, I'm looking for a regular expression to recognize natural language words with umlauts but without numbers. While \w with re.U does recognize words with umlauts, it also matches numbers, which I do not want. Is there a better way than an exhaustive enumeration such as [-a-zàáâãäåæ...]? I guess there should be a better way as \w appears to know about alphabetical characters... Thanks in advance Jens -- http://mail.python.org/mailman/listinfo/python-list
Re: Regular Expression for words (with umlauts, without numbers)
On 2011-05-13, Peter Otten wrote: > Jens Lechtenboerger wrote: > >> I'm looking for a regular expression to recognize natural language >> words with umlauts but without numbers. While \w with re.U does >> recognize words with umlauts, it also matches numbers, which I do >> not want. >> >> Is there a better way than an exhaustive enumeration such as >> [-a-zàáâãäåæ...]? >> >> I guess there should be a better way as \w appears to know about >> alphabetical characters... > > How about [^\W\d] ? Brilliant. Thanks Jens -- http://mail.python.org/mailman/listinfo/python-list
Re: Installing Python on Windows 7
Am Freitag, 28. Oktober 2016 01:24:18 UTC+2 schrieb Karen Hermann: > I just downloaded Python 3.5.2 for Windows, which I installed on a Windows 7 > laptop. Disclaimer up front, I am a former lapsed programmer that has been > away from it and all things Windows since Windows XP. :)I’m back to > being a bit of a newbie again. > > It’s a relatively clean laptop, just got it, and rarely use it, and there’s > nothing else on it except Microsoft Office 2013. I downloaded Python, > installed it, tried to run it, and got this message: The program can’t > start because api-ms-win-crd-runtime-l1-1-0.dll is missing from your > computer. Try reinstalling the program to fix this problem. > > I’ve redownloaded it and reinstalled it several times, each time deleting all > traces of it before trying again. I’ve even disabled my virus-scanner > (Sophos), thinking that might be interfering but that didn’t help either. > > Nothing is helping, I can’t get past this. > > Can you help please, is there something wrong with my system / setup, should > I be downloading a different Python version? > > Thanks in advance for any help you can give! Hi Karen, i suggest the following steps: 1. Download C runtime Libraries from Microsoft https://support.microsoft.com/de-de/kb/2999226 for Windows 7 2. Install it 3. Try again to install Python. 4. Have fun. Greetings. JJ -- https://mail.python.org/mailman/listinfo/python-list
beginner's refcount questions
Hello, python uses gc only where refcounts alone haven't yet done the job. Thus, the following code class Foo: def __del__(self): print "deled!" def foo(): f = Foo() foo() print "done!" prints deled! done! and not the other way round. In c++, this is a central technique used for all sorts of tasks, whereas in garbage collected languages it's usually not available. Is there a reason not to rely on this in Python? For example, are there alternative Python implementations that behave differently? Or some other subtle problems? And some other minor question: Is there a way to query the use count of an object? This would be useful for debugging and testing. -- Cheers, Jens -- http://mail.python.org/mailman/listinfo/python-list
Re: calling python from C#...
There is IronPython which compiles to .NET. And there was another project bridging the .NET runtime with the standard Python interpreter of which I forgot the name. Jens -- http://mail.python.org/mailman/listinfo/python-list
Re: writing large files quickly
Donn wrote: >> How the heck does that make a 400 MB file that fast? It literally takes >> a second or two while every other solution takes at least 2 - 5 minutes. >> Awesome... thanks for the tip!!! > Because it isn't really writing the zeros. You can make these > files all day long and not run out of disk space, because this > kind of file doesn't take very many blocks. The blocks that > were never written are virtual blocks, inasmuch as read() at > that location will cause the filesystem to return a block of NULs. Under which operating system/file system? As far as I know this should be file system dependent at least under Linux, as the calls to open and seek are served by the file system driver. Jens -- http://mail.python.org/mailman/listinfo/python-list
Re: writing large files quickly
Ivan wrote: > Steven D'Aprano wrote: >> Isn't this a file system specific solution though? Won't your file system >> need to have support for "sparse files", or else it won't work? > Yes, but AFAIK the only "modern" (meaning: in wide use today) file > system that doesn't have this support is FAT/FAT32. I don't think ext2fs does this either. At least the du and df commands tell something different. Actually I'm not sure what this optimisation should give you anyway. The only circumstance under which files with only zeroes are meaningful is testing, and that's exactly when you don't want that optimisation. On compressing filesystems such as ntfs you will get this behaviour as a special case of compression and compression makes more sense. Jens -- http://mail.python.org/mailman/listinfo/python-list
Re: writing large files quickly
Donn wrote: > Because it isn't really writing the zeros. You can make these > files all day long and not run out of disk space, because this > kind of file doesn't take very many blocks. The blocks that > were never written are virtual blocks, inasmuch as read() at > that location will cause the filesystem to return a block of NULs. Are you sure that's not just a case of asynchronous writing that can be done in a particularly efficient way? df quite clearly tells me that I'm running out of disk space on my ext2fs linux when I dump it full of zeroes. Jens -- http://mail.python.org/mailman/listinfo/python-list
Re: Python vs C for a mail server
Nicolas wrote: > If it's just a way to throw a programming challenge at your friend's > face, then you should check whether it's okay to use Python rather than > C/C++, otherwise he could be charged of cheating by using a more > productive language :). Though this comment of mine is likely to start a religious war, it might also bring up some useful points. I'm a bit relucted to swallow the common view of Python being a so- productive language, especially compared to C++. I value C++ productiveness actually much higher than it's performance. To stick to the mailserver example, I'm pretty sure I'd do it in C++. I got very exited about Python when I first saw it, but I've encountered several problems that hindered productivity dramatically. One thing is the lack of static types. The problem is not only that the compiler can't tell you very basic things you're doing wrong but also that code isn't intrinsically documented: def send_mail(mail): ... What can I do with mail? In C++, you're looking up what type it is (presumably by pressing M-x in Emacs on the word before it) and have a look on it's type definition. In Python, it can be quite difficult to tell what you can do with it because the information of what will be passed in can be several layers up. Also there is not even symbol-safety: self.not_defined will never rise a compile-time error. And to address the memory management critisism about C++: Unless you have cyclic structures (you probably won't have in a mail server), just use smart pointers and you don't have to be concerned more about it than you'd have to be in Python. I aggree on C++ libraries being weak on unicode strings though, or even generally weak in the libraries (you have the C libraries, but they're not very type safe or elegant to use). I'm aware that C++ is a horrible monstrosity, an argument whiches weight depends on the OP's friends C++ experience. Please don't be offended, but if anyone could make a point of how Python's disadvantages in these regards could be alleviated, I'd be very interested. Jens -- http://mail.python.org/mailman/listinfo/python-list
Re: writing large files quickly
Ivan wrote:
> ext2 is a reimplementation of BSD UFS, so it does. Here:
> f = file('bigfile', 'w')
> f.seek(1024*1024)
> f.write('a')
> $ l afile
> -rw-r--r-- 1 ivoras wheel 1048577 Jan 28 14:57 afile
> $ du afile
> 8 afile
Interesting:
cp bigfile bigfile2
cat bigfile > bigfile3
du bigfile*
8 bigfile2
1032bigfile3
So it's not consumings 0's. It's just doesn't store unwritten data. And I
can think of an application for that: An application might want to write
the biginning of a file at a later point, so this makes it more efficient.
I wonder how other file systems behave.
> I read somewhere that it has a use in database software, but the only
> thing I can imagine for this is when using heap queues
> (http://python.active-venture.com/lib/node162.html).
That's an article about the heap efficient data structure. Was it your
intention to link this?
Jens
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python vs C for a mail server
Nicolas wrote: > http://nicolas.lehuen.com/ > My two latest problems with coding in C++ are due to the environments : > libraries using different string types and the whole problem with the > building system. I love the language, but I get a much better leverage > through Python and Java due to the quality and ease of use of their > built-in and third party libraries. I use C++ only for my core data > structure (namely a tuned version of a ternary search tree which I use > to build full text indices). Those points are all valid. I'm using Python for that reason. And there is another point that there are good Python bindings for the the more important C libraries, but usually no decent C++ wrapper for it. Jens -- http://mail.python.org/mailman/listinfo/python-list
Re: Python vs C for a mail server
Alex wrote:
> <http://www.artima.com/weblogs/viewpost.jsp?thread=4639>
> <http://www.mindview.net/WebLog/log-0025>
> Since Robert Martin and Bruce Eckel (the authors of the two documents
> linked above) are both acknowledged gurus of statically typechecked
> languages such as C++, the convergence of their thinking and experience
> indicated by those documents is interesting.
Indeed, especially Eckels article shed some light about testing as an
alternative to static typing. I still can't quite understand why you can't
do both. Clearly unit tests should be part of any software, not only
Python software.
Test failures, however, don't tell you anything about the current usage of
your program - just about the inteded usage at the point where the test
was writte. Clearly you can't test _anything_? And clearly you can never
be sure that all you collegues did so as well? This not only about type
safety, but simply name safety.
What do you do when you want to no if a certain method or function is
actually used from somewhere, say "foobar", it a language which allows
(and even encourages) that it could be called by:
getattr(obj, "foo" + "bar")()
?
There is no systematic way to find this call.
In C++, just commend out the definition and the compiler will tell you.
I'm pretty sure I red a PEP about static type safety in Python at some
point. It was even thinking about generics I think.
> The "but without declaration it can't be self-documenting" issue is a
> red herring. Reading, e.g.:
> int zappolop(int frep) { ...
> gives me no _useful_ "self-documenting" information
That's true. If the programmer wants to obfuscate his intention, I'm sure
neither Python nor C++ can stop him. The question is how much more work is
to write comprehensible code in one language or the other. I'm a bit
afraid about Python on that matter.
Python provides ways to easy literal documentation. But I'd really like to
have a way of indicating what I'm talking about in a way that's ensured to
be in-sync with the code. Tests are not where the code is. I have
difficulties remembering the type of a lot of symbols, and looking at
testing code to fresh up usage is more difficult that just jumping to the
definition (which the development envirnment is likely to be able to).
> [smart pointers and GC]
As you say, smart pointers are not full-blown garbage collection, which is
usually what you want, isn't it? I my (admittedly short) life as a
professional developer I have not yet come accross a situation where
reference counting was not sufficient to model the memory management.
As for the locking: Apart from locking your whatever you need to lock in
your user code, I don't think any special locking is necessary for the
memory management. Smart pointer can increment and decrement their ref
counts atomically as far as I know.
We use boost::shared_ptr in multi-threaded code just out of the box. We
also use a reference counted string implementation in multi threaded code.
> At Google, we collectively have rather a lot of experience in these
> issues, since we use three general-purpose languages: Python, Java, C++.
I have no doubt that goolge know what they're doing, and if you're working
there then you're likely to know what you're talking about.
I found it especially astonishing what you had to say against the use of
smart pointers.
Jens
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python vs C for a mail server
Jay wrote: > You can do both, but why? *Especially* in a language like C++, where > thanks to pointers and casting, there really isn't any type safety > anyway. How much time in your C/C++ code is spent casting and trying to > trick the compiler into doing something that it thinks you shouldn't be > doing? Not much frankly. Though I have no doubt that there is a lot of code that does, but more so in older C++ code. > How does type safety tell you anything about the current usage of your > program? Quite a bit; of course, it's doesn't cover everything and clearly testing the semantics is still needed, but a lot of what code is used in what way is described by the type system. And I admit that I'm used to use the type system as documentation of what's going on. > And unit tests *might* not tell about the current usage, but > integration tests certainly do. Of course tests will cover a lot what static typing does and more. I'm just claiming that they can support each other. > I don't think I've ever seen anyone advocating calling a function like > getattr(obj "foo" + "bar")(). You can do some very powerful things with > getattr, thanks to Python's dynamic nature, but I don't think anyone is > recommending calling a function like that. A lot of people got me wrong on that, please see Paul's postings. I really didn't mean that literally. > And is that fear based simply on "feeling", or on actual experience. The former, that's why I did start this branch of the thread, though I'm already regretting it. > Because in all of my own industry experience, it's been MUCH easier to > jump into someone else's Python code than someone else's C++ code (and > at my last job, I had to do a lot of both). I find Python to be much > more self-documenting, because there's not so much scaffolding all over > the place obfuscating the true intention of the code. That really depends on who's code you're looking at, as in any language. I can believe there are more C++ obfuscaters out there than ones for Python. I usually have little problems jumping into C++ code of other people but the principle reason for that will be that the people I'm working with have a very clean and expressive coding style. > You need to look at doctest: > http://docs.python.org/lib/module-doctest.html > With doctest, tests are EXACTLY where the code is. I've used doctest > with incredibly successful results, in industry. That's indeed a good point for Python. > Reference counting by itself is not necessarily sufficient (because of > circular references). That's why even Python, with its reference > counting based system, has additional capabilities for finding circular > references. Whenever I encountered the need for circular references it was because an object, that was in some sense owned by another, needed a pointer back to it's owner. That solved easily with non-owning C-style pointers or weak pointers. If you have an example where this is not sufficient, I'd be *very* keen on hearing it (it may be easy, I don't know). > I believe that Alex's official job title at Google is "Uber Technical > Lead". I'm sure I'll quickly be corrected if that's wrong, but trust me > (and everyone else who's spent significant time reading c.l.p.) that > Alex Martelli knows what he's talking about. You seem to be thinking that I was ironic. That was certainly not my intention. I was just trying to minimise the amount of flames I'll be getting. > A lot of your arguments are the very typical arguments that people levy > against Python, when they're first around it. And that's fine, most > people go through that. They are taught programming in C++ or Java, and > that that *that* is the way you're supposed to program. Then they see > that Python does things in different ways, and automatically assume > that Python is doing it wrong. I'm sorry to hear this, and whilst I'm certainly lacking experience in Python, I'm not one of those people. > All I can say is that if you spend time with Python (and more > importantly, read quality, well-established Python code that's already > out there), you'll see and understand the Python way of doing things. I will. Jens -- http://mail.python.org/mailman/listinfo/python-list
Re: Python vs C for a mail server
Paul wrote: > Or should I be looking for some other context here? Three people were looking at the wrong one, thanks for putting this right. I really should not have given my point that briefly. Jens -- http://mail.python.org/mailman/listinfo/python-list
Graphical introspection utilities?
Hello, as it would be so obviously a good thing to have a graphical (or maybe curses-base) browser through the dynamic state of a Python program, it's probably there. Can someone point me to something? Cheers, Jens -- http://mail.python.org/mailman/listinfo/python-list
Re: Another try at Python's selfishness
n.estner wrote: > Yes, I 100% agree to that point! > But the point is, the current situation is not newbie-friendly (I can > tell, I am a newbie): I declare a method with 3 parameters but when I > call it I only pass 2 parameters. That's confusing. If I declare a > member variable, I write: "self.x = ValueForX", why can't I be equally > explicit for declaring member functions? For someone new to OO in general it might as well be something good, so he realises that there actually really is a hidden parameter. After all, there is something to understand with "self", and this discrapency between the number of arguments and parameters puts newbies to it. Jens -- http://mail.python.org/mailman/listinfo/python-list
Custom wxPython Widget
What is the best source code to read? Any tips; suggestions? Thanks in advance Jens Thiede -- http://mail.python.org/mailman/listinfo/python-list
Re: Looking for a wxPython GUI builder
On Jun 16, 3:18 am, Dick Moores <[EMAIL PROTECTED]> wrote: > How about SPE? > > Any others? > > And which ones do people > actually use? Commercial or Freeware. > > Thanks, > > Dick Moores There's Boa Constructor... (http://boa-constructor.sourceforge.net/ Screenshots). I haven't actually used it though. -- http://mail.python.org/mailman/listinfo/python-list
Metaclass v.s. Property function.
I don't like the property function, usable in the new-style classes, because having to remember to manage a list of "foo = property(...)" assignments just plain sucks, so I wrote a metaclass that does things a little differently. Please have a look and tell me whether this is useful or impractical. The metaclass is here: http://pastebin.com/m5b06b571 and some simple testcode is here: http://pastebin.com/m382f2ae9. Notice the first line though. Thanks for any replies, Jens Thiede. -- http://mail.python.org/mailman/listinfo/python-list
Re: Metaclass v.s. Property function.
On Aug 11, 6:06 pm, Duncan Booth <[EMAIL PROTECTED]> wrote: > Jens Thiede <[EMAIL PROTECTED]> wrote: > > I don't like the property function, usable in the new-style classes, > > because having to remember to manage a list of "foo = property(...)" > > assignments just plain sucks, so I wrote a metaclass that does things > > a little differently. Please have a look and tell me whether this is > > useful or impractical. The metaclass is here: > >http://pastebin.com/m5b06b571and some simple testcode is here: > >http://pastebin.com/m382f2ae9. Notice the first line though. > > Here's something I posted a while back which lets you use property as a > decorator: > > class C(object): > def __init__(self, colour): > self._colour = colour > > @property.set > def colour(self, value): > self._colour = value > > @property.get > def colour(self): > return self._colour > > @property.delete > def colour(self): > self._colour = 'none' > > See:http://groups.google.co.uk/group/comp.lang.python/browse_thread/threa... > > Whether you like that style is of course a matter of personal opinion. Thanks, a very simple but a great and more "pythonic" idea. -- http://mail.python.org/mailman/listinfo/python-list
Embedding Python in a multithreaded managed C++ application
Hi experts, I am currently embedding the Python interpreter 2.3.4 in a managed C++ application (I currently use an unmanaged dll which is linked to Python23.lib). Forget about Ironpython - no binary compatibility for extension modules, forget about a decent C API, by the way... After a lot of work I managed to embed the interpreter with the Global Python Interpreter lock correctly used from each thread. I have, however, still a deadlock which I am not able to find and is related to the dll I use. Does anybody have an idea how to debug the exact source of this deadlock by means of the Python debug options. Thanks a lot for any help. ThreadShredder -- http://mail.python.org/mailman/listinfo/python-list
idea for testing tools
Hello, I find it annoying that one has to write self.assertEqual(x, y) rather than just assert x == y when writing tests. This is a nuisance in all the programming languages I know of (which are not too many). In Python however, there appears to be a better alternative. The piece of code below gives the benefit of printing the violating values in case of a testing failure as well as the concise syntax: The snippet def test_test(): def foo(x): return x + 3 x = 1 y = 2 assert foo(x) < y + x try: test_test() except AssertionError: analyse() would give: Traceback (most recent call last): File "./ast-post.py", line 138, in ? test_test() File "./ast-post.py", line 134, in test_test assert foo(x) < y + x AssertionError failure analysis: foo: x: 1 ( x ): 1 foo ( x ): 4 y: 2 x: 1 y + x: 3 foo ( x ) < y + x: False The code that makes this possible relies only on code present in the standard library (being traceback, inspect and the parsing stuff) while being as short as: #!/usr/bin/python import sys, types import traceback, inspect import parser, symbol, token import StringIO def get_inner_frame(tb): while tb.tb_next: tb = tb.tb_next return tb.tb_frame def visit_ast(visitor, ast): sym = ast[0] vals = ast[1:] assert len(vals) > 0 is_simple = len(vals) == 1 is_leaf = is_simple and type(vals[0]) != types.TupleType if not is_leaf: visitor.enter() for val in vals: visit_ast(visitor, val) visitor.leave() if is_leaf: visitor.leaf(sym, vals[0]) elif is_simple: visitor.simple(sym, vals[0]) else: visitor.compound(sym, vals) class ast_visitor: def enter(self): pass def leave(self): pass def leaf(self, sym, val): pass def simple(self, sym, val): pass def compound(self, sym, vals): pass class simple_printer(ast_visitor): def __init__(self, stream): self.stream = stream def leaf(self, sym, val): print >>self.stream, val, def str_from_ast(ast): s = StringIO.StringIO() visit_ast(simple_printer(s), ast) return s.getvalue() class assertion_collector(ast_visitor): def __init__(self, statements): self.statements = statements def compound(self, sym, vals): if sym == symbol.assert_stmt: # two nodes: the "assert" name and the expression self.statements.append(vals[1]) class pretty_evaluate(ast_visitor): def __init__(self, globals_, locals_): self.globals = globals_ self.locals = locals_ def _expr(self, expression): code = compile(expression, '', 'eval') try: result = eval(code, self.globals, self.locals) except Exception, e: result = e print '%50s: %s' % (expression, str(result)) def compound(self, sym, vals): ast = [ sym ] ast.extend(vals) expression = str_from_ast(ast) self._expr(expression) def leaf(self, sym, val): if sym == token.NAME: self._expr(val) def analyse(): type_, exc, tb = sys.exc_info() frame = get_inner_frame(tb) try: filename, line, fun, context, index = ( inspect.getframeinfo(frame, 1) ) ast = parser.suite(context[0].lstrip()).totuple() assert_statements = [ ] visit_ast(assertion_collector(assert_statements), ast) traceback.print_exc() print "\nfailure analysis:\n" for statement in assert_statements: visit_ast( pretty_evaluate(frame.f_globals, frame.f_locals), statement) finally: del frame -- Cheers, Jens -- http://mail.python.org/mailman/listinfo/python-list
Re: idea for testing tools
Bruno Desthuilliers <[EMAIL PROTECTED]> writes: > http://codespeak.net/py/current/doc/test.html#assert-with-the-assert-statement Ok, I didn't come across this before. I didn't work for me though, even the simple case #!/usr/bin/python a = 1 b = 2 def test_some(): assert a == b didn't reveal the values for a and b, though some more complex cases showed something. -- Cheers, Jens -- http://mail.python.org/mailman/listinfo/python-list
Pyexcelerator
Hi, i have a question with the pyexcelerator Modul. I´m using the Version 0.6.3a. Now I want to know how I can change the Colour Palette of Excel. I want to have my own colours for pattern_fore_colour and things like this. I want to build the colours dynamically. I have the RGB values for the colours in my XML script. Is there anybody who have done this? Best regards Jens Kabella -- http://mail.python.org/mailman/listinfo/python-list
Speeding up network access: threading?
Hello, what would be best practise for speeding up a larger number of http-get requests done via urllib? Until now they are made in sequence, each request taking up to one second. The results must be merged into a list, while the original sequence needs not to be kept. I think speed could be improved by parallizing. One could use multiple threads. Are there any python best practises, or even existing modules, for creating and handling a task queue with a fixed number of concurrent threads? Thanks and regards! -- http://mail.python.org/mailman/listinfo/python-list
Re: Speeding up network access: threading?
Hello, The fairly obvious thing to do is use a queue.queue for tasks and another for results and a pool of threads that read, fetch, and write. Thanks, indeed. Is a list thrad-safe or do I need to lock when adding the results of my worker threads to a list? The order of the elements in the list does not matter. Jens -- http://mail.python.org/mailman/listinfo/python-list
Re: Speeding up network access: threading?
Hello, The fairly obvious thing to do is use a queue.queue for tasks and another for results and a pool of threads that read, fetch, and write. Thanks, indeed. Is a list thrad-safe or do I need to lock when adding the results of my worker threads to a list? The order of the elements in the list does not matter. Jens -- http://mail.python.org/mailman/listinfo/python-list
Re: Speeding up network access: threading?
Hi and sorry for double posting - had mailer problems, Terry said "queue". not "list". Use the Queue class (it's thread-safe) in the "Queue" module (assuming you're using Python 2.x; in Python 3.x it's called the "queue" module). Yes yes, I know. I use a queue to realize the thread pool queue, that works all right. But each worker thread calculates a result and needs to make it avaialable to the application in the main thread again. Therefore, it appends its result to a common list. This seems works as well, but I was thinking of possible conflict situations that maybe could happen when two threads append their results to that same result list at the same moment. Regards, Jens -- http://mail.python.org/mailman/listinfo/python-list
decode(..., errors='ignore') has no effect
Hi,
I try to decode a string,e.g.
u'M\xfcnchen, pronounced [\u02c8m\u028fn\xe7\u0259n]'.decode('cp1252',
'ignore')
but even thoug I use errors='ignore'
I get UnicodeEncodeError: 'charmap' codec can't encode character u'\u02c8'
in position 21: character maps to
How come?
Thanks,
Jens
--
http://mail.python.org/mailman/listinfo/python-list
Re: decode(..., errors='ignore') has no effect
To convert unicode into str you have to *encode()* it.
u"...".decode(...) will implicitly convert to ASCII first, i. e. is
equivalent to
u"...".encode("ascii").decode(...)
Hence the error message
Ah - yes of course.
And how can you use the system's default encoding with errors=ignore?
The default encoding is the one that is used if no parameters are given to
"encode".
Thanks again!
--
http://mail.python.org/mailman/listinfo/python-list
Re: Adding a cookie
Hi,
I am creating a webscraper for a specific web site for an
application.
Now, that website has a specific cookie which needs to be set in
the request. Otherwise, the website is redirected.
I have been trying for the last 6 hours to add a cookie to the
HTTP request, but to no avail.
You just need to send the cookie's value in the HTTP header.
E.g. urllib2 can do this like this:
request = urllib2.Request(url, None, {'Cookie': '$Version="1";
cookieName="CookieValue"; $Path=/cookiePath'})
Regards,
Jens
--
http://mail.python.org/mailman/listinfo/python-list
Re: Making safe file names
Andrew Berg wrote:
> Currently, I keep Last.fm artist data caches to avoid unnecessary API calls
> and have been naming the files using the artist name. However, artist names
> can have characters that are not allowed in file names for most file systems
> (e.g., C/A/T has forward slashes). Are there any recommended strategies for
> naming such files while avoiding conflicts (I wouldn't want to run into
> problems for an artist named C-A-T or CAT, for example)? I'd like to make
> the files easily identifiable, and there really are no limits on what
> characters can be in an artist name. --
It's not clear what the context that you need this for. You
could e.g. replace all characters not allowed by the file
system by their hexidecimal (ASCII) values, preceeded by a
'%" (so '/' would be changed to '%2F', and also encode a '%'
itself in a name by '%25'). Then you have a well-defined
two-way mapping ("isomorphic" if I remember my math-lear-
nining days correctly) between the original name and the
way you store it. E.g.
"C/A/T" would become "C%2FA%2FT"
and
"C%2FA/T" would become "C%252FA%2FT"
You can translate back and forth between them with not too
much effort.
Of course, that assumes that '%' is a character allowed by
your file system - otherwise pick some other one, any one
will do in principle. It's a bit harder for a human to in-
terpret but rathe likely not that much of a problem. You
probably will have seen that kind of scheme used in URLs.
The concept is rather old and called 'escape character',
i.e. have one character that assumes some special meaning
and also "escaped" it.
If, on the hand, those names are never to be translated back
to the original name another strategy would be to use the SHA1
hash value of the artists name. Since clashes between SHA1 hash
values are rather hard to produce it's a rather safe method of
converting something (i.e. the artists name) to a number. The
drawback, of course, is that you can't translate back from the
hash value to the original name (if that would be simple the
whole thing wouldn't work;-)
Regards, Jens
--
\ Jens Thoms Toerring ___ [email protected]
\__ http://toerring.de
--
http://mail.python.org/mailman/listinfo/python-list
Re: in need of some help...
Alex Norton wrote: > On Wednesday, 1 May 2013 13:15:28 UTC+1, Jens Thoms Toerring wrote: > > Of course, it might be nicer to have a "result" label some- > > where in the graphical interface which you set to the text > > instead of printing it out to the console. And you also will > > probably add some "Quit" button to end the game. > > how would i go about adding print outcomes of all options to a label ? If you have a QLabel you can set its text to anything you want using its setText() method. Regaeds, Jens PS: If I may ask you a favor: consider refraining from using Google's completely broken interface to newsgroups - your post consists of nearly 200 lines of text containing all I wrote, with an empty line inserted between each of them, and a single line of text you wrote. It's rather annoying to have to sieve through that much of unrelated stuff just to find thar one line that's re- levant. And this Google groups crap seems to make it nearly impossible to do it any other way. If you don't believe me see e.g. http://wiki.python.org/moin/GoogleGroupsPython There are much better alternatives to "Google groups", using a real usenet news server and a program that does not mess up content of news group postings. They've been developed with 30 years of experience with newsgroups. If I'd be conspiracy theorist I would conclude that Google is up to something bad in trying to make using newsgroups nearly impossible by their badly broken stuff (and, to add credibility to such a claim, their complete disregard for all the criticism they got over the years, actually making each version of Google groups even worse), but it's rather likely just another case of pure incompetence (or a "why should we care" attitude:-( -- \ Jens Thoms Toerring ___ [email protected] \__ http://toerring.de -- http://mail.python.org/mailman/listinfo/python-list
Re: in need of some help...
Chris “Kwpolska” Warrick wrote: > (slightly offtopic, sorry.) > On Sun, May 12, 2013 at 12:20 AM, Jens Thoms Toerring > wrote: > > PS: If I may ask you a favor: consider refraining from using Google's > > completely broken interface to newsgroups - your post consists > > of nearly 200 lines of text containing all I wrote, with an empty > > line inserted between each of them, and a single line of text > > you wrote. It's rather annoying to have to sieve through that > > much of unrelated stuff just to find thar one line that's re- > > levant. > Gmail automatically hides long quotes. This is helpful in situations > like this one. More mail software should implement that > functionality. Seriously: once you go Gmail, you never go back. i giess you mean Gougle groups and not Gmail, which I can't comment on since I don't use it. You still have to "un-hide" the message when you want to under- stand what the other person is respoding to. And then you're back to square one (and the double-spacing issue seems to remain). That is why it has been a good tradition to remove everything when you reply that isn't relevant to what you're replying to. Of course, with the sorry excuse for an editor you habe in a web based form that's more difficult to do than with a real editor. That's one of the reasons I would advice to use a specialized program that can be made to use the editor of your choice. Another extremly annoying thing when reading via Google groups is that there's no reasonable threading, i.e. it's not immediately obvious what is meant as a reply to a specific post. That makes Google groups basically useless for longer discussions. And then Google groups overflow with spam messages that any self-respecting news server would discard, so you never get to see them. > > And this Google groups crap seems to make it nearly > > impossible to do it any other way. If you don't believe me see > > e.g. > > > > http://wiki.python.org/moin/GoogleGroupsPython > > > > There are much better alternatives to "Google groups", > > using a real usenet news server and a program that does > > not mess up content of news group postings. They've been > > developed with 30 years of experience with newsgroups. > Or something even better: a mailing list. > http://mail.python.org/mailman/listinfo/python-list is where you can > find it. Much friendlier than Usenet, and the software itself is > developed by the FLUFL. Mailing lists are quite fine and I'm on quite a number of them. But I don't want many more to further fill up my inbox. And there you again have the problem that there's no reasonable threading of messages when these messages arrive in your mail folder. > > If I'd be conspiracy theorist I would conclude that Google > > is up to something bad in trying to make using newsgroups > > nearly impossible by their badly broken stuff (and, to add > > credibility to such a claim, their complete disregard for > > all the criticism they got over the years, actually making > > each version of Google groups even worse), but it's rather > > likely just another case of pure incompetence (or a "why > > should we care" attitude:-( > They shouldn’t care because Usenet users often yell “Get off my > lawn!”. Young people don’t use newsgroups. They don’t even know what > Usenet is. Well, it's a pity when younger people don't use newsgroups (no idea if this is true, though) since they can be extremely useful for all kinds of purposes - I e.g. learned a huge amount from just lurking. And while some groups may be nicer than others, that's no excuse for Google at all to actively trying to destroy them. I'd rather prefer Google to simply get rid of Google groups and put the long time archives they obtained from DajaNews into better hands that do care. Regards, Jens -- \ Jens Thoms Toerring ___ [email protected] \__ http://toerring.de -- http://mail.python.org/mailman/listinfo/python-list
Re: in need of some help...
Chris Angelico wrote: > On Mon, May 13, 2013 at 7:56 AM, Jens Thoms Toerring wrote: > > Chris “Kwpolska” Warrick wrote: > >> Gmail automatically hides long quotes. This is helpful in situations > >> like this one. More mail software should implement that > >> functionality. Seriously: once you go Gmail, you never go back. > > > > i giess you mean Gougle groups and not Gmail, which I can't comment > > on since I don't use it. > No, Chris (not me, the other Chris... *an*other Chris okay, one of > the chorus of Chrises of this list!) did mean Gmail, the Google > webmail client. It does threading (and does it better than > SquirrelMail does), and it does the hiding of long quotes, long > signatures, etc. Ok, sorry then about that - as I said I never have used Gmail (and don't plan using it for other reasons than usability - and then I would hardly consider anything with a web interface for a text medium to be very usable;-). But, as far as I understand, Gmail is about email, so I'm a bit at a loss to understand what got this to do with news groups and Google groups (were the post I originally was responing to according to the header seemed to be coming from) that I intended this to be about? Best regards, Jens -- \ Jens Thoms Toerring ___ [email protected] \__ http://toerring.de -- http://mail.python.org/mailman/listinfo/python-list
Re: How to create new python file with increament number, if doesn't exist?
Avnesh Shakya wrote:
>I want to create a new python file like 'data0.0.5', but if it is already
> exist then it should create 'data0.0.6', if it's also exist then next like
> 'data0.0.7'. I have done, but with range, please give me suggestion so that
> I can do it with specifying range.
> I was trying this way and it's working also..
> i = 0
> for i in range(100):
> try:
> with open('Data%d.%d.%d.json'%(0,0,i,)): pass
> continue
> except IOError:
> edxCorrectDataFile = file('Data%d.%d.%d.json'%(0,0,i,), 'a+')
> break
> But here I have defined range 100, Is it possible without range it create
> many required files?
What about something as simple as this?
i = 0
while os.path.exists( 'Data{0}.{1}.{2}.json'.format( 0, 0, i ) ) :
i += 1
f = open( 'Data{0}.{1}.{2}.json'.format( 0, 0, i ), 'w' )
For your code you'ld make it e.g.
i = 0
try:
while True :
with open('Data%d.%d.%d.json'%(0,0,i)):
i += 1
except IOError:
edxCorrectDataFile = file('Data%d.%d.%d.json'%(0,0,i), 'a+')
Note that I don't see how all this trying to open a file and
catching an exeception if it doesn't exist is any better
than simply using os.path.exists(). You may have read some-
where that it avoids a race condition and would thus be more
secure. That's not the case here, an attacker still could
create e.g. a symbolic link with the name of the file you're
going to open between the time the exception is thrown and your
program getting around to open the file you expect not to exist,
so nothing is gained by using this somewhat convoluted method.
What would be needed for avoiding a race condition is an addi-
tional flag to be passed to open() like the for example the
O_EXCL flag that can be passed to Unix' open() system function.
But that isn't supported by Pythons open() function which
rather likeky is based on C's fopen() function). The best
advice to avoid such problems is probably not to open files
with a predictable name (or better to use one of the methods
to create files with names guaranteed to be unique by the
system) in directories to which other users than you also
have write permission.
Regards, Jens
--
\ Jens Thoms Toerring ___ [email protected]
\__ http://toerring.de
--
http://mail.python.org/mailman/listinfo/python-list
Re: Updating a filename's counter value failed each time
In article you wrote:
> After a user selects a file from the form, that sleection of his can be
> found form reading the variable 'filename'
> If the filename already exists in to the database i want to update its
> counter and that is what i'm trying to accomplish by:
> ---
> if form.getvalue('filename'):
> cur.execute('''UPDATE files SET hits = hits + 1, host = %s, lastvisit
> =
> %s WHERE url = %s''', (host, lastvisit, filename) )
There are (single) quotes missing around (at least) the file
name (the 'url' column) which I'm rather sure is a string -
you need them around all strings you use in SQL statements.
I don't know which database and interface you're using but I would
guess that many have the ability to inserting quotes where neces-
sary etc. E.g. with sqlite3 you would use
cur.execute('UPDATE files SET hits = hits + 1, host = ?, lastvisit = ? '
'WHERE url = ?', (host, lastvisit, filename) )
and the quotes required around (at least) the 'filename' string
will be inserted automatically.
Also take care to check the filename you insert - a malicous
user might cobble together a file name that is actually a SQL
statement and then do nasty things to your database. I.e. never
insert values you received from a user without checking them.
> For some reason this never return any data, because for troubleshooting
> i have tried:
> data = cur.fetchone()
There's nothing that your SQL statement (if correct) would return,
so what do you expect to have returned by the fetchone() method?
Perhaps there's something like the 'rowcount' property in sqlite3
which returns the number of rows modified by an INSERT or UPDATE.
> Since for sure the filename the user selected is represented by a record
> inside 'files' table why its corresponding counter never seems to get
> updated?
I would guess because you forgot the uotes around string
values in your SQL statement which thus wasn't executed.
Regards, Jens
--
\ Jens Thoms Toerring ___ [email protected]
\__ http://toerring.de
--
http://mail.python.org/mailman/listinfo/python-list
Re: Updating a filename's counter value failed each time
Νίκος wrote:
> On 17/6/2013 8:54 μμ, Jens Thoms Toerring wrote:
> > Also take care to check the filename you insert - a malicous
> > user might cobble together a file name that is actually a SQL
> > statement and then do nasty things to your database. I.e. never
> > insert values you received from a user without checking them.
> Yes in generally user iput validation is needed always, but here here
> the filename being selected is from an html table list of filenames.
> But i take it you eman that someone might tried it to pass a bogus
> "filename" value from the url like:
> http://superhost.gr/cgi-bin/files.py?filename="Select.";
> Si that what you mean?
Well, you neer wrote where this filename is coming from.
so all I could assume was that the user can enter a more
or less random file name. If he only can select one from
a list you put together there's probably less of a problem.
> But the comma inside the execute statement doesn't protect me from such
> actions opposed when i was using a substitute operator?
> > I would guess because you forgot the uotes around string
> > values in your SQL statement which thus wasn't executed.
> i tried you suggestions:
> cur.execute('''UPDATE files SET hits = hits + 1, host = %s, lastvisit =
> %s WHERE url = "%s"''', (host, lastvisit, filename) )
> seems the same as:
> cur.execute('''UPDATE files SET hits = hits + 1, host = %s, lastvisit =
> %s WHERE url = %s''', (host, lastvisit, filename) )
> since everything is tripled quoted already what would the difference be
> in "%s" opposed to plain %s ?
As I wrote you need *single* quotes around strings in
SQL statements. Double quotes won't do - this is SQL
and not Python so you're dealing with a different lan-
guage and thus different rules apply. The triple single
quotes are seen by Python, but SQL needs its own.
Regards, Jens
--
\ Jens Thoms Toerring ___ [email protected]
\__ http://toerring.de
--
http://mail.python.org/mailman/listinfo/python-list
Re: Updating a filename's counter value failed each time
MRAB wrote: > On 17/06/2013 19:32, Jens Thoms Toerring wrote: > > As I wrote you need *single* quotes around strings in > > SQL statements. Double quotes won't do - this is SQL > > and not Python so you're dealing with a different lan- > > guage and thus different rules apply. The triple single > > quotes are seen by Python, but SQL needs its own. > > > The query looks safe to me as he _is_ using a parametrised query. Perhaps - the OP never told which API (or database) he is using. What about some API that simply connects the first argument of exxecute() with the second with just a simple '%' to construct the string for the SQL state- ment? In that case there would be no single quotes a- round strings, or would there? Regards, Jens -- \ Jens Thoms Toerring ___ [email protected] \__ http://toerring.de -- http://mail.python.org/mailman/listinfo/python-list
Strange effect with import
Hi, I hope that this isn't a stupid question, asked already a hundred times, but I haven't found anything definitive on the problem I got bitten by. I have two Python files like this: S1.py -- import random import S2 class R( object ) : r = random.random( ) if __name__ == "__main__" : print R.r S2.p( ) S2.py -- import S1 def p( ) : print S1.R.r and my expectation was that the static variable 'r' of class R would be identical when accessed from S1.py and S2.py. Unfortunately, that isn't the case, the output is different (and R seems to get instantiated twice). But when I define R in S2.py instead S1.py -- import S2 print S2.R.r S2.p( ) S2.py -- import random class R( object ) : r = random.random( ) def p( ) : print R.r or, alternatively, if I put the defintion of class R into a third file which I then import from the other 2 files, things suddenly start to work as expected/ Can someone explain what's going one here? I found this a bit sur- prising. This is, of course, not my "real" code - it would be much more sensible to pass the number to the function in the second file as an argument - but is the smallest possinle program I could come up with that demonstrate the prob- lem. In my "real" code it's unfortunately not possible to pass that number to whatever is going to use it in the other file, I have to simulate a kind of global variable shared between different files. Best regards, Jens -- \ Jens Thoms Toerring ___ [email protected] \__ http://toerring.de -- http://mail.python.org/mailman/listinfo/python-list
Re: Strange effect with import
Thanks a lot to all three of you: that helped me understand the errors of my ways! You just saved me a few more hours of head-scratching;-) A few replies to the questions and comments by Steven: Steven D'Aprano wrote: > On Thu, 20 Dec 2012 20:39:19 +0000, Jens Thoms Toerring wrote: > > and my expectation was that the static variable 'r' of class R > The terminology we prefer here is "class attribute", not "static > variable". Attributes are always assigned in dynamic storage, whether > they are per-instance or on the class. I'm comimg from C/C++ and that's were my terminology is from, I know I still have to learn a lot more about Python;-) > > In my "real" code it's unfortunately not > > possible to pass that number to whatever is going to use it in the > > other file, I have to simulate a kind of global variable > > shared between different files. > Well, I find that hard to believe. "Not convenient"? I could believe > that. "Difficult"? Maybe. "Tricky"? I could even believe that. But "not > possible"? No, I don't believe that it is impossible to pass variables > around as method arguments. You are rather likely right and I probably should have written: "I don't see any way to pass that variable to the object that is supposed to use it". Perhaps you have an idea how it could be done correctly when I explain the complete picture: I'm writing a TCP server, based on SocketServer: server = SocketServer.TCPServer((192.168.1.10, 12345), ReqHandler) where ReqHandler is the name of a class derived from SocketServer.BaseRequestHandler class ReqHandler(SocketServer.BaseRequestHandler): ... A new instance of this class is gernerated for each connection request to the server. In the call that creates the server I can only specify the name of the class but no arguments to be passed to it on instantiation - at least I found nothing in the docu- mentation. On the other hand I need to get some information into this class and thus the only idea I came up with was to use some kind of global variable for the purpose. Perhaps there's a much better way to do that but I haven't found one yet. Or perhaps it is an omission in the design of SocketServer or (more likely) my mis-understanding of the documentation (as I wrote I'm relatively new to Python). Thnak you and best regards, Jens -- \ Jens Thoms Toerring ___ [email protected] \__ http://toerring.de -- http://mail.python.org/mailman/listinfo/python-list
Re: Strange effect with import
Terry Reedy wrote: > > server = SocketServer.TCPServer((192.168.1.10, 12345), ReqHandler) > > > > where ReqHandler is the name of a class derived from > > SocketServer.BaseRequestHandler > You misunderstood the doc. You pass the class, not the name of the class. > From 21.19.4.1. socketserver.TCPServer Example > server = socketserver.TCPServer((HOST, PORT), MyTCPHandler) Yes, I meant "the class", but I'm a bit weak on nomenclature in Python;-) > > A new instance of this class is gernerated for each connection > > request to the server. In the call that creates the server I can > > only specify the name of the class but no arguments to be passed > Code those arguments directly into the handle method of your version of > MyTCPhandler. Or if you need to override multiple methods and use the > same values in multiple methods, override __init__ and add self.x = > x-value statements. Sorry, you lost me there: what means "code those arguments directly into the handle method"? According to the documen- tation (or at least to my understanding of it;-) the handle() method is suppose to accept just one argument, 'self'. And even if I would change the method to accept more arguments and that wouldnt blow up into my face, where would they be coming from (and from where would I pass them)? Best regards, Jens -- \ Jens Thoms Toerring ___ [email protected] \__ http://toerring.de -- http://mail.python.org/mailman/listinfo/python-list
Re: Strange effect with import
Hans Mulder wrote: > What happens if instead of a class you pass a function that > takes the same arguments as the SocketServer.BaseRequestHandler > constructor and returns a new instance of your ReqHandler? > That's not quite what the documentaion clls for, but I'd hope > it's close enough. Interesting idea - I'm not yet at a level of Python wizardry that I would dare to do something that's not explicitely bles- sed be the documentation;-) > Maybe something like this: > class ReqHandler(SocketServer.BaseRequestHandler): > def __init__(self, request, client_address, server, ham, spam) > super(SocketServer, self).__init__( > self, request, client_address, server) > self.ham = ham > self.spam = spam > > And later: > import functools > server = SocketServer.TCPServer((192.168.1.10, 12345), >functools.partial(ReqHandler, ham="hello", spam=42)) Ok, that's still way over may head at the moment;-) I will hhave to read up on functools tomorrow, it's the first time I heard of it but it looks quite interesting at a first glance. Thank you for these ideas, I'll need a bit of time to figure out these new concepts and I don't think I'm up to it tonight any- more;-) Best regards. Jens -- \ Jens Thoms Toerring ___ [email protected] \__ http://toerring.de -- http://mail.python.org/mailman/listinfo/python-list
Re: Strange effect with import
Hans Mulder wrote: > Maybe something like this: > class ReqHandler(SocketServer.BaseRequestHandler): > def __init__(self, request, client_address, server, ham, spam) > super(SocketServer, self).__init__( > self, request, client_address, server) > self.ham = ham > self.spam = spam > The only thing I had to change about this was to assign the additional class variables before calling super() because in the __init__() method of the base class my overloaded handle() method is already called which needs those extra variables. > And later: > import functools > server = SocketServer.TCPServer((192.168.1.10, 12345), >functools.partial(ReqHandler, ham="hello", spam=42)) Thanks a lot, that's now all working perfectly well and I got rid of those pesky global variables;-) Probably the guys that wrote the SocketServer module indeed didn't expect people as dense as me to use their module and thus didn't mention that passing additional information to a handler object can be done this way... Best regards, Jens -- \ Jens Thoms Toerring ___ [email protected] \__ http://toerring.de -- http://mail.python.org/mailman/listinfo/python-list
Re: how to solve complex equation?
Usama Khan wrote: > how to solve complex equation in pyhton? and then use it to make a program. > . i have created new post as my last post is i guessed ranked as a cheater. > .:( > i know very litle about python as well as programing. . > which equation am taliking u will be thinking. . i am giving u the link > kindly c that equation. . n kindly let me know the way. . > https://groups.google.com/forum/?fromgroups=#!topic/comp.lang.python/cxG7DLxXgmo First of all, the equation given there is unusable (even the number of parentheses doesn't match up). Propbably it's meant to be the one someone else posted a link to: http://classes.engr.oregonstate.edu/cce/winter2012/ce492/Modules/06_structural_design/06-3_body.htm This thingy can't be solved on paper so you need some iterative algorithm to find the solution. So waht you do is modify the equa- tion so that you have 0 on one side and then consider the other side to be a function of SN+1. Now the problem you're left with is to find the value(s) of SN+1 (and thus of SN) where the func- tion has a zero-crossing. A commonly use algorithms for finding zero-crossings is Newton's method. You can find lots of sites on the internet describing it in all neccessary detail. It boils down to start with some guess for the result and then calculate the next, better approximation via xn+1 = xn - f(xn) / f'(xn) where f(xn)n) is the value of the function at point xn and f'(xn) the value of the derivative of f (with respect to x) also at xn. You repeat the process until the difference be- tween xn an the next, better approximation, xn+1, has become as small as you need it. So it's very simple to implement and the ugliest bit is pro- bably calculating the required derivative of the function with respect to SN+1 (wbich you can take to be x). Regards, Jens -- \ Jens Thoms Toerring ___ [email protected] \__ http://toerring.de -- http://mail.python.org/mailman/listinfo/python-list
Messing with the GC
Hi,
triggered by some problems I had with PySide I got a bit
confused about what the GC may do in certain situations.
Here's a small test program I cobbled together:
import sys
class X( object ) :
def __init__( self, parent, cnt ) :
print( "In constructor for {0} {1}".format( self, cnt ),
file = sys.stderr )
self.parent = parent
self.cnt = cnt
def __del__( self ) :
print( "In destructor for {0} {1}".format( self, self.cnt ),
file = sys.stderr )
def foo( self ) :
print( "Before", file = sys.stderr )
self.parent.z = X( self.parent, 2 ) # Is this bad?
print( "After", file = sys.stderr )
class Y( object ) :
def __init__( self ) :
print( "In constructor for {0}".format( self ),
file = sys.stderr )
self.z = X( self, 1 )
def __del__( self ) :
print( "In destructor for {0}".format( self ),
file = sys.stderr )
Y( ).z.foo( )
Have a look at the line with the comment. At this point the
only reference in existence to the X class instance, of which
a method is just being executed, goes out of scope. Thus I
would assume that the GC could now kick any time, possibly
even before the following call of print() or before the method
call returns. That, in turn might result in a crash of the
script.
Is my assumption about this flawed and there are no potential
dangers? Perhaps with
Y( ).z.foo( )
a temporary second reference is created that keeps the GC
for removing the X instance...
Another thing I'm puzzled about is the output of the
script:
In constructor for <__main__.Y object at 0x2919210>
In constructor for <__main__.X object at 0x2919310> 1
Before
In constructor for <__main__.X object at 0x2919350> 2
After
In destructor for <__main__.X object at 0x2919310> 1
Ok, the destrucor for the first instance of the X class is
called only after printing out "After", so the GC didn't
delete the object before. But then there are obviously no
calls of the destructors of neither the second instance
of the X class nor of the Y class instance. Shouldn't
they be invoked before the program ends?
Thanks and best regards, Jens
--
\ Jens Thoms Toerring ___ [email protected]
\__ http://toerring.de
--
http://mail.python.org/mailman/listinfo/python-list
Re: Messing with the GC
Hi, thank you for the explanations. I had overlooked the cyclic nature of what I had produced here and, of course, the GC can't be blamed for not collecting objects that are part of a cycle. The other question about the last refe- rence to an object vanishing within a method call (which, as I now clearly understand, can't happen and wouldn't make much sense) was triggered by a segmentation fault I get when I do something similar in PySide, so I was getting worried if it might be due to a GC issue. Now I know its got to be something different;-) Thanks and best regards, Jens -- \ Jens Thoms Toerring ___ [email protected] \__ http://toerring.de -- http://mail.python.org/mailman/listinfo/python-list
Re: Messing with the GC
Terry Reedy wrote: > On 1/20/2013 3:09 PM, Jens Thoms Toerring wrote: > > thank you for the explanations. I had overlooked the > > cyclic nature of what I had produced here and, of course, > > the GC can't be blamed for not collecting objects that are > > part of a cycle. The other question about the last refe- > > rence to an object vanishing within a method call (which, > > as I now clearly understand, can't happen and wouldn't make > > much sense) was triggered by a segmentation fault I get > > when I do something similar in PySide, so I was getting > > worried if it might be due to a GC issue. Now I know its > > got to be something different;-) > Perhaps the hardest part of writing C extensions to CPython directly in > C (versus something like Cython) is properly balancing increfs and > decrefs. An incref without a later decref can lead to a memory leak. A > decref without a preceding incref (so CPython thinks the object can be > deleted, when it should not be) can lead to segfaults. Definitely - I got started with Python having to write glue code to get Python to work with a C++ library. And keeping track of which side thinks it owns an object can sometimes be a bit of a challenge... > So I would report PySide code leading to segfaults to the > PySide people. Now that I'm more sure that it's unlikely to be a Python GC related issue (or my not understanding what I'm doing, to be precise) this is on my to-do list. But first I have to distill things down to a very short example program still exhibiting the problem - and experience tells me that this will most li- kely result in the realization that it's not a PySide issue at all but some misunderstanding on my side;-) Best regards, Jens -- \ Jens Thoms Toerring ___ [email protected] \__ http://toerring.de -- http://mail.python.org/mailman/listinfo/python-list
Output: Number of digits in exponent?
How can I control the number of digits in the exponent when writing floats to a file? It seems that Python2.4.2(winXP) prints three digits anyway. >>> print 1.0e50 1e+050 >>> print '%e' % (1.0e50) 1e+050 I would prefer two digits in the exponent. Usually 3 digits is not a problem, except for waisted disk space in 99.99% of all practical cases. But this time I have to write an interface file to a program written in an other programming language, and the output format requires a fix fortran format with 2 digits in the exponent. Can this be done in Python? Speed is an issue so I don't like the idea of rolling my own output function. It should be possible to do: >>> print '%12.5.2e' % (1.0e50) 1.000e+50 Thanks for any help! Jens -- http://mail.python.org/mailman/listinfo/python-list
2.4 or 2.3.4 for 2.3 software?
Can we expect the current release of 2.4 to be just as reliable as 2.3.4 for 2.3 compliant software? Thanks for any help! Jens -- http://mail.python.org/mailman/listinfo/python-list
Re: Help needed to create a Python extension library for an existing shared memory hash table library
Simon Hardy-Francis wrote:
> Hi Python fans, I just released my first open source project ever called
> SharedHashFile [1]. It's a shared memory hash table written in C. Some guy
> on Quora asked [2] whether there's an extension library for Python coming
> out. I would like to do one but I know little about Python. I was wondering
> if anybody in this group has experience writing extension libraries for
> Python?
There are, as far as I know, a number of tools that can help you
to create extension modules from C/C++ libraries (i.e. Python bin-
dings for the library). One that I have used successfully with a
largish C++ library of mine is shortly described here
http://www.riverbankcomputing.com/software/sip/intro
and the documentation and download are at
http://pyqt.sourceforge.net/Docs/sip4/
http://www.riverbankcomputing.com/software/sip/download
Since I haven't used any of the alternatives I can't comment on
how good they are. A list of them can be found here
https://wiki.python.org/moin/IntegratingPythonWithOtherLanguages
Regards, Jens
--
\ Jens Thoms Toerring ___ [email protected]
\__ http://toerring.de
--
https://mail.python.org/mailman/listinfo/python-list
Re: installing libraries like numpy scipy matplotlib
PIYUSH KUMAR wrote: > I have never used linux in my life.. only windows based computing.. So I > have problems in installing third party libraries in python. It depends. One question is if there's already a ready-for-use package for the third party library you want to install. If that is the case then the next question is which distro you're using - there are different package-management systems. Since you mentioned Pyke: if you got Ubuntu or Debian there's a package for it you can simply install it using the command sudo apt-get install python-pyke (The 'sudo' bit is for temporarily assuming the permissions to install the package, you're going to be asked for your password. 'apt-get' is the program for installing and de- installing packages. And 'python-pyke' is the name of the package._ If you also want the documentation installed add 'python-pyke-doc'. If you have some other distribution there might be a different program for installing new packages. And there's basically always also some program with a graphical user interface, wrapped around that and which shows you which packages exist (thousands). If there's no package for what you want you need to download the sources and install them yourself. For the details there usually is a file named README.txt and/or INSTALL.txt (or similar). Often all you need to use is the three commands ./configure make sudo make install If what you want to install depends on further software you also need to install that before creating and installing of what you need will succeed. So, again check if a ready-made package for that dependencies available or download, compile and install that. This can, in some cases, become a bit of work if A need B, B in turn need C and D, D needs E etc.;-) > So can somebody just explain me how many softwares or other python packages > I have to install before installing any of these. Sorry, but I don't understand this sentence. But in case for what you want to install there's a package for your distro you don't need to worry at all - all dependencies will get installed automatically (so if you want to install package A and that needs package B, B will installed automatically before A is installed). > I also had problems installing PYKE last month. What were these problems? Best regards, Jens -- \ Jens Thoms Toerring ___ [email protected] \__ http://toerring.de -- https://mail.python.org/mailman/listinfo/python-list
Re: installing libraries like numpy scipy matplotlib
memilanuk wrote: > On 07/04/2015 07:58 AM, Jens Thoms Toerring wrote: > > PIYUSH KUMAR wrote: > >> I have never used linux in my life.. only windows based computing.. So I > >> have problems in installing third party libraries in python. > > > > It depends. One question is if there's already a ready-for-use > > package for the third party library you want to install. If that > > is the case then the next question is which distro you're using > I could be wrong, but I think the point was that he's not using Linux, > while the majority of instructions for getting and using various > scientific libraries for Python assume that the user *is* using Linux. Reading it the first time I understood it too mean that the OP is completely new to Linux (otherwise why mention Linux at all and not ask directly "How do I install something under Windows?"). But on second reading I see that it could also have beenmeant the other way round and it's not about Linux at all;-) Best regards, Jens -- \ Jens Thoms Toerring ___ [email protected] \__ http://toerring.de -- https://mail.python.org/mailman/listinfo/python-list
Writing to same file from two threads
Hi,
I noticed in someone elses program that it writes single
lines to the same file from (what I call for loss of a better
name) the "main thread" of the program and from a thread sub-
sequentally started. This got me worried if it might result
in garbled output (i.e. having some output from A inside a
line written by B or vice versae) because the "main thread" or
the other thread could be interrupted during a call of write().
Is this a valid concern (and thus locking the file object is
required before writing to it) or am I guaranteed that this
can't happen? In the latter case I would be grateful for an
explanation what mechanism is responsible for this never to
happen.
Thanks and best regards, Jens
PS: I already have determined experimentally that a context
switch definitely can happen between two calls of write()
(and I expected nothing else), what I'm worried about are
context switches somewhere within the very innards of what
write() does.
--
\ Jens Thoms Toerring ___ [email protected]
\__ http://toerring.de
--
http://mail.python.org/mailman/listinfo/python-list
Re: Writing to same file from two threads
Paul Rubin wrote: > [email protected] (Jens Thoms Toerring) writes: > > in garbled output (i.e. having some output from A inside a > > line written by B or vice versae) because the "main thread" or > Yes they do get garbled like that. Preferred Python style is put a > single thread in charge of all the i/o to that file, and communicate > with it by message passing through Queue objects. That is safer than > directly using locks. Thank you for confirmig my suspicion;-) But you have induced another question: why is using a Queue safer than locking (not that I doubt that it might be more elegant etc.). Is it "safer" because it's less likely that one gets it wrong (e.g. by for- grtting to acquire the lock) or is there something inherently unsafe about locks? Thank you and best regards, Jens -- \ Jens Thoms Toerring ___ [email protected] \__ http://toerring.de -- http://mail.python.org/mailman/listinfo/python-list
Re: Writing to same file from two threads
Antoine Pitrou wrote: > Jens Thoms Toerring toerring.de> writes: > > > > Paul Rubin nospam.invalid> wrote: > > > jt toerring.de (Jens Thoms Toerring) writes: > > > > in garbled output (i.e. having some output from A inside a > > > > line written by B or vice versae) because the "main thread" or > > > > > Yes they do get garbled like that. Preferred Python style is put a > > > single thread in charge of all the i/o to that file, and communicate > > > with it by message passing through Queue objects. That is safer than > > > directly using locks. > > > > Thank you for confirmig my suspicion But you have induced > > another question: why is using a Queue safer than locking (not > > that I doubt that it might be more elegant etc.). Is it "safer" > > because it's less likely that one gets it wrong (e.g. by for- > > grtting to acquire the lock) or is there something inherently > > unsafe about locks? > For the record, binary files are thread-safe in Python 3, but text files > are not. > Locks are safe if you use them well. As you point out, if you forget > to acquire your lock, or if you devise a situation where there is a > deadlock between competing locks, you can have difficult to diagnose > issues. Queues have their internal locking all done for you. Thank you for your kind answers! Best regards, Jens -- \ Jens Thoms Toerring ___ [email protected] \__ http://toerring.de -- http://mail.python.org/mailman/listinfo/python-list
Re: subprocess.call
Ombongi Moraa Fe wrote:
> [-- text/plain, encoding 7bit, charset: ISO-8859-1, 19 lines --]
> In my python script, I have this:
> command="lynx -dump
> 'phpscript?param1=%s¶m2=%s¶m3=%s¶m4=%s¶m5=%s'"%(value1,value2,value3,value4)
> result=subprocess.call(command,shell=True)
> print 'xml message'
> However, the response from running the php script is also printed on output
> screen. I don't want this output.
> How can i ensure that only the last print 'xml response' is returned?
You mean is printed out? Use subprocess.Popen() and redirect
stdout to a pipe, similar to this:
p = subprocess.Popen(command, stdout=subprocess.PIPE)
r = p.communicate()
print r[0] # This is the output (to stdout)
The return value of the Popen objects communicate() method is
a tuple with two elements, first contains what got written to
stdout (if redirected to a pipe, otherwise Nome), the second
what went to stderr (again if redirected to a pipe). If you
also redirect stdin then you can pass what you want to send
to the process spawned via Popen() as an argument to commu-
nicate().
BTW, according to the dicumentation you should split the
command line into its componenents and pass that as a list
to the call() or Popen() subprocess methods, so it would
seem to reasonable to use e.g.
command = [ 'lynx', '-dump',
( "'phpscript?param1={0}¶m2={1}¶m3={2}"
"¶m4={3}¶m5={4}'" )
.format( value1, value2, value3, value4, value5 ) ]
Note that there was one value for creating the string to be
passed to lynx was mising.
Regards, Jens
--
\ Jens Thoms Toerring ___ [email protected]
\__ http://toerring.de
--
http://mail.python.org/mailman/listinfo/python-list
Re: Nested For loop not running full
inshu chauhan wrote:
> I have this part of my code where I am trying to traverse over an image by
> running a for loop for both x and y co-ordinate axis. But the loop is
> terminating by just reading first pixel. Can think of a reason why this is
> happening ?
> The code is:
> for sy in xrange(0, segimage.height):
> for sx in xrange(0, segimage.width):
> if segimage[sy,sx] == (0.0, 0.0, 0.0):
> continue
> else:
> seg_color = segimage[sy,sx]
> blue = int(seg_color[0])
> green = int(seg_color[1])
> red = int(seg_color[2])
> reg_num = blue + 256 * green + 65536 * red
> for l in f:
> sp = l.split(",")
> if len(sp) == 14:
> print sy, sx # for checking which pixel its
> reading currently
> print reg_num, sp[0] # for checking whats
> happening
> if reg_num == int(sp[0].strip()):
> print reg_num, sp[0].strip() # for checking
> whats happening
> classification = int(sp[13].strip())
> The inside "for loop" is for reading a csv format file from which I am
> extracting some information.
Are you sure that the loop is only run once? In that case the most
likely thing is that the image consists of only a single pixel
(or all except the first one are black, then it might also look
as if the loop would be run only once;-)
But what looks strange is the innermost loop. You never tell what
exactly 'f' is but I would tend to assume that it is a file object
for your CSV file, which you opened somewhere before. And now you
read it in completely when dealing with the very first pixel of
your image. Afterwards, when dealing with the other pixels of the
image, there's nothing left to be read in, so the inner loop won't
be run again, making it appear as if the outer loops would only be
run once.
If my assumptions are correct and you want to read in the file
again and again for each pixel then you should either open it
again and again for each pixel or, probably better, reset the
file object so that it "points" back to the start of the file
before the start of the innermost loop, using the seek() method
- a simple "f.seek(0)" should do the job (assuming that this is
a normal file, i.e. one that can be "rewound" and not e.g. a re-
directed pipe).
An even better solution (if you have enough memory) might be to
read in the whole file into a list and iterate over that instead
of the file itself. And better than that might be to build a
dictionary of values in the file that you can use later on, so
you don't have to run over the whole file again and again:
d = { }
for l in f :
sp = split( l, ',' )
if len( sp ) == 14 :
d[ int( sp[ 0 ].strip( ) ) ] = int( sp[ 13 ].strip( ) )
Then you can later check directly if some color value (what
you have named 'reg_num') is in the file by using
if reg_num in d :
and the corresponding value from the file (what you assign
to 'classification') is simply the value of the dictionary
for the key given by 'reg_num'. i.e.
classification = d[ reg_num ]
Regards, Jens
--
\ Jens Thoms Toerring ___ [email protected]
\__ http://toerring.de
--
http://mail.python.org/mailman/listinfo/python-list
Re: File Read issue by using module binascii
Tim Roberts wrote:
> Jimmie He wrote:
> >When I run the readbmp on an example.bmp(about 100k),the Shell is become to
> >"No respose",when I change f.read() to f.read(1000),it is ok,could someone
> >tell me the excat reason for this?
> >Thank you in advance!
> >
> >Python Code as below!!
> >
> >import binascii
> >
> >def read_bmp():
> >f = open('example.bmp','rb')
> >rawdata = f.read() #f.read(1000) is ok
> >hexstr = binascii.b2a_hex(rawdata) #Get an HEX number
> >bsstr = bin (int(hexstr,16))[2:]
> I suspect the root of the problem here is that you don't understand what
> this is actually doing. You should run this code in the command-line
> interpreter, one line at a time, and print the results.
> The "read" instruction produces a string with 100k bytes. The b2a_hex then
> produces a string with 200k bytes. Then, int(hexstr,16) takes that 200,000
> byte hex string and converts it to an integer, roughly equal to 10 to the
> 240,000 power, a number with some 240,000 decimal digits. You then convert
> that integer to a binary string. That string will contain 800,000 bytes.
> You then drop the first two characters and print the other 799,998 bytes,
> each of which will be either '0' or '1'.
> I am absolutely, positively convinced that's not what you wanted to do.
> What point is there in printing out the binary equavalent of a bitmap?
> Even if you did, it would be much quicker for you to do the conversion one
> byte at a time, completely skipping the conversion to hex and then the
> creation of a massive multi-precision number. Example:
> f = open('example.bmp','rb')
> rawdata = f.read()
> bsstr = []
> for b in rawdata:
> bsstr.append( bin(ord(b)) )
> bsstr = ''.join(bsstr)
> or even:
> f = open('example.bmp','rb')
> bsstr = ''.join( bin(ord(b))[2:] for b in f.read() )
Exactly my idea at first. But then I started to time it (using
the timeit module) by comparing the following functions:
# Original version
def c1( rawdata ) :
h = binascii.b2a_hex( rawdata )
z = bin( int( h, 16 ) )[ 2 : ]
return '0' * ( 8 * len( r ) - len( z ) ) + z
# Convert each byte directly
def c2( rawdata ) :
return ''.join( bin( ord( x ) )[ 2 : ].rjust( 8, '0' ) for x in r )
# Convert each byte using a list for table look-up
def c3( rawdata ) :
h = [ bin( i )[ 2 : ].rjust( 8, '0' ) for i in range( 256 ) ]
return ''.join( h[ ord( x ) ] for x in rawdata )
# Convert each byte using a dictionary for table look-up (avoids
# lots of ord() calls)
def c4( rawdata ) :
h = { chr( i ) : bin( i )[ 2 : ].rjust( 8, '0' ) for i in range( 256 ) }
return ''.join( h[ x ] for x in rawdata )
As you can see I even in c3() and c4() tried to speed things up
further by using a table look-up instead if calling bin() etc.
on each byte. But the results was that c2() is nearly 15 times
slower than c1(), c3() about 3 times and c4() still more than 2
times slower! So the method the OP uses seems to be quite a bit
more efficient than one might be tempted to assume.
I would guess that the reason is that c1() does just a small
number of calls of functions that probably aren't implemented
in Python but in C and thus can be a lot faster then anything
you could achieve with Python, while the other functions use a
for loop in Python, which seems to account for a good part of
the CPU time used. To test for that I split the 'rawdata' string
into a list of character (i.e. single letter strings) and re-
assembled it using join() and a for loop:
r = list( rawdata( )
z = ''.join( x for x in r )
The second line alone took about 1.7 times longer than the
whole, seemingly convoluted c1() function!
What I take away from this is that a lot of the assumption one
is prone to make when coming from e.g. a C/C++ background can
be quite misleading when extrapolating to Python (or other in-
terpreted languages)...
Best regards, Jens
--
\ Jens Thoms Toerring ___ [email protected]
\__ http://toerring.de
--
http://mail.python.org/mailman/listinfo/python-list
Re: in need of some help...
Alex Norton wrote: > thanks... ill take a look at the Qt event handling It's rather simple: instead of the program running through a sequence of steps, the program normally is basically doing nothing. It just reacts to events that normally come from the user, i.e. the user clicks on some icon or widget, or (s)he enters something via the keyboard. You etermine which of all the possible events to which widget are relevant to you, write handler functions for them and tell the widget to call some function when an event happens. The simplest case is a button: you want to react to it, so you write a function for what's to be done when it's clicked on and then tell Qt to call that function once it gets clicked (there are different events even for a simple push button, it can be clicked, just pushed, released etc.). And if you have set up everything for that you tell Qt to start waiting for events. So the steps are: 1. Tell Qt that this is a program using it app = QtGui.QApplication( sys.argv ) 2. Create your graphical interface (what you seem to have done more or less) 3. Connect desired events (what's called "signals" in Qt lingo) for a certain widget to the function to be called with something like your_widget.clicked.connect( your_function ) (replace 'clicked' with e.g. 'pushed' or 'released' when interested in a push or release signal instead) 4. Start the event loop (i.e. have Qt wait for the user to do something and call one of your functions if the user did something you're interested in) with app.exec_( ) When this returns the game is over. So you don't wait for keyboard input with input() like in your original program but instead tell Qt to do the waiting for you and call the appropriate function you defined when something interesting happens. What you probably will have to change about the graphical interface is that instead of using QLabel widgets for 'Air', 'Earth', 'Fire', 'Water' to use e.g. QPushButtons since QLabels are rather static objects - they don't receive any "click" events and it's rather likely some kind of event like this is what you're going to want to react to. And for that QPushButtons seem to be the simplest choice to start with. So have an 'Air' button (let's call it 'bAir' and then do bAir.clicked.connect( air_clicked ) after defining a function air_clicked() in which you deal with that case. that might be as simple as def air_clicked( ) : # Randomly pick one of 'air', 'fire', 'water' or 'earth' z = [ 'air', 'fire', 'water', earth' ][ random.randrange( 4 ) ] if z == 'air' : print( 'Stalemate' ) elif z == 'water' : print( 'Air removes Water, you win!' ) ... Now, when during the game the 'Air' button is clicked this function will get called. Of course, it might be nicer to have a "result" label some- where in the graphical interface which you set to the text instead of printing it out to the console. And you also will probably add some "Quit" button to end the game. Regards, Jens -- \ Jens Thoms Toerring ___ [email protected] \__ http://toerring.de -- http://mail.python.org/mailman/listinfo/python-list
Re: Sum operation in numpy arrays
Ana Dionísio wrote: > Hello! I have several numpy arrays in my script and i want to add them. For > example: > a=[1,2,3,4,5] > b=[1,1,1,1,1] > c=[1,0,1,0,1] > for i in range(5): > d[i]=a[i]+b[i]+c[i] > print d > [3,3,5,5,7] > I did it like that but I get an error: "TypeError: unsupported operand > type(s) for +: 'float' and 'numpy.string_'" > How can I sum my arrays? There's no numpy array at all in use in your example program, you only have normal lists. If you had numpy arrays you wouldn't need to loop over the indivindual elements, you could do just d = a + b + c to add up the (numpy) arrays. The error message you get may indicate that somewhere in your real code (not the one you posted) one of the elements of one of the (numpy) arrays is not a number but a string, e.g. by doing some- thing like a = numpy.array( [ 1, '2', 3, 4, 5 ] ) (note the single quotes around 2, that will result in the element having a type of 'numpy.string'. But if it's like that is impossible to tell without seeing the real code you're using;-) I guess you will have to give us what you actually use if you want something more helpful. Regards, Jens -- \ Jens Thoms Toerring ___ [email protected] \__ http://toerring.de -- http://mail.python.org/mailman/listinfo/python-list
Re: Help with loading file into an array
peter berrett wrote: > I am trying to build a program that can find comets in a series of > astronomical images. I have already written functions to find the comet in a > series of images, the data of which is stored in embedded lists. > The area I am having difficulty with is take a standard gif file (1024 x > 1024) and reading it into an array or embedded lists. > In a nutshell here is an example of what I want to do > Let's say I have a gif file called 20130428__c2_1024.gif in a folder > called c:\comets > I want to read the data from that gif file taking the red data (excluding > the green and blue data) and store that in an array called Image[][] which > is a nested array length 1024 with a list in each item of 1024 length (ie > 1024 x 1024) > Could someone please provide a piece of code to do the above so I can then > go on to modify it to pick up different files from different folders? In > particular I am keen to seen how you read in the data and also how you > change the directory from which you are reading the image. > the following should do the trick using, as Fábio already suggested, the Python Image Library (PIL): 8<- #!/ur/bin/env python import Image im = Image.open( 'c:/comets/20130428__c2_1024.gif' ) w, h = im.size red_arr = [ ] for y in range( h ) : red_line = [ ] for x in range( w ) : red_line.append( im.getpixel( ( x, y ) )[ 0 ] ) red_arr.append( red_line ) 8<- For obvious reasons I couldn't use 'Image' as the name of the list of lists, so it's 'red_arr' instead. This is probably not the fas- test solution, but it's simple and hopefully will get you started. Concerning reading other files: here I may not understand your problem since it looks rather trivial to me by simply passing the open() method of 'Image' the name of a file in a different directory. Regards, Jens -- \ Jens Thoms Toerring ___ [email protected] \__ http://toerring.de -- http://mail.python.org/mailman/listinfo/python-list
Re: Why do Perl programmers make more money than Python programmers
In comp.lang.python Steven D'Aprano wrote: > On Sun, 05 May 2013 12:11:11 -0500, Ignoramus16992 wrote: > > According to CIO.com, Python programmers make only $83,000 per year, > > while Perl programmers make $93,000 per year. > > > > http://www.cio.com/slideshow/detail/97819?source=ifwartcio#slide10 > > http://www.cio.com/slideshow/detail/97819?source=ifwartcio#slide11 > > > > I would like to know, what explains the discrepancy. > Perl is much harder to use, so the average Perl programmer burns out > after a few years and takes up a less stressful career, like going > undercover in the Russian mob or the Taliban. So only the most dedicated, > brilliant and extreme programmers last long enough to become a Perl > expert, and consequently can demand higher pay, while any idiot can learn > to program Python, as I have. > Also, Perl programmers are an unprincipled, devious bunch, always looking > for an opportunity to blackmail their employers into paying them extra. > Python programmers are a decent, law-abiding people with a strong moral > code who would never stoop to the sort of things that Perl coders are > proud of doing. Now you got me badly worried, using both Perl and Python (and other, unspeakable languages, but not VB I promise!) Will I end up as a Python hacker for the mob or worse - or is there a chance of redemption (perhaps after a few years in Guanta- namo bay)? And should I, while it lasts, get the Perl or the Python salary, or the mean or both combined? Got to consider that when applying for my next job! Regards, Jens -- \ Jens Thoms Toerring ___ [email protected] \__ http://toerring.de -- http://mail.python.org/mailman/listinfo/python-list
Re: Why do Perl programmers make more money than Python programmers
In comp.lang.python Rainer Weikusat wrote: > [email protected] (Jens Thoms Toerring) writes: > > In comp.lang.python Steven D'Aprano > > wrote: > >> On Sun, 05 May 2013 12:11:11 -0500, Ignoramus16992 wrote: > > > >> > According to CIO.com, Python programmers make only $83,000 per year, > >> > while Perl programmers make $93,000 per year. > >> > > >> > http://www.cio.com/slideshow/detail/97819?source=ifwartcio#slide10 > >> > http://www.cio.com/slideshow/detail/97819?source=ifwartcio#slide11 > >> > > >> > I would like to know, what explains the discrepancy. > > > >> Perl is much harder to use, so the average Perl programmer burns out > >> after a few years and takes up a less stressful career, like going > >> undercover in the Russian mob or the Taliban. So only the most dedicated, > >> brilliant and extreme programmers last long enough to become a Perl > >> expert, and consequently can demand higher pay, while any idiot can learn > >> to program Python, as I have. > > > >> Also, Perl programmers are an unprincipled, devious bunch, always looking > >> for an opportunity to blackmail their employers into paying them extra. > >> Python programmers are a decent, law-abiding people with a strong moral > >> code who would never stoop to the sort of things that Perl coders are > >> proud of doing. > > > > Now you got me badly worried, using both Perl and Python (and > > other, unspeakable languages, but not VB I promise!) Will I > > end up as a Python hacker for the mob or worse > https://en.wikipedia.org/wiki/Strange_Case_of_Dr_Jekyll_and_Mr_Hyde > [SCNR] Well, that didn't have a happy ending:-( Should have listened to my parents when they told me again and again "Never use Perl, just say no!". Seems I'm doomed - what's the proper way to apply for a job with the mob? -- \ Jens Thoms Toerring ___ [email protected] \__ http://toerring.de -- http://mail.python.org/mailman/listinfo/python-list
Re: Why do Perl programmers make more money than Python programmers
In comp.lang.python Roy Smith wrote: > In article , > [email protected] (Jens Thoms Toerring) wrote: > > Well, that didn't have a happy ending:-( Should have listened to > > my parents when they told me again and again "Never use Perl, just > > say no!". Seems I'm doomed - what's the proper way to apply for a > > job with the mob? > I don't think you apply. If they want you, they'll find you. I see, that's what's called headhuntering, isn't it? -- \ Jens Thoms Toerring ___ [email protected] \__ http://toerring.de -- http://mail.python.org/mailman/listinfo/python-list
Re: ANN: Quest for the Holy Grail (a PyGame game)
very nice game, i like it. -Horst -- http://mail.python.org/mailman/listinfo/python-list
Re: join a samba domain
Toff wrote: d = c.Win32_ComputerSystem d.JoinDomainOrWorkGroup(None, 3, "mydom", "mydompw", r"admin\\mydom") Shouldn't r"admin\\mydom" be "admin\\mydom" or r"admin\mydom". Or maybe just "admin" /Jens Henrik -- http://mail.python.org/mailman/listinfo/python-list
Re: flock seems very unsafe, python fcntl bug?
Your problem is that open(...,'w') is not locked.
Use something like:
lockf = open('aaa', 'a')
fnctl.flock(lockf,fnctl.LOCK_EX)
file = open('aaa', 'w')
file.write('asdf')
file.close()
lockf.close()
[EMAIL PROTECTED] wrote:
I ran following 2 programs (lock1, lock2) at almost same time,
to write either "123456", or "222" to file "aaa" at the same time.
But I often just got "222456" in "aaa" .
Is this a bug of python fcntl module ? See 2 programs I ran:
#!/usr/bin/env python
import fcntl, time
file = open('aaa', "w")
fcntl.flock(file, fcntl.LOCK_EX)
file.write('123456')
time.sleep(10)
file.close()
#!/usr/bin/env python
import fcntl, time
file = open('aaa', "w")
fcntl.flock(file, fcntl.LOCK_EX)
file.write('222')
time.sleep(10)
file.close()
--
http://mail.python.org/mailman/listinfo/python-list
