Re: msvcr90.dll is MIA?
Maybe this is a stupid question. I don't know, I don't use windows. But what's so special about msvcr and visual studio compiler? Python compiles fine with gcc under unixes, so is it a problem to compile python interpreter with mingw and get rid of the proprietary runtime dependecies? Or does the windows python port make heavy use of ms' libs? -- http://mail.python.org/mailman/listinfo/python-list
RE: [melbourne-pug] array matching
You could use dictionary of lists
test = {
A: [1, 3],
B: [2, 4]
}
print test[A]
[1, 3]
test[A].append(5)
print test[A]
[1, 3, 5]
Cheers,
Fil
_
From: [email protected]
[mailto:[email protected]] On Behalf Of
Bill Jordan
Sent: Friday, 30 April 2010 3:58 AM
To: [email protected]; [email protected]; [email protected];
[email protected]; [email protected]
Subject: [melbourne-pug] array matching
Hey guys,
I am sorry if this is not the right list to post some questions. I have a
simple question please and would appreciate some answers as I am new to
Python.
I have 2 D array: test = [[A,1],[B,2],[A,3][B,4]]
I want to arrang this array in different arrays so each one will have what
is attached to. For example I want the output:
A =[1,3] and B=[2,4]
Thanks,
Bill
--
http://mail.python.org/mailman/listinfo/python-list
python fast HTML data extraction library
Hello, Sometime ago I was searching for a library that would simplify mass data scraping/extraction from webpages. Python XPath implementation seemed like the way to go. The problem was that most of the HTML on the net doesn't conform to XML standards, even the XHTML (those advertised as valid XHTML too) pages. I tried to fix that with BeautifulSoup + regexp filtering of some particular cases I encountered. That was slow and after running my data scraper for some time a lot of new problems (exceptions from xpath parser) were showing up. Not to mention that BeautifulSoup stripped almost all of the content from some heavily broken pages (50+KiB page stripped down to some few hundred bytes). Character encoding conversion was a hell too - even UTF-8 pages had some non- standard characters causing issues. Cutting to the chase - that's when I decided to take the matter into my own hands. I hacked together a solution sporting completely new approach overnight. It's called htxpath - a small, lightweight (also without dependencies) python library which lets you to extract specific tag(s) from a HTML document using a path string which has very similar syntax to xpath (but is more convenient in some cases). It did a very good job for me. My library, rather than parsing the whole input into a tree, processes it like a char stream with regular expressions. I decided to share it with everyone so there it is: http://code.google.com/p/htxpath/ I am aware that it is not beautifully coded as my experience with python is rather brief, but I am curious if it will be useful to anyone (also it's my first potentially [real-world ;)] useful project gone public). In that case I promise to continue developing it. It's probably full of bugs, but I can't catch them all by myself. regards, Filip Sobalski -- http://mail.python.org/mailman/listinfo/python-list
Re: python fast HTML data extraction library
On Jul 23, 3:53 am, Paul McGuire wrote:
> # You should use raw string literals throughout, as in:
> # blah_re = re.compile(r'sljdflsflds')
> # (note the leading r before the string literal). raw string
> literals
> # really help keep your re expressions clean, so that you don't ever
> # have to double up any '\' characters.
Thanks, I didn't know about that, updated my code.
> # Attributes might be enclosed in single quotes, or not enclosed in
> any quotes at all.
> attr_re = re.compile('([\da-z]+?)\s*=\s*\"(.*?)\"', re.DOTALL |
> re.UNICODE | re.IGNORECASE)
Of course, you mean attribute's *value* can be enclosed in single/
double quotes?
To be true, I haven't seen single quote variant in HTML lately but I
checked it and it seems to be in the specs and it can be even quite
useful (man learns something every day).
Thank you for pointing that one out, I updated the code accordingly
(just realized that condition check REs need an update too :/).
As far as the lack of value quoting is concerned, I am not so sure I
need this - It would significanly obfuscate my REs and this practice
is rather deprecated, considered unsafe
and I've seen it only in very old websites.
> How would you extract data from a table? For instance, how would you
> extract the data entries from the table at this
> URL:http://tf.nist.gov/tf-cgi/servers.cgi? This would be a good example
> snippet for your module documentation.
This really seems like a nice example. I'll surely explain it in my
docs (examples are surely needed there ;)).
> Try extracting all of the sldjlsfjd links from
> yahoo.com, and see how much of what you expect actually gets matched.
The library was used in my humble production environment, processing a
few hundred thousand+ of pages and spitting out about 1 SQL
records so it does work quite good with a simple task like extracting
all links. However, I can't really say that the task introduced enough
diversity (there were only 9 different page templates) to say that the
library is 'tested'...
On Jul 26, 5:51 pm, John Machin wrote:
> On Jul 23, 11:53 am, Paul McGuire wrote:
>
> > On Jul 22, 5:43 pm, Filip wrote:
>
> > # Needs re.IGNORECASE, and can have tag attributes, such as > CLEAR="ALL">
> > line_break_re = re.compile('', re.UNICODE)
>
> Just in case somebody actually uses valid XHTML :-) it might be a good
> idea to allow for
>
> > # what about HTML entities defined using hex syntax, such as &#;
> > amp_re = re.compile('\&(?![a-z]+?\;)', re.UNICODE | re.IGNORECASE)
>
> What about the decimal syntax ones? E.g. not only and
> but also
>
> Also, entity names can contain digits e.g. ¹ ¾
Thanks for pointing this out, I fixed that. Although it has very
little impact on how the library performs its main task (I'd like to
see some comments on that ;)).
--
http://mail.python.org/mailman/listinfo/python-list
ANN: fathom 0.4.0, fathom-tools 0.1.0, qfathom 0.1.0
Hello everyone! I would like to announce new version of fathom as well as two more packages built on top fathom. All this code is still very young and experimental, so it is not supposed to be production ready. However, I would greatly appreciate any advice or ideas on what could be useful and what would you expect of those packages. fathom 0.4.0 Fathom is python 3 library for inspecting database schema. It produces objective model of all database objects and thus allows easy database inspection. Changes: * Indices are top level objects and provide information about index uniqueness * Procedure objects provide information about arguments for MySQL * Rather than raise database specific exception library strives to raise FathomError http://code.google.com/p/fathom/ fathom-tools This is a separate package with utilities that were earlier in fathom package. It includes: * fathom2django which produces django models from database schema and now handles many to many fields, which makes it better than Django's inspectdb * fathom2graphviz which produces graphviz dot files with entity-relationship diagrams * utilities for building new tools based on fathom library http://code.google.com/p/fathom-tools qfathom QFathom is graphical tool written in python3 and PyQt for inspecting the database. It is basically a graphical frontend for fathom. http://code.google.com/p/qfathom/ And once again any kind of ideas or comments would be greatly appreciated. -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
ANN: geany-pyflakes 1.0
Hello everyone, I have just published a small plugin for Geany IDE that adds Pyflakes error detection to the editor. If there are people using Geany for Python development I would be very grateful for opinions and suggestions. The plugin can be found here: http://code.google.com/p/geany-pyflakes/ -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Re: ANN: geany-pyflakes 1.0
> fyi - the downloaded geany-pyflakes-1.0.tar.gz is in fact not gzipped and > should either be gizzped or simply named geany-pyflakes-1.0.tar. > > However, after activating the plugin, asking geany for a new file breaks > geany and it rudely closes so I'm unable to actually do anything with it. I have posted a gzipped version with a fix. Would you care to take a look? -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Re: ANN: geany-pyflakes 1.0
> OK - it no longer crashes, but it doesn't appear to do anything. Is there > something further I need to do to configure it? The Pyflakes tab does > appear in the message window and I've created a new testpyflakes.py with > code similar to your example. OK, so there must be something wrong. Do you have pyflakes installed? Where are they installed? > Also, just FYI, the plugin manager now shows the version as 1.0.1 while the > tar file name indicates 1.0.2. Yeah, I should make a configure file with version and use it in c code, to never make that mistake again, but I really, really don't wanto to touch autotools again. -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
function annotations in open source projects
Do you know any open source python3 projects that use function annotations? I would like to see some real use, so maybe I find them useful to use in my own project. -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Re: function annotations in open source projects
Thanks for the links. -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
ANN: fathom 0.1.0
Hello everyone! I have created a small python package for python3 called fathom that provides database inspection. It is in a very early stage, rather a proof of concept right now. It provides basic information about database schema and works with Sqlite3, PostgreSQL and MySQL. I am looking for comments and additional ideas for new features.. Documentation can be found here: http://code.google.com/p/fathom/wiki/Manual Package can be downloaded from PyPi or from here: http://code.google.com/p/fathom/downloads/list -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Re: ANN: fathom 0.2.0
I forgot links: Homepage: http://code.google.com/p/fathom/ Documentation: http://code.google.com/p/fathom/wiki/Manual -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
ANN: fathom 0.2.0
Hi, I have released version 0.2.0 of fathom, python3 package for database inspection. Fathom supports retrieving database schema from Sqlite3, PostgreSQL and MySQL. This is still very early version and I am experimenting with different approaches. I would be very thankful for any input and suggestions about what would be useful in the library or tools. **Changes in Version 0.2.0** * get_database convenience function, that doesn't require knowledge of database type * Index object provides information about columns, that are indexed * Table provides information about foreign keys * Procedure objects provide return value and procedure's body * Database object provides trigger list * Utility function get_database_type, that tries to guess database type * Tools: -> fathom2django that generates django models from database schema -> fathom2graphviz that generates entity-relationship diagrams in graphviz dot language, that can be turned into pdf or image files -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
ANN: Fathom 0.3.0
Hi, I have released version 0.2.0 of fathom, python3 package for database inspection. Fathom supports retrieving database schema from Sqlite3, PostgreSQL, MySQL and Oracle. This is still very early version and I am experimenting with different approaches. As always I would be very thankful for any input and suggestions about what would be useful in the library or tools. Here you can download the package: http://code.google.com/p/fathom/downloads/list **Changes in 0.3.0:** -> Oracle support :-) -> Triggers provide information about event upon which they are fired -> utility function for finding procedures that access a table -> fathom2graphviz draws lines between columns for foreign key -> fathom2graphviz and fathom2django can now print output to a file, rather than to std -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Fwd: How can I build embeddable zip python on Linux
Dear Python team, I have tried to build the GDB with Python support for windows. Because I didn’t want to have any Python dependence on building system, I decided to embed python in GDB using Windows x86_64 embeddable zip file and with it I had success. When I tried to build GDB with Python support on Linux, I realized I couldn’t find embeddable python for Linux. I have tried to find some article how can I build embeddable Python from source in Linux and only what I have found is this: https://stackoverflow.com/questions/42280553/how-to-build-embeddable-python but again it is only for Windows. Please, can you give me some instructions how can I build embeddable Python from source in Linux? Is it possible in the Linux environment? Please let me know if I can assist any further. With best regards, M.Eng. Filip Bascarevic -- https://mail.python.org/mailman/listinfo/python-list
Re: Extend Python
> My Question: > Swig offers some great features but is to basic for us. Is there > another program that creates more readble code that can be easily > edited? How much work is it to write our own wrappers? Not too much, and practicaly not at all if you want to wrap procedures (not objects or types): - marshaslling an argument list from Python to C and from C to Python takes exactly 1 line of code, - handling errors takes 2 lines of code (unless you want to do something fancy with them; I mean just catching an exception and getting the error message). - entry in the PyMethodDef is one simple line Bottomline: you need 5 additional lines of C code per procedure to make it usable from Python. Unless you have hundreds of procedures, there is no point in using special tools to do that. Especially if you need full control over the results. regards, Filip Dreger -- Akademia Linuksa - http://www.akademia.linuksa.pl -- http://mail.python.org/mailman/listinfo/python-list
func_code vs. string problem
Each function has a func_code property that is suposed to contain the pure bytecode of the function. All the context (including reference to relevant namespaces) is stored in different fields of the function object. Since 'exec' is able to execute any string or bytecode in the current scope, it would seem possible to execute code of any function in any namespace. But no matter how I tried, I could not do it. There must be something I am missing. Here's what I do:(if anyone wants to help, I placed the source under http://www.bajobongo.net/foo.py - tested on Python 2.4.1) 1. I declare a function. In the next steps I will try to run its code from inside a class: def myfunction(): print abc self.test() 2. I declare a class foo, with two methods. The first one tries to reach some local variables from a string passed to exec. The other one tries to do the same from inside a bytecode (from myfunction). IMHE this should make no difference to 'exec' - [spoiler: it does]. class foo: def test(self): print "ABC" def checkfunction(self): abc=10 exec myfunction.func_code def checkstring(self): abc=10 exec "print abc;self.test()" 3. I test the both methods. Sadly, the 'checkfunction' fails to find the correct namespace (id does not see 'abc' nor 'self'). Adding things like: "exec myfunction.func_code in globals(),locals()" does not help. i=foo() i.checkstring() i.checkfunction() # this throws exception; why??? 4. I try to find some help here, and hope to also gain better undesrtanding of how Python works :-) Thanks for any suggestions, regards, Filip Dreger -- http://mail.python.org/mailman/listinfo/python-list
Re: func_code vs. string problem
I came up with a simpler description of the problem. It's all in the simple source: # we define 'print b' in three different ways: as a string, # a bytecode and a function string="print b" code=compile(string,'string','exec') def function(): print b # now we make functions that test if it is possible to execute 'print b' # in some local scope def execstring(): b=5 exec string def execfunction(): b=5 exec function.func_code def execcode(): b=5 exec code execstring() # works execcode() # works execfunction() # throws name error exception... My problem is that both 'string' and 'code' are references to code objects, so they _should_ behave in the same way... I am close to giving up... I am trying to find a way of executing functions without creating a nested scope, so they can share local and global namespace (even if they are declared in some other module). I _could_ turn them into strings and pass around as compiled objects, butthis would be very ugly. I am sure Python has some better, cleaner way to do this. regards, Filip Dreger -- http://mail.python.org/mailman/listinfo/python-list
Re: func_code vs. string problem
Uzytkownik "Steven Bethard" <[EMAIL PROTECTED]> napisal w wiadomosci news:[EMAIL PROTECTED] > Filip Dreger wrote: >> I am trying to find a way of executing functions without creating a >> nested scope, so they can share local and global namespace (even if >> they are declared in some other module). > > Why? Can you explain what the use case is? I have a large set of objects of a single class - 'actor'. They are created all at once, and they all live in a sort of container written in C. The methods of the functions are few and general. One of them is simply 'act()', that is called by the C program every X seconds (tics, whatever). Each object can play 0..N roles, and the roles played by an object often change during its lifetime. The roles should be played in the act() method. Also, the roles should be defined in some simple way, and in an external python file (so adding/removing roles does not require recompiling of the parts embedded in C). If I had a magic function 'exec in current scope' I would implement it like this: class actor: def __init__(): self.roles=[] def act(): for i in self.roles: exec i in current scope then the roles would simply be functions defined in any importable file. For example creating an actor that logs each time it is called would be as simple as: import actor def log(): self.counter+=1 print "called %d times"%self.counter a=actor.actor() a.counter=0 a.roles.append(log) Let me recapitulate (is that the right English word?): 1. I need to keep the definitions of roles in a separate file (they can not simply be additional methods) 2. I need the roles to have full access to global and local namespace of the actor object (sometimes they have to change it, and sometimes they have to use some external modules) - the roles should work like plugins. 3. I guess I also need a fresh look at the problem. Maybe I am simply going the wrong way? I am more than willing to change my design, if someone shows me the light :-) regards, Filip Dreger -- http://mail.python.org/mailman/listinfo/python-list
Re: func_code vs. string problem
Uzytkownik "Steven Bethard" <[EMAIL PROTECTED]> napisal w wiadomosci news:[EMAIL PROTECTED] > See the documentation: > > http://docs.python.org/ref/dynamic-features.html > > """The eval(), execfile(), and input() functions and the exec > statement do not have access to the full environment for resolving > names. Names may be resolved in the local and global namespaces of > the caller. Free variables are not resolved in the nearest enclosing > namespace, but in the global namespace.""" Thank you! I feel silly I have not found the piece of instruction myself. And even worse, as I still do not understand: if exec() resolves free bariable names in the global namespace, how come it works well with code in a string? Or with a code in a compiled string? > Note the last sentence, which tells you that your free variable, > 'abc', will be resolved in the *global* namespace. In your > particular problem, you can solve this by substituting your local > namespace for the global namespace. This particular problem - sadly - is a simplified version of the real one. I need access to both: global and local namespaces, so this solution is not for me. The manual says I can pass two dictionaries to exec, one for global namespace and one for local. But, strange as it seems, I could not get it to work. In the example I gave, changing exec f.func_code to exec f.func_code in globals(),locals() does not help a bit. > One possible workaround might be: > > py> class foo: > ... def test(self): > ... print "ABC" > ... def checkfunction(self): > ... abc=10 > ... l = locals() > ... l.update(globals()) > ... exec myfunction.func_code in l > ... def checkstring(self): > ... abc=10 > ... exec "print abc;self.test()" > ... > py> foo().checkfunction() > __main__.foo > 10 > ABC I thought about that... I don't know why, but it seems wrong. Maybe the updating dictionaries is not very expensive, but there must be some better way. This code has a 'workaround - fixme' written all over it. Up to now, each time I encountered such a workaroundish solution, it turned out Python has a cleaner way to do it. I sure hope this is also the case here :-). > But I'd have to know what your real use case is to tell you whether > or not this is worth the trouble. Why do you want to exec the > func_code anyway? Why can't you just call the function? I put the description in the other post. Perhaps it's jkust my design that's broken. Thanks again. regards, Filip Dreger -- http://mail.python.org/mailman/listinfo/python-list
Re: func_code vs. string problem
Uzytkownik "Steven Bethard" <[EMAIL PROTECTED]> napisal w wiadomosci news:[EMAIL PROTECTED] > Any reason you can't define it like: > > class actor(object): > def __init__(self): > self.roles = [] > def act(self): > for role_func in self.roles: > role_func(self) [snip] > By importing actor, they have full access to the global namespace of > the actor.actor object, by simply accessing the actor module > attributes. > > So the issue here is really about full *local* namespace access. Do > you really need *full* local namespace access? Why isn't access to > the actor.actor instance sufficient? !!! Yep, of course it is sufficient. Abondoning the obvious role_func() must have had some good reasons some time ago, but now I can not even remember them, probably they were not so important :-) Thanks a million, Filip -- http://mail.python.org/mailman/listinfo/python-list
Re: Can .py be complied?
U¿ytkownik "monkey" <[EMAIL PROTECTED]> napisa³ w wiadomo¶ci news:[EMAIL PROTECTED] > If using Jython to complie, is the end-user need JRE instead of > Python > installed, or need both of them? Only JRE. Just like Java. >> I don't know the exact details, but try using the compiled Python >> scripts (bytecode). I believe they are semi-optimized and platform >> independent. They are the .pyc and .pyo files generated when the >> script >> is run. > Is that means a .py convert to .pyc or .pyo, without the need of > "make file" > as using py2exe? .pyc files are generated every time a module (any .py file can be a module) is imported. So if you have a program, say, example.py, you just start the python interpreter and write: >>> import example And then example.pyc will appear beside example.py. This new file does not require example.py (you can even delete it), and works on any computer with Python installed (on Windows you can just double-click it) If you start the Python interpreter using: python -OO (if you are using Windows, you shoud start the interpreter from the command line, probably something like: c: cd \ python24\python -OO) and then import your example.py, you will get a file example.pyo, which is also stripped of any documentation strings (a bit harder to decode). regards, Filip Dreger -- http://mail.python.org/mailman/listinfo/python-list
Re: why "import wx" doesn't work?
U¿ytkownik "monkey" <[EMAIL PROTECTED]> napisa³ w wiadomo¶ci news:[EMAIL PROTECTED] >> Which version of wxPython are you running? What do you mean by >> "does not >> work"...does the import fail or is your code giving errors? > > It is the current new version 2.6. The error message said that the > class > wxApp is not defined... This is very good! wxApp is never defined if you use "import wx". You must use "wx.wxApp" instead. If you import a module using "import anything", then all the names imported from the module must begin with "anything.". If you import wx using "import wx", then ALL the wx commands, classes and variables (all the names) MUST begin with 'wx.". Change them, and your program will work. "from wx import *" is a special shortcut, allowing you to use all the names without "wx.". If you change "from something import *" to "import something", your code will always break, this is normal. regards, Filip Dreger -- http://mail.python.org/mailman/listinfo/python-list
Python CI and CD support available on Semaphore (feedback appreciated)
HI folks, We have recently launched Python support on our continuous integration and deployment service and are looking for communities feedback. If you're up for it, please give a test drive to our service with your Python projects and give us your thoughts on what we could further improve upon. Official blog post announcing Python support: http://bit.ly/1KYrVsT Our website: https://semaphoreci.com/ Thanks in advance! BTW we offer free plans for open source projects and have limited free plans for private projects. -- https://mail.python.org/mailman/listinfo/python-list
Re: sending mailing list with smtplib
On 15 Aug 2006 13:41:53 -0700, 3KWA <[EMAIL PROTECTED]> wrote: > What would be the best way to go about it then? Instantiate a new msg > in the loop? > > I guess I must read the doc more carefully, thanks for your time (if > you can spare some more I would be grateful). You can reuse your message object, but you need to delete the old header before setting a new one: <http://docs.python.org/lib/module-email.Message.html#l2h-3843> -- filip salomonsson -- http://mail.python.org/mailman/listinfo/python-list
Re: Visibility against an unknown background
On 10/24/06, Sergei Organov <[EMAIL PROTECTED]> wrote: > I'd be very upset to see, say, 5-6 highly intersecting > scientific plots on the same picture drawn using the > "marching ants" approach. I'd be a bit upset to see scientific plots *on a picture* at all, regardless of approach. -- filip salomonsson -- http://mail.python.org/mailman/listinfo/python-list
Re: Exploiting Dual Core's with Py_NewInterpreter's separated GIL ?
robert wrote: > I'd like to use multiple CPU cores for selected time consuming Python > computations (incl. numpy/scipy) in a frictionless manner. > > Interprocess communication is tedious and out of question, so I thought about > simply using a more Python interpreter instances (Py_NewInterpreter) with > extra GIL in the same process. > I expect to be able to directly push around Python Object-Trees between the 2 > (or more) interpreters by doing some careful locking. I don't want to discourage you but what about reference counting/memory management for shared objects? Doesn't seem fun for me. Take a look at IPython1 and it's parallel computing capabilities [1, 2]. It is designed to run on multiple systems or a single system with multiple CPU/multi-core. It's worker interpreters (engines) are loosely coupled and can utilize several MPI modules, so there is no low-level messing with GIL. Although it is work in progress it already looks quite awesome. [1] http://ipython.scipy.org/moin/Parallel_Computing [2] http://ipython.scipy.org/moin/Parallel_Computing/Tutorial fw -- http://mail.python.org/mailman/listinfo/python-list
Re: python socket proxy
[EMAIL PROTECTED] wrote: > Hi all > > I am trying to create a lighweight tcp proxy server. [...] There is a bunch of nice recipies in the Python Cookbook on port forwarding. In the [1] and [2] case it should be fairly simple to add an extra authentication step with pyOpenSSL. [1] http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/483730 [2] http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/114642 [3] http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/483732 best, fw -- http://mail.python.org/mailman/listinfo/python-list
Re: Update on Memory problem with NumPy arrays
sonjaa wrote: > Hi > > last week I posted a problem with running out of memory when changing > values in NumPy arrays. Since then I have tried many different > approaches and > work-arounds but to no avail. [...] Based on the numpy-discussion this seems to be fixed in the SVN now(?). Anyway, you can use 'where' function to eliminate the loops: from numpy import * y = ones((501,501)) z = zeros((501,501)) it = 50 for kk in xrange(it): y[1,1] = 4 y[1,2] = 4 y[1,0] = 4 y[2,1] = 6 print "Iteration #:%s" %(kk) y = where((y == 4) | (y == 6), 2, y) best, fw -- http://mail.python.org/mailman/listinfo/python-list
Re: String negative indices?
[EMAIL PROTECTED] wrote: > Logically, I should be able to enter x[-2:-0] to get the last and next to > last characters. However, since Python doesn't distinguish between positive > and negative zero, this doesn't work. Instead, I have to enter x[-2:]. Hooray! Logically there is no such thing as positive or negative zero, or did I miss something in the primary? PS. x[len(x)-2 : len(x)-0] cheers, fw -- http://mail.python.org/mailman/listinfo/python-list
Re: String negative indices?
Steven D'Aprano wrote: > On Fri, 23 Jun 2006 02:17:39 -0700, Filip Wasilewski wrote: > > > [EMAIL PROTECTED] wrote: > > > >> Logically, I should be able to enter x[-2:-0] to get the last and next to > >> last characters. However, since Python doesn't distinguish between > >> positive and negative zero, this doesn't work. Instead, I have to enter > >> x[-2:]. > > > > Hooray! Logically there is no such thing as positive or negative zero, > > or did I miss something in the primary? > > No, not in the primary, or even in the secondary, but possibly in the > tertiary. > > For many purposes, it doesn't make sense to distinguish +0 from -0. But > for other purposes, it does. I believe you will agree that this is mostly a matter of notation of some fact rather than really trying to sign zero. For example I use to denote numbers that approach zero from "the right side" by x -> [0 uppercase +], which is definitely harder to type and display in poor man's text editor than x -> +0. Otherwise I should also rethink the meaning of positive (>0) and negative (<0) numbers (at least where such relations are defined). [...] > In statistical mechanics, some systems can have negative absolute > temperatures, including negative zero. Counter-intuitively, negative > absolute temperatures aren't colder than absolute zero, but hotter than > any positive temperature. So, strangely enough, a temperature of -0K is > hotter than a infinitely hot temperature! Yeah, this is really cool ;-) -- Filip "During board meeting: We have great news. Our earnings reached highest value ever - a positive 0." -- http://mail.python.org/mailman/listinfo/python-list
Re: String negative indices?
Steven D'Aprano wrote:
> On Sat, 24 Jun 2006 05:36:17 -0700, Filip Wasilewski wrote:
>
> > Steven D'Aprano wrote:
> >> On Fri, 23 Jun 2006 02:17:39 -0700, Filip Wasilewski wrote:
> >>
> >> > [EMAIL PROTECTED] wrote:
> >> >
> >> >> Logically, I should be able to enter x[-2:-0] to get the last and next
> >> >> to last characters. However, since Python doesn't distinguish between
> >> >> positive and negative zero, this doesn't work. Instead, I have to enter
> >> >> x[-2:].
> >> >
> >> > Hooray! Logically there is no such thing as positive or negative zero,
> >> > or did I miss something in the primary?
> >>
> >> No, not in the primary, or even in the secondary, but possibly in the
> >> tertiary.
> >>
> >> For many purposes, it doesn't make sense to distinguish +0 from -0. But
> >> for other purposes, it does.
> >
> > I believe you will agree that this is mostly a matter of notation of
> > some fact rather than really trying to sign zero.
>
> No.
>
> In floating point, +0 *really is* different from -0 -- the two floats
> have different bit patterns. Floating point libraries must be specifically
> programmed to ignore that difference, making zero treated differently than
> other floats. That's -- usually -- a good thing.
So does the fact that the "-0.0" and "+0.0" strings can be converted by
C compilator to floats and stored with different bit patterns means
that they can behave much different in computation and comparison than
real numbers written in an expression on a piece of paper? Rather not.
Floating point is only a (very useful) finite precision model (there
are others) of real (rational) numbers and these differences are
implementation details for me.
And as far as the division by signed floating point zero and similar
operations in IEEE 754 are concerned, again, this is only a model in
which 0.0 means a value that is either equal to real zero or greater,
but too small to be distinguished from zero using finite amount of bits
(similarly -0.0), so the expressions like
1.0/-0.0 -> -1.#INF
1.0/0.0 -> 1.#INF
have some justification, although being meaningless in ordinary
arithmetic.
To be not so off-topic, I also expect from Python's integer, float and
Decimal computer formats to behave as much similar to their equivalents
known from math lessons as possible, which in particular means '=='
equality of -0 and 0, -0.0 and 0.0, and Decimal("-0") and Decimal("0"),
no matter what.
> In mathematics, well, maybe... certainly in the Real number system, there
> is no difference, and +0 and -0 are just two ways of writing the same
> thing. In the hyperreals, +0 and -0 are the same, but there are
> infinitesimals which are different, and signed. I don't know enough about
> the surreals to comment. In matrix maths, there are an infinite number of
> different matrices where all the elements are zero -- they are all
> distinct, different, zeroes.
>
> And in "the real world", all numbers are an abstraction anyway, so I'm
> not too worried about correspondence to reality. If mathematicians find a
> use for distinguishing +0 from -0 (as the statistical physicists have
> done) they will treat them as "really" different. If they don't, they
> won't.
As far as math stuff is concerned I will make it short. You can't deny
that there is still unique additive identity in an additive group.
I also wouldn't mix symbols of physics states or others with ordinary
math expressions as this is only a matter of notation of some concept
(feel free to use any symbol you want).
Thanks for stimulating my neural cells, cheers,
fw
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to build extensions on Windows?
Kevin D. Smith wrote: > I've written a simple Python extension for UNIX, but I need to get it > working on Windows now. I'm having some difficulties figuring out how > to do this. I've seen web pages that say that MS Visual Studio is > required, and other that say that's not true, that MinGW will work. > Then there is Mike Fletcher's web page > (http://www.vrplumber.com/programming/mstoolkit/) that describes in > detail how to build extensions, but most of the links to external > software are no longer valid. I think it's safe to say that I am > completely lost, as there appears to be no authoritative, up-to-date > description on how to make this work. There is an easy way to build Python extensions on Windows with MinGW and it works fine for me. Just follow these steps: 1. Get MinGW gcc and/or g++, preferably via MinGW installer from [1]. You may have to restart your computer or manually edit PATH system environment variable to include MinGW's bin directory (default is c:\mingw\bin). Then check if it is there by typing `path` in the cmd window. 2. Get pexports-0.42h.zip from [2] and extract pexports.exe file 3. Prepare MinGW compatible .a library file pexports.exe c:\WINDOWS\system32\python24.dll > python24.def c:\mingw\bin\dlltool.exe --dllname python24.dll --def python24.def --output-lib libpython24.a 4. Place the new libpython24.a file in the Python's libs directory (but not in the Lib dir), default is c:\python24\libs 5. Build your extension by executing your setup script with `--compiler=mingw32` parameter. python setup.py build --compiler=mingw32 Additionally you may wish to put a distutils.cfg file in the c:\python\lib\distutils dir containing following entries: [build] compiler = mingw32 This will tell the distutils script to use MinGW compiler by default when executing `python setup.py build` command. best, fw [1] http://sourceforge.net/projects/mingw/ [2] http://starship.python.net/crew/kernr/mingw32/pexports-0.42h.zip -- http://mail.python.org/mailman/listinfo/python-list
Re: How to build extensions on Windows?
Jarek Zgoda wrote: > Filip Wasilewski napisa³(a): > > > There is an easy way to build Python extensions on Windows with MinGW > > and it works fine for me. Just follow these steps: > > It was brougt to my attention that mingw-compiled extensions for Python > 2.4 use other malloc() that Python 2.4, is it true? Can this have any > impact on stability? I have inspected the dll dependencies for python24.dll (ActiveState build, but I think this is also true for regular one) and my custom extensions built with MinGW and both of them use malloc() from msvcr71.dll. I think that as long as the extensions are linked against proper version of msvcr (7.1) there should not be such surprises. Other thing is that Python uses it's own memory allocator [1] and to avoid problems with memory corruption one should not try to allocate memory with PyMem_Malloc() and free it with free() and so forth. cheers, fw [1] http://www.python.org/doc/2.4.3/api/memoryOverview.html -- http://mail.python.org/mailman/listinfo/python-list
Re: Searching for patterns on the screen
Jerry Hill wrote:
> Hello all,
>
> I have a piece of code I could use some help optimizing. What I'm
> attempting to do is periodically grab a screenshot, and search for 2D
> patterns of black pixels in it. I don't care about any color other
> than black. Here's some simple code that simulates my worst-case
> scenario, scanning the whole screen for a sequence that does not
> exist:
>
> import ImageGrab # From the PIL library
>
> def removeColor(rgb):
> r, g, b = rgb
> return (r == 0 and g == 0 and b == 0)
>
> BMP = ImageGrab.grab.getdata()
> x = map(removeColor, BMP)
>
> The idea is to search for sequences of black pixels on a background
> that can change colors. To that end, I transform the screengrab into
> a sequence of either black or nonblack pixels as I search them. Note
> that in my actual code, I use imap so I only transform pixels as I
> search them, and I'm using the KnuthMorrisPratt algorithm from
> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/117214 to do
> the actual searching.
>
> >From some testing using the timeit module:
> map(None, BMP) takes about 0.6 seconds on a 1600x1200 screengrab.
> map(removeColor, BMP) takes about 1.5 seconds.
>
> I'd love to speed things up, if possible. It seems like what I'm
> doing would probably be a fairly well defined problem, but I don't
> know enough about the field to even know where to start my research,
> so even a list of keywords that would point me to discussion of
> similar topics would be welcome.
>
> This is being done in Python 2.5c2 using PIL 1.1.5
>
> --
> Jerry
Calling Python function for every pixel results in significant
overhead. Instead of that you can use numpy to do the preprocesing.
im = ImageGrab.grab()
a = numpy.fromstring(im.tostring(), dtype=[('r','|u1'), ('g','|u1'),
('b','|u1')])
a.shape = im.size[::-1]
black = (a['r'] == 0) & (a['g'] == 0) & (a['b'] == 0)
s = black.tostring() # sequence of '\x00' and '\x01'
You can also take a look at scipy ndimage
(http://www.scipy.org/doc/api_docs/scipy.ndimage.html) if in need of
more specific routines.
cheers,
fw
--
http://mail.python.org/mailman/listinfo/python-list
Re: xmlrpc, extract data from http headers
Milos Prudek wrote: > I perform a XML-RPC call by calling xmlrpclibBasicAuth which in turn calls > xmlrpclib. This call of course sends a HTTP request with correct HTTP > headers. The response is correctly parsed by xmlrpclib, and I get my desired > values. > > However, I also need to get the raw HTTP headers from the HTTP response. There > is a cookie in the HTTP response and I need to read that cookie. > > How could I do that? Overload the _parse_response method of Transport in your BasicAuthTransport and extract headers from raw response. See the source of xmlrpclib.py in the standard library for details. cheers, fw -- http://mail.python.org/mailman/listinfo/python-list
Re: xmlrpc, extract data from http headers
Milos Prudek wrote:
> > Overload the _parse_response method of Transport in your
> > BasicAuthTransport and extract headers from raw response. See the
> > source of xmlrpclib.py in the standard library for details.
>
> Thank you.
>
> I am a bit of a false beginner in Python. I have written only short scripts. I
> want to read "Dive into Python" to improve my knowledge. Your advice is
> perfect. It is my fault that I need a little more guidance.
>
> I am not sure how the headers will be passed from Transport to the instance of
> ServerProxy. That is, if I change the _parse_response method, how do I
> retreive the headers using the ServerProxy command?
Erm, now I see that my previous response was incorrect, sorry. The
headers are not passed to the _parse_response method.
A better solution would be to extract cookies from headers in the
request method and return them with response (see the code below). I
still wonder if there is an easy way to use CookieJar from cookielib
with xmlrpclib.
class CookieTransport(xmlrpclib.Transport):
def request(self, host, handler, request_body, verbose=0):
h = self.make_connection(host)
if verbose:
h.set_debuglevel(1)
self.send_request(h, handler, request_body)
self.send_host(h, host)
self.send_user_agent(h)
self.send_content(h, request_body)
errcode, errmsg, headers = h.getreply()
if errcode != 200:
raise ProtocolError(
host + handler,
errcode, errmsg,
headers
)
self.verbose = verbose
cookies = self.get_cookies(headers)
try:
sock = h._conn.sock
except AttributeError:
sock = None
response = self._parse_response(h.getfile(), sock)
if len(response) == 1:
response = response[0]
return response, cookies
def _get_cookies(self, headers):
import Cookie
c = []
for v in headers.getheaders('set-cookie'):
c.append(Cookie.SimpleCookie(v))
return c
server = xmlrpclib.ServerProxy("http://localhost:8000";,
transport=CookieTransport())
result, cookies = server.something.call()
best,
fw
--
http://mail.python.org/mailman/listinfo/python-list
Re: xmlrpc, extract data from http headers
Milos Prudek wrote: > > A better solution would be to extract cookies from headers in the > > request method and return them with response (see the code below). I > > Full solution! Wow! Thank you very much. I certainly do not deserve such > kindness. Thanks a lot Filip! Glad to help. All in all this is the purpose of this group, isn't it? cheers, fw -- http://mail.python.org/mailman/listinfo/python-list
Re: A critique of cgi.escape
On 25 Sep 2006 15:13:30 GMT, Jon Ribbens <[EMAIL PROTECTED]> wrote: > > Here's a point for you - the documentation for cgi.escape says that > the characters "&", "<" and ">" are converted, but not what they are > converted to. If the documentation isn't clear enough, that means the documentation should be fixed. It does _not_ mean "you are free to introduce new behavior because nobody should trust what this function does anyway". -- filip salomonsson -- http://mail.python.org/mailman/listinfo/python-list
Re: About alternatives to Matlab
sturlamolden wrote: > Boris wrote: > > Hi, is there any alternative software for Matlab? Although Matlab is > > powerful & popular among mathematical & engineering guys, it still > > costs too much & not publicly open. So I wonder if there's similar > > software/lang that is open & with comparable functionality, at least > > for numerical aspect. Thanks! > > I have used Matlab for years, and has recently changed to Python. In > addition one needs NumPy and Matplotlib, and perhaps SciPy. Other > useful packages are PyGTK for GUI, PyGame for multimedia, PyOpenGL for > 3D graphics, Mpi4Py for parallel computation, etc. You will find python > packages for nearly any conceivable task. Unlike Matlab, Python is a > general purpose programming language, and a distant cousin of Lisp. The > Python language is fare more expressive and productive than Matlab, yet > even more easy to use. Huh, so you don't think it's just damn sexy to do OOP, networking and databases by multiplying matrices?;-) > The NumPy package is the core requirement for numerical work in Python. > It is quite different form Matlab, but I think it is more powerful. > Particularly, arrays are passed by reference (not by value), and > indexing creates view matrices. For those interested in drilling down on this topic there is a Numpy for Matlab Users guide at http://scipy.com/NumPy_for_Matlab_Users > To compare Matlab with NumPy we can e.g. use the D4 discrete wavelet > transform. I have here coded it in Matlab and Python/NumPy using Tim > Swelden's lifting scheme. [...] Actually you have not. The algorithm you presented gives completely wrong results. Have a look at quick&dirty(TM) implementation bellow. import math import numpy def d2_lwt(x, axis=-1, level=None, copy=False): """ Daubechies2 (4 point support) Lifting Wavelet Transform in NumPy """ C1 = 1.7320508075688772# sqrt(3) C2 = 0.4330127018922193# sqrt(3)/4 C3 = -0.066987298107780702 # (sqrt(3)-2)/4) C4 = 0.51763809020504137 # (sqrt(3)-1)/sqrt(2) C5 = 1.9318516525781364# (sqrt(3)+1)/sqrt(2) if not isinstance(x, numpy.ndarray) or x.dtype.kind != 'f': x = numpy.array(x, dtype=numpy.float64) elif copy: x = x.copy() max_level = int(math.floor(math.log(x.shape[axis],2))) if level is None: level = max_level else: assert level <= max_level, "Level param too high" coeffs = [] cA = x.swapaxes(0, axis) while level > 0: level -= 1 # lazy even = cA[::2] odd = cA[1::2] # dual # using `even = even + C1*odd` may speed up things on # some machines (only for not in-place transform). even += C1*odd # primal odd[0] -= C2*even[0] + C3*even[-1] odd[1:] -= C2*even[1:] + C3*even[:-1] # dual even[:-1] -= odd[1:] even[-1] -= odd[0] # scale even *= C4 odd *= C5 cA, cD = even, odd coeffs.append(cD.swapaxes(axis, 0)) coeffs.append(cA.swapaxes(axis, 0)) coeffs.reverse() return coeffs if __name__ == "__main__": d = [1,5,3,2,4,8,5,2] data = [ numpy.array(d, dtype=numpy.float32), numpy.array(d, dtype=numpy.float64), numpy.array([d]*2, dtype=numpy.float64), numpy.array([d]*2, dtype=numpy.float32), numpy.array(d, dtype=numpy.int32), [d]*2, [[d,d]]*3, ] for i,x in enumerate(data): print "Case %d:" % (i+1) print "x in:\n", x coeffs = d2_lwt(x, axis=-1, level=None, copy=False) print "coeffs:" for c in coeffs: print c, c.dtype print "x out:\n", x print Please excuse me not including Matlab version here, I would like to have a free weekend. As far as the speed comparison is concerned I totally agree that NumPy can easily outperform Matlab in most cases. Of course one can use compiled low-level extensions to speed up specific computations in Matlab, but it's a lot easier and/or cheaper to find very good tools for Python. > If anyone wonders why I think Travis Oliphant and the NumPy team should > be knighted, then this is the answer. Would love to see the ceremony:) cheers, fw -- http://mail.python.org/mailman/listinfo/python-list
Re: numpy: frequencies
robert wrote: > I have an integer array with values limited to range(a,b) like: > > ia=array([1,2,3,3,3,4,...2,0,1]) > > and want to speedly count the frequencies of the integers into get a density > matrix. > Is this possible without looping? See numpy.bincount (for integers >= 0) if you mean 'without writing looping code in Python' or please specify your question. > Question 2: is it possible to compute a "moving maximum" without python > looping > > ia=array([4,2,1,5,3,2,2,0,1,1]) > -> mvmax(ia,3) -> > [4,4,4,5,5,5,3,2,2,1]) I haven't seen a ready solution but this can be easily converted into Pyrex/C looping. cheers, fw -- http://mail.python.org/mailman/listinfo/python-list
Re: About alternatives to Matlab
sturlamolden wrote: [...] > Here is the correct explanation: > > The factorization of the polyphase matrix is not unique. There are > several valid factorizations. Our implementations corresponds to > different factorizations of the analysis and synthesis poyphase > matrices, and both are in a sence correct, although numerically > different. Please correct me if I'm missing something but I'm pretty sure that choice of polyphase matrix factorization for given wavelet filter bank has influence only on speed and numerical stability of computation and not on the result itself. Particularly it should be possible to reconstruct analysis and synthesis filters from polyphase matrices regardless of the chosen factorization and both the discrete wavelet transform and the wavelet lifting scheme should give corresponding results for chosen wavelet (one can be rewritten in the form of other). cheers, fw -- http://mail.python.org/mailman/listinfo/python-list
Re: About alternatives to Matlab
sturlamolden wrote: > > Actually, there was a typo in the original code. I used d1[l-1] where I > should have used d1[l+1]. Arrgh. Here is the corrected version, the > Matlab code must be changed similarly. It has no relevance for the > performance timings though. > > > def D4_Transform(x, s1=None, d1=None, d2=None): > """ > D4 Wavelet transform in NumPy > (C) Sturla Molden > """ > C1 = 1.7320508075688772 > C2 = 0.4330127018922193 > C3 = -0.066987298107780702 > C4 = 0.51763809020504137 > C5 = 1.9318516525781364 > if d1 == None: >d1 = numpy.zeros(x.size/2) >s1 = numpy.zeros(x.size/2) >d2 = numpy.zeros(x.size/2) > odd = x[1::2] > even = x[:-1:2] > d1[:] = odd[:] - C1*even[:] > s1[:-1] = even[:-1] + C2*d1[:-1] + C3*d1[1:] > s1[-1] = even[-1] + C2*d1[-1] + C3*d1[0] > d2[0] = d1[0] + s1[-1] > d2[1:] = d1[1:] + s1[:-1] > even[:] = C4 * s1[:] > odd[:] = C5 * d2[:] If you swap C4 with C5 then our solutions give identical results and both match the classic dwt approach: even[:] = C5 * s1[:] odd[:] = C4 * d2[:] > if x.size > 2: > D4_Transform(even,s1[0:even.size/2], > d1[0:even.size/2],d2[0:even.size/2]) cheers, fw -- http://mail.python.org/mailman/listinfo/python-list
Re: About alternatives to Matlab
Jon Harrop wrote: [...] > I first wrote an OCaml translation of the Python and wrote my own > little "slice" implementation. I have since looked up a C++ solution and > translated that into OCaml instead: > > let rec d4_aux a n = > let n2 = n lsr 1 in > let tmp = Array.make n 0. in > for i=0 to n2-2 do > tmp.(i) <- a.(i*2)*.h0+.a.(i*2+1)*.h1+.a.(i*2+2)*.h2+.a.(i*2+3)*.h3; > tmp.(i+n2) <- a.(i*2)*.g0+.a.(i*2+1)*.g1+.a.(i*2+2)*.g2+.a.(i*2+3)*.g3; > done; > tmp.(n2-1) <- a.(n-2)*.h0 +. a.(n-1)*.h1 +. a.(0)*.h2 +. a.(1)*.h3; > tmp.(2*n2-1) <- a.(n-2)*.g0 +. a.(n-1)*.g1 +. a.(0)*.g2 +. a.(1)*.g3; > Array.blit tmp 0 a 0 n; > if n > 4 then d4_aux a (n lsr 1) > > let d4 a = d4_aux a (Array.length a) > > Not only is that shorter than the Python and makes my eyes bleeding, but it's only my personal feeling. Try reading the code above aloud and you will see why `shorter` is not necessarily considered a merit, especially on this group. Besides of that this code is irrelevant to the original one and your further conclusions may not be perfectly correct. Please learn first about the topic of your benchmark and different variants of wavelet transform, namely difference between lifting scheme and dwt, and try posting some relevant examples or use other topic for your benchmarks. Neglecting this issues, I wonder if it is equally easy to juggle arbitrary multidimensional arrays in a statically typed language and do operations on selected axis directly from the interactive interpreter like in the NumPy example from my other post - http://groups.google.com/group/comp.lang.python/msg/a71a5bf00a88ad57 ? I don't know much OCaml so it would be great if you could elaborate on this. > , it is much faster: > > 0.56s C++ (direct arrays) > 0.61s F# (direct arrays) > 0.62s OCaml (direct arrays) > 1.38s OCaml (slices) > 2.38s Python (slices) > 10s Mathematica 5.1 > > Note that all implementations are safe (e.g. C++ uses a.at(i) instead of > a[i]). [...] So why not use C++ instead of all others if the speed really matters? What is your point here? Could you please share full benchmark code, so we could make conclusions too? I would like to get to know about your benchmark methodology. I wonder if you are allocating the input data really dynamically or whether it's size is a priori knowledge for the compiler. > In this specific context (discrete wavelet transform benchmark), I'd have > said that speed was the most important thing after correctness. In this very specific context: Yes, if you are targeting specific one-shot issue like JPEG2000 encoder, where you are using only 2 types of wavelets. In that case you would probably draw back to low-level C or C++ and use a good optimizing compiler for a specific hardware. Not necessarily, if you are doing research with different kinds of wavelets and need a general, flexible and easily adaptable tool or just the computation speed is not considered a bottleneck. Language speed is a great advantage, but if it always was the most important, people would talk directly to the CPUs or knit DSP chips in theirs backyards whenever they need to solve a calculation problem. Please don't make this discussion heading towards `what is the fastest way of doing d4 transform and why OCaml rules` instead of `what is the optimal way of doing things under given circumstances and still have a free afternoon ;-)`. Different tasks need different, well-balanced measures. Neither Python nor OCalm nor any other language is a panacea for every single problem. cheers, fw -- http://mail.python.org/mailman/listinfo/python-list
Re: About alternatives to Matlab
Jon Harrop wrote: > Filip Wasilewski wrote: > > Besides of that this code is irrelevant to the original one and your > > further conclusions may not be perfectly correct. Please learn first > > about the topic of your benchmark and different variants of wavelet > > transform, namely difference between lifting scheme and dwt, and try > > posting some relevant examples or use other topic for your benchmarks. > > Your lifting implementation is slightly longer and slower than the > non-lifting one but its characteristics are identical. For n=2^25 I get: Jon, both Python and Matlab implementations discussed here use the lifting scheme, while yours is a classic convolution based approach. These are two *different* algorithms for computing wavelet transforms. Although they have similar names and give similar results, it does not mean you can use them interchangeably in one benchmark! It just does not make sense. What's more, taking only very big input 'n' gives only very limited information about true performance. How do you think who is faster, a sprinter or a long distance runner? If you want to draw objective conclusions about algorithm implementation you should measure timing and memory usage characteristic for different input lengths. This will also give you some idea about call overhead and possible memory bandwidth influence. For example, the 4-tap db2 lifting transform should be about 50% faster than the one using subband filtering. That's theory. In practice, due to specific memory usage, the speedup gained with reduction of number of arithmetic operations is easily wasted by huge cache miss overhead (C implementation, measured on x86 architecture) for arrays which don't fit entirely in the CPU cache. See my point? > 1.88s C++ (816 non-whitespace bytes) > 2.00s OCaml (741 b) > 2.33s F# (741 b) > 9.83s Your Python (1,002 b) > > The original python was 789 bytes. Is the byte count a standard measure you apply to describe code quality? I don't think you would be much happier to see totally obfuscated golf one-liners. > > Neglecting this issues, I wonder if it is equally easy to juggle > > arbitrary multidimensional arrays in a statically typed language and do > > operations on selected axis directly from the interactive interpreter > > like in the NumPy example from my other post - > > http://groups.google.com/group/comp.lang.python/msg/a71a5bf00a88ad57 ? > > I don't know much OCaml so it would be great if you could elaborate on > > this. > > It is probably just as easy. Instead of dynamic typing you have parametric > polymorphism. If you want to make your function generic over arithmetic > type then you can pass in the arithmetic operators. Probably? Using the interactive interpreter, how can I easily create two n-dimensional arrays of arbitrary data type, add them and multiply by 17? > >> In this specific context (discrete wavelet transform benchmark), I'd have > >> said that speed was the most important thing after correctness. > > > > Not necessarily, if you are doing research with different kinds of > > wavelets and need a general, flexible and easily adaptable tool or just > > the computation speed is not considered a bottleneck. > > Brevity is probably next. Memory usage, development time, cost, portability, scalability, readability, level of abstraction, usage experience and many many others, not necessarily in this order. I do not intend getting into advocacy of any specific language or feature here. I just prefer fair comparisons and clean use cases instead of marketing like that and I don't believe anyone will buy your story about OCaml superior brevity after seeing one fairly irrelevant loop with few arithmetic operations as a killer argument. > > Please don't make this discussion heading towards `what is the fastest > > way of doing d4 transform and why OCaml rules` instead of `what is the > > optimal way of doing things under given circumstances and still have a > > free afternoon ;-)`. > > Comments like that seem to be based on the fundamental misconception that > writing C++ like this is hard. Not harder than writing a loop in bazillion of other languages. When developing software do you only count time spent on typing because you are idle during build process? > Here's my complete OCaml: ... > and C++: > #include ... You didn't mention your platform and compiler settings. Using vector at() method instead of plain C/C++ array indexing increases the timings by factor of ~1.5 to 10 or more (GCC, MSVC), depending on the optimizations set, so it doesn't seem to be the best possible solution. I was also unable to run your OCaml example for n >= 2^21 (win32, out of the box OCaml installation). Is there some 16MB memory limit imposed on Array? # let q = Array.make (1 lsl 21) 0.0;; Exception: Invalid_argument "Array.make". cheers, fw -- http://mail.python.org/mailman/listinfo/python-list
Re: About alternatives to Matlab
Jon Harrop wrote: > Filip Wasilewski wrote: > > Jon, both Python and Matlab implementations discussed here use the > > lifting scheme, while yours is a classic convolution based approach. > > I've done both in OCaml. The results are basically the same. Have you tried taking advantage of the 50% reduction of arithmetic operations for the lifting scheme? > > These are two *different* algorithms for computing wavelet transforms. > > Although they have similar names and give similar results, it does not > > mean you can use them interchangeably in one benchmark! It just does > > not make sense. > > It makes sense because they solve the same problem. When you're comparing > non-trivial problems between disparate languages you are not likely to use > the same algorithms. Using built-in hash functions is an obvious example. I don't even ask if you have checked the output result coefficients because I'm pretty sure you have not. They solve similar problem but in different ways and give a bit different results, for example the layout of coefficients in the memory is different as well as border distortion handling. Please don't tell me it's better to do something else instead of presenting a relevant solution. By the way, how do you think why there are both implementations included in Matlab Wavelet Toolbox and there are people paying lots of money for it? > > What's more, taking only very big input 'n' gives only very limited > > information about true performance. > > I've presented times for 2 different "n". I agree that it would be better to > present infinite different "n" but I only had finite time. A simple loglog plot with 25 or so dyadic lengths would be just fine. I don't expect you to measure times for non-dyadic lengths because neither of the solutions presented here so far support this. [...] > > This will > > also give you some idea about call overhead and possible memory > > bandwidth influence. For example, the 4-tap db2 lifting transform > > should be about 50% faster than the one using subband filtering. That's > > theory. In practice, due to specific memory usage, the speedup gained > > with reduction of number of arithmetic operations is easily wasted by > > huge cache miss overhead (C implementation, measured on x86 > > architecture) for arrays which don't fit entirely in the CPU cache. See > > my point? > > No. The effects you are talking about are swamped by the interpreted vs > compiled effect. Have I already mentioned it was observed with low-level C implementation on x86? I would really appreciate you took some time on exploring the unfortunate topic of this benchmark before posting another comment like that. > >> 1.88s C++ (816 non-whitespace bytes) > >> 2.00s OCaml (741 b) > >> 2.33s F# (741 b) > >> 9.83s Your Python (1,002 b) > >> > >> The original python was 789 bytes. > > > > Is the byte count a standard measure you apply to describe code > > quality? > > You can use bytes, lines, words, tokens or just look at the code. Whatever > you do, Python loses on this benchmark in terms of brevity, clarity and > performance. Jon, as I have already said, you did not even show a fully comparable example yet. What I would really like to see is a lifting scheme implementation that can take 1- to 10-dimensional arrays (single and double precision floats, convert on the fly from integer types when necessary and handle strided, non-contiguous arrays), axis and maximum decomposition level as input and return list of views (or arrays if necessary) with transform coefficients for every level of decomposition. Can do? > > I don't think you would be much happier to see totally > > obfuscated golf one-liners. > > That doesn't even make sense. Squeezing code onto one line doesn't improve > byte count. Yeah, right ;-) > >> It is probably just as easy. Instead of dynamic typing you have > >> parametric polymorphism. If you want to make your function generic over > >> arithmetic type then you can pass in the arithmetic operators. > > > > Probably? Using the interactive interpreter, how can I easily create > > two n-dimensional arrays of arbitrary data type, add them and multiply > > by 17? > > In F#: > > open Math.Matrix.Generic > let a = init n m (fun i j -> ...) > let b = init n m (fun i j -> ...) > 17. $* (a + b) > > In OCaml you'd either use an array of arrays and define your own functions > over them, or you'd use the built-in Bigarray.Array2 and define some > functions over that. Either way, you have to use different operator names > for dif
Re: About alternatives to Matlab
Jon Harrop wrote: > Filip Wasilewski wrote: > > Jon Harrop wrote: > >> Filip Wasilewski wrote: > >> > Jon, both Python and Matlab implementations discussed here use the > >> > lifting scheme, while yours is a classic convolution based approach. > >> > >> I've done both in OCaml. The results are basically the same. > > > > Have you tried taking advantage of the 50% reduction of arithmetic > > operations for the lifting scheme? > > Yes. The time taken is dominated by memory accesses. The amount of > arithmetic is almost irrelevant. I can say from my experience, that this depends much on the input size and the CPU cache size. For arrays of about n^18 or less and 2MB cache the speedup for the straightforward C lifting implementation is around 50%, while all operations are done in-place. For bigger arrays traversing memory several times becomes too expensive, at least until the input size is so big that the memory gets swapped and the inplace method becomes faster again. I suppose that such behavior should be quite similar for other low-level implementations as well. Actually I started to wonder if I could automate conversion of multi-loop lifting schemes [1] into one-pass loop. This could be done by delaying computation of the "coming next" step just by few units, until values from the preceding step are ready to use. This should reduce the random memory access effect for several cases and make the timings quite stable. [1] http://en.wikipedia.org/wiki/Lifting_scheme > >> It makes sense because they solve the same problem. When you're comparing > >> non-trivial problems between disparate languages you are not likely to > >> use the same algorithms. Using built-in hash functions is an obvious > >> example. > > > > I don't even ask if you have checked the output result coefficients > > because I'm pretty sure you have not. > > I checked the coefficients against other DWTs from the web. > > > They solve similar problem but in > > different ways and give a bit different results, for example the > > layout of coefficients in the memory is different as well as border > > distortion handling. Please don't tell me it's better to do something > > else instead of presenting a relevant solution. > > I used a more efficient approach because I could. Using a compiled language > gave me that choice. We should compare both approaches in both languages. I am getting a bit tired of this discussion. I am not arguing which approach is faster, because it very much depends on the use case. I just ask you to do fair comparison using the original algorithm presented here instead of forcing your arguments with the not-so-relevant variant convenient for you. Every language (or at least vast majority of them) that has loops and basic arithmetic gives you such choice, no matter compiled or not. These are pretty simple algorithms when limited to single D4 case, but the D4 is not the only wavelet transform known to mankind and there are situations when you do not have such choice, because some problems just can't be expressed using the classic approach. If the OCaml is so superior as you claim, please show me how to do the lifting scheme (preferably in the http://groups.google.com/group/comp.lang.python/msg/a71a5bf00a88ad57 form with features described in my previous post) so I could actually learn something new and maybe even start to consider it as a possible alternative to C-ish language family. As far as the DWT only is concerned, well, If I were to choose the most efficient approach (in both the CPU time and the development time terms) for Python I would use a ready extension instead of developing the wheel once again. Let me think, that should go something like this: import pywt coeffs = pywt.wavedec(x, 'db2', 'per') > >> No. The effects you are talking about are swamped by the interpreted vs > >> compiled effect. > > > > Have I already mentioned it was observed with low-level C > > implementation on x86? I would really appreciate you took some time on > > exploring the unfortunate topic of this benchmark before posting > > another comment like that. > > We are talking about this Python implementation, not some "low-level C > implementation on x86". With Python, it is the overhead of the interpreter > that makes it slow and encourages you to obfuscate your code in order to > make it run in a reasonable amount of time. I was talking about why you should not limit your benchmarks to the very single case and gave example how the choice of parameters can influence the results. It has nothing to do with the interpreted vs. compiled harangue. Would you argue that some O(n^2) algorithm is always better than
Re: Why doesn't Python's "robotparser" like Wikipedia's "robots.txt" file?
On 02/10/2007, John Nagle <[EMAIL PROTECTED]> wrote: > > But there's something in there now that robotparser doesn't like. > Any ideas? Wikipedia denies _all_ access for the standard urllib user agent, and when the robotparser gets a 401 or 403 response when trying to fetch robots.txt, it is equivalent to "Disallow: *". http://infix.se/2006/05/17/robotparser It could also be worth mentioning that if you were planning on crawling a lot of Wikipedia pages, you may be better off downloading the whole thing instead: <http://download.wikimedia.org/> (perhaps adding <http://code.google.com/p/wikimarkup/> to convert the wiki markup to HTML). -- filip salomonsson -- http://mail.python.org/mailman/listinfo/python-list
Re: Using msvcrt (in Windows), how to catch Enter key?
On Oct 29, 11:26 am, Dick Moores <[EMAIL PROTECTED]> wrote: > Windows XP Pro, Python 2.5.1 > > import msvcrt > while True: > if msvcrt.kbhit(): > key = msvcrt.getch() > if key == 'Enter' > do something > > Is there a way to catch the pressing of the 'Enter' key? Yes there is. Just open the Python shell and see what is being returned by `getch` or `getche` functions when you press Enter: >>> import msvcrt >>> msvcrt.getch() '\r' Also try to avoid `busy waiting` and calling msvcrt.kbhit in a loop without a sleep statement. I don't know your case but probably this should be enough: while True: if msvcrt.getch() == '\r': ... fw -- http://mail.python.org/mailman/listinfo/python-list
Re: achieving performance using C/C++
On Nov 5, 7:40 am, sandipm <[EMAIL PROTECTED]> wrote: > I did fair amount of programming in python but never used c/c++ as > mentioned below. > any good tutorials for using C/C++ to optimize python codebase for > performance? > how widely do they use such kind of mixed coding practices? [...] Since you are fluent in Python I also suggest you to start with Pyrex[1]/Cython[2]. This will let you to express your ideas more naturally (in a Python-based language) and interface with C code base without delving much into reference counting, manual parameters conversion and low-level exception handling. You will find some tutorials and links to resources and mailing lists on the projects websites, and if you specify your problem a bit more then probably someone will be able to give you more precise references. For alternatives take a look at [3] and [4]. Try also scanning the Python Package Index[5] for 'binding', 'wrapper', 'lib' or 'pyrex' keywords or browse for C/C++ Programming Language category as this will return some examples of Python extensions. [1] http://www.cosc.canterbury.ac.nz/greg.ewing/python/Pyrex/ [2] http://www.cython.org/ [3] http://www.python.org/doc/faq/extending/#writing-c-is-hard-are-there-any-alternatives [4] http://docs.python.org/lib/module-ctypes.html [5] http://pypi.python.org fw -- http://mail.python.org/mailman/listinfo/python-list
Re: Calling a class instance like a function
__call__ 2009/2/19 Uberman : > I'm wondering if there's a way to invoke a "function" operator on a Python > class instance. For example, given a class instance: > >myClass = MyClass() > > I want to call that instance like a function, with an argument value: > >myClass(5.0) > > I can override the "()" operator in C++, so I'm wondering if there's a way to > do it in Python as well. > > Thanks! > -- > http://mail.python.org/mailman/listinfo/python-list > -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Project structure - Best practices
This is first time that I am building python application that is larger than a single module and I would like to do it right. I google it a bit, finding some stuff about not using src directory (which I have seen so many times, that I believed it be standard) and using packages. Still, there are few things, that I would like to achieve with this structure: * being able to use pychecker a lot - catching all typos in one shot instead of running app many times really saves me a lot of time * being able to write some unit tests * having clean division of code among packages and modules (I have seen some projects, where modules are pretty large - I would like to keep them logically divided, event if they stay smaller) My project is a tool for people interested in role playing games. My current structure looks something like this: /src rpgDirectory.py (main script, running the app) src/rpg plans.py support.py gui.py iosystem.py src/rpg/character model.py sheet.py gui.py handlers.py requirements.py The problem is, that modules from src/rpg/character use classes defined in support.py. Therefore I have to use absolute paths to import it and this works only, when I run rpgDirectory.py. When I use pychecker, it can't import this module and fails. Any suggestions, how can I avoid this and what structure should I use? -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Re: Project structure - Best practices
> http://jcalderone.livejournal.com/39794.html That's exactly what I have read before posting here ;-) -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Do more imported objects affect performance
I have following question: if I use from module import * instead from module import Class am I affecting performance of my program? I believe, that all those names must be stored somewhere, when they are imported and then browsed when one of them is called. So am I putting a lot of "garbage" to this storage and make those searches longer? -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Re: pydoc enforcement.
Hey! If you are interested, I have written a small tool for declaring variables and attributes. It's not very sophisticated, because I have written it solely for own use. It might be useful though. You can download it from you: http://code.google.com/p/pyver/downloads/list For a small example, it works like this: def foo(): a = 5 class Bar: def __init__(self): self.a = 5 Errors: Line 2: Variable a in function foo was not specified; Line 7: Field a in function __init__ was not specified; To have it working fine, you have to declare fields and variables like this: def foo(): #| a | a = 5 class Bar: #| a | def __init__(self): self.a = 5 But as soon as I finished I realized, I don't really need this since there are pychecker and pylint that do all what I need and I don't have to type all those declarations. But I have learned how to parse python code using tools from stdlib, so it wasn't really wasted time. Hope this helps :-) If you want to know something more about this, don't hesitate to ask. 2008/12/1 [EMAIL PROTECTED] <[EMAIL PROTECTED]>: > I've been thinking about implementing (although no idea yet *HOW*) the > following features/extension for the python compile stage and would be > interested in any thoughts/comments/flames etc. > > Basically I'm interested adding a check to see if: > 1) pydoc's are written for every function/method. > 2) There are entries for each parameter, defined by some > predetermined syntax. > > My idea is that as much as I love dynamic typing, there are times when > using some modules/API's that have less than stellar documentation. I > was thinking that if it was possible to enable some switch that > basically forced compilation to fail if certain documentation criteria > weren't met. > > Yes, it should be up to developers to provide documentation in the > first place. Or, the client developer might need to read the source > (IF its available)... but having some "forced" documentation might at > least ease the problem a little. > > For example (half borrowing from Javadoc). > > class Foo( object ): > > def bar( self, ui ): > pass > > > Would fail, since the bar method has an "unknown" parameter called > "ui". > What I think could be interesting is that the compiler forces some > documentation such as: > > class Foo( object ): > > def bar( self, ui ): >""" >@Param: ui : blah blah blah. >""" > pass > > > The compiler could check for @Param matching each parameter passed to > the method/function. Sure, a lot of people might just not put a > description in, so we'd be no better off. But at least its getting > them *that* far, maybe it would encourage them to actually fill in > details. > > Now ofcourse, in statically typed language, they might have the > description as "Instance of UIClass" or something like that. For > Python, maybe just a description of "Instance of abstract class UI" or > "List of Dictionaries"... or whatever. Sure, precise class names > mightn't be mentioned (since we mightn't know what is being used > then), but having *some* description would certainly be helpful (I > feel). > > Even if no-one else is interested in this feature, I think it could > help my own development (and would be an interested "first change" > into Python itself). > > Apart from bagging the idea, does anyone have a suggestion on where in > the Python source I would start for implementing such an idea? > > Thanks > > Ken > > -- > http://mail.python.org/mailman/listinfo/python-list > -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Re: Do more imported objects affect performance
I see. Thanks for a really good explanation, I like to know, how to do things in the proper way :) 2008/12/1 Nick Craig-Wood <[EMAIL PROTECTED]>: > Rafe <[EMAIL PROTECTED]> wrote: >> On Dec 1, 7:26?am, "Filip Gruszczy?ski" <[EMAIL PROTECTED]> wrote: >> > I have following question: if I use >> > >> > from module import * >> > >> > instead >> > >> > from module import Class >> > >> > am I affecting performance of my program? I believe, that all those >> > names must be stored somewhere, when they are imported and then >> > browsed when one of them is called. So am I putting a lot of "garbage" >> > to this storage and make those searches longer? >> >> Why use it if you don't need it? Your post implies a choice and the >> '*' import can really make things muddy if it isn't actually necessary >> (rare). Why not just import the module and use what you need? It is >> way easier to read/debug and maintains the name-space. > > Importing the module is actualy slower... If you import the name into > your namespace then there is only one lookup to do. If you import the > module there are two. > > $ python -m timeit -s 'from timeit import Timer' 'Timer' > 1000 loops, best of 3: 0.0784 usec per loop > > $ python -m timeit -s 'import timeit' 'timeit.Timer' > 100 loops, best of 3: 0.243 usec per loop > > I'm not suggestion you should ever use "from module import *" only > ever import the things you actually need, eg > "from module import MyClass, my_function" > > And here is the test again, actually calling something with the same > difference in execution speed :- > > $ python -m timeit -s 'from os import nice' 'nice(0)' > 100 loops, best of 3: 1.21 usec per loop > > $ python -m timeit -s 'import os' 'os.nice(0)' > 100 loops, best of 3: 1.48 usec per loop > > -- > Nick Craig-Wood <[EMAIL PROTECTED]> -- http://www.craig-wood.com/nick > -- > http://mail.python.org/mailman/listinfo/python-list > -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Removing None objects from a sequence
Hi! I would like to iterate over a sequence nad ignore all None objects. The most obvious way is explicitly checking if element is not None, but it takes too much space. And I would like to get something faster. I can use [ sth for sth in self.__sth if not sth is None ], but I don't know if that's the best way. I checked itertools, but the only thing that seemed ok, was ifilter - this requires seperate function though, so doesn't seem too short. How can I get it the shortest and fastest way? -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
I don't mean memory, but space in code ;-) I'll try this generator :) -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
I am not doing it, because I need it. I can as well use "if not elem is None", but I just wanted to know, if there is some better way of doing this. I like to know :-) And I can't understand why you are becoming so aggressive because of this. Just because I asked for that, doesn't mean, that I will put some obfuscated code into my project. I just wanted to learn something new - I checked itertools, I googled a bit, now I wanted to ask here if someone knew some really cool way of this. All the other assumptions weren't really necessary. Thanks for those ideas, however. I like the last one a lot :) -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
Just to be clear, I decided to use generator by alex23, as it seems simple, short and understandable. Still reading this thread was quite interesting, thanks :-) -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
deleting a method
I am trying to delete a method from a class. It's easy to delete other attributes, but when I try: >>> class A: ... def foo(): ... pass ... >>> a = A() >>> del a.foo I get Traceback (most recent call last): File "", line 1, in AttributeError: A instance has no attribute 'foo' Why is it so and how may still delete it? -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Re: deleting a method
> To answer the second question: since 'foo' is an attribute of the > class 'A', you can delete the attribute from the class. > >>>> class A(object): >... def foo(self): >... pass >... >>>> a = A() >>>> 'foo' in dir(a) >True >>>> del A.foo >>>> 'foo' in dir(a) >False Thanks, now I see, what happens, but don't exactly know why. Could you point me to some good explanation how object creation is performed in Python? I browsed language reference and google, but couldn't find good explanation. -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Tree views - Best design practices
Hi! I have certain design problem, which I cannot solve elegantly. Maybe you know some good design patterns for this kind of tasks. Task: We have a model which has two kinds of objects: groups and elements. Groups can hold other groups (subgroups) and elements. It's a simple directory tree, for example. We would like to display it in a tree view (which sound good for this kind of model). What is more required, for groups and elements there are different sets of operations, which should be available under right click. For example for group, there should be operations: 'add element' and 'add group', and for element there should be 'change properties'. Do you know any smart way to achieve this? The simplest way is to ask for the class and display operations accordingly. But from the first day with OO programming I have heard, that asking for class is wrong. But I can hardly see any easy and more maintainable solution for this problem. Could you help me with this? -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Re: Tree views - Best design practices
>>>> class Element(object): >operations = "Element operations" > > >>>> class Group(object): >operations = "Group operations" > > >>>> e = Element() >>>> g = Group() >>>> >>>> e.operations > 'Element operations' >>>> g.operations > 'Group operations' But this is the same as asking for a class, except for having to write a method giving some form of a class name and then basing on this classname display operations. I know this solution, but this is what I would like to evade. -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Re: Tree views - Best design practices
I'd love to have operations methods, which would return names of operations and references to proper methods. The problem is that certain operations require more, than just running some operation on the Group or Element. Let's say, you want to add new group to an existing one - you can't do that to an element. But adding a group cannot be just done by calling group.addGroup. You must first construct the group or at least get information require to construct a group, so you must for example call a dialog which will ask for information about the group. And that's in conflict with model/view pattern. And that's not all. I am using qt and when I want to add this group to another one, I have to first call beginInsertRows on the model used by the tree view (it's a layer between the gui and the real model). I tried building a wrapper around my model which would provide additional info, but then I had to keep relations between objects in two places (relations between real groups and elements and wrapper groups and wrapper elements), which wasn't the right way to do. I just can figure a simple, elegant way to do this. -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Re: Tree views - Best design practices
>class Element(object): >@staticmethod >def is_leaf(): return True >@staticmethod >def is_branch(): return False > >class Group(object): >@staticmethod >def is_leaf(): return False >@staticmethod >def is_branch(): return True > > Of course, you have to give priority to one or the other, in case an > object thinks it is both. > >if thing.is_branch(): ># Treat it like a branch >elif thing.is_leaf(): ># Treat it like a leaf > > I believe this is a simpler method than checking a single attribute > for a name. Is it really any better than asking for class? I mean, if I need to add another class to a hierarchy which behaves differently, will it be more flexible (actually you have to add another method to every class and check for in the gui). I believe it's just the same as asking for the class, but we hide it under static methods. It's no different though. -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Re: Tree views - Best design practices
> Yes. There is a difference between the interface of an object (namely, > what methods and attributes it has and what their semantic meaning is) > and the class of an object (what methods and attributes it has and how > they are implemented.) > > In general, you shouldn't be asking about an object's class. Down the > road, you may want to use an object that isn't of the same class but > does support the interface. > > Consider how the file object is used in Python. Pretty much every > place you can use a file object you can use a StringIO, right? That's > because StringIO supports the file interface while it isn't a file. > > You may want to read up on 'duck typing' to get a better sense of why > this is important. I have read also your next message and these are good arguments for dynamically typed languages and really convinces me - I mean that they provide some flexibility at the cost of writing a little more. They provide flexibility though in a different field. Duck typing is great, because it allows completely use of a completely different hierarchy, which only has to have the same interface. But I am looking for a different type of flexibility. I would like to be able to add more classes to my hierarchy and not have to change my code in many places when I add new class to the hierarchy. If I have to change every class in the hierarchy because I add new class, then it's not something I would like to do. And idea how I can avoid this? -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Two instances share an attribute
I really don't understand, what's happening with the following code.
Am I doing something wrong?
#!/usr/bin/python
class EnumeratedContent:
def __init__(self, values = []):
self.values_ = values
def values(self):
return self.values_
def addValue(self, value):
self.values_.append(value)
x = EnumeratedContent()
x.addValue('ff')
print x.values()
x = EnumeratedContent()
x.addValue('gg')
print x.values()
This code prints:
['ff']
['ff', 'gg']
Why the heck self.__values keeps old list? Can anyone explain it to me?
I am using:
Python 2.5.2 (r252:60911, Sep 14 2008, 23:49:00)
[GCC 4.1.2 (Gentoo 4.1.2 p1.0.2)] on linux2
--
Filip Gruszczyński
--
http://mail.python.org/mailman/listinfo/python-list
Re: Two instances share an attribute
Every day something new. Thanks a lot :) 2008/11/15 Cameron Simpson <[EMAIL PROTECTED]>: > On 15Nov2008 22:41, Filip Gruszczyński <[EMAIL PROTECTED]> wrote: > | I really don't understand, what's happening with the following code. > | Am I doing something wrong? > > Yes. This is a common mistake: > > | class EnumeratedContent: > | def __init__(self, values = []): > | self.values_ = values > > The "values = []" happens at class definition time, not instance > definition time. So when "values" is not supplied, the same list > is reused as the default value. > > The usual idiom is this: > > class EnumeratedContent: > def __init__(self, values = None): > if values is None: > values = [] > self.values_ = values > > which makes a new [] during the instance creation. > > Cheers, > -- > Cameron Simpson <[EMAIL PROTECTED]> DoD#743 > http://www.cskk.ezoshosting.com/cs/ > > If you don't live on the edge, you're taking up too much space. - t-shirt > -- > http://mail.python.org/mailman/listinfo/python-list > -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Script in /usr/bin doesn't see my packages
I have used distutils to install a python package and a script, that runs a program. Everything seemed ok, when I run interpreter I can import stuff, that I have just installed. But when I run script that was installed into /usr/bin it can't import anything. This happens also with my test scripts: i put an import statement there and run them (they are outside of development directory) and they import stuff just the way they should. But when I put them into /usr/bin they stop. Could anyone help me on this? -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Script for a project inside own directory
I am having a project built like this: project module1.py module2.py packages1/ module3.py etc. I have script that uses objects from those modules/packages. If I keep this script inside project directory it's ok and it works. But I would like to move it to own scripts directory and from there it doesn't work. I think I understand why it doesn't work (the root of local packages and modules is there and it can't see what it above it), but I would like to ask if there is any workaround? I would like to keep all my scripts in separate dir instead of main dir. I like to keep it clean. -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Re: Script for a project inside own directory
Works great. Thanks a lot.
2009/3/22 Maxim Khitrov :
> 2009/3/22 Filip Gruszczyński :
>> I am having a project built like this:
>>
>> project
>> module1.py
>> module2.py
>> packages1/
>> module3.py
>>
>> etc.
>>
>> I have script that uses objects from those modules/packages. If I keep
>> this script inside project directory it's ok and it works. But I would
>> like to move it to own scripts directory and from there it doesn't
>> work. I think I understand why it doesn't work (the root of local
>> packages and modules is there and it can't see what it above it), but
>> I would like to ask if there is any workaround? I would like to keep
>> all my scripts in separate dir instead of main dir. I like to keep it
>> clean.
>
> import sys
> sys.path.append('')
>
> If project directory is one level up, you can do something like this:
>
> import os
> import sys
> sys.path.append(os.path.realpath('..'))
>
> - Max
>
--
Filip Gruszczyński
--
http://mail.python.org/mailman/listinfo/python-list
Too early implementation
I am not a very disciplined person. Usually I rush to my next assignment and code furiously, hoping that my initial understanding of the stated problem will be just fine. And Python does very little to stop me ;-) If I had to do something in C++, I know I would have to write all those header files, fill stubs, etc., which is a lot of work, which I don't want to do twice. So I take a while and start to think, because it's better to be safe, than sorry. With Python you rarely are sorry, because you can do everything so quickly. And yet, at some point you see, that flaws in design get so annoying, that you need to do something about them. Usually at that point it's a bit problematic ;-) So, do you know some good methods to prevent myself from just starting coding (which I like very much) and do some thinking about the problem (which I like a little less ;-))? -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Re: Too early implementation
Yep, I have heard a lot about test driven development. I am now programming a lot using DJango and I would like to use its test framework to try it. However, I have little experience with this (as most people I know). I also have no idea, how to apply this, when I write code heavily focused on user interface. I thought, that prototyping is cool, especially with python, where you can create something functional pretty quickly. But recently I was met with a situation, when after writing 1k lines of prototype and then writing main program > 3k lines I noticed, that there is something really wrong with the design. It was then, that I started wondering: maybe I should first prepare some good design documents, I don't know some SAD or something. -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Pythonic emptiness checking
One of the Python Zen rules is Explicit is better implicit. And yet it's ok to do: if x: do_sth when x is string or list. Since it's very comfy, I've got nothing against though. I am just curious, why is it so? And one more thing: is it ok to do if x: instead of if x is not None: Because I often encounter it and would like to know, if I can simplify it. Especially that I liked similar construction in C/C++. -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Re: Pythonic emptiness checking
Yes, I get the difference. If x is [], than if x: won't be executed and if x is not None: will be. Thanks for clarifying. -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
ECG segmentation
Hi! I need to create a script, that performs ECG segmentation, but I can hardly find any useful materials on the web. Did anyone try to do this and could point me to some good materials/snippets about this? -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Re: Fill memeory with endless loop?
On Út, kvě 13, 2008 at 06:49:33 +0200, globalrev wrote: > if i do something like > while 1: > print "x" > > will the program ever stop because it runs out of memory? No, there is no reason to run out of memory. This will simply make an everlasting x-printer and there is no need to store anything (except output buffers, terminal scrollback, etc., but all of this is temporary and of limited size). In case of adding elements to a list, the memory usage can, of course, grow unlimitedly. On linux, if the process eats up too much memory, it gets killed. I do not know about windows, but they would probably crash with a BSOD or something of that sort. -- http://mail.python.org/mailman/listinfo/python-list
Ideas for master's thesis
Hello there, I am student of CS at the University of Warsaw, currently 4th year. I am attending Object Oriented Programming seminar and it is about time, I started looking for an idea of my master's degree project. As I like Python very much, I would like to do something with this language, yet I don't know if there are any needs/science fields, that could be used as a basis for a thesis. Therefore I would like to ask, if there is any way to access any person involved in development, who would help me find some field that need to be developed/researched (rather the first one) and would be close enough to the OOP, so I could present it to a mentor (he is cool with doing something for OSS community). -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
XML validation in stdlib?
Hi! I took a look at the standard library and tried to find some validation against schema tools, but found none. I googled, but found only links to external libraries, that can do some validation. Does it mean, that there is no validation in stdlib or have I just missed something? -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Explicit variable declaration
Hello everyone! It is my first message on this list, therefore I would like to say hello to everyone. I am fourth year student of CS on the Univeristy of Warsaw and recently I have become very interested in dynamically typed languages, especially Python. I would like to ask, whether there is any way of explicitly declaring variables used in a function? While I am pretty sure, that there is no such way in the language itself, I would like to know, whether there are any third-party tools to do that. This would be very useful for me during development, so I am looking for such a tool. -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Re: Explicit variable declaration
> You mean the type? Not in 2.x, but in 3.x, there are function > annotations: > > def a_function(arg1: int, arg2: str) -> None: pass Nope, I don't like types ;-) 3.x seems pretty revolutionary, and this typing can be appreciated by some people. > Declaring what about them? If you mean declaring the type, remember > that Python deliberately allows any name to be bound to any object; > type declarations can't be enforced without losing a lot of the power > of Python. Just declaring, that they exist. Saying, that in certain function there would appear only specified variables. Like in smalltalk, if I remember correctly. -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Re: Explicit variable declaration
Wow! This is extremely easy and seems to do exactly what I need. Those
decorators are pretty powerful then. Thanks for your help, I'll try to
use this.
> def uses(names):
> def decorator(f):
> used = set(f.func_code.co_varnames)
> declared = set(names.split())
> undeclared = used-declared
> unused = declared-used
> if undeclared:
> raise ValueError("%s: %s assigned but not declared"
>% (f.func_name, ','.join(undeclared)))
> if unused:
> raise ValueError("%s: %s declared but never used"
>% (f.func_name, ','.join(unused)))
> return f
> return decorator
>
> Used something like this:
>
> >>> @uses("x y")
> def f(x):
> y = x+1
> return z
>
> >>> @uses("x y z")
> def f(x):
> y = x+1
> return z
>
>
> Traceback (most recent call last):
> File "", line 1, in
> @uses("x y z")
> File "", line 10, in decorator
> raise ValueError("%s: %s declared but never used" % (f.func_name,
> ','.join(unused)))
> ValueError: f: z declared but never used
> >>> @uses("x")
> def f(x):
> y = x+1
> return z
>
>
> Traceback (most recent call last):
> File "", line 1, in
> @uses("x")
> File "", line 8, in decorator
> raise ValueError("%s: %s assigned but not declared" % (f.func_name,
> ','.join(undeclared)))
> ValueError: f: y assigned but not declared
>
>
> >>>
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
--
Filip Gruszczyński
--
http://mail.python.org/mailman/listinfo/python-list
Re: Explicit variable declaration
> If you want to just declare that name exist, but doesn't want to > declare the type, why don't you just do this: > > def somefunc(): > nonlocal = nonlocal > local = 0 # or None or [] or an initial value > # > return nonlocal * local Err.. I don't quite get. How it may help me? Could you explain? -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Re: Explicit variable declaration
> In Python the standard patten for "declaring" variables is just to assign to
> them as they are needed. If you want the effect of a declaration as you
> would do in C, you can just define the variable and initialize it to 0 or
> None. (Or {} for a new dictionary, or [] for a new list.)
Yep, I get it and I absolutely love this about Python. What I don't
actually love is situation, where I misspell and instead of setting
one variable, other is created. This is why I would like to declare
variables first and say, that these are the only ones, that can be set
in certain function (the same with fields in a class).
I am finishing a small tool, that allows to create such declarations
in similar manner to Smalltalk declarations. I hope I can post a link
to it soon.
--
Filip Gruszczyński
--
http://mail.python.org/mailman/listinfo/python-list
Re: Unimport statement
The main problem are references to objects within a module, because we can NEVER be sure there aren't any, even though we cleaned up everything, that's just a consequence of Python nature. We can keep the old objects referenced and it would make an equivalent to the reload() builting, just without loading the module, in effect the same as removing it from sys.modules and deleting imported references. I do not see any possibe clean way to unload a module... On Ne, kvě 11, 2008 at 10:02:15 +0200, [EMAIL PROTECTED] wrote: > We should have that statement, so that problems, expressed in > http://groups.google.com/group/comp.lang.python/browse_thread/thread/3bda1fc4895ec886/bc5fe40cfbd10124?lnk=raot#bc5fe40cfbd10124, > would not occur. -- http://mail.python.org/mailman/listinfo/python-list
Re: Make Python create a tuple with one element in a clean way
Hello, > so it would be clean if Python would convert anything put into ( ) to > be a tuple, even if just one value was put in (without having to use > that ugly looking comma with no value after it). if it worked that way, it will absolutely mess up Python syntax, because we mathematicians are used to use parentheses to force explicit operator precedence, to group expressions, etc. Should (1+1)*2 yield 4 or (2,2) ? -- http://mail.python.org/mailman/listinfo/python-list
Which is more pythonic?
I have just written a very small snippet of code and started thinking, which version would be more pythonic. Basically, I am adding a list of string to combo box in qt. So, the most obvious way is: for choice in self.__choices: choicesBox.addItem(choice) But I could also do: map(self.__choices, choicesBox.addItem) or [choicesBox.addItem(choice) for choice in self.__choices] I guess map version would be fastest and explicit for is the slowest version. However, the first, most obvious way seems most clear to me and I don't have to care about speed with adding elements to combo box. Still, it's two lines instead of one, so maybe it's not the best. So, which one is? -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
ANN: logbuilder 0.1.0a
This is a one-time post to annouce the creation of logbuilder project, an open source tool for change log creation based on version control commit messages. Using conventions in commit messages logbuilder detects messages that can be imported into the change log and painlessly creates on for every version of software. Home page: http://code.google.com/p/logbuilder/ Features: * Subersion support * Mercurial support * Filtering using wildcards, regular expressions and string matching * (planned) Using templates for displaying results Requirements: * Python 2.6+ * pysvn or mercurial For more information, please come to http://code.google.com/p/logbuilder/. -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Validating positional arguments in optparse
optparse module is quite smart, when it comes to validating options, like assuring, that certain option must be an integer. However, I can't find any information about validating, that positional arguments were provided and I can't find methods, that would allow defining positional argument in OptionParser. Is it possible to do this, or was it intentionaly left this way? -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Re: Validating positional arguments in optparse
> That being said, I still stick with optparse. I prefer the dogmatic > interface that makes all my exe use the exact same (POSIX) convention. I > really don't care about writing /file instead of --file I would like to keep POSIX convention too, but just wanted OptionParser to do the dirty work of checking that arguments are all right for me and wanted to know the reason, it doesn't. I guess I should write a subclass, which would do just that. -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
urllib2 urlopen takes too much time
I have encountered a performance problem using suds, which was traced down to _socket.recv. I am calling some web services and each of them uses about 0.2 sec and 99% of this time is spent on urllib2.urlopen, while the rest of the call is finished in milliseconds. Because of this, my web app works really slow, especially when it must do many ws calls. I could of course decrease the number of calls and do a lot of caching (which would be quite perilous and I wouldn't like to delve into this), but it seems as treating symptoms, rather than the illness itself. The machine I am connecting to through ws is in the same building, the connection should be really fast and mostly it is - except for using suds. Is it possible, that I could done something wrong and it hangs on this socket for too long? Have anyone encountered similar problems? Oh, and we did some profiling using also other tool written in C#, that also uses those web services and it worked much faster, so it doesn't seem to be connection problem. -- Filip Gruszczyński -- http://mail.python.org/mailman/listinfo/python-list
Specific iterator in one line
This is purely sport question. I don't really intend to use the answer
in my code, but I am wondering, if such a feat could be done.
I have a following problem: I have a list based upon which I would
like to construct a different one. I could simply use list
comprehensions, but there is an additional trick: for some elements on
this list, I would like to return two objects. For example I have a
list of 0s and 1s and for 0 I would like to add 1 'a' and for 1 I
would like to add 2 'b', like this:
[1, 0, 0, 1] -> ['b', 'b', 'a', 'a', 'b', 'b']
The easy way is to return a tuple ('b', 'b') for 1s and then flatten
them. But this doesn't seem very right - I'd prefer to create a nice
iterable right away. Is it possible to achieve this? Curiosly, the
other way round is pretty simple to achieve, because you can filter
objects using if in list comprehension.
--
Filip Gruszczyński
--
http://mail.python.org/mailman/listinfo/python-list
Re: Specific iterator in one line
Oh, and there is additional requirement: it must be a one liner with
at most 80 characters ;-)
W dniu 30 czerwca 2009 10:44 użytkownik Filip Gruszczyński
napisał:
> This is purely sport question. I don't really intend to use the answer
> in my code, but I am wondering, if such a feat could be done.
>
> I have a following problem: I have a list based upon which I would
> like to construct a different one. I could simply use list
> comprehensions, but there is an additional trick: for some elements on
> this list, I would like to return two objects. For example I have a
> list of 0s and 1s and for 0 I would like to add 1 'a' and for 1 I
> would like to add 2 'b', like this:
>
> [1, 0, 0, 1] -> ['b', 'b', 'a', 'a', 'b', 'b']
>
> The easy way is to return a tuple ('b', 'b') for 1s and then flatten
> them. But this doesn't seem very right - I'd prefer to create a nice
> iterable right away. Is it possible to achieve this? Curiosly, the
> other way round is pretty simple to achieve, because you can filter
> objects using if in list comprehension.
>
>
> --
> Filip Gruszczyński
>
--
Filip Gruszczyński
--
http://mail.python.org/mailman/listinfo/python-list
