Where to put data
I'm having a disagreement with a buddy on the packaging of a program we're doing in Python. It's got a number of modules and large number of library files. The library stuff is data, not code. I'd like to put the modules in /usr/lib/pythonX.Y/mymodules or wherever setup.py decides. And the data in /usr/share/lib/myprogram. My buddy says, that it'll be hard to be consistant in the /usr/share/.. when we consider platforms other than linux. So, he wants: /usr/lib/pythonX.Y/myprogram mymodules ... mydata I've got 2 issues with this: 1. I don't know if putting data in the python tree is "legit". 2. I'd have to do a lot of rewritting. My modules currently use: include mymodules.foobar x=mymodules.foobar.func() and I would need to change that to: include myprogram.mymodules.foobar x=myprogram.mymodules.foobar.func() unless there is python way to drop the "myprogram" bit? -- http://mail.python.org/mailman/listinfo/python-list
Re: Where to put data
Right now my program does a search for modules in "all the normal places", which seems to work for windows, mac and linux. Once the modules are found I just insert that location into sys.path[0]. Which permits the modules to reside anywhere on the HDD. However, I have feeling that this isn't quite pythonic. -- http://mail.python.org/mailman/listinfo/python-list
Re: Where to put data
> I would not put anything in the toplevel Python folder. You need to > place everything under site-packages --> "Python27\Lib\site-packages > \PackageName\blah". Of course client created files should be saved to > a more accessible place. Oh. Just looking at my setup (Ubunutu 11.10) and I see that /usr/lib/python2.7 doesn't have a site-packages directory. However, /usr/local/lib/python2.7 has both dist-packages and site-packages. So, my stuff should probably go into /usr/local/lib/python2.7/site-packages? Interesting (?) that these are empty dirs right now? Also, if I look at my sys.path value I see that /usr/local/lib/python2.7/dist-packages is in the path; but site-packages is not. > Considering mymodules is a valid python package, you can do: > py> from mymodules import foobar Yes. Understand that part. And then I can just call 'foobar()'. What I was wondering is if there was a way to set something in __init__.py to shorten the calls. So, if I have: /usr/local/lib/python2.7/dist-packages/myprogram mymods __init__.py mod1.py mod2.py mylibs __init__.py Is there some magic I can put into myprogram/__init__.py which forces modules to be imported from mymods instead of myprogram/mymods? Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: Where to put data
On Wednesday, January 25, 2012 8:30:54 PM UTC-7, Michael Torrie wrote: > Unless you are writing a python library that will be used by others, I > don't think that where you put your files has anything to do with being > "pythonic" or not. Just do what works for your OS. Yes. I agree and it's nice to have a confirmation. So far I've been putting all my program into /usr/local/share/MYPROGRAM and then simply inserting an entry into sys.path. Then, for other systems, I check a few common locations until I find the installation. I'm getting mangled by the debian maintainers and friends who seem to believe that python modules need to go into /usr/lib/python... -- http://mail.python.org/mailman/listinfo/python-list
Re: Where to put data
On Thursday, January 26, 2012 8:20:24 PM UTC-7, Michael Torrie wrote: > > > I'm getting mangled by the debian maintainers and friends who seem to > > believe that python modules need to go into /usr/lib/python... > > I guess the maintainers aren't distinguishing between python apps and > their submodules and general python modules (libraries), which is pretty > silly. Even as a mere user I would not like my /usr/lib/python > directory cluttered with python code that is not useful generally but is > only for specific apps. Namespace collisions are inevitable with other > python apps (not libraries) if folks insist on doing this. Well, I might be wrong in my assumptions. Never got invited to join the inner circle so I'm looking at all this from the outside. > > Calibre appears to be in the Ubuntu standard repositories. I just Yeah, I looked at that as well after your earlier post. > checked and in calibre proper (not talking about dependent libraries and > things that would be useful outside of calibre), there are no python > files installed in /usr/lib/python/. Calibre modules that belong to But, when you dl from the calibre site the default location is /opt. > calibre proper are in /usr/lib/calibre. Recipes (really just python > scripts) are in /usr/share/calibre. Maybe Ubuntu is doing things > differently than Debian, but I'm hard pressed to see the logic in > forcing everything ever written in python, such as submodules, installed > to /usr/lib/python. Baffles the mind. I completely agree. Mind you, one consolation in putting things in, for example, /usr/lib/pythonX.Y are: - you can let setup find that that magic location - you don't need to worry about your app finding the lib (python modules). But, I've pretty much decided that the easy way (and dare I say the correct way?) is to let my packager decide where to install the modules. My program really doesn't care (nor do I). And, then I'll end up with my program's stuff something like: myprogram mymodules ... mylib ... program-bin And then have a link in the users path which is a link or a one line call to program-bin. With modules in a directory at the same level as program-bin I don't have to do any module searches, etc. Seems to be a simple and sort-of-elegant solution. I've tried this with one-line callers and links and it seems to work in all cases. Any gotchas? -- http://mail.python.org/mailman/listinfo/python-list
Re: Where to put data
On Friday, January 27, 2012 3:15:44 PM UTC-7, John Nagle wrote: > On 1/25/2012 9:26 AM, bvdp wrote: > > I'm having a disagreement with a buddy on the packaging of a program > > we're doing in Python. It's got a number of modules and large number > > of library files. The library stuff is data, not code. > > How much data? Megabytes? Gigabytes? > > I have some modules which contain nothing but big > constants, written by a program in Python format. > > John Nagle A couple of hundred files totaling about 2 meg. Not a lot. Gosh, I remember when this was several full floppies :) -- http://mail.python.org/mailman/listinfo/python-list
Raise X or Raise X()?
Which is preferred in a raise: X or X()? I've seen both. In my specific case I'm dumping out of a deep loop: try: for ... for ... for ... if match: raise StopInteration() else ... except StopInteration: print "found it" -- http://mail.python.org/mailman/listinfo/python-list
Re: Raise X or Raise X()?
Thanks all for the comments. > Personally, I used "raise X" to mean "this doesn't need arguments and > should never have any" and "raise X()" to mean "this needs arguments but > I'm too lazy to provide them right now". Think of it as a FIXME. Yes, that makes as much sense as anything else :) -- http://mail.python.org/mailman/listinfo/python-list
TK program problem
I've just done an update to my system here to Ubuntu 11.04. Mostly no
problems ... but I have an important (to me) python/TK program that's
stopped working. Well, it works ... mostly.
The python version is 2.7.1+ (no idea what the + means!).
I _think_ I have traced the problem to certain menus which call a
class. The calls appear to be ignored.
Basically, what I have is a line like:
bf = makeButtonBar(root, row=0, column=0, buttons=(
("Quit", self.quitall ),
("Stop", self.stopPmidi ),
("New Dir", self.chd),
("Load Playlist", self.playList),
("Favorites", selectFav),
("Options", setOptions) ) )
To create a menu bar. The function makeButtonBar() creates the buttons
with:
for txt, cmd in buttons:
Button(bf, text=txt, height=1, command=cmd).grid(column=c,
row=0, pady=5)
All this is fine (and worked perfectly before my upgrade). The menu
items which are ordinary functions continue to work. BUT the callbacks
which are classes are just ignored when they are clicked.
A cut from one of the ignored classes:
class selectFav:
def __init__(self):
...
And I've inserted some prints in the __init__() and nothing is
printed. Also, converted the class to new-style () but no change there
either.
Either python/tk has changed or my system is totally $*(#*#.
Suggestions welcome!
--
http://mail.python.org/mailman/listinfo/python-list
Re: TK program problem
> I'm not a tk user, but it sounds like it has regressed from accepting
> arbitrary callables as callbacks to accepting functions specifically.
>
> What happens if you replace:
>
> ("Favorites", selectFav),
>
> with:
>
> ("Favorites", lambda: selectFav()),
Okay, this works. Great and thanks! Seems to me that the way I was
doing it should be alright ... and I've got some other programs
exhibiting the same problem.
Before I go "fixing" the issue ... is this known or even a bug?
--
http://mail.python.org/mailman/listinfo/python-list
Re: TK program problem
Probably the fix is to use a function :) > The docs [1] say that a callback is a function, so I guess that if it > worked before it was just luck. You should bring it up on the tkinter > list and see what they have to say about it, though. > > I'm a bit confused about why you would want to use a class as a > callback anyway. It looks like when the button is clicked it > instantiates the class and then throws it away? I have no idea why I used a class like this, expect that it seemed to work at the time. Yes, the class is instantiated when the button is clicked. Then, the user stays in the class and uses its methods until he hits in the class. So, I guess what's happening is that I'm treating the button click much like a call to a new program/window which sets some options, etc. in the main program. You mention the tkinter group. Ummm, what group is that??? Best, -- http://mail.python.org/mailman/listinfo/python-list
Re: TK program problem
On May 20, 4:29 pm, Ian Kelly wrote: > On Fri, May 20, 2011 at 5:07 PM, bvdp wrote: > > You mention the tkinter group. Ummm, what group is that??? > > http://tkinter.unpythonic.net/wiki/TkinterDiscuss Thanks. New one for me. I'll subscribe and see if they know about this. Best, -- http://mail.python.org/mailman/listinfo/python-list
Re: TK program problem
Thanks, Peter, for the detailed explanation. I was going to write a bit of sample/minimal code to demo this, but you nicely beat me to it! > Here's a minimal script to reproduces the problem: > > $ cat tkcallclass.py > import Tkinter as tk > > root = tk.Tk() > root.withdraw() > > class Classic: > def __init__(self): > print "hello" > > button = tk.Button(root, command=Classic) > button.invoke() > $ python2.6 tkcallclass.py > hello > $ python2.7 tkcallclass.py > Traceback (most recent call last): > File "tkcallclass.py", line 11, in > button.invoke() > File "/usr/local/lib/python2.7/lib-tk/Tkinter.py", line 2081, in invoke > return self.tk.call(self._w, 'invoke') > _tkinter.TclError: invalid command name "__main__.Classic" > $ Any idea why I'm not getting any traceback in my program? It just runs and appears to ignore the callback. > In 2.7 the Tkinter code was changed to use hasattr(obj, "__call__") instead > of callable(obj) to recognize callbacks. This gives different results for > oldstyle classes > > >>> class A: pass > ... > >>> callable(A) > True > >>> hasattr(A, "__call__") > > False > > ...and they are no longer registered automatically with Tkinter. In theory > you could register them explicitly yourself > > $ cat tkcallclass2.py > import Tkinter as tk > > root = tk.Tk() > root.withdraw() > > class Classic: > def __init__(self): > print "hello" > > button = tk.Button(root, command=root.register(Classic)) > button.invoke() > $ python2.7 tkcallclass2.py > hello > > but in practice changing them to newstyle (i. e. have them inherit from > object) or wrapping them in a lambda appears convenient. Yes, I can confirm that both the lambda and setting the class to: class selectFav(object): works. > Personally, I would reconsider whether using a class as a callback is really > necessary. Replacing the class with a function should be a straightforward > process. IIRC, I used the class method since it nicely encapsulates a set of operations: - create/raise a window - list a set of configurable options - have and buttons, both of which destroy the window. Having a function as the callback certainly works as well. Not sure which is the best method in the long run ... I'm trying to use classes more in my programming since it's nice to wrap a bunch of functions up like this. Thanks again. -- http://mail.python.org/mailman/listinfo/python-list
Re: TK program problem
On May 20, 4:37 pm, rantingrick wrote: > Thats sounds to me a lot like hammering square pegs though round > holes... Perhaps you should explain first in "plain english" what Ahh, but what fun would the Internet, Usenet and programming be without round holes and square pegs. I thought my English was pretty good. > problem your application is intended to solve, then how it is expected I'm trying very much to focus on the problem I'm having with a particular bit of python code. I'm not trying to have the community solve my coding problems. > to interact with the user, and finally, what exactly is NOT working > correctly. I would suffix such a documentary with the current source > code, verbatim. You want me to attach several hundred/thousand lines of source code to demonstrate that a particular bit of python has changed behavior between versions? -- http://mail.python.org/mailman/listinfo/python-list
Re: TK program problem
> One of the purposes and advantages of Python 3 is having only one class > system. Best to always use new-style classes in Python 2.2+ unless you > understand and need old-style classes (and need should be never for most > people). > Thanks for this. I'll keep it in mind! One thing I really don't understand ... is there a difference between the old/new forms: class foo: class foo(): In cases where I've played with them, they _appear_ to work the same? Also, where does one find the magic that says that for a tkinter class you should use: class foo(object): Not really sure where "object" comes from. Thanks and best, -- http://mail.python.org/mailman/listinfo/python-list
from xx import yy
I'm having a conceptual mind-fart today. I just modified a bunch of code to use "from xx import variable" when variable is a global in xx.py. But, when I change/read 'variable' it doesn't appear to change. I've written a bit of code to show the problem: mod1.py myvar = 99 def setvar(x): global myvar myvar = x test1.py import mod1 mod1.myvar = 44 print (mod1.myvar) mod1.setvar(33) print (mod1.myvar) If this test1.py is run myvar is fine. But, if I run: test2.py from mod1 import myvar, setvar myvar = 44 print (myvar) setvar(33) print (myvar) It doesn't print the '33'. I thought (apparently incorrectly) that import as would import the name myvar into the current module's namespace where it could be read by functions in the module -- https://mail.python.org/mailman/listinfo/python-list
Re: from xx import yy
On Sunday, November 12, 2017 at 7:18:04 PM UTC-7, bvdp wrote: > I'm having a conceptual mind-fart today. I just modified a bunch of code to > use "from xx import variable" when variable is a global in xx.py. But, when I > change/read 'variable' it doesn't appear to change. I've written a bit of > code to show the problem: > > mod1.py > myvar = 99 > def setvar(x): > global myvar > myvar = x > > test1.py > import mod1 > mod1.myvar = 44 > print (mod1.myvar) > mod1.setvar(33) > print (mod1.myvar) > > If this test1.py is run myvar is fine. But, if I run: > > test2.py > from mod1 import myvar, setvar > myvar = 44 > print (myvar) > setvar(33) > print (myvar) > > It doesn't print the '33'. > > I thought (apparently incorrectly) that import as would import the name myvar > into the current module's namespace where it could be read by functions in > the module Thanks all for confirming that I was wrong to use "from .. import". Hmmm, perhaps for functions it might be okay. But, in most cases it's a lot more obvious to use module.function() when calling. Maybe a bit slower, but I'm sure it's negligible in most cases. And, yes, I am trying to share state info between modules. Is this a bad thing? I guess I would write getter() and setter() functions for all this. But that does seem to remind me too much of some other language :) -- https://mail.python.org/mailman/listinfo/python-list
Re: from xx import yy
On Tuesday, November 14, 2017 at 2:53:22 PM UTC-7, Cameron Simpson wrote: > On 13Nov2017 08:58, bvdp wrote: > >On Sunday, November 12, 2017 at 7:18:04 PM UTC-7, bvdp wrote: > >> I'm having a conceptual mind-fart today. I just modified a bunch of code > >> to use "from xx import variable" when variable is a global in xx.py. But, > >> when I change/read 'variable' it doesn't appear to change. I've written a > >> bit of code to show the problem: > >> > >> mod1.py > >> myvar = 99 > >> def setvar(x): > >> global myvar > >> myvar = x > >> > >> test1.py > >> import mod1 > >> mod1.myvar = 44 > >> print (mod1.myvar) > >> mod1.setvar(33) > >> print (mod1.myvar) > >> > >> If this test1.py is run myvar is fine. But, if I run: > >> > >> test2.py > >> from mod1 import myvar, setvar > >> myvar = 44 > >> print (myvar) > >> setvar(33) > >> print (myvar) > >> > >> It doesn't print the '33'. > >> > >> I thought (apparently incorrectly) that import as would import the name > >> myvar into the current module's namespace where it could be read by > >> functions in the module > > > >Thanks all for confirming that I was wrong to use "from .. import". Hmmm, > >perhaps for functions it might be okay. But, in most cases it's a lot more > >obvious to use module.function() when calling. Maybe a bit slower, but I'm > >sure it's negligible in most cases. > > You're wrong to use it for this particular special case, and ony because you > lose the reference to the module itself, which means you're no longer > accessing > the _reference_ "mod1.myvar", you're accessing a copy of the reference. So > the > wrong reference gets changed. > > In the general case, this isn't something people do a great deal. For most > imports you want access to things from the module with no intention of > changing > those references in the source module. > > So "from xx import yy" is perfectly find for that, the most common use case. > > Consider: > > from os.path import basename, dirname > > Is your code more readable with: > > from os.path import basename, dirname > base_of_parent_dir = basename(dirname(some_path)) > > or as: > > import os.path > base_of_parent_dir = os.path.basename(os.path.dirname(some_path)) > > particularly when you make lots of such calls? I much prefer the former. > > >And, yes, I am trying to share state info between modules. Is this a bad > >thing? I guess I would write getter() and setter() functions for all this. > >But > >that does seem to remind me too much of some other language :) > > In controlled situations it can be ok. Usually I define a small class for > this > kind of thing and keep all the state in class instances. That way it can > scale > (keeping multiple "states" at once). > > However, it does depend on your particular situation. I find situations where > I > want to directly change "global" values in other modules very rare, though > not > unknown. > > Cheers, > Cameron Simpson (formerly [email protected]) In my original case, I think (!!!), the problem was that I had a variable in mod1.py and when I did the "from mod1 import myvarible" all was fine. Python create a new local-to-the-module variable and initialized it to the value it was set to in mod1. And at this point all is well. But, when mod1 changed the value of myvariable the change didn't get passed to the other modules. Of course, the reason for my confusion is that I'm thinking that python is using pointers :) Opps. Maybe we need pointers in python . -- https://mail.python.org/mailman/listinfo/python-list
"normalizing" a value
Not sure what this is called (and I'm sure it's not normalize). Perhaps "scaling"? Anyway, I need to convert various values ranging from around -50 to 50 to an 0 to 12 range (this is part of a MIDI music program). I have a number of places where I do: while x < 0: x += 12 while x >= 12: x -= 12 Okay, that works. Just wondering if there is an easier (or faster) way to accomplish this. -- https://mail.python.org/mailman/listinfo/python-list
Re: "normalizing" a value
On Wednesday, July 1, 2015 at 6:27:57 PM UTC-7, [email protected] wrote: > On Wed, Jul 1, 2015, at 20:12, bvdp wrote: > > Not sure what this is called (and I'm sure it's not normalize). Perhaps > > "scaling"? > > > > Anyway, I need to convert various values ranging from around -50 to 50 to > > an 0 to 12 range (this is part of a MIDI music program). I have a number > > of places where I do: > > > >while x < 0: x += 12 > >while x >= 12: x -= 12 > > And this gives you what you want? With e.g. 13=1, 14=2, 22=10, 23=11, > 24=0, 25 = 1, etc. Seems unusual that that's what you would want. > > Also note this gives an 0 to 11 range for the results, not 0 to 12. > > Anyway, x %= 12 will give the same results. Thanks guys. Yes, that is exactly what I want. I have a number of places where a MIDI note value is being generated. MIDI should be 0..127, but the process creates notes outside the range. Guess that's another question: if the value I have is <0 or >127 I add/subtract 12 'til it's in range. Don't see using modulo working on this??? As far as the original question: Yes, that's what I need. At times I need to take a note (say 14) and map it into a single octave range. So, the 12 becomes 2. Both 14 and 2 are numeric values for note "d", just an octave apart. Interesting that negative values translate properly. That's an non-intuitive result to me. Guess I should have studied that math stuff harder way back when! -- https://mail.python.org/mailman/listinfo/python-list
Re: "normalizing" a value
On Wednesday, July 1, 2015 at 7:15:28 PM UTC-7, Steven D'Aprano wrote: > On Thu, 2 Jul 2015 10:12 am, bvdp wrote: > > > Not sure what this is called (and I'm sure it's not normalize). Perhaps > > "scaling"? > > > Could be normalising, could be scaling. > > > Anyway, I need to convert various values ranging from around -50 to 50 to > > an 0 to 12 range (this is part of a MIDI music program). I have a number > > of places where I do: > > You say "around" -50 to 50. Can you get 51? 151? How do you want to treat > such out of range numbers? > > Are the values integer valued, or can they include fractional values like > 27.356? > > >while x < 0: x += 12 > >while x >= 12: x -= 12 > > > > Okay, that works. Just wondering if there is an easier (or faster) way to > > accomplish this. > > > One approach is to just re-scale the numbers from the range -50...50 to > 0...12 inclusive. That is: > > before => after > -50 => 0 > 0 => 6 > 50 => 12 > > > and everything in between is scaled equivalently. Given x between -50 and 50 > inclusive, calculate: > > y = (x+50)/100.0*12 > > > (Note that in Python 2, you need to make at least one of those values a > float, otherwise you may get unexpected results.) > > That will give you y values from the range 0.0 to 12.0 inclusive. If x is > less than -50.0 or more than +50.0 y will likewise be out of range. You can > clip the result: > > y = min(12.0, max(0.0, y)) > > If your x values are integer values (no fractional values) between -50 and > +50 you can use clock arithmetic. Think of a clock marked 0 to 12 (so there > are 13 values), once you reach 12 adding 1 takes you back to 0. > > 0, 13, 26, 39 => 0 > 1, 14, 27, 40 => 1 > 2, 15, 28, 41 => 2 > ... > 12, 25, 38, 51 => 12 > > > Extending that to negative values in the most obvious fashion: > > -1, -14, -27, -40 => 12 > ... > -12, -25, -38, -51 => 1 > -13, -26, -39, -52 => 0 > > > We can do that easily with the % (modulo) operator: > > y = x % y > > > Modulo even works with non-integer values: > > py> 13.5 % 13 > 0.5 > > > > -- > Steven Thanks for this Steven. However, I think it's not correct in this case. Like I said in another message X%12 is working just fine. No matter what the value is. -- https://mail.python.org/mailman/listinfo/python-list
Re: "normalizing" a value
On Wednesday, July 1, 2015 at 7:23:19 PM UTC-7, [email protected] wrote: > On Wed, Jul 1, 2015, at 21:49, bvdp wrote: > > Interesting that negative values translate properly. That's an > > non-intuitive result to me. Guess I should have studied that math stuff > > harder way back when! > > There are multiple interpretations of the operation, and not all > languages behave the same way as Python does with negative operands. > Python is the odd one out when one considers C/C++, C#, and Java which > all behave a different way. > > In general, almost all languages behave in a way so that given q, r = a > // b, a % b; q * b + r == a. However, this simply changes the question > to how division results involving negative operands are rounded. > > Here's an article by GvR about why python behaves the way it does: > http://python-history.blogspot.com/2010/08/why-pythons-integer-division-floors.html Interesting link. Thanks. I always thought that modulo was modulo. Guess this is another example of why converting code between languages is hard :) Anyway, far as shoving my MIDI notes into a single octave, x % 12 seems to be perfect. -- https://mail.python.org/mailman/listinfo/python-list
Re: "normalizing" a value
On Wednesday, July 1, 2015 at 8:37:18 PM UTC-7, Dennis Lee Bieber wrote: > On Wed, 1 Jul 2015 18:49:34 -0700 (PDT), bvdp declaimed > the following: > > > > >Thanks guys. Yes, that is exactly what I want. I have a number of places > >where a MIDI note value is being generated. MIDI should be 0..127, but the > >process creates notes outside the range. Guess that's another question: if > >the value I have is <0 or >127 I add/subtract 12 'til it's in range. Don't > >see using modulo working on this??? > > > >As far as the original question: Yes, that's what I need. At times I need to > >take a note (say 14) and map it into a single octave range. So, the 12 > >becomes 2. Both 14 and 2 are numeric values for note "d", just an octave > >apart. > > > > Modulo will give you the "note", but throws out the octave. > > Your original post specified -50..50, which spans only 101 values, whereas > MIDI spans 128 values -- so what do you really consider out of range? And > what MIDI value is -50 supposed to map against. Is input 0 supposed to > represent middle-C with negative values being notes on the bass clef and > positive values being treble clef? > > MIDI middle-C is note 60. Presuming your 0 is supposed to be middle-C, I'd > do the transformation as: > > midiNote = invalue + 60 > if midiNote < 0: midiNote = midiNote % 12 > if midiNote > 127: midiNote = (midiNote % 12) + 115 > > which actually means input values of -60..+67 are shifted directly to a > midi note number, and values outside of that range are shadowed as the > lowest or highest octave. > > >>> for i in range(-70, 80, 4): > ... midiNote = i + 60 > ... if midiNote < 0: midiNote = midiNote % 12 > ... if midiNote > 127: midiNote = ((midiNote - 5) % 12) + 113 > ... print i, midiNote, "CcDdEFfGgAaB"[midiNote % 12], divmod(midiNote, > 12) > ... > -70 2 D (0, 2) > -66 6 f (0, 6) > -62 10 a (0, 10) > -58 2 D (0, 2) > -54 6 f (0, 6) > -50 10 a (0, 10) > -46 14 D (1, 2) > -42 18 f (1, 6) > -38 22 a (1, 10) > -34 26 D (2, 2) > -30 30 f (2, 6) > -26 34 a (2, 10) > -22 38 D (3, 2) > -18 42 f (3, 6) > -14 46 a (3, 10) > -10 50 D (4, 2) > -6 54 f (4, 6) > -2 58 a (4, 10) > 2 62 D (5, 2) > 6 66 f (5, 6) > 10 70 a (5, 10) > 14 74 D (6, 2) > 18 78 f (6, 6) > 22 82 a (6, 10) > 26 86 D (7, 2) > 30 90 f (7, 6) > 34 94 a (7, 10) > 38 98 D (8, 2) > 42 102 f (8, 6) > 46 106 a (8, 10) > 50 110 D (9, 2) > 54 114 f (9, 6) > 58 118 a (9, 10) > 62 122 D (10, 2) > 66 126 f (10, 6) > 70 118 a (9, 10) > 74 122 D (10, 2) > 78 114 f (9, 6) > >>> > > The oddity for the >127 case is that 127 is not an even octave break point. > divmod() gives you the "octave" and the note. > -- > Wulfraed Dennis Lee Bieber AF6VN > [email protected]://wlfraed.home.netcom.com/ Thanks. I like your solution for values <0 >127. Makes more sense than mindlessly adding/subtracting 12 to bring into range. In this case I'm just trying to take an out-of-range note to the top/bottom of the valid midi range. -- https://mail.python.org/mailman/listinfo/python-list
Dealing with exceptions
Every time I write a program with exception handling (and I suppose that includes just about every program I write!) I need to scratch my brain when I create try blocks. For example, I'm writing a little program do copy specific files to a USB stick. To do the actual copy I'm using: try: shutil.copy(s, os.path.join(usbpath, songname)) except ... now, I need to figure out just what exceptions to handle. Let's see: IOError that means that the disk is full or otherwise buggered. Better dump out of the loop. But, I know there can be other errors as well. Doing some tests, I know that certain filenames are invalid (I think a "?" or unicode char is invalid when writing to a FAT32 filesystem). And, so what exception is that? Without actually creating the error, I can't figure it out. In this case, I can run the program an number of times and parse out the errors and write code to catch various things. But, I think I'm missing something completely. Guess what I'm looking for is a list of possible (probable?) errors for the shutil.copy() command. And, in a much bigger manual, for most other commands. Maybe I'm just venting about FAT32 filesystems :) -- http://mail.python.org/mailman/listinfo/python-list
Re: Dealing with exceptions
> > IOError and OSError should cover all copy problems, I think. How do you know that? I can figure it out as well by running the program, but I'd like to make the determination of what to catch when I'm writing the code. -- http://mail.python.org/mailman/listinfo/python-list
Re: Dealing with exceptions
> > Here's a bit of a left-field thought: Maybe none of them. > Not far left at all :) > > What are you actually doing when you get an exception? Can you > > plausibly recover? If not - that is, if you're going to abort the > > whole operation anyway - then save yourself the trouble of writing the > > try/catch, and just let the exception propagate up (to the console, if > > nowhere else). > My first cut of the program did exactly that. Just abort the whole thing, figuring that the disk was full or buggered. What I ran into was that half way through the process I ended up with a filename which the FAT32 didn't like. So, my brain-dead idea was to catch those (and ignore them) and continue on. But then I have to distinguish between a bad filename and "real" errors. But, you are probably correct if you say I should check the filename first :) Is there a module for that? -- http://mail.python.org/mailman/listinfo/python-list
Circular import problem
I'm going quite nutty here with an import problem. I've got a fairly complicated program (about 12,000 lines in 34 modules). I just made some "improvements" and get the following error: bob$ mma Traceback (most recent call last): File "/usr/local/bin/mma", line 55, in import MMA.main File "/usr/local/share/mma/MMA/main.py", line 33, in import MMA.docs File "/usr/local/share/mma/MMA/docs.py", line 34, in import MMA.grooves File "/usr/local/share/mma/MMA/grooves.py", line 41, in import MMA.auto File "/usr/local/share/mma/MMA/auto.py", line 35, in import MMA.parse File "/usr/local/share/mma/MMA/parse.py", line 2052, in 'AUTHOR': MMA.docs.docAuthor, AttributeError: 'module' object has no attribute 'docs' I can fix this by deleting a line in docs.py. Just take out the "import MMA.grooves" and all works. What I really don't get is that in the middle of this module which is now NOT loading "grooves.py" I have a command to access a function in that module and it works. Yes, grooves.py has an import docs.py. And there are lots of other such imports in the program. So, I have to assume that the error is being generated by some other modules loading in the grooves.py stuff ... but really have no idea. Is there anything I can do to trace though this and get it working properly? Oh, this is python 2.5.1 on Ubuntu Linux. Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: Circular import problem
> Seehttp://effbot.org/zone/import-confusion.htm > Try to move the circular references later in the code (maybe inside a > function, when it is required), or much better, refactor it so there is no > circularity. > > -- > Gabriel Genellina Yes, thanks. I'd read that page before posting. Helpful. But, I still don't understand how python can access a function in a file I have NOT included. In this case, to get things to work, I DO NOT "import MMA.grooves" but later in the module I access a function with "xx=MMA.grooves.somefunc()" and it finds the function, and works just fine. It shouldn't work. I have tried to delay the import, and that does work. But, from a stylistic view I really to like to have all my imports at the top of the module. Maybe some old assembler/C habits on my part. -- http://mail.python.org/mailman/listinfo/python-list
Re: Circular import problem
Just as a bit of a followup, I have fixed the problem in my code. I changed the order of some of the imports in some other modules. What I was doing was more guesswork and good luck ... but it works. I really wonder if there is a better way to figure these problems out. Reading a few of the other discussions on the 'net about this it seems that I'm not the 1st to get hit with the problem ... and the solutions I find aren't any prettier than mine :) -- http://mail.python.org/mailman/listinfo/python-list
Re: Circular import problem
> > But, I still don't understand how python can access a function in a > > file I have NOT included. In this case, to get things to work, I DO > > NOT "import MMA.grooves" but later in the module I access a function > > with "xx=MMA.grooves.somefunc()" and it finds the function, and works > > just fine. It shouldn't work. > > That depends a bit on what is "MMA" and what is "grooves". > MMA.grooves means "look for an attribute named grooves inside MMA". If MMA > is a module, and MMA.grooves is a class/function defined inside the > module, you don't need to import it before using it. > But if MMA is a package, and MMA.grooves is a module inside that package, > you need to import it first (unless MMA/__init__.py does that already) In this case (and I should have been more clear) MMA is a package and grooves is a module. It is working now, but before I moved other imports around I was able to do the following: 1. NOT include MMA.gooves, 2. call the function MMA.grooves.somefunc() and have it work. The point I was trying to make was that something in my imports was causing the namespace to contain MMA.grooves.* when it hadn't been imported in that module. I suspect that my reordering effected some initialization of modules, probably the creation of some classes. But that is a guess. I've just used an empty __init__.py file. I will have to read up a bit more on packages and see what advantage there is to import in that. > > I have tried to delay the import, and that does work. But, from a > > stylistic view I really to like to have all my imports at the top of > > the module. Maybe some old assembler/C habits on my part. > > Sure, it's considered good style. Which is considered good style? Loading at the top or loading when needed? -- http://mail.python.org/mailman/listinfo/python-list
Re: Building musical chords starting from (a lot of) rules
Mr.SpOOn wrote: Hi, I'm writing a method to create musical chords. This method must follow a specific set of syntax rules. At least, this is my idea, but maybe there's a better way. Anyway, in the code I have class Chord which is a set. The costrunction of a chord is based on a root note and a structure, so by default, giving just a note, it creates a major chord adding the third and fifth note. So, in pseudo-code: Chord(C) -> [C, E, G] And this is the base case. All the other rules must specify what kind of note to add or which one should be modified. For example: C min, means to add the third minor note: C Eb G C 9 is a base chord plus a the ninth note, but this implies the presence of the seventh too, so it results in: C E G B D But I can also say: C 9 no7, so it should just be: C E G D, without the seventh. There are also more complex rules. For the eleventh, for example, it should add also the ninth and seventh note. In the normal case it adds their major version, but I can specify to add an augmented nine, so this modification must have precedence over the base case. Anyway, I think I can use a chain of if-clauses, one per rule and at the end remove the notes marked with "no". But this seems to me a very bad solution, not so pythonic. Before I proceed for this way, do you have any suggestion? Hope the problem is not too complicated. Thanks, Carlo Have a look at my program MMA which does just what I think you want. Well, probably for different purposes, but same end result with the chords. All table based, BTW.http://www.mellowood.ca/mma -- http://mail.python.org/mailman/listinfo/python-list
Re: Building musical chords starting from (a lot of) rules
Mr.SpOOn wrote: So for example in jazz music it is more common the minor seventh than the major, so writing just G7 you mean the dominant seventh chord (with minor seventh) and you have to write just the major one with maj7. A minor 7th has a flatted 3rd (ie. C, Eb, G, Bb). Don't confuse your readers and yourself by thinking that "minor" equals "dominant" in this context. It never does. Spelling is usually Cm7. A 7th is always a dominant (C, E, G, Bb). The only spelling I've ever seen for this is C7. A major 7th is just that (C, E, G, B). Spelling include CM7 (note the uppercase M) and Cmaj7. And then there are things like minor major 7th (C, Eb, G, B) ... and the problem with all this is that the spellings really aren't standard. It varies with different music book publishers, geographic locales and time. There are a number of books around (some out of print) which discuss this in great detail (insert the word tedious if you want). One good reference, if you can find it, is: Carl Brandt and Clinton Roemer. Standardized Chord Symbol Notation. Roerick Music Co. Sherman Oaks, CA. -- Listen to my CD at http://www.mellowood.ca/music/cedars Bob van der Poel ** Wynndel, British Columbia, CANADA ** EMAIL: [EMAIL PROTECTED] WWW: http://www.mellowood.ca -- http://mail.python.org/mailman/listinfo/python-list
unescape escapes in strings
When reading lines of data from a file in the from (no quotes!)
foo\x20bar
and I assign to a variable in a line line like:
f = file('infile', 'r')
for a in f:
print a
the string is read in as string with the literal characters 'f', 'o' ...
'x' , '2' ...
as compared to an assignment like:
a="foo\x20bar"
which is identical to
a="foo bar"
Okay, so far ... I think this is what I want since my program is using
space characters as delimiters and I'm trying to use the \x20 notation
to avoid splitting.
But, now the problem. When I finally assign the string with the \x20 to
a variable the literals are still there. And here I really want them all
nicely converted to the desired values.
So, the question is: is there an "unescape()" for strings so that
"foo\x20bar" is converted to "foo bar"
--
http://mail.python.org/mailman/listinfo/python-list
Re: unescape escapes in strings
MRAB wrote:
bvdp wrote:
When reading lines of data from a file in the from (no quotes!)
foo\x20bar
and I assign to a variable in a line line like:
f = file('infile', 'r')
for a in f:
print a
the string is read in as string with the literal characters 'f', 'o'
... 'x' , '2' ...
as compared to an assignment like:
a="foo\x20bar"
which is identical to
a="foo bar"
Okay, so far ... I think this is what I want since my program is using
space characters as delimiters and I'm trying to use the \x20 notation
to avoid splitting.
But, now the problem. When I finally assign the string with the \x20
to a variable the literals are still there. And here I really want
them all nicely converted to the desired values.
So, the question is: is there an "unescape()" for strings so that
"foo\x20bar" is converted to "foo bar"
>>> a = r"foo\x20bar"
>>> print a
foo\x20bar
>>> a = a.decode("string-escape")
>>> print a
foo bar
Thanks ... I think in my original testing I tried decode() but it didn't
work. Testing more ...
the file has 2 lines:
foo bar
foo\x20bar
and the program to read is:
f=file('in', 'r')
for a in f:
a = a.strip()
a=a.decode()
print list(a)
I get:
python read.py
[]
[u'f', u'o', u'o', u' ', u'b', u'a', u'r']
[u'f', u'o', u'o', u'\\', u'x', u'2', u'0', u'b', u'a', u'r']
So, the \x20 is still literal.
Any other ideas??? I suppose I could write a re expression ... but
surely that is not needed???
--
http://mail.python.org/mailman/listinfo/python-list
Re: unescape escapes in strings
Perfect ... thanks.
>>> a = a.decode("string-escape")
Using "string-escape" does the trick!
Wonderful, this python. And the quick answers on this group.
--
http://mail.python.org/mailman/listinfo/python-list
more on unescaping escapes
So, we think something is working and send of a bug fix to our client :)
I'm not sure I understand this at all and wonder if there is bug?
>>> a="c:\\Program\x20Files\\test"
>>> a
'c:\\Program Files\\test'
so far, so good.
>>> a.decode("string-escape")
'c:\\Program Files\test'
Umm, not so good? The \\ before the P is okay, but the \\t is change to \t
and
>>> print a.decode("string-escape")
c:\Program Filesest
Now, decode() converted the \\t to a \t and print expanded the \t to a tab.
I would have thought that the \\t would have the same result as the \\P ???
Obviously my brain is missing something (hopefully obvious).
--
http://mail.python.org/mailman/listinfo/python-list
Re: more on unescaping escapes
I'm getting hopelessly lost in a series of \\\ s :)
Let's see if this makes sense:
>>> a='c:\\Program Files\\test'
>>> a.decode('string-escape')
'c:\\Program Files\test'
In this case there are still 2 '\'s before the P; but only 1 before the
't'. Now, when it comes time to open the file windows accepts double
'\'s in the filename. So, life is fine. But, the essential part here is
that we're lucky we can use '\\' or '\' in a path. What if this wasn't true?
The following shows a bit of difference:
>>> a='c:\Program Files\test'
>>> a
'c:\\Program Files\test'
In this case the interpreter has changed the '\P' to '\\P'. And if one
lists the string the '\t' really is a tab. No decode() at all in any of
this.
I guess the general rule would be to double up '\'s in filenames and (in
my program's case) to use the \x20 for spaces.
Thanks.
--
http://mail.python.org/mailman/listinfo/python-list
Re: more on unescaping escapes
andrew cooke wrote: do you know that a string with the letter "r" in front doesn't escape slashes? it's intended for regular expressions, but would simplify things for you here too. just do a=r'c:\\Program Files\test' Yes, I knew that. Unfortunately in my program loop I really don't have the opportuity to use a raw string. But, good reminder. Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: more on unescaping escapes
Tim Wintle wrote:
On Mon, 2009-02-23 at 17:00 -0700, bvdp wrote:
Let's see if this makes sense:
>>> a='c:\\Program Files\\test'
>>> a.decode('string-escape')
'c:\\Program Files\test'
Hint: try running
print a
and see what's written - I think that the interpreter adds extra "\"
characters to escape things and make things more easy to read.
i.e.
a = "c:\\test\\t"
a
'c:\\test\\t'
print a
c:\test\t
so when it displays strings in the interpreter it includes escape
characters, when it is printed though the output is straight to stdout
and isn't escaped.
Hope that helps,
Tim Wintle
Not sure if it's more clear or not :)
>>> a="c:\\Program\x20Files\\test"
>>> a
'c:\\Program Files\\test'
>>> print a
c:\Program Files\test
Which is all fine. And I didn't need to use decode().
So, in this case I'm assuming that the interpreter is converting the
escapes on assignment. And, in this case the string has single \s in it.
So far this is making sense. So, when I do a decode() the \t is
converted to a tab, etc.
I think my "problem" may be in viewing an assignment like that above as
opposed to reading a line from a file with '\'s and '\x20's in it. In
this case the assignment doesn't do the conversion (and that is a good
thing!). Now, when I go to save the variable with the escapes I use
decode() and it works.
Just a matter of consistantly using a method I suppose. Still, confusing
between my program reading data from a file and doing testing of strings
at the interpreter. Best to do one or the other, not both.
--
http://mail.python.org/mailman/listinfo/python-list
Re: more on unescaping escapes
Chris Rebert wrote: On Mon, Feb 23, 2009 at 4:26 PM, bvdp wrote: [problem with Python and Windows paths using backslashes] Is there any particular reason you can't just internally use regular forward-slashes for the paths? They work in Windows from Python in nearly all cases and you can easily interconvert using os.pathsep if you want the path to be pretty when you show it to (or get it from) the user or whatever. Cheers, Chris Just because I never really thought too much about it :) I'm doing my work on a linux box and my user is on windows ... and he's used to using '\' ... but, you are absolutely right! Just use '/' on both systems and be done with it. Of course I still need to use \x20 for spaces, but that is easy. Thanks for the suggestion! -- http://mail.python.org/mailman/listinfo/python-list
Re: more on unescaping escapes
Bear in mind that it's the string as it really is that is being operated on, not the representation of it that you displayed Yes, that is the confusion ... what is displayed and what's actually in the string. I think I understand it all now :) Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: more on unescaping escapes
Gabriel Genellina wrote: En Mon, 23 Feb 2009 22:46:34 -0200, bvdp escribió: Chris Rebert wrote: On Mon, Feb 23, 2009 at 4:26 PM, bvdp wrote: [problem with Python and Windows paths using backslashes] Is there any particular reason you can't just internally use regular forward-slashes for the paths? They work in Windows from Python in nearly all cases and you can easily interconvert using os.pathsep if you want the path to be pretty when you show it to (or get it from) the user or whatever. Just because I never really thought too much about it :) I'm doing my work on a linux box and my user is on windows ... and he's used to using '\' ... but, you are absolutely right! Just use '/' on both systems and be done with it. Of course I still need to use \x20 for spaces, but that is easy. Why is that? "\x20" is exactly the same as " ". It's not like %20 in URLs, that becomes a space only after decoding. py> '\x20' == ' ' True py> '\x20' is ' ' True (ok, the last line might show False, but being True means that both are the very same object) I need to use the \x20 because of my parser. I'm reading unquoted lines from a file. The file creater needs to use the form "foo\x20bar" without the quotes in the file so my parser can read it as a single token. Later, the string/token needs to be decoded with the \x20 converted to a space. So, in my file "foo bar" (no quotes) is read as 2 tokens; "foo\x20bar" is one. So, it's not really a problem of what happens when you assign a string in the form "foo bar", rather how to convert the \x20 in a string to a space. I think the \\ just complicates the entire issue. -- http://mail.python.org/mailman/listinfo/python-list
Re: more on unescaping escapes
Gabriel Genellina wrote: En Mon, 23 Feb 2009 23:31:20 -0200, bvdp escribió: Gabriel Genellina wrote: En Mon, 23 Feb 2009 22:46:34 -0200, bvdp escribió: Chris Rebert wrote: On Mon, Feb 23, 2009 at 4:26 PM, bvdp wrote: [problem with Python and Windows paths using backslashes] Is there any particular reason you can't just internally use regular forward-slashes for the paths? [...] you are absolutely right! Just use '/' on both systems and be done with it. Of course I still need to use \x20 for spaces, but that is easy. Why is that? "\x20" is exactly the same as " ". It's not like %20 in URLs, that becomes a space only after decoding. I need to use the \x20 because of my parser. I'm reading unquoted lines from a file. The file creater needs to use the form "foo\x20bar" without the quotes in the file so my parser can read it as a single token. Later, the string/token needs to be decoded with the \x20 converted to a space. So, in my file "foo bar" (no quotes) is read as 2 tokens; "foo\x20bar" is one. So, it's not really a problem of what happens when you assign a string in the form "foo bar", rather how to convert the \x20 in a string to a space. I think the \\ just complicates the entire issue. Just thinking, if you was reading the string from a file, why were you worried about \\ and \ in the first place? (Ok, you moved to use / so this is moot now). Just cruft introduced while I was trying to figure it all out. Having to figure the \\ and \x20 at same time with file and keyboard input just confused the entire issue :) Having the user set a line like c:\\Program\x20File ... works just fine. I'll suggest he use c:/program\x20files to make it bit simple for HIM, not my parser. Unfortunately, due to some bad design decisions on my part about 5 years ago I'm afraid I'm stuck with the \x20. Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: more on unescaping escapes
Adam Olsen wrote:
On Feb 23, 7:18 pm, bvdp wrote:
Gabriel Genellina wrote:
En Mon, 23 Feb 2009 23:31:20 -0200, bvdp escribió:
Gabriel Genellina wrote:
En Mon, 23 Feb 2009 22:46:34 -0200, bvdp escribió:
Chris Rebert wrote:
On Mon, Feb 23, 2009 at 4:26 PM, bvdp wrote:
[problem with Python and Windows paths using backslashes]
Is there any particular reason you can't just internally use regular
forward-slashes for the paths? [...]
you are absolutely right! Just use '/' on both systems and be done
with it. Of course I still need to use \x20 for spaces, but that is
easy.
Why is that? "\x20" is exactly the same as " ". It's not like %20 in
URLs, that becomes a space only after decoding.
I need to use the \x20 because of my parser. I'm reading unquoted
lines from a file. The file creater needs to use the form "foo\x20bar"
without the quotes in the file so my parser can read it as a single
token. Later, the string/token needs to be decoded with the \x20
converted to a space.
So, in my file "foo bar" (no quotes) is read as 2 tokens; "foo\x20bar"
is one.
So, it's not really a problem of what happens when you assign a string
in the form "foo bar", rather how to convert the \x20 in a string to a
space. I think the \\ just complicates the entire issue.
Just thinking, if you was reading the string from a file, why were you
worried about \\ and \ in the first place? (Ok, you moved to use / so
this is moot now).
Just cruft introduced while I was trying to figure it all out. Having to
figure the \\ and \x20 at same time with file and keyboard input just
confused the entire issue :) Having the user set a line like
c:\\Program\x20File ... works just fine. I'll suggest he use
c:/program\x20files to make it bit simple for HIM, not my parser.
Unfortunately, due to some bad design decisions on my part about 5 years
ago I'm afraid I'm stuck with the \x20.
Thanks.
You're confusing the python source with the actual contents of the
string. We already do one pass at decoding, which is why \x20 is
quite literally no different from a space:
'\x20'
' '
However, the interactive interpreter uses repr(x), so various
characters that are considered formatting, such as a tab, get
reescaped when printing:
'\t'
'\t'
len('\t')
1
It really is a tab that gets stored there, not the escape for one.
Finally, if you give python an unknown escape it passes it leaves it
as an escape. Then, when the interactive interpreter uses repr(x), it
is the backslash itself that gets reescaped:
'\P'
'\\P'
len('\P')
2
list('\P')
['\\', 'P']
What does this all mean? If you want to test your parser with python
literals you need to escape them twice, like so:
'c:Program\\x20Filestest'
'c:Program\\x20Filestest'
list('c:Program\\x20Filestest')
['c', ':', '\\', '\\', 'P', 'r', 'o', 'g', 'r', 'a', 'm', '\\', 'x',
'2', '0', 'F', 'i', 'l', 'e', 's', '\\', '\\', 't', 'e', 's', 't']
'c:Program\\x20Filestest'.decode('string-escape')
'c:\\Program Files\\test'
list('c:Program\\x20Filestest'.decode('string-escape'))
['c', ':', '\\', 'P', 'r', 'o', 'g', 'r', 'a', 'm', ' ', 'F', 'i',
'l', 'e', 's', '\\', 't', 'e', 's', 't']
However, there's an easier way: use raw strings, which prevent python
from unescaping anything:
r'c:\\Program\x20Files\\test'
'c:Program\\x20Filestest'
list(r'c:\\Program\x20Files\\test')
['c', ':', '\\', '\\', 'P', 'r', 'o', 'g', 'r', 'a', 'm', '\\', 'x',
'2', '0', 'F', 'i', 'l', 'e', 's', '\\', '\\', 't', 'e', 's', 't']
Thank you. That is very clear. Appreciate your time.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Newb question: underscore
My guess would be someone has used the common convention of naming the "get the corresponding localised version of this string from the application's gettext database" function as '_' for convenience. Funny that this comes up. I just noticed this in some code I was looking at the other day. A number of statements in the form: print _(something) My question is: Why would anyone decide to obfuscate something as easy to read as Python??? At first I thought that they were making a function out of print (which makes some sense), but I don't think that is the case. I tried (not very hard) to trace back the code to figure out where _() is being assigned, but gave up. Oh, this is in the gdesklets package if anyone is interested. Listen to my CD at http://www.mellowood.ca/music/cedars Bob van der Poel ** Wynndel, British Columbia, CANADA ** EMAIL: [EMAIL PROTECTED] WWW: http://www.mellowood.ca -- http://mail.python.org/mailman/listinfo/python-list
Simple and safe evaluator
Is there a simple/safe expression evaluator I can use in a python program. I just want to pass along a string in the form "1 + 44 / 3" or perhaps "1 + (-4.3*5)" and get a numeric result. I can do this with eval() but I really don't want to subject my users to the problems with that method. In this use I don't need python to worry about complex numbers, variables or anything else. Just do the math on a set of values. Would eval() with some restricted list of permitted operators do the trick? I'm feeling too lazy to write/debug my own parser for this :) Thanks, Bob. -- http://mail.python.org/mailman/listinfo/python-list
Re: Simple and safe evaluator
Matimus wrote:
On Jun 11, 1:25 pm, bvdp <[EMAIL PROTECTED]> wrote:
Is there a simple/safe expression evaluator I can use in a python
program. I just want to pass along a string in the form "1 + 44 / 3" or
perhaps "1 + (-4.3*5)" and get a numeric result.
I can do this with eval() but I really don't want to subject my users to
the problems with that method.
In this use I don't need python to worry about complex numbers,
variables or anything else. Just do the math on a set of values. Would
eval() with some restricted list of permitted operators do the trick?
I'm feeling too lazy to write/debug my own parser for this :)
Thanks, Bob.
Here is something that I wrote using the _ast module. It works pretty
well, and might be a good example for others wanting to experiment
with the _ast module. On a related note... if anybody wants to provide
feedback on this code it would be much appreciated. It involves a lot
of if/elif branches, and feels ugly.
Matt
[code]
import _ast
class SafeEvalError(Exception):
pass
class UnsafeCode(SafeEvalError):
pass
# safe types:
# Sequences:
# list, tuple, dict, set, frozen_set*
# Literals: str, unicode, int, long, complex, float
def safe_eval(text):
"similar to eval, but only works on literals"
ast = compile(text, "", 'exec', _ast.PyCF_ONLY_AST)
return _traverse(ast.body[0].value)
def _traverse(ast):
if isinstance(ast, _ast.List):
return [_traverse(el) for el in ast.elts]
elif isinstance(ast, _ast.Tuple):
return tuple(_traverse(el) for el in ast.elts)
elif isinstance(ast, _ast.Dict):
return dict(
zip(
(_traverse(k) for k in ast.keys),
(_traverse(v) for v in ast.values)
)
)
elif isinstance(ast, _ast.Str):
return ast.s
elif isinstance(ast, _ast.Num):
return ast.n
elif isinstance(ast, _ast.Expr):
return _traverse(ast.value)
elif isinstance(ast, _ast.BinOp):
if isinstance(ast.op, _ast.Add):
return _traverse(ast.left) + _traverse(ast.right)
elif isinstance(ast.op, _ast.Sub):
return _traverse(ast.left) - _traverse(ast.right)
elif isinstance(ast.op, _ast.Div):
return _traverse(ast.left) / _traverse(ast.right)
elif isinstance(ast.op, _ast.FloorDiv):
return _traverse(ast.left) // _traverse(ast.right)
elif isinstance(ast.op, _ast.Mod):
return _traverse(ast.left) % _traverse(ast.right)
elif isinstance(ast.op, _ast.Mult):
return _traverse(ast.left) * _traverse(ast.right)
elif isinstance(ast.op, _ast.Pow):
return _traverse(ast.left) ** _traverse(ast.right)
elif isinstance(ast.op, _ast.BitAnd):
return _traverse(ast.left) & _traverse(ast.right)
elif isinstance(ast.op, _ast.BitOr):
return _traverse(ast.left) | _traverse(ast.right)
elif isinstance(ast.op, _ast.BitXor):
return _traverse(ast.left) ^ _traverse(ast.right)
elif isinstance(ast.op, _ast.LShift):
return _traverse(ast.left) << _traverse(ast.right)
elif isinstance(ast.op, _ast.RShift):
return _traverse(ast.left) >> _traverse(ast.right)
elif isinstance(ast, _ast.BoolOp):
if isinstance(ast.op, _ast.And):
return all(_traverse(v) for v in ast.values)
if isinstance(ast.op, _ast.Or):
return any(_traverse(v) for v in ast.values)
elif isinstance(ast, _ast.UnaryOp):
if isinstance(ast.op, _ast.Invert):
return _traverse(ast.operand)
if isinstance(ast.op, _ast.USub):
return -_traverse(ast.operand)
if isinstance(ast.op, _ast.UAdd):
return +_traverse(ast.operand)
if isinstance(ast.op, _ast.Not):
return not _traverse(ast.operand)
raise UnsafeCode()
if __name__ == "__main__":
print safe_eval("[1,2,3,{'hello':1}, (1,-2,3)], 4j, 1+5j, ~1+2*3")
[/code]
Oh, this is interesting. Similar to some other code I found on the web
which grabs a list of permitted token values using the dis module:
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/286134
I was really hoping for a builtin on this :)
Thanks.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Simple and safe evaluator
Simon Forman wrote: On Jun 11, 1:25 pm, bvdp <[EMAIL PROTECTED]> wrote: Is there a simple/safe expression evaluator I can use in a python program. I just want to pass along a string in the form "1 + 44 / 3" or perhaps "1 + (-4.3*5)" and get a numeric result. I can do this with eval() but I really don't want to subject my users to the problems with that method. In this use I don't need python to worry about complex numbers, variables or anything else. Just do the math on a set of values. Would eval() with some restricted list of permitted operators do the trick? I'm feeling too lazy to write/debug my own parser for this :) Thanks, Bob. Funny, I need exactly the same kind of parser myself right now. Fredrik Lundh has posted some code-and-explanation on an excellent simple parser that's easy to extend. http://effbot.org/zone/simple-iterator-parser.htm Just make it recognize the operator tokens you're interested in and if a string parsers w/o errors then you know it's safe to eval(). I probably won't get to writing this myself for a few days or a week, but if you do will you post it here (or send me a copy)? I'll do the same if I get to it sooner. Regards, ~Simon I'll have to read Fredrik's code a few more times, but I think it makes as much sense as anything else. Of course, I could take the lazy man's way out and just to a left->right evaluation without any ()s, etc., which in my project would work. But, honestly, I thought it'd be easier. I was going to use eval() until I realized that it was not a good idea. Darn shame we have to work so hard to prevent some jerk's malicious code from effecting our stuff. Oh well, that's life. -- http://mail.python.org/mailman/listinfo/python-list
Re: Simple and safe evaluator
I'm finding my quest for a safe eval() quite frustrating :)
Any comments on this: Just forget about getting python to do this and,
instead, grab my set of values (from a user supplied text file) and call
an external program like 'bc' to do the dirty work. I think that this
would avoid someone from embedding os.system("rm ...") in what I thought
would be a math expression and having it maybe do damage? Perhaps I'm
getting too paranoid in my old age.
I guess this would slow things down a bit, but that is not a big
concern. Bigger concern would be that I'm not sure if 'bc' or whatever
is guaranteed to be on other platforms than *nix. And if I want to be
really paranoid, I could worry that someone had planted a bad 'bc' on
the target.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Simple and safe evaluator
Matimus wrote:
The solution I posted should work and is safe. It may not seem very
readable, but it is using Pythons internal parser to parse the passed
in string into an abstract symbol tree (rather than code). Normally
Python would just use the ast internally to create code. Instead I've
written the code to do that. By avoiding anything but simple operators
and literals it is guaranteed safe.
Just wondering ... how safe would:
eval(s, {"__builtins__":None}, {} )
be? From my testing it seems that it parses out numbers properly (int
and float) and does simple math like +, -, **, etc. It doesn't do
functions like int(), sin(), etc ... but that is fine for my puposes.
Just playing a bit, it seems to give the same results as your code using
ast does. I may be missing something!
--
http://mail.python.org/mailman/listinfo/python-list
Re: Simple and safe evaluator
Matimus wrote:
On Jun 11, 9:16 pm, George Sakkis <[EMAIL PROTECTED]> wrote:
On Jun 11, 8:15 pm, bvdp <[EMAIL PROTECTED]> wrote:
Matimus wrote:
The solution I posted should work and is safe. It may not seem very
readable, but it is using Pythons internal parser to parse the passed
in string into an abstract symbol tree (rather than code). Normally
Python would just use the ast internally to create code. Instead I've
written the code to do that. By avoiding anything but simple operators
and literals it is guaranteed safe.
Just wondering ... how safe would:
eval(s, {"__builtins__":None}, {} )
be? From my testing it seems that it parses out numbers properly (int
and float) and does simple math like +, -, **, etc. It doesn't do
functions like int(), sin(), etc ... but that is fine for my puposes.
Just playing a bit, it seems to give the same results as your code using
ast does. I may be missing something!
Probably you do; within a couple of minutes I came up with this:
s = """
... (t for t in 42 .__class__.__base__.__subclasses__()
... if t.__name__ == 'file').next()('/etc/passwd')
... """>>> eval(s, {"__builtins__":None}, {} )
Traceback (most recent call last):
File "", line 1, in
File "", line 3, in
IOError: file() constructor not accessible in restricted mode
Not an exploit yet but I wouldn't be surprised if there is one. Unless
you fully trust your users, an ast-based approach is your best bet.
George
You can get access to any new-style class that has been loaded. This
exploit works on my machine (Windows XP).
[code]
# This assumes that ctypes was loaded, but keep in mind any classes
# that have been loaded are potentially accessible.
import ctypes
s = """
(
t for t in 42 .__class__.__base__.__subclasses__()
if t.__name__ == 'LibraryLoader'
).next()(
(
t for t in 42 .__class__.__base__.__subclasses__()
if t.__name__ == 'CDLL'
).next()
).msvcrt.system('dir') # replace 'dir' with something nasty
"""
eval(s, {"__builtins__":None}, {})
[/code]
Matt
Yes, this is probably a good point. But, I don't see this as an exploit
in my program. Again, I could be wrong ... certainly not the first time
that has happened :)
In my case, the only way a user can use eval() is via my own parsing
which restricts this to a limited usage. So, the code setting up the
eval() exploit has to be entered via the "safe" eval to start with. So,
IF the code you present can be installed from within my program's
scripts ... then yes there can be a problem. But for the life of me I
don't see how this is possible. In my program we're just looking at
single lines in a script and doing commands based on the text.
Setting/evaluating macros is one "command" and I just want a method to
do something like "Set X 25 * 2" and passing the "25 * 2" string to
python works. If the user creates a script with "Set X os.system('rm
*')" and I used a clean eval() then we could have a meltdown ... but if
we stick with the eval(s, {"__builtins__":None}, {}) I don't see how the
malicious script could do the class modifications you suggest.
I suppose that someone could modify my program code and then cause my
eval() to fail (be unsafe). But, if we count on program modifications to
be doorways to exploits then we might as well just pull the plug.
Bob.
--
http://mail.python.org/mailman/listinfo/python-list
Re: howto split string with both comma and semicolon delimiters
dmitrey wrote:
hi all,
howto split string with both comma and semicolon delimiters?
i.e. (for example) get ['a','b','c'] from string "a,b;c"
I have tried s.split(',;') but it don't work
Thx, D.
Howabout:
s = s.replace(";", ",")
s = s.split(",")
--
http://mail.python.org/mailman/listinfo/python-list
Re: Simple and safe evaluator
George Sakkis wrote: You probably missed the point in the posted examples. A malicious user doesn't need to modify your program code to have access to far more than you would hope, just devise an appropriate string s and pass it to your "safe" eval. Oppps, I did miss the point. I was assuming that the modifying stuff was being done before the call to the eval(). I was wrong. I'll have to get the ast based code incorporated into my code and just use it. Darn, but it seems that each and every time one sees a simple solution to a simple problem ... :) Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: Simple and safe evaluator
Okay guys. I have the _ast based safe eval installed and working in my
program. It appears to be working just fine. Thanks for the help.
Now, a few more questions:
1. I see that _ast is a 2.5 module?? So, for folks using my code with
<2.5 I could do something like this:
# I've got some imports here to look after the error() and warning()
funcs
emsg_done = 0
etx = ""
def unsafe_eval(s):
""" safe eval for < python 2.5 (lacks _ast) """
global emsg_done
if not emsg_done:
warning("You are using an unsafe eval() function. Please
upgrade to Python version 2.5 or greater.")
emsg_done=1
# need error trap here as well ...
return eval(s, {"__builtins__":None}, {} )
def safe_eval(text):
"similar to eval, but only works on numerical values."
global etx
try:
ast = compile(text, "", 'eval', _ast.PyCF_ONLY_AST)
except:
error("Expression error in '%s'" % text)
etx = text # for error reporting, bvdp
return _traverse(ast.body)
try:
import _ast
num_eval = safe_eval
except:
num_eval = unsafe_eval
# rest of matt's ast code follows.
Which appears to do the following: if there isn't an _ast module we just
define an alternate, not-so-safe, function and warn the user; otherwise
we use the safe version. I'm a bit uncomfortable with the import _ast
being after the function which uses the code, but it seems to work.
2. I thought I'd be happy with * / + -, etc. Of course now I want to add
a few more funcs like int() and sin(). How would I do that?
Thanks. This is looking very nice indeed.
Bob.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Simple and safe evaluator
George Sakkis wrote:
On Jun 16, 4:47 pm, bvdp <[EMAIL PROTECTED]> wrote:
2. I thought I'd be happy with * / + -, etc. Of course now I want to add
a few more funcs like int() and sin(). How would I do that?
For the builtin eval, just populate the globals dict with the names
you want to make available:
import math
globs = {'__builtins__' : None}
# expose selected builtins
for name in 'True False int float round abs divmod'.split():
globs[name] = eval(name)
# expose selected math constants and functions
for name in 'e pi sqrt exp log ceil floor sin cos tan'.split():
globs[name] = getattr(math,name)
return eval(s, globs, {})
Thanks. That was easy :)
The change to the _ast version is left as an exercise to the reader ;)
And I have absolutely no idea on how to do this. I can't even find the
_ast import file on my system. I'm assuming that the _ast definitions
are buried in the C part of python, but that is just a silly guess.
Bob.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Simple and safe evaluator
[EMAIL PROTECTED] wrote: On Jun 17, 8:02 am, bvdp <[EMAIL PROTECTED]> wrote: Thanks. That was easy :) The change to the _ast version is left as an exercise to the reader ;) And I have absolutely no idea on how to do this. I can't even find the _ast import file on my system. I'm assuming that the _ast definitions are buried in the C part of python, but that is just a silly guess. Bob. If you just need numeric expressions with a small number of functions, I would suggest checking the expression string first with a simple regular expression, then using the standard eval() to evaluate the result. This blocks the attacks mentioned above, and is simple to implement. This will not work if you want to allow string values in expressions though. import re def safe_eval( expr, safe_cmds=[] ): toks = re.split( r'([a-zA-Z_\.]+|.)', expr ) bad = [t for t in toks if len(t)>1 and t not in safe_cmds] if not bad: return eval( expr ) Yes, this appears to be about as good (better?) an idea as any. Certainly beats writing my own recursive decent parser for this :) And it is not dependent on python versions. Cool. I've run a few tests with your code and it appears to work just fine. Just a matter of populating the save_cmds[] array and putting in some error traps. Piece of cake. And should be fast as well. Thanks!!! Bob. -- http://mail.python.org/mailman/listinfo/python-list
Re: Simple and safe evaluator
Aahz wrote:
In article <[EMAIL PROTECTED]>,
Simon Forman <[EMAIL PROTECTED]> wrote:
FWIW, I got around to implementing a function that checks if a string
is safe to evaluate (that it consists only of numbers, operators, and
"(" and ")"). Here it is. :)
What's safe about "1000 ** 1000"?
Guess it depends on your definition of safe. I think that in most cases
folks looking for "safe" are concerned about a malicious interjection of
a command like "rm *" ... your example hangs the system for a long time
and eventually will error out when it runs out of memory, but (probably)
doesn't cause data corruption.
It would be nice if in a future version of Python we could have a
safe/limited eval() ... which would limit the resources.
--
http://mail.python.org/mailman/listinfo/python-list
Function to import module to namespace
Is it possible to do this from a function: import a module and append the defs in that module to an existing module/namesapce. So, in my code I have something like: # main code import mods def loadmore(n): import_module(n, mods) # end of main this will permit the addition of the the stuff in file 'n.py' to 'mods'. Assuming that foo1() is defined in newmod, I should now be able to do something like mods.foo1(). Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: Function to import module to namespace
Terry Reedy wrote: bvdp wrote: Is it possible to do this from a function: import a module and append the defs in that module to an existing module/namesapce. So, in my code I have something like: # main code import mods def loadmore(n): import_module(n, mods) # end of main this will permit the addition of the the stuff in file 'n.py' to 'mods'. Assuming that foo1() is defined in newmod, I should now be able to do something like mods.foo1(). Do you mean something like this? >>> import string >>> dir(string) ['Formatter', 'Template', '_TemplateMetaclass', '__builtins__', '__doc__', '__file__', '__name__', '__package__', '_multimap', '_re', 'ascii_letters', 'ascii_lowercase', 'ascii_uppercase', 'capwords', 'digits', 'hexdigits', 'maketrans', 'octdigits', 'printable', 'punctuation', 'whitespace'] >>> import math >>> math.__dict__.update(string.__dict__) >>> dir(math) ['Formatter', 'Template', '_TemplateMetaclass', '__builtins__', '__doc__', '__file__', '__name__', '__package__', '_multimap', '_re', 'acos', 'acosh', 'ascii_letters', 'ascii_lowercase', 'ascii_uppercase', 'asin', 'asinh', 'atan', 'atan2', 'atanh', 'capwords', 'ceil', 'copysign', 'cos', 'cosh', 'degrees', 'digits', 'e', 'exp', 'fabs', 'factorial', 'floor', 'fmod', 'frexp', 'hexdigits', 'hypot', 'isinf', 'isnan', 'ldexp', 'log', 'log10', 'log1p', 'maketrans', 'modf', 'octdigits', 'pi', 'pow', 'printable', 'punctuation', 'radians', 'sin', 'sinh', 'sqrt', 'sum', 'tan', 'tanh', 'trunc', 'whitespace'] tjr Yes, I think that's what I might need. I'll give it a go in my code and see if that does work. Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: Function to import module to namespace
Terry Reedy wrote:
>
>
> Do you mean something like this?
> >>> math.__dict__.update(string.__dict__)
> >>> dir(math)
> ['Formatter', 'Template', '_TemplateMetaclass', '__builtins__',
I think this is working First off, 2 module files:
funcs.py
def func1():
print "I'm func1 in funcs.py"
more.py
def func2():
print "I'm func2 in 'more.py'"
and my wonderful main program:
xx.py
import funcs
def addnewfuncs(p):
x = __import__(p)
funcs.__dict__.update(x.__dict__)
funcs.func1()
addnewfuncs('more')
funcs.func2()
The first problem I had was getting import to accept a variable. It
doesn't seem to, so I used __import__(). Then, I had to remember to
assign this to a variable ... and then it appears to work just fine.
Did I miss anything in this???
Thanks for the pointer!
Bob.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Function to import module to namespace
John Machin wrote:
Good questions. Short answer ... probably 'cause I've not thought the
problem though completely :)
> You are updating with *everything* in the 'more' module, not just the
> functions. This includes such things as __name__, __doc__, __file__.
> Could have interesting side-effects.
>
> One quick silly question: why do you want to do this anyway?
>
I'm writing a "simple" macro expander. I've got some mainline code which
reads an input file, parses out macros, and expands them. So, in my
input file I might have something like:
a fjas j kj sdklj sdkl jfsdkl [ link somewhere sometext ]
So, I need a function link() to evaluate and return "http://";
Instead of putting funcs like link() in my mainline I've stuck them in a
module of their own. In the mainline I
import funcs
and then when I need to expand I have the following code:
if not cmd in vars(funcs):
error("Unknown function/variable '%s'" % cmd)
if type(vars(funcs)[cmd]) == type(parse):
txt = eval("""funcs.%s("%s")""" % (cmd, arg))
else: # not a func, just expand the variable
if arg:
error("Argument to variable '%s' not permitted." % cmd)
txt = str(eval("funcs.%s" % cmd ))
Of course, the question comes up ... what if a user (probably me) wants
to add more functions? Easy enough to just edit funcs.py I suppose, but
I thought it'd be nice to use more modules.
So, I suppose that rather than adding the 2ndary module stuff to the
default 'funcs.py' I could just as well have a look and check for the
needed function in all the modules I've imported.
Sorry, *two* quick silly questions: are the add-on modules under your
control, or do you want to be able to do this with arbitrary modules?
[If under your control, you could insist that such modules had an
__all__ attribute with appropriate contents]
Why would I want to do that ... and how?
A third: why do you want to import into an existing namespace? Now
that you know about __import__, why just not call the functions where
they are?
Yeah, that would probably be cleaner (safer).
Thanks.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Function to import module to namespace
John Machin wrote:
On Jun 30, 11:45 am, bvdp <[EMAIL PROTECTED]> wrote:
John Machin wrote:
Good questions. Short answer ... probably 'cause I've not thought the
problem though completely :)
> You are updating with *everything* in the 'more' module, not just the
> functions. This includes such things as __name__, __doc__, __file__.
> Could have interesting side-effects.
>
> One quick silly question: why do you want to do this anyway?
>
I'm writing a "simple" macro expander. I've got some mainline code which
reads an input file, parses out macros, and expands them. So, in my
input file I might have something like:
a fjas j kj sdklj sdkl jfsdkl [ link somewhere sometext ]
So, I need a function link() to evaluate and return "http://";
Instead of putting funcs like link() in my mainline I've stuck them in a
module of their own.
Why? Do they have any use at all other than to be called by your
mainline?
In the mainline I
import funcs
and then when I need to expand I have the following code:
Build a dictionary *ONCE*, not each time you want to look in it. E.g.
funcs_vars = vars(funcs)
... much later:
if not cmd in funcs_vars:
if not cmd in vars(funcs):
error("Unknown function/variable '%s'" % cmd)
if type(vars(funcs)[cmd]) == type(parse):
txt = eval("""funcs.%s("%s")""" % (cmd, arg))
Any particular reason why you are going to the trouble of laboriously
building a statement, compiling it, and running it, when you could
simply do:
txt = funcs_vars[cmd](arg)
?
else: # not a func, just expand the variable
if arg:
error("Argument to variable '%s' not permitted." % cmd)
txt = str(eval("funcs.%s" % cmd ))
txt = str(funcs_vars[cmd])
Of course, the question comes up ... what if a user (probably me) wants
to add more functions? Easy enough to just edit funcs.py I suppose, but
I thought it'd be nice to use more modules.
You may already have one more module than you need. If the only thing
different about the extra functions is that they are an afterthought,
then put them in the same module as the others.
So, I suppose that rather than adding the 2ndary module stuff to the
default 'funcs.py' I could just as well have a look and check for the
needed function in all the modules I've imported.
Sounds quite unnecessary, inefficient and pointless to me.
Sorry, *two* quick silly questions: are the add-on modules under your
control, or do you want to be able to do this with arbitrary modules?
[If under your control, you could insist that such modules had an
__all__ attribute with appropriate contents]
Why would I want to do that ... and how?
Why: so that you append only the functions that you want to the target
namespace, and don't import stuff like __name__. How: read about
__all__. Just append the stuff that's listed in __all__. However it
sounds like you have seen the light and won't be doing anything like
appending to a namespace.
A third: why do you want to import into an existing namespace? Now
that you know about __import__, why just not call the functions where
they are?
Yeah, that would probably be cleaner (safer).
Much cleaner, safer, etc is just to import, or even not to have a
separate module at all.
HTH,
John
Thanks. Much food for thought here. Let me digest and see what comes of
it all.
--
http://mail.python.org/mailman/listinfo/python-list
Pop return from stack?
Assuming I have a module 'foo.py' with something like this: def error(s): print "Error", s sys.exit(1) def func(s): ... do some processing ... call error() if bad .. go to system exit. ... more processing and then I write a new program, test.py, which: import foo def myerror(s): print "new error message" foo.error = myerror a = foo.func(..) Now, if an error is encountered myerror() is called. Fine. But execution resumes in func(). Not exactly what I wanted. I can "fix" this simply by wrapping the call to foo.func() in a try/ expect and have myerror() raise an exception. This appears to work, but I'm hesitant to use this out of fear that I'm building up some kind of stack overflow or something which will bite me later. Is there a better way? Simplest for an old assembler guy like me would be pop a return address off the stack ... but python isn't assembler :) I don't want to change stuff in the foo.py module since it's part of an existing program. But, if I must, I suppose I could. I'd prefer to just short-circuit this if possible. Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: Pop return from stack?
> An exception will walk up the stack, calling any cleaning-up code that needs > to be done (removing object references, executing finally: blocks, exiting > context managers properly. It won't break anything. Don't be afraid of > Python's high-level features! Okay, I believe you (and the rest of the gang. In my trivial program the exception in working ... so I'll leave it alone. Thanks. > - Thomas -- http://mail.python.org/mailman/listinfo/python-list
Re: Pop return from stack?
On Aug 14, 5:23 pm, Steven D'Aprano wrote: > This general technique is called "monkey patching". > New term for me :) > > Now, if an error is encountered myerror() is called. Fine. But execution > > resumes in func(). Not exactly what I wanted. > > Of course it does. Your new error handler fails to exit, so execution > resumes like it does after any other function. I guess I wasn't being clear. I don't want to exit in my new bit of code. Just continue a loop (which I didn't show in the example). > Exceptions are the standard way of doing things. That's what sys.exit() > does -- it raises SystemExit exception. Okay, didn't know that exit() was really an exception. Good to know. But, like I said, I'm not looking to exit. > > With very few exceptions, if you're writing your own error handlers like > this, you're doing it wrong. Your error handler throws away useful > debugging information, and it gives you no extra information that a > standard Python traceback couldn't give. Yeah, but I really don't want a traceback printed out for a user just because a file can't be found, or he's got a bad bit of syntax in his file. So, that's why I have the specific error routine. Works fine in the main program. > Oh my ... I've seen people writing Java in Python, C++ in Python, Perl in > Python, even VB in Python, but this is the first time I've meet some one > who wants to write assembler in Python :) Naw, I had my fun with assembler in the good old days. Never want to write another line of it :) -- http://mail.python.org/mailman/listinfo/python-list
Re: Pop return from stack?
On Aug 15, 12:52 pm, John Nagle wrote: > On 8/14/2010 4:05 PM, bvdp wrote: > > > Assuming I have a module 'foo.py' with something like this: > > > def error(s): > > print "Error", s > > sys.exit(1) > > > def func(s): > > ... do some processing > > ... call error() if bad .. go to system exit. > > ... more processing > > Fix "func". That's terrible Python. No standard Python library > module calls system exit to handle an error. So that must be in > your code. Standard procedure for errors is to raise an > exception. Not to belabor the point .. but "func" is not a standard lib module. It's part of a much larger application ... and in that application it makes perfect sense to terminate the application if it encounters an error. I fail to see the problem with this. Why would an APPLICATION raise a error or not exit to the system? Does it help to note that error() as defined in the application prints out a helpful message, etc? The whole problem I was having is that I was trying to tie a small application (an helper to the main application) to use a bit of the existing code as a pseudo-library. Certainly, if the code I was interfacing with was a standar Python module ... well, then this thread would not exist in the first place. However, I have gotten hit with more than one comment like yours. So, could you please clarify? Is it bad form to exit an application with sys.exit(1) when an error in a file the application is processing is found? Honestly, I'm not trying to be argumentative ... just trying to understand. Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: Pop return from stack?
On Aug 20, 6:41 pm, Gregory Ewing wrote: > bvdp wrote: > > The whole problem I was having is that I was trying to tie a small > > application (an helper to the main application) to use a bit of the > > existing code as a pseudo-library. > > This is precisely the reason that it's a bad idea to > directly terminate the program from somewhere deep inside > the code. It makes it hard to re-use the code in another > context. > > It's much better to raise an exception containing an > appropriate error message, catch it at the top level > of the application and print the message and exit there. > Then you can easily re-use any of the code in a context > where it's not appropriate to have it exit out from > under you. > > -- > Greg Thanks Greg. That makes a lot of sense ... for the next program I write :) -- http://mail.python.org/mailman/listinfo/python-list
Creating slice notation from string
I'm trying to NOT create a parser to do this and I'm sure that it's easy if I could only see the light! Is it possible to take an arbitrary string in the form "1:2", "1", ":-1", etc. and feed it to slice() and then apply the result to an existing list? For example, I have a normal python list. Let's say that x = [1,2,3,4] and I have a string, call it "s', in the format "[2:3]". All I need to do is to apply "s" to "x" just like python would do. I can, of course, convert "x" to a list with split(), convert the 2 and 3 to ints, and then do something like: x[a:b] ... but I'd like something more general. I think the answer is in slice() but I'm lost. Thanks. -- http://mail.python.org/mailman/listinfo/python-list
