numpy - 2D matrix/array - initialization like in Matlab...
See this:
==
In [5]: Dx = numpy.matrix('1 0 0; 0 0.5 -0.5; 0 -0.5 1.5')
In [6]: Dx
Out[6]:
matrix([[ 1. , 0. , 0. ],
[ 0. , 0.5, -0.5],
[ 0. , -0.5, 1.5]])
==
Ok... So now test = 33 and instead of the value 1.5 I want to use the
value of "test" and put it directly into the matrix (or array):
==
In [7]: test=33
In [8]: Dx = numpy.matrix('1 0 0; 0 0.5 -0.5; 0 -0.5 test')
---
NameError Traceback (most recent call last)
/home/user/something/ in ()
> 1 Dx = numpy.matrix('1 0 0; 0 0.5 -0.5; 0 -0.5 test')
/usr/lib/python2.7/dist-packages/numpy/matrixlib/defmatrix.pyc in
__new__(subtype, data, dtype, copy)
252
253 if isinstance(data, str):
--> 254 data = _convert_from_string(data)
255
256 # now convert data to an array
.. etc...
==
So obviously it doesn't understand that I want this:
==
In [21]: Dx[2,2]=test
In [22]: Dx
Out[22]:
matrix([[ 1. , 0. , 0. ],
[ 0. , 33. , -0.5],
[ 0. , -0.5, 33. ]])
==
Without having to manually change all the individual places using my
variables (test is actually many variables, not just one but I think you
should understand the problem now).
How to initialize my array directly using variables ?
It could also be that I wanted:
test11 = 1
test12 = 1.5
test13 = 2
test21 = 0
test22 = 5
Dx = numpy.matrix('test11 test12 test13; test21 test22 -0.5; 0 -0.5 1.5')
Etc... for many variables...
Appreciate ANY help, thank you very much!
--
http://mail.python.org/mailman/listinfo/python-list
Re: numpy - 2D matrix/array - initialization like in Matlab...
On 10/15/2012 11:26 PM, MRAB wrote:
On 2012-10-15 22:09, someone wrote:
See this:
==
In [5]: Dx = numpy.matrix('1 0 0; 0 0.5 -0.5; 0 -0.5 1.5')
In [6]: Dx
Out[6]:
matrix([[ 1. , 0. , 0. ],
[ 0. , 0.5, -0.5],
[ 0. , -0.5, 1.5]])
==
Ok... So now test = 33 and instead of the value 1.5 I want to use the
value of "test" and put it directly into the matrix (or array):
==
In [7]: test=33
In [8]: Dx = numpy.matrix('1 0 0; 0 0.5 -0.5; 0 -0.5 test')
---
NameError Traceback (most recent call
last)
/home/user/something/ in ()
> 1 Dx = numpy.matrix('1 0 0; 0 0.5 -0.5; 0 -0.5 test')
/usr/lib/python2.7/dist-packages/numpy/matrixlib/defmatrix.pyc in
__new__(subtype, data, dtype, copy)
252
253 if isinstance(data, str):
--> 254 data = _convert_from_string(data)
255
256 # now convert data to an array
.. etc...
==
So obviously it doesn't understand that I want this:
==
In [21]: Dx[2,2]=test
In [22]: Dx
Out[22]:
matrix([[ 1. , 0. , 0. ],
[ 0. , 33. , -0.5],
[ 0. , -0.5, 33. ]])
==
Without having to manually change all the individual places using my
variables (test is actually many variables, not just one but I think you
should understand the problem now).
How to initialize my array directly using variables ?
It could also be that I wanted:
test11 = 1
test12 = 1.5
test13 = 2
test21 = 0
test22 = 5
Dx = numpy.matrix('test11 test12 test13; test21 test22 -0.5; 0 -0.5 1.5')
Etc... for many variables...
Appreciate ANY help, thank you very much!
What it prints should give you a hint:
>>> Dx = numpy.matrix([[test11, test12, test13], [test21, test22,
-0.5], [0, -0.5, 1.5]])
>>> Dx
matrix([[ 1. , 1.5, 2. ],
[ 0. , 5. , -0.5],
[ 0. , -0.5, 1.5]])
Uh, great - thank you very much!
As you maybe see, I'm only a python newbie so I'm not so good at
understanding the error messages and reading the source code yet.
Thank you very much for the solution to the problem! It's highly
appreciated. Thanks.
--
http://mail.python.org/mailman/listinfo/python-list
pygame - importing GL - very bad...
See this code (understand why I commented out first line): # from OpenGL.GL import * from OpenGL.GL import glEnable, GL_DEPTH_TEST, \ glShadeModel, GL_SMOOTH, glClearColor, \ GL_CULL_FACE, GL_BLEND, glBlendFunc, \ GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA, \ glClear, GL_COLOR_BUFFER_BIT, GL_DEPTH_BUFFER_BIT, \ glLoadIdentity, glTranslate, glRotate, \ glMultMatrixf, glPushMatrix, glCallList, \ glPopMatrix, glDisable, GL_LIGHTING The reason why I commented out the first line is that I use "pylint" and it reports: "[W] Redefining built-in 'format'" for this line. From: http://www.logilab.org/card/pylintfeatures tell: W0621: Redefining name %r from outer scope (line %s) Used when a variable's name hide a name defined in the outer scope. I don't like to redefine already defined names so therefore I had to outcomment first line and then keep on adding stuff until I could run my program... But this SUCKS! I can see that pygame hasn't been updated for a long while - not many users use it? I'm not very happy about this... Any good / clever solution to this problem, so I avoid this nasty crappy work-around? Any ideas / suggestions ? Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: pygame - importing GL - very bad...
On 01/01/2013 12:13 PM, Chris Angelico wrote: > On Tue, Jan 1, 2013 at 10:00 PM, someone wrote: >> See this code (understand why I commented out first line): >> >> # from OpenGL.GL import * >> from OpenGL.GL import glEnable, GL_DEPTH_TEST, \ >> glShadeModel, GL_SMOOTH, glClearColor, \ >> GL_CULL_FACE, GL_BLEND, glBlendFunc, \ >> GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA, \ >> glClear, GL_COLOR_BUFFER_BIT, GL_DEPTH_BUFFER_BIT, \ >> glLoadIdentity, glTranslate, glRotate, \ >> glMultMatrixf, glPushMatrix, glCallList, \ >> glPopMatrix, glDisable, GL_LIGHTING >> >> Any good / clever solution to this problem, so I avoid this nasty crappy >> work-around? > > You could simply > > import OpenGL.GL as GL > > and then use all those names as GL.glPopMatrix, GL.GL_LIGHTING, etc. I > don't know if that's better or worse. You're right - but I forgot to write that even though this maybe should/is recommended many places then I've seen a lot of opengl code on the internet and IMHO NOBODY does that and it'll be a lot slower to type that in front of all the opengl commands... So this solution is not something I like too... But I can see some other people came up with good solutions, which I didn't knew about.. Thank you. -- http://mail.python.org/mailman/listinfo/python-list
Re: pygame - importing GL - very bad...
On 01/01/2013 12:49 PM, Steven D'Aprano wrote: > On Tue, 01 Jan 2013 12:00:32 +0100, someone wrote: > >> See this code (understand why I commented out first line): >> >> # from OpenGL.GL import * > [...] >> The reason why I commented out the first line is that I use "pylint" and >> it reports: "[W] Redefining built-in 'format'" for this line. >> >> From: http://www.logilab.org/card/pylintfeatures tell: W0621: Redefining >> name %r from outer scope (line %s) Used when a variable's name hide a >> name defined in the outer scope. >> >> I don't like to redefine already defined names so therefore I had to >> outcomment first line and then keep on adding stuff until I could run my >> program... But this SUCKS! I can see that pygame hasn't been updated for >> a long while - not many users use it? I'm not very happy about this... > > from pygame import * > del format Are you sure about this? Because I'm not (OTOH I'm maybe not as experienced in python as some of you)... Ipython log: In [6]: test=format(43) In [7]: type(test) Out[7]: str In [8]: from pygame import * In [9]: test=format(43) In [10]: type(test) Out[10]: str In [11]: del format --- NameError Traceback (most recent call last) in () > 1 del format NameError: name 'format' is not defined What does this mean? Why does it say 'format" cannot be deleted after I did the wildcard import ? > pylint may still complain, but you can ignore it. By deleting the name > "format", that will unshadow the builtin format. Are you sure? -- http://mail.python.org/mailman/listinfo/python-list
Re: pygame - importing GL - very bad...
On 01/01/2013 01:56 PM, Peter Otten wrote: > someone wrote: > It turns out pylint is lying. The situation is equivalent to > > $ cat module.py > for format in [42]: > pass > del format > > $ cat main.py > original_format = format > from module import * > assert format is original_format > $ python main.py > > The assert doesn't trigger, so format is not redefined. But pylint complains > anyway: > > $ pylint main.py -rn > No config file found, using default configuration > * Module main > W: 2: Redefining built-in 'format' > C: 1: Missing docstring > C: 1: Invalid name "original_format" (should match (([A-Z_][A-Z0-9_]*)| > (__.*__))$) > W: 2: Wildcard import module > > If you can ignore the warning about the wildcard import you should be able In the case of opengl import, I'll ignore the wildcard import warning. But in other cases I'll likely look a bit into it and see if I can avoid it or at least change it to something like: "import OpenGL.GL as GL" etc. > to ignore the "redefining built-in" warning, too. Personally I would avoid > putting magic comments like > > from module import * # pylint: disable=W0622 Oh, I just learned something new now... How come I cannot type "#pylint: enable=W0622" in the line just below the import ? Not so intuitively/logically for me... > $ pylint main.py -rn > No config file found, using default configuration > * Module main > I: 2: Locally disabling W0622 > C: 1: Missing docstring > C: 1: Invalid name "original_format" (should match (([A-Z_][A-Z0-9_]*)| > (__.*__))$) > W: 2: Wildcard import module > > into the code. Thank you very much... Another thing is that I don't understand this warning: Invalid name "original_format" (should match (([A-Z_][A-Z0-9_]*)| > (__.*__))$) I get it everywhere... I don't understand how it wants me to label my variables... Maybe I should disable this warning to get rid of it... -- http://mail.python.org/mailman/listinfo/python-list
Re: pygame - importing GL - very bad...
On 01/01/2013 11:39 PM, alex23 wrote: On Jan 1, 9:00 pm, someone wrote: I can see that pygame hasn't been updated for a long while - not many users use it? It helps if you look in the right place: pygame "Last Updated 2012-12-29": https://bitbucket.org/pygame/pygame/src pygame2: http://code.google.com/p/pgreloaded/ Maybe... But if you look for stable releases here: http://www.pygame.org/download.shtml You'll find the top option: 1.9.1 Packages (August 6th 2009) And then previous releases is just below 1.9.1: pygame-1.9.0release.tar.gz ~ 1.4M - August 1, 2009 pygame-1.8.1release.tar.gz ~ 1.4M - July 30, 2008 pygame-1.8.0release.tar.gz ~ 1.4M - March 29, 2008 pygame-1.7.1release.tar.gz ~ 1.3M - August 16, 2005 1.7.0 ~ no source release was made. pygame-1.6.2.tar.bz2 ~ 1140 kb - pygame-1.6.tar.gz ~ 832 kb - October 23, 2003 pygame-1.5.tar.gz ~ 736 kb - May 30, 2002 pygame-1.4.tar.gz ~ 808 kb - Jan 30, 2002 pygame-1.3.tar.gz ~ 731 kb - Dec 19, 2001 pygame-1.2.tar.gz ~ 708 kb - Sep 4, 2001 pygame-1.1.tar.gz ~ 644 kb - Jun 23, 2001 pygame-1.0.tar.gz ~ 564 kb - Apr 5, 2001 pygame-0.9.tar.gz ~ 452 kb - Feb 13, 2001 pygame-0.5.tar.gz ~ 436 kb - Jan 6 14, 2001 pygame-0.4.tar.gz ~ 420 kb - Dec 14, 2000 pygame-0.3b.tar.gz ~ 367 kb - Nov 20, 2000 pygame-0.2b.tar.gz ~ 408 kb - Nov 3, 2000 pygame-0.1a.tar.gz ~ 300 kb - Oct 28, 2000 Back to year 2000... Maybe they should get a grip on themselves and distribute a new stable releases in year 2013 - then it would at least SEEM to look as the project is not dead. But in any case, I'm happy with it - haven't experienced any big issues with pygame yet, so don't take this as I don't value what they do. Maybe they've made a great version back in 2009 and it's so good that there wasn't any need for a newer stable version before 2013. But it gives the impression that nothing happens, when so many years pass on... Anyway, thanks a lot to all... (And sorry I accidentally replied privately to some of you - in thunderbird I should hit the "followup"-button but maybe they've removed it and instead I keep on hitting "reply" - very confusing that the first button in thunderbird is reply instead of followup, which is what I always prefer to use (so other people can see the answers). Thanks you for pointing out that (at least) something did happen on 2012-12-29, when it looks a bit dead on the official homepage. -- http://mail.python.org/mailman/listinfo/python-list
Re: pygame - importing GL - very bad...
On 01/02/2013 04:01 AM, Nobody wrote: On Wed, 02 Jan 2013 00:49:36 +0100, someone wrote: In [11]: del format --- NameError Traceback (most recent call last) in () > 1 del format NameError: name 'format' is not defined What does this mean? Why does it say 'format" cannot be deleted after I did the wildcard import ? You can't delete built-in names. Ah, ok - and cannot overwrite it too, I guess... A shame that pylint didn't knew about this. It has nothing to do with the wildcard import. The PyOpenGL modules delete "format" from the module's variables as soon as they are finished with it, so the set of names created by the wildcard import doesn't include "format". Ok, sounds to me like I can safely ignore this pylint warning in any case... Thanks! -- http://mail.python.org/mailman/listinfo/python-list
Re: how to solve complex equation?
On 01/01/2013 10:49 PM, Jens Thoms Toerring wrote: Usama Khan wrote: how to solve complex equation in pyhton? and then use it to make a program. . i have created new post as my last post is i guessed ranked as a cheater. .:( i know very litle about python as well as programing. . which equation am taliking u will be thinking. . i am giving u the link kindly c that equation. . n kindly let me know the way. . https://groups.google.com/forum/?fromgroups=#!topic/comp.lang.python/cxG7DLxXgmo First of all, the equation given there is unusable (even the number of parentheses doesn't match up). Propbably it's meant to be the one someone else posted a link to: http://classes.engr.oregonstate.edu/cce/winter2012/ce492/Modules/06_structural_design/06-3_body.htm This thingy can't be solved on paper so you need some iterative algorithm to find the solution. So waht you do is modify the equa- tion so that you have 0 on one side and then consider the other side to be a function of SN+1. Now the problem you're left with is to find the value(s) of SN+1 (and thus of SN) where the func- tion has a zero-crossing. A commonly use algorithms for finding zero-crossings is Newton's method. You can find lots of sites on the internet describing it in all neccessary detail. It boils down to start with some guess for the result and then calculate the next, better approximation via xn+1 = xn - f(xn) / f'(xn) where f(xn)n) is the value of the function at point xn and f'(xn) the value of the derivative of f (with respect to x) also at xn. You repeat the process until the difference be- tween xn an the next, better approximation, xn+1, has become as small as you need it. So it's very simple to implement and the ugliest bit is pro- bably calculating the required derivative of the function with respect to SN+1 (wbich you can take to be x). Exactly. I think a 5th semester engineering student should know this. If not, it's not the python-skills that is the problem. It's the (lack of) mathematical insight. I think maybe the OP should not demand people here to "show the solution", but begin with something much simpler and then as he becomes better and better with python, he can move towards more programmatically advanced code. The basic idea about an iterating loop, defining an objective function etc is not really advanced python. But I think the OP should start out with something more simple than what he tries to do here. Just starting out by iteratingly finding the solution to x^2-9 = 0 is a great starting place. After having studied, googled python examples on the internet, I think most people should be able to solve x^2-9=0 in a day for a guy who knows about programming in an arbitrary programming language other than python. It looks like laziness, when we don't see any attempts to show what has been tried to do, from the OP's point of view. The best way to get good feedback (IMHO) is to post some code and then post any errors/warnings that python is giving back. Don't just write: "Give me the code for doing this". -- http://mail.python.org/mailman/listinfo/python-list
Re: pygame - importing GL - very bad...
On 01/02/2013 10:52 AM, alex23 wrote: On Jan 2, 1:01 pm, Nobody wrote: You can't delete built-in names. Actually, you can. If you ever need to shoot yourself in the foot in this particular way, you can always do: del __builtins__.format Not saying you _should_, just that you _can_ :) Ok, good to know (not saying I would ever try it) :-) -- http://mail.python.org/mailman/listinfo/python-list
Re: pygame - importing GL - very bad...
On 01/02/2013 08:39 AM, Steven D'Aprano wrote: On Wed, 02 Jan 2013 00:49:36 +0100, someone wrote: What does this mean? Why does it say 'format" cannot be deleted after I did the wildcard import ? It means that there is no "format" in the current scope, which implies that pygame no longer has a "format" which can be imported. You don't need an import to shadow built-ins. See for example: py> format py> format = "NOBODY expects the Spanish Inquisition!" py> format # this shadows the built-in "format" 'NOBODY expects the Spanish Inquisition!' py> del format # get rid of the Spanish Inquisition py> format py> del format Traceback (most recent call last): File "", line 1, in NameError: name 'format' is not defined Ok, thank you very much - that was/is very illustrative... When a name is not discovered in the current scope, the builtin scope is checked before Python gives up and reports a NameError. But del only works on the current scope, to stop you from accidentally deleting the wrong object. Ok, I'll remember that in the future, thank you. > pylint may still complain, but you can ignore it. By deleting the > name "format", that will unshadow the builtin format. Are you sure? Since it turns out that pylint was actually wrong to complain, no format was actually imported, then yes you can safely ignore it :-) Ok, I can see from your example that you're right. Nice to know the real explanation, thank you very much. :-) -- http://mail.python.org/mailman/listinfo/python-list
Re: pylint, was Re: pygame - importing GL - very bad...
On 01/02/2013 01:07 PM, Peter Otten wrote: someone wrote: On 01/01/2013 01:56 PM, Peter Otten wrote: from module import * # pylint: disable=W0622 Oh, I just learned something new now... How come I cannot type "#pylint: enable=W0622" in the line just below the import ? With what intended effect? If I have a section with A LOT OF warnings and I don't want those in that section to show up ? Isn't that valid enough? Another thing is that I don't understand this warning: Invalid name "original_format" (should match (([A-Z_][A-Z0-9_]*)| > (__.*__))$) I get it everywhere... I don't understand how it wants me to label my variables... Maybe I should disable this warning to get rid of it... pylint wants global names to be uppercase (what PEP 8 recommends for constants) or "special" (two leading and two trailing underscores): THATS_OK = 42 __thats_ok_too__ = object() but_thats_not = "spam" OMG... I don't want to type those underscores everywhere... Anyway, thank you very much for explaining the meaning of what it wants... -- http://mail.python.org/mailman/listinfo/python-list
Re: pygame - importing GL - very bad...
On 01/02/2013 10:57 PM, Michael Torrie wrote: On 01/01/2013 04:49 PM, someone wrote: On 01/01/2013 12:13 PM, Chris Angelico wrote: > You could simply > > import OpenGL.GL as GL You're right - but I forgot to write that even though this maybe should/is recommended many places then I've seen a lot of opengl code on the internet and IMHO NOBODY does that and it'll be a lot slower to type that in front of all the opengl commands... So this solution is not something I like too... But I can see some other people came up with good solutions, which I didn't knew about.. Why is this solution not to your liking? Python has namespaces for a Because the amount of opengl-functions is HUGE, many people (at least on the internet) do as I and (IMHO) it takes up too much time to change a lot of code plus sometimes I grab/modify small code pieces from the internet and it makes my development SO MUCH faster just to make an exception here with star-import for opengl-commands. reason. They both keep code separated and modular. Use them. At most you should import the most commonly-used symbols only, and refer to the rest through their respective namespaces (with whatever alias you've given the import). There is no additional typing burden. There are SO MANY "common-only used" symbols, but I also once believed that I should do as you write (which I agree with you, is the correct way to do it). I'm not saying you're incorrect - I just say that it speeds up my development significantly to only use star-import for opengl-stuff. Despite your opinion, it is completely false that "NOBODY does [this]." In other words a decent python programmer rarely does "from blah import *." There's a reason why pylint flags this. Frankly the code you've seen on the internet that does this is not setting a good example. It's bad programming practice, plain and simple. I'm a bit surprised that others on this list in this thread intimated that it's okay to do import *. The only place where I've seen an import * that actually belonged Generally you're completely correct. After having worked with opengl for some time however, I must say that my personal opinion is that the benefits of making an exception here outweights the disadvantages a lot - but only for the many MANY opengl-commands. was in an __init__.py that brought sub-module symbols into the main package namespace, and even then I figure there's got to be a better way. Maybe this is your own private pet project, but if you ever plan to involve others in your development, leaving the OpenGL symbols in their own namespaces will make code maintenance a lot easier down the line. As said, you're completely correct. Until now it's my own private project, though I'm considering making it open-source after some time. Right now I just need to develop as quick as possible because I have a lot of work to do. Later on another developer may come along and if it's not easy to tell at a glance if a symbol is provided by another module or if it's something you defined, he's going to have a lot of unnecessary work. Don't worry about that - opengl programmers immediately see and recognize opengl-commands... This, I deem is absolutely not a problem - opengl programmers easily recognize opengl commands immediately. They also usually begin with GL_ - hence it's quite easy to see what is an opengl command and everything that isn't an opengl-command is (as you suggest) NOT imported using "star"-import. -- http://mail.python.org/mailman/listinfo/python-list
Re: pygame - importing GL - very bad...
On 01/02/2013 11:30 PM, Andrew Berg wrote: On 2013.01.02 15:57, Michael Torrie wrote: *. The only place where I've seen an import * that actually belonged was in an __init__.py that brought sub-module symbols into the main package namespace, and even then I figure there's got to be a better way. This. I have some code that imports multiprocessing.connection and I do actually type out multiprocessing.connection.Client and it doesn't bother me one bit. Code is read a lot more than it is written, even if only one person ever sees it. The whole "less typing" thing is absurd, especially when IDEs have completion features and stdlib modules share Until about a week ago, actually I hadn't setup emacs to use code completion - maybe this will change now because it'll speed things up and code completion is just a wonderful thing making things easier + quicker to do. I also setup emacs to do use TAGS and a lot more things now. similar or exact function names (is it subprocess.Popen or os.popen? I guess the author wanted to save 2 seconds typing while anyone who reads it has to spend 5-10 to find out which is being used) . I've been using Well, this is not a problem for opengl-commands, I think... Normally I do as you suggest and for the exact same reasons as you write. full namespaces since I started learning Python, and I even do it at the interactive interpreter because it's just a habit. IMO, "from foo import *" should only ever be used for /intentional/ namespace pollution (and even then, there are probably better ways to do it). But I don't do any pollution - only this "format", which was a false alert. And so far I'm so consequent to only use it for the opengl-commands. I would deem that this is what most opengl-programmers do. Try to search for opengl-code out there on the internet... Most people do as I write I do here, IMHO. But you have a point and generally I totally agree with you. This is just a particular exception (and the only one I have) where I disagree, because I do as I think the majority of opengl-programmers do - based on what I see posted on the internet. -- http://mail.python.org/mailman/listinfo/python-list
Re: pylint, was Re: pygame - importing GL - very bad...
On 01/02/2013 03:26 PM, Dave Angel wrote: On 01/02/2013 09:09 AM, someone wrote: On 01/02/2013 01:07 PM, Peter Otten wrote: OMG... I don't want to type those underscores everywhere... Anyway, thank you very much for explaining the meaning of what it wants... Global const values should be ALL_CAPS, so it's obvious that nobody intends to modify them. It's the non-const global attributes that expect to be underscored. You shouldn't have to use those underscores very often. After all, there is seldom a need for a non-const global value, right? Don't think I suppose you're right. of it as a pylint problem, but as a hint from pylint that perhaps you should use fewer globals. I had some bad code which I improved greatly now with thanks to pylint. I'll remember what you've written the next time I look at it - I think I don't use that many global non-const values now - I wrapped a lot of things into a class now. This is much better and I guess the correct "object-oriented" thing to do. I hope/think I'll get this warning a lot fewer times in the future. Thanks a lot for the explanation. -- http://mail.python.org/mailman/listinfo/python-list
Re: pylint, was Re: pygame - importing GL - very bad...
On 01/03/2013 12:52 AM, Steven D'Aprano wrote:
On Wed, 02 Jan 2013 09:26:32 -0500, Dave Angel wrote:
Global const values should be ALL_CAPS, so it's obvious that nobody
intends to modify them.
Like math.pi I suppose? *wink*
:-)
It's the non-const global attributes that expect to be underscored.
Pylint is wrong here.
Ok, forget my previous post - now I looked a bit deeper into it again.
Consider this as an example:
# Global mouse states = global constants:
M_LEFT = 1
M_MIDDLE = 2
M_RIGHT = 3
M_WHEEL_UP = 4
M_WHEEL_DOWN = 5
class somethingWork:
""" OpenGL something class """
def __init__(self, someInputFile):
self.inputFile = someInputFile
self.running = False
# self.viewport = (800,600)
self.viewport = (1024, 768)
self.lightDone = False
self.rx = 0 # rotation x
self.ry = 0 # rotation y
self.rz = 0 # rotation z
etc ...
What pylint says is:
1) class somethingWork: Invalid name "somethingWork" (should match
[A-Z_][a-zA-Z0-9]+$), I'm not that good at regular exps, but I suppose
it wants my class name to start with a capital letter ?
2) self.lightDone: Invalid name "lightDone" (should match
[a-z_][a-z0-9_]{2,30}$)
So I can now understand that pylint doesn't like my naming convention
with a capital letter in the middle of the variable name, like:
"lightDone" = a boolean value. I suppose pylint wants me to use (a
little longer method) an underscore to separate words in long variable
names...
3) self.rx / rself.ry / self.rz: Invalid name "rx" (should match
[a-z_][a-z0-9_]{2,30}$) - so I suppose it wants this name to end with an
underscore ?
I have a lot of these warnings...
The double-leading-and-trailing-underscore naming scheme is reserved for
Python itself. PEP 8 explicitly states not to invent your own "dunder"
names:
__double_leading_and_trailing_underscore__: "magic" objects or
attributes that live in user-controlled namespaces. E.g. __init__,
__import__ or __file__. Never invent such names; only use them as
documented.
I think I would also never use __something__ names...
The section on global variables does not say to use dunder names:
http://www.python.org/dev/peps/pep-0008/#id31
Thanks a lot for this reference...
If pylint says that global variables should be named like "__variable__",
that is explicitly going against PEP 8.
I don't think that is what it's saying... But I think it wants me to use
more underscores, i.e. self.lightDone => self.light_done I think...
You shouldn't have to use those underscores very often. After all,
there is seldom a need for a non-const global value, right? Don't think
of it as a pylint problem, but as a hint from pylint that perhaps you
should use fewer globals.
That at least is good advice.
Ok, sorry for my previous post. This post better explains how I've named
my variables. I don't like it complains about a simple and good variable
name as self.rx, self.ry, self.rz and so on. These are IMHO very good
variable names: rotation about x, y and z. Short and precise/accurate.
I'm not sure if I'll agree with all the warnings it comes up with. But I
think I could maybe introduce more use of underscores in the middle of
my variable names, in the future...
Thanks for the extra + good explanations.
--
http://mail.python.org/mailman/listinfo/python-list
Re: pylint, was Re: pygame - importing GL - very bad...
On 01/02/2013 08:31 PM, Ian Kelly wrote: On Wed, Jan 2, 2013 at 10:57 AM, Chris Angelico wrote: Yeah, same applies to most linters I think. You end up disagreeing with the author on half the points. Oh well. Doesn't make the tool useless, just means you need to fiddle with it to get it how you want it. It's a lot less work to disable a check than to implement a desired check that is missing, so to me it's better that a linter do too much by default than not enough. I just started using pylint and some of the stuff it came up with is REALLY good - so I'll definately use pylint, pep8 (and friends) more in the future. And I think I'll also get to a point where I'll disable some of the checks - as one of you wrote: How I name my variables is (maybe) my own business and for instance I like a short variable name once in a while, e.g. "rx", "ry", "rz" for rotation around x- y- and z-axises and these variable names should not be changed. But can I ask you something: English is not my native language and I looked up what "linter" means - but it's not in my dictionary. What doet "linter" mean ? I don't suppose these exlanations are the same as you would give, in the context you're using? http://www.wordreference.com/definition/linter http://www.collinsdictionary.com/dictionary/american/linter http://www.merriam-webster.com/dictionary/linter ? -- http://mail.python.org/mailman/listinfo/python-list
Re: pylint, was Re: pygame - importing GL - very bad...
On 01/03/2013 03:55 AM, Ian Kelly wrote:
On Wed, Jan 2, 2013 at 7:24 PM, someone wrote:
3) self.rx / rself.ry / self.rz: Invalid name "rx" (should match
[a-z_][a-z0-9_]{2,30}$) - so I suppose it wants this name to end with an
underscore ?
It wants the name to be at least 3 characters long.
Uh, ok, thank you. I'll remember that.
Doesn't this "[ ... ]" mean something optional?
What does {2,30}$ mean?
I think $ means that the {2,30} is something in the end of the sentence...
--
http://mail.python.org/mailman/listinfo/python-list
Re: pylint, was Re: pygame - importing GL - very bad...
On 01/03/2013 10:00 AM, Peter Otten wrote:
Terry Reedy wrote:
[a-z_][a-z0-9_]{2,30}$) - so I suppose it wants this name to end with an
underscore ?
No, it allows underscores. As I read that re, 'rx', etc, do match. They
No, it's one leading letter or underscore [a-z_] plus at least two letters,
underscores or digits [a-z0-9_]{2,30}
Ah, [a-z0-9_]{2,30} means there should be at least two characters and
maximum 30 characters here ?
--
http://mail.python.org/mailman/listinfo/python-list
Re: pylint, was Re: pygame - importing GL - very bad...
On 01/03/2013 05:52 PM, Terry Reedy wrote: That seems like a improper error message from the tool. "Invalid name" does *not* properly describe that situation. The name is *not* "Invalid" in any sense of the word, and a "checker" that tells you it is is creating needless false-positives. An error checker should be saying something like: "self.lightDone: Does not match PEP8 recommended style" making it clear that this is *not* an error, it is a *style* related *warning*. I quite agree. Wanting 3 chars for attribute names is not even PEP-8 style but pylint-author style. I was really surprised at that. In that case, 'Does not match pylint recommended style.' or even 'configured styles'. I have not used pylint or pychecker as of yet. I agree with you all... Thanks, everyone - now I shall investigate pylint and friends in more detail on my own :-) -- http://mail.python.org/mailman/listinfo/python-list
Re: pygame - importing GL - very bad...
On 01/03/2013 03:09 PM, Mike C. Fletcher wrote: On 13-01-02 08:53 PM, someone wrote: So this solution is not something I like too... But I can see some other people came up with good solutions, which I didn't knew about.. Why is this solution not to your liking? Python has namespaces for a Because the amount of opengl-functions is HUGE, many people (at least on the internet) do as I and (IMHO) it takes up too much time to change a lot of code plus sometimes I grab/modify small code pieces from the internet and it makes my development SO MUCH faster just to make an exception here with star-import for opengl-commands. I'd agree on it being rather impractical/pointless/verbose to have every single OpenGL entry point and constant have an extra gl. or glu. or glut. added to the front. OpenGL/GLU/GLUT is already namespaced, but Good to hear, I'm not alone, thanks. using C-style prefix namespacing (that is gl* glu* glut* and GL_*, GLU_*, GLUT_*), so adding Python style namespacing to the front of that makes it very verbose. OpenGL-using code is *littered* with OpenGL entry points and constants (and yes, I intend the slight slight), so that's going to make it rather annoying to work with. PyOpenGL's current approach is mostly attempting to maintain backward compatibility with the older revisions. wxPython actually rewrote its whole interface to go from * imports into namespaced lookups and then wrote a little migration tool that would attempt to rewrite your code for the new version. They also provided a transitional API so that code could mix-and-match the styles. For PyOpenGL that would look something like this: from OpenGL import gl, glu, glut gl.Rotate(...) gl.Clear(gl.COLOR_BUFFER_BIT) Ok, that's interesting. However, I like it the way it is, where I can copy/paste C-code from the web and change some small things and it'll work and fit into my needs. BUT I didn't know there was such a transitional API - interesting. I however don't want to be a first-mover - let's see if sufficiently many people will use this and then I'll consider doing it too. At the moment, I still think the star-import is good because it makes it easy to look for C-code and program it (=do it) like people would do it in C. or, if you really needed PEP-8 compliance, and don't mind making the API look nothing like the original, we might even go to: from opengl import gl, glu, glut gl.rotate(...) gl.clear(gl.COLOR_BUFFER_BIT) Erhm, that's the same as above. Is that what you meant to write? Either of which would *also* make it possible for us to lazy-load the entry points and symbols (that would save quite a bit of ram). But I'm not actually likely to do this, as it makes it far more annoying to work with C-oriented references (and since PyOpenGL is primarily used by new OpenGL coders who need to lean heavily on references, that's a big deal). Currently you can often copy-and-paste C code into PyOpenGL and have it work properly as far as the OpenGL part is concerned (arrays and the like need to be rewritten, but that's not something I can control, really). People are already confused by the small variations Agree - that's something I like. from C OpenGL, making the API look entirely different wouldn't be a good direction to move, IMO. Well, I'm sometimes a bit annoyed that python doesn't give as many warnings/errors as one gets in C - for instance sometimes I copy/paste and forget to remove the trailing semi-colons. However after I began to use pylint and friends, this error will be caught. Then sometimes I forgot to add () for function calls, which in C would cause an error by the compiler but which python allows so one can get the object (which maybe is also a good reason, but at least in the beginning when I learned python, this was very annoying + confusing to me). Thanks for the comments about this transitional opengl-stuff - I was unaware of that. Currently it's not so interesting to me, I like it the way I do it now so I can quickly grab code pieces from C/C++ from the web and change it to suit my needs... -- http://mail.python.org/mailman/listinfo/python-list
Re: pylint, was Re: pygame - importing GL - very bad...
On 01/03/2013 12:27 PM, Chris Angelico wrote:
On Thu, Jan 3, 2013 at 10:19 PM, someone wrote:
Doesn't this "[ ... ]" mean something optional?
What does {2,30}$ mean?
I think $ means that the {2,30} is something in the end of the sentence...
You can find regular expression primers all over the internet, but to
answer these specific questions: [...] means any of the characters in
the range; the $ means "end of string"; and {2,30} means at least two,
and at most thirty, of the preceding character. So you have to have
one from the first group, then 2-30 from the second, for a total of
3-31 characters in your names.
Got it, thanks!
--
http://mail.python.org/mailman/listinfo/python-list
Re: Regular expression syntax, was Re: pylint, was Re: pygame - importing GL - very bad...
On 01/03/2013 12:39 PM, Peter Otten wrote:
someone wrote:
On 01/03/2013 10:00 AM, Peter Otten wrote:
Terry Reedy wrote:
[a-z_][a-z0-9_]{2,30}$) - so I suppose it wants this name to end with
[an
underscore ?
No, it allows underscores. As I read that re, 'rx', etc, do match. They
No, it's one leading letter or underscore [a-z_] plus at least two
letters, underscores or digits [a-z0-9_]{2,30}
Ah, [a-z0-9_]{2,30} means there should be at least two characters and
maximum 30 characters here ?
Yes. See
http://docs.python.org/2/library/re.html#regular-expression-syntax
Thanks - it's on my TODO-list to learn more about how to use these
regexps...
--
http://mail.python.org/mailman/listinfo/python-list
Re: pylint, was Re: pygame - importing GL - very bad...
On 01/03/2013 03:56 AM, Dave Angel wrote: The first lint program I recall hearing of was available in the early 1980's, and was for the C language. At the time, the C language was extremely flexible (in other words, lots of ways to shoot yourself in the foot) and the compiler was mostly of the philosophy - if there's a way to make sense of the statement, generate some code, somehow. Anyway, lint made sense to me as the crud that gets mixed in with the real fabric. And a linter is a machine that identifies and removes that crud. Well, the lint program didn't remove anything, but it identified a lot of it. I didn't hear the term linter till decades later. Aah, now I understand this "lintering" and where it came from - thanks a lot! :-) -- http://mail.python.org/mailman/listinfo/python-list
Re: pygame - importing GL - very bad...
On 01/05/2013 02:30 AM, Dave Angel wrote: from opengl import gl, glu, glut gl.rotate(...) gl.clear(gl.COLOR_BUFFER_BIT) Erhm, that's the same as above. Is that what you meant to write? No, it's not the same; here he did not capitalize the function names. Previously they look like class instantiations. Ah, you're right. Sorry :-) Well, I'm sometimes a bit annoyed that python doesn't give as many warnings/errors as one gets in C - for instance sometimes I copy/paste and forget to remove the trailing semi-colons. Trailing semi colons are legal in most cases. The semi-colon is a separator between statements, when one wants to put multiple statements on one line. However after I began to use pylint and friends, this error will be caught. Then sometimes I forgot to add () for function calls, which in C would cause an error by the compiler Actually no. In C, a function name without parentheses is also a function pointer. Not exactly the same as a function object, though C++ gets a lot closer. But the real reason C catches that typo is that the types most likely don't match, depending on what you meant to do with the return value. Ok, I think you're right. At least I find that C-compilers catches many errors/warnings which python don't say anything about. But also C require me to define/declarer the types of variables before I use them... OTOH I guess I like that python is faster to code in, compared to C... but which python allows so one can get the object (which maybe is also a good reason, but at least in the beginning when I learned python, this was very annoying + confusing to me). Function objects are enormously useful, as you get more adept at using Python. Ok, I'll look forward to that. Recently I had some problems with pass-by-value vs pass-by-reference. I googled the problem and found that by default python passes by reference. I then debugged my program and finally found out that I should make a copy (a new object) of the variable, before I passed it to my function. And THIS solved my problem. But in C/C++ I think the default is to pass by value, so this error in my program was a bit unexpected... Anyway, I feel python is a great language for doing things much faster than I could possibly do in C/C++. I also have on my todo-list to take my opengl-program and make it into an executable. I mainly sit on a linux-pc but I want to distribute my opengl program for windows (which has most users). I've found something on google for py2app, cx_Freeze, bbfreeze and Freeze and I hope this cross-platform thing does not cause too many problems... I tried one of these tools a few weeks ago but I think I only succeeded with a very simple hello-world program... Anyway, that's a problem I'll investigate in a few months when I think/hope my opengl program is finished... -- http://mail.python.org/mailman/listinfo/python-list
Re: New in Python , Need a Mentor
On 01/02/2013 05:30 PM, Chris Angelico wrote: On Thu, Jan 3, 2013 at 3:24 AM, Wolfgang Strobl wrote: Chris Angelico : I strongly recommend IDLE - much better editing/recall facilities than the command-line Python has), and work through the tutorial: Well, this is certainly a matter of taste. I'd recommend using some small, language independent programmers editor and some modern distributed version control system right at the beginning. Put your code, even the smallest snippets, under version control, make that a habit. Write small doctests for your code from the very beginning. Try to construct your code so that it works equally well as a module and as a standalone script Don't start developing web applications, write some small utilities for your own needs, first. Personally, I suggest SciTE and TortoiseHG on Windows, but that too is, as I said, a matter of taste. I don't edit code in IDLE, I just use it for interactive work. For actual script editing, agreed (though I use git rather than hg), but it really does help to have a way to *very* quickly test a line or two of code. I really REALLY like debugging with "eric"... http://eric-ide.python-projects.org/ -- http://mail.python.org/mailman/listinfo/python-list
Re: pygame - importing GL - very bad...
On 01/05/2013 12:47 PM, Chris Angelico wrote: C has typed variables, so it's a compile-time error to try to put any other type into that variable. Python doesn't. That flexibility comes at the cost of error-catching. There are hybrid systems, but in general, type declarations imply variable declarations, and that's something that Python doesn't want. (I'm of the opinion that declarations aren't such a bad thing; they make some aspects of scoping easier. However, that's a design decision that Python is as unlikely to reverse as indentation-defined blocks.) Understood. Ok, I'll look forward to that. Recently I had some problems with pass-by-value vs pass-by-reference. I googled the problem and found that by default python passes by reference. No, it doesn't. You can find good references on the subject in various places, but call-by-reference as implemented in Pascal simply doesn't exist in most modern languages, because its semantics are way confusing. The name given to this technique varies; here's a couple of links: http://effbot.org/zone/call-by-object.htm http://en.wikipedia.org/wiki/Evaluation_strategy#Call_by_sharing If you don't like calling it "pass-by-reference", perhaps you prefer calling it: “call by object reference“... From: http://effbot.org/zone/call-by-object.htm In any case I think we understand each other. -- http://mail.python.org/mailman/listinfo/python-list
Re: pylint, was Re: pygame - importing GL - very bad...
On 01/05/2013 01:49 PM, Jan Riechers wrote:
On 05.01.2013 03:11, someone wrote:
But about the regular expressions (a bit deeper look into that):
Like said of Chris:
[a-z]
defines a "catching group", in this case all ascii lowercase letters
ranging from "a" to "z". If noting else is provided, the rule matches
one letter if there is no range defined by something like:
{} -> Target a range of a match/hit
There are also a
? -> Lazy match
* -> Gready match
[A-Z][0-9]{1,3} means translated:
Look for any uppercase letter in ascii(!) (not "öäü" or similiar)
ranging from "A" to "Z".
Now look for any digit (2nd catching group) with the addition to satisfy
the rule ONLY if there are at least 1 to 3 digits found.
Note: If there are 4 or more digits - the catching rule is still
satisfied and will provide a match/hit.
Ok, thanks a lot for the elaboration... I think I need to work with it
myself at some time to be sure of understanding it...
If there is a follow up group, the next evaluation is gone through if
present and so forth. If the expression is satisfied, the match is
returned.
The lazy "?" and greedy "*" matches try to satisfy, as the naming
implies, to match as less or as much of what you have asked for.
For example the regular expression is valid:
0* -> Look for a zero, and be greedy as of how many zeros you want match
which might follow.
Regular expressions don't have to include catching groups in order to work.
But when you use them yourself somehow, its quite simple I think.
I guess you are anyhow busy mangling with pyLint, PEP-Standards and
pyOpenGL - so good luck with that :)
You're right - I'm a bit "overbooked" at the moment - but thanks a lot
for clarifyring this with the regexps :-)
--
http://mail.python.org/mailman/listinfo/python-list
Re: pygame - importing GL - very bad...
On 01/05/2013 02:27 PM, Chris Angelico wrote: On Sun, Jan 6, 2013 at 12:06 AM, someone wrote: In any case I think we understand each other. That's one of the links I just posted :) It's not just a naming difference, though. With Pascal's pass-by-reference semantics, this code would act differently: def foo(x): x = 5 a = 2 foo(a) print(a) Python prints 2, because the assignment to x just rebinds the name inside foo. True pass-by-reference actually changes the caller's variable. C can achieve this by means of pointers; in Python, you can I thought that python also used "true" pass-by-reference, although I haven't figured out exactly when I have this problem. I can just see that sometimes I get this problem and then I need to copy the variable, if I don't want the original data of the variable to be overwritten... pass and mutate a list, thus: def foo(x): x[0] = 5 # Dereference the pointer, kinda x=[None] # Declare a pointer variable, ish x[0] = 2 foo(x) # Don't forget to drop the [0] when passing the pointer to another function print(x[0]) # Prints 5. See? We have pass-by-reference! Yes, something like this has happened to me in my python code... Not sure if my example was exactly like this, but I remember something where I found this to be a problem that I had to fix. But otherwise, rebinding names in the function has no effect on anything outside. Among other things, this guarantees that, in any hmm. ok So it's not true pass-by-reference like I thought... That's interesting. situation, a name referencing an object can be perfectly substituted for any other name referencing the same object, or any other way of accessing the object. def foo(lst): lst[0]=len(lst) x = [10,20,30] y = x foo(x) # These two... foo(y) # ... are identical! This is something I've experienced from my own coding, I think... This is a philosophy that extends through the rest of the language. A function returning a list can be directly dereferenced, as can a list literal: def foo(): return [0,1,4,9,16] print( ["Hello","world!","Testing","Testing","One","Two","Three"][foo()[2]] ) That prints out "One"... I think I understand - that's interesting too... This is a flexibility and power that just doesn't exist in many older languages (C and PHP, I'm looking at you). Object semantics make more sense than any other system for a modern high level language, which is why many of them do things that way. I agree, that I think python is really great and it's fast to do something useful. Not sure I still understand exactly all aspects of this pass-by-value and by-references, but in any case I know enough to watch out for this problem and once I see I have a problem, I can also take care of it and solve it by making a copy. I think maybe after 3-6-9 months more of working with python, I should be fully confident with this. Thanks for taking the time to explain a bit of this to me... -- http://mail.python.org/mailman/listinfo/python-list
Re: pygame - importing GL - very bad...
On 01/06/2013 12:37 PM, alex23 wrote:
On Jan 6, 5:49 am, someone wrote:
I thought that python also used "true" pass-by-reference, although I
haven't figured out exactly when I have this problem. I can just see
that sometimes I get this problem and then I need to copy the variable,
if I don't want the original data of the variable to be overwritten...
Generally I find it easier to call variables 'labels' or 'references';
you're not stashing a value into a slot, you're giving a name to an
object. So you're never really passing values around, just labels that
refer to an object.
Ok.
The confusion kicks in because there are two types of object: mutable
and immutable. Mutable objects can change, immutable objects cannot.
Yes, I've seen that described before.
Operations on an immutable object will return a _new_ immutable
object; the label used in an operation on an immutable object will
refer to the new object, any other references to the original object
will continue to refer to the original. Strings, numbers, tuples and
frozensets are all immutable built-in types.
>>> a = "alpha"
>>> b = a
>>> a = a + "bet"
>>> a
'alphabet'
>>> b
'alpha'
Ok, I think I knew some of these things, however I didn't think so much
about them before now.
With immutable types, you're safe to pass them into a function, act on
them, and not have references in the outer scope reflect the change:
>>> def double(x): return x * 2
...
>>> a = "alpha"
>>> double(a)
'alphaalpha'
>>> a
'alpha'
This is exactly what I've found out happens to me some times. Until now
I've managed to fix my problems. But it's good to remember this thing
with immutable vs. mutable types, which was something I didn't think
much about before. I'll think about this difference in the future, thank
you.
Everything else, including user defined objects, tend to be mutable:
>>> a = dict(foo=1,bar=2)
>>> b = a
>>> a['foo'] = 99
>>> a
{'foo': 99, 'bar': 2}
>>> b
{'foo': 99, 'bar': 2}
Yes, I've noticed this a lot of times in my own coding...
With mutable objects, you're _always_ operating on the _same_ object
that everything is referring to, even when you pass it into a
different scope:
>>> def toggle_foo( x ): x['foo'] = not x['foo']
...
>>> a = dict(foo=True)
>>> toggle_foo(a)
>>> a
{'foo': False}
Exactly, what I've also experienced a few times.
Note that the `toggle_foo` function doesn't return anything, nor is
the value of a re-assigned. The object that a refers to is passed into
`toggle_foo`, modified in place, and all references to it remain
pointing to the same, now modified object.
Yes, I notice that, thanks.
This is one of the big causes of unexpected behaviour to new Python
users, but as you get a better understanding of how Python's object
model works, you'll see it's really quite consistent and predictable.
It was a bit surprising to me in the beginning - though I'm still a
beginner (or maybe now almost an "intermediate" user), the good
explanation you come with now wasn't something I've thought so much of
before now. But I've clearly experienced many of the examples you write
about here, in my own coding and I've usually been very careful about
checking if my original object was modified un-intentionally. I think I
can deal with this now.
There are a couple of ways you can ensure you're always working with a
copy of an object if you need to. For lists, the canonical way is:
>>> a = [1,2,3]
>>> b = a
>>> a = [1, 2, 3]
>>> b = a[:] # shallow copy of a
>>> b.append(99)
>>> b
[1, 2, 3, 99]
>>> a
[1, 2, 3]
`b = a[:]` uses slice notation to create a new list that contains
everything in the original list, from start to end. This is called a
"shallow" copy; `a` and `b` both refer to _different_ lists, but the
objects within are the _same_ objects. For numbers, this isn't
Ok, good to know.
important, as they're immutable. For mutable objects, however, it's
something you need to bear in mind:
>>> a = [ [1,2], [3, 4] ] # list of lists
>>> b = a[:]
>>> b[0][0] = 99
>>> b
[[99, 2], [3, 4]]
>>> a
[[99, 2], [3, 4]]
So here, while `b` refers to copy of `a`, that copy contains the
_same_ list object
Newbie, homework help, please.
Ok, this is my dillema, not only am I new to this programming buisness, before
the last few days, I did not even know what python was, and besides opening up
the internet or word documents, that is most of what I know. Yet, I have a
professor who should be on Psych medication for giving us 3 projects, 2 of
which I have not listed here to do. I was able to do research over the last 3
days, and I have spent 3 days on this project, by borrowing others ideas on
this project. Below, you will find my professors assignment (oh, and due in one
week right before finals, so I am stressing out so much, cause I don't know why
he is crazy enough to assign crap like this a week before finals when I have
Calculus final,chem final, etc. I have figured out most of the assignment, and
below, it will be posted after the teacher's post of the assignment. What I
need help with, and I have tried relentlessly to find, is how to put freaking
stars(asterisks) as border around a list without installing any other program
to a portable python, of course, this is where my problem lies. Below, you will
see what I have done, please, help!!!
You are required to complete and submit the following programming projects in
Python by the indicated deadline:
Standard Header Information project (5 pts):
Write a program that will:
1) Ask the user for the following information:
- name of file to be created for storing SHI
- user’s name (as part of SHI)
- user’s course and section (as part of SHI)
- user’s semester and year (as part of SHI)
- user’s assignment title (as part of SHI)
2) Write the above SHI data to a text (.txt) file with the name chosen by the
user (above)
3) Close the file that the SHI data was written to
4) Open the file with the SHI data (again)
5) Read the data into different (from part 1) variable names
6) Display the SHI data read from the file in the interpreter with a border
around the SHI data (include a buffer of 1 line/space between the border and
SHI data). An example might look like:
***
* *
* First Name and Last *
* ENGR 109-X *
* Fall 2999 *
* Format Example *
* *
***
textfile=input('Hello, we are about to create a text file. An example would be:
(sample.txt) without the parenthesis. What ever you do name it, it needs to end
in (.txt). What would you like to name your textfile?')
userinput=[input('What is your name?'),input('What is your Course Section and
Course number?'),input('What is the Semester and year?'),input('What is the
title of this class assignment?')]
for item in userinput:
openfile=open(textfile,'w');openfile.writelines("%s\n" % item for item in
userinput);openfile.close()
x=textfile;indat=open(x,'r');SHI=indat.read()
def border(Sullivan):
string=SHI
stringlength=len(string)
stringlength=stringlength("%s\n" % item for item in stringlength) + 2 * (3
+ 3)
hBorder=stringlength//2*"* "+"*"[:stringlength%2]
spacer="*"+" "*(stringlength - 2)+"*"
fancyText="* "+string+" *"
return(hBorder,spacer,fancyText,hBorder)
textTuple = border(SHI)
for lines in textTuple:
print (lines)
--
http://mail.python.org/mailman/listinfo/python-list
Re: Newbie, homework help, please.
On Saturday, April 21, 2012 12:28:33 PM UTC-5, someone wrote:
> Ok, this is my dillema, not only am I new to this programming buisness,
> before the last few days, I did not even know what python was, and besides
> opening up the internet or word documents, that is most of what I know. Yet,
> I have a professor who should be on Psych medication for giving us 3
> projects, 2 of which I have not listed here to do. I was able to do research
> over the last 3 days, and I have spent 3 days on this project, by borrowing
> others ideas on this project. Below, you will find my professors assignment
> (oh, and due in one week right before finals, so I am stressing out so much,
> cause I don't know why he is crazy enough to assign crap like this a week
> before finals when I have Calculus final,chem final, etc. I have figured out
> most of the assignment, and below, it will be posted after the teacher's post
> of the assignment. What I need help with, and I have tried relentlessly to
> find, is how to put freaking stars(asterisks) as border around a list without
> installing any other program to a portable python, of course, this is where
> my problem lies. Below, you will see what I have done, please, help!!!
> You are required to complete and submit the following programming projects in
> Python by the indicated deadline:
>
> Standard Header Information project (5 pts):
> Write a program that will:
> 1) Ask the user for the following information:
> - name of file to be created for storing SHI
> - user’s name (as part of SHI)
> - user’s course and section (as part of SHI)
> - user’s semester and year (as part of SHI)
> - user’s assignment title (as part of SHI)
> 2) Write the above SHI data to a text (.txt) file with the name chosen by the
> user (above)
> 3) Close the file that the SHI data was written to
> 4) Open the file with the SHI data (again)
> 5) Read the data into different (from part 1) variable names
> 6) Display the SHI data read from the file in the interpreter with a border
> around the SHI data (include a buffer of 1 line/space between the border and
> SHI data). An example might look like:
>
> ***
> * *
> * First Name and Last *
> * ENGR 109-X *
> * Fall 2999 *
> * Format Example *
> * *
> ***
>
>
> textfile=input('Hello, we are about to create a text file. An example would
> be: (sample.txt) without the parenthesis. What ever you do name it, it needs
> to end in (.txt). What would you like to name your textfile?')
> userinput=[input('What is your name?'),input('What is your Course Section and
> Course number?'),input('What is the Semester and year?'),input('What is the
> title of this class assignment?')]
> for item in userinput:
> openfile=open(textfile,'w');openfile.writelines("%s\n" % item for item in
> userinput);openfile.close()
> x=textfile;indat=open(x,'r');SHI=indat.read()
> def border(Sullivan):
> string=SHI
> stringlength=len(string)
> stringlength=stringlength("%s\n" % item for item in stringlength) + 2 *
> (3 + 3)
> hBorder=stringlength//2*"* "+"*"[:stringlength%2]
> spacer="*"+" "*(stringlength - 2)+"*"
> fancyText="* "+string+" *"
> return(hBorder,spacer,fancyText,hBorder)
>
> textTuple = border(SHI)
> for lines in textTuple:
> print (lines)
almost forgot, it has to have a 1 inch border around the top, bottom, left, and
right, with it being aligned to the left. In the picture above, that is not how
it actually looks, the stars to the right are aligned on the right, not right
next to each other. Thanks.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Newbie, homework help, please.
On Saturday, April 21, 2012 12:28:33 PM UTC-5, someone wrote:
> Ok, this is my dillema, not only am I new to this programming buisness,
> before the last few days, I did not even know what python was, and besides
> opening up the internet or word documents, that is most of what I know. Yet,
> I have a professor who should be on Psych medication for giving us 3
> projects, 2 of which I have not listed here to do. I was able to do research
> over the last 3 days, and I have spent 3 days on this project, by borrowing
> others ideas on this project. Below, you will find my professors assignment
> (oh, and due in one week right before finals, so I am stressing out so much,
> cause I don't know why he is crazy enough to assign crap like this a week
> before finals when I have Calculus final,chem final, etc. I have figured out
> most of the assignment, and below, it will be posted after the teacher's post
> of the assignment. What I need help with, and I have tried relentlessly to
> find, is how to put freaking stars(asterisks) as border around a list without
> installing any other program to a portable python, of course, this is where
> my problem lies. Below, you will see what I have done, please, help!!!
> You are required to complete and submit the following programming projects in
> Python by the indicated deadline:
>
> Standard Header Information project (5 pts):
> Write a program that will:
> 1) Ask the user for the following information:
> - name of file to be created for storing SHI
> - user’s name (as part of SHI)
> - user’s course and section (as part of SHI)
> - user’s semester and year (as part of SHI)
> - user’s assignment title (as part of SHI)
> 2) Write the above SHI data to a text (.txt) file with the name chosen by the
> user (above)
> 3) Close the file that the SHI data was written to
> 4) Open the file with the SHI data (again)
> 5) Read the data into different (from part 1) variable names
> 6) Display the SHI data read from the file in the interpreter with a border
> around the SHI data (include a buffer of 1 line/space between the border and
> SHI data). An example might look like:
>
> ***
> * *
> * First Name and Last *
> * ENGR 109-X *
> * Fall 2999 *
> * Format Example *
> * *
> ***
>
>
> textfile=input('Hello, we are about to create a text file. An example would
> be: (sample.txt) without the parenthesis. What ever you do name it, it needs
> to end in (.txt). What would you like to name your textfile?')
> userinput=[input('What is your name?'),input('What is your Course Section and
> Course number?'),input('What is the Semester and year?'),input('What is the
> title of this class assignment?')]
> for item in userinput:
> openfile=open(textfile,'w');openfile.writelines("%s\n" % item for item in
> userinput);openfile.close()
> x=textfile;indat=open(x,'r');SHI=indat.read()
> def border(Sullivan):
> string=SHI
> stringlength=len(string)
> stringlength=stringlength("%s\n" % item for item in stringlength) + 2 *
> (3 + 3)
> hBorder=stringlength//2*"* "+"*"[:stringlength%2]
> spacer="*"+" "*(stringlength - 2)+"*"
> fancyText="* "+string+" *"
> return(hBorder,spacer,fancyText,hBorder)
>
> textTuple = border(SHI)
> for lines in textTuple:
> print (lines)
Thanks your Bart for trying, I don't understand how it works or if you tried to
place my script in python to see if it would work, unfortunately, I tried 10
different ways plus yours, and I don't see the connection. Unless you were
trying to help me see the way and I did not, sorry, but thanks for trying. The
characters did show me a little more.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Newbie, homework help, please.
On Saturday, April 21, 2012 12:28:33 PM UTC-5, someone wrote:
> Ok, this is my dillema, not only am I new to this programming buisness,
> before the last few days, I did not even know what python was, and besides
> opening up the internet or word documents, that is most of what I know. Yet,
> I have a professor who should be on Psych medication for giving us 3
> projects, 2 of which I have not listed here to do. I was able to do research
> over the last 3 days, and I have spent 3 days on this project, by borrowing
> others ideas on this project. Below, you will find my professors assignment
> (oh, and due in one week right before finals, so I am stressing out so much,
> cause I don't know why he is crazy enough to assign crap like this a week
> before finals when I have Calculus final,chem final, etc. I have figured out
> most of the assignment, and below, it will be posted after the teacher's post
> of the assignment. What I need help with, and I have tried relentlessly to
> find, is how to put freaking stars(asterisks) as border around a list without
> installing any other program to a portable python, of course, this is where
> my problem lies. Below, you will see what I have done, please, help!!!
> You are required to complete and submit the following programming projects in
> Python by the indicated deadline:
>
> Standard Header Information project (5 pts):
> Write a program that will:
> 1) Ask the user for the following information:
> - name of file to be created for storing SHI
> - user’s name (as part of SHI)
> - user’s course and section (as part of SHI)
> - user’s semester and year (as part of SHI)
> - user’s assignment title (as part of SHI)
> 2) Write the above SHI data to a text (.txt) file with the name chosen by the
> user (above)
> 3) Close the file that the SHI data was written to
> 4) Open the file with the SHI data (again)
> 5) Read the data into different (from part 1) variable names
> 6) Display the SHI data read from the file in the interpreter with a border
> around the SHI data (include a buffer of 1 line/space between the border and
> SHI data). An example might look like:
>
> ***
> * *
> * First Name and Last *
> * ENGR 109-X *
> * Fall 2999 *
> * Format Example *
> * *
> ***
>
>
> textfile=input('Hello, we are about to create a text file. An example would
> be: (sample.txt) without the parenthesis. What ever you do name it, it needs
> to end in (.txt). What would you like to name your textfile?')
> userinput=[input('What is your name?'),input('What is your Course Section and
> Course number?'),input('What is the Semester and year?'),input('What is the
> title of this class assignment?')]
> for item in userinput:
> openfile=open(textfile,'w');openfile.writelines("%s\n" % item for item in
> userinput);openfile.close()
> x=textfile;indat=open(x,'r');SHI=indat.read()
> def border(Sullivan):
> string=SHI
> stringlength=len(string)
> stringlength=stringlength("%s\n" % item for item in stringlength) + 2 *
> (3 + 3)
> hBorder=stringlength//2*"* "+"*"[:stringlength%2]
> spacer="*"+" "*(stringlength - 2)+"*"
> fancyText="* "+string+" *"
> return(hBorder,spacer,fancyText,hBorder)
>
> textTuple = border(SHI)
> for lines in textTuple:
> print (lines)
Thanks for your reply Mr. Roy Smith. Also, thanks for the tip. Maybe I did not
make myself as clear or maybe you did not understand my post. It states
homework help, and I am doing this post to get help before I pay somebody to
show me how to do it, because, obviously our professor is not kind enough to
show us how to do this less than two weeks before the due date with 3 projects
(and now I have basically less than a week before it is due- and I am not
waiting till the last minute), and this just so happens to be the easier one. I
understand the concept of him trying to give us deadlines and making us learn,
but I don't appreciate paying for a class where I am not taught anything or
told the right direction due to his belief that he is helping me to get a
stronger work ethic, I have 5 classes this semester, and stupid me, I had to
take the hardest ones I could not realizing the depth of the struggles I would
face. Again, thanks for the tip, but I never asked you to do my homework, it
clearly states that I did research over the last few days, over the internet,
because our brilliant professor is giving us just th
Re: Newbie, homework help, please.
On Saturday, April 21, 2012 12:28:33 PM UTC-5, someone wrote:
> Ok, this is my dillema, not only am I new to this programming buisness,
> before the last few days, I did not even know what python was, and besides
> opening up the internet or word documents, that is most of what I know. Yet,
> I have a professor who should be on Psych medication for giving us 3
> projects, 2 of which I have not listed here to do. I was able to do research
> over the last 3 days, and I have spent 3 days on this project, by borrowing
> others ideas on this project. Below, you will find my professors assignment
> (oh, and due in one week right before finals, so I am stressing out so much,
> cause I don't know why he is crazy enough to assign crap like this a week
> before finals when I have Calculus final,chem final, etc. I have figured out
> most of the assignment, and below, it will be posted after the teacher's post
> of the assignment. What I need help with, and I have tried relentlessly to
> find, is how to put freaking stars(asterisks) as border around a list without
> installing any other program to a portable python, of course, this is where
> my problem lies. Below, you will see what I have done, please, help!!!
> You are required to complete and submit the following programming projects in
> Python by the indicated deadline:
>
> Standard Header Information project (5 pts):
> Write a program that will:
> 1) Ask the user for the following information:
> - name of file to be created for storing SHI
> - user’s name (as part of SHI)
> - user’s course and section (as part of SHI)
> - user’s semester and year (as part of SHI)
> - user’s assignment title (as part of SHI)
> 2) Write the above SHI data to a text (.txt) file with the name chosen by the
> user (above)
> 3) Close the file that the SHI data was written to
> 4) Open the file with the SHI data (again)
> 5) Read the data into different (from part 1) variable names
> 6) Display the SHI data read from the file in the interpreter with a border
> around the SHI data (include a buffer of 1 line/space between the border and
> SHI data). An example might look like:
>
> ***
> * *
> * First Name and Last *
> * ENGR 109-X *
> * Fall 2999 *
> * Format Example *
> * *
> ***
>
>
> textfile=input('Hello, we are about to create a text file. An example would
> be: (sample.txt) without the parenthesis. What ever you do name it, it needs
> to end in (.txt). What would you like to name your textfile?')
> userinput=[input('What is your name?'),input('What is your Course Section and
> Course number?'),input('What is the Semester and year?'),input('What is the
> title of this class assignment?')]
> for item in userinput:
> openfile=open(textfile,'w');openfile.writelines("%s\n" % item for item in
> userinput);openfile.close()
> x=textfile;indat=open(x,'r');SHI=indat.read()
> def border(Sullivan):
> string=SHI
> stringlength=len(string)
> stringlength=stringlength("%s\n" % item for item in stringlength) + 2 *
> (3 + 3)
> hBorder=stringlength//2*"* "+"*"[:stringlength%2]
> spacer="*"+" "*(stringlength - 2)+"*"
> fancyText="* "+string+" *"
> return(hBorder,spacer,fancyText,hBorder)
>
> textTuple = border(SHI)
> for lines in textTuple:
> print (lines)
Thanks Bart for trying, it helped me out a little more by showing me a little
more than I knew, but I tried and I am not sure if it does fit my example due
to it was too many stars in between the lines, and not match up, but if you see
any more help, that would be great, or see something I am doing wrong, or
figure something out, let me know, have a great day, again, thanks
--
http://mail.python.org/mailman/listinfo/python-list
Re: Newbie, homework help, please.
On Saturday, April 21, 2012 3:44:49 PM UTC-5, BartC wrote:
> "someone" wrote in message
> news:9071485.2215.1335040139144.JavaMail.geo-discussion-forums@yniw15...
>
> > Thanks Bart for trying, it helped me out a little more by showing me a
> > little more than I knew, but I tried and I am not sure if it does fit my
> > example due to it was too many stars in between the lines, and not match
> > up, but if you see any more help, that would be great, or see something I
> > am doing wrong, or figure something out, let me know, have a great day,
> > again, thanks
>
> Here's the example you posted:
>
> ***
> * *
> * First Name and Last *
> * ENGR 109-X *
> * Fall 2999 *
> * Format Example *
> * *
> ***
>
> And here's the output from both bits of code I posted, which has 'one inch'
> of white space all around; both examples must be viewed with a fixed-pitch
> font (or in 'code' mode):
>
>
>
>
>
>
> ***
> * *
> * First Name and Last *
> * ENGR 109-X *
> * Fall 2999 *
> * Format Example *
> * *
> ***
>
>
>
>
>
> Please post the output you're getting that has too many asterisks.
>
> --
> Bartc
Hi, Bart: Thank you, your post is working now, maybe, I did something wrong,
unfortunately, you are right, my setup for getting the file to pull up
correctly now is an issue, At first, I got a Vertical line with it working,
then I tried to tinker with it, and it fratched, lol
def border(text):
maxwidth=0
for s in text:
if len(s)>maxwidth: maxwidth=len(s)
vertinchlines=6# assume 6 lines/inch
hozinchchars=10# assume 10 chars/inch
hozmargin=" "*hozinchchars
newtext=[]
for i in range(vertinchlines):
newtext.append("")
newtext.append(hozmargin+"*"*(maxwidth+4)+hozmargin)
newtext.append(hozmargin+"* "+" "*maxwidth+" *"+hozmargin)
for s in text:
newtext.append(hozmargin+"* "+s+" "*(maxwidth-len(s))+" *"+hozmargin)
newtext.append(hozmargin+"* "+" "*maxwidth+" *"+hozmargin)
newtext.append(hozmargin+"*"*(maxwidth+4)+hozmargin)
for i in range(vertinchlines):
newtext.append("")
return newtext
x=textfile;indat=open(x,'r');SHI=indat.read()
textTuple = border(SHI)
for lines in textTuple:
print ("%s\n" %textTuple)
The issue is trying to get the SHI to work right, but omg, this is the closes I
have gotten, you are awsome, thank you very much, i guess i will just keep
messing with it till i get it
--
http://mail.python.org/mailman/listinfo/python-list
Re: Newbie, homework help, please.
On Saturday, April 21, 2012 5:48:29 PM UTC-5, BartC wrote:
> "someone" wrote in message
> news:9533449.630.1335042672358.JavaMail.geo-discussion-forums@ynmf4...
> > On Saturday, April 21, 2012 3:44:49 PM UTC-5, BartC wrote:
>
> > Hi, Bart: Thank you, your post is working now, maybe, I did something
> > wrong, unfortunately, you are right, my setup for getting the file to pull
> > up correctly now is an issue, At first, I got a Vertical line with it
> > working, then I tried to tinker with it, and it fratched, lol
> > def border(text):
> > maxwidth=0
> > for s in text:
Thanks, Bart, it worked perfectly: I was the one doing wrong due to I was doing
the indentation incorrectly, I messed it up and thought it was the code when it
was me. Thank you so much, and have a blessed day.
> > if len(s)>maxwidth: maxwidth=len(s)
> > vertinchlines=6# assume 6 lines/inch
> > hozinchchars=10# assume 10 chars/inch
> > hozmargin=" "*hozinchchars
> > newtext=[]
> > for i in range(vertinchlines):
> > newtext.append("")
> > newtext.append(hozmargin+"*"*(maxwidth+4)+hozmargin)
> > newtext.append(hozmargin+"* "+" "*maxwidth+" *"+hozmargin)
> > for s in text:
> > newtext.append(hozmargin+"* "+s+" "*(maxwidth-len(s))+" *"+hozmargin)
> > newtext.append(hozmargin+"* "+" "*maxwidth+" *"+hozmargin)
> > newtext.append(hozmargin+"*"*(maxwidth+4)+hozmargin)
> > for i in range(vertinchlines):
> > newtext.append("")
> > return newtext
> > x=textfile;indat=open(x,'r');SHI=indat.read()
> > textTuple = border(SHI)
> > for lines in textTuple:
> > print ("%s\n" %textTuple)
> >
> >
> > The issue is trying to get the SHI to work right, but omg, this is the
> > closes I have gotten, you are awsome, thank you very much, i guess i will
> > just keep messing with it till i get it
>
> I had to use this code to make this work right from a file (in additon to
> the border() function):
>
> textfile="kkk4" # (don't use this; this was my test input)
>
> x=textfile;indat=open(x,'r');
>
> SHI=indat.readlines()
> indat.close()
>
> for i in range(len(SHI)):# remove trailing '\n' from each line
> s=SHI[i]
> SHI[i]=(s[0:len(s)-1])
>
> textTuple = border(SHI)
>
> for lines in textTuple:
> print (lines)
>
> Your indat.read() seemed to read all the lines as one long string. I used
> indat.readlines() instead. However each line has a newline char '\n' at the
> end. I put in a loop to get rid of that (I'm sure there's a one-line fix to
> do that, but as I said don't know Python).
>
> The input file I used had to have this format:
>
> First Name and Last
> ENGR 109-X
> Fall 2999
> Format Example
>
> --
> Bartc
--
http://mail.python.org/mailman/listinfo/python-list
numpy (matrix solver) - python vs. matlab
Hi, Notice cross-post, I hope you bear over with me for doing that (and I imagine that some of you also like python in the matlab-group like myself)... -- Python vs. Matlab: -- Python: from numpy import matrix from numpy import linalg A = matrix( [[1,2,3],[11,12,13],[21,22,23]] ) print "A=" print A print "A.I (inverse of A)=" print A.I A.I (inverse of A)= [[ 2.81466387e+14 -5.62932774e+14 2.81466387e+14] [ -5.62932774e+14 1.12586555e+15 -5.62932774e+14] [ 2.81466387e+14 -5.62932774e+14 2.81466387e+14]] Matlab: >> A=[1 2 3; 11 12 13; 21 22 23] A = 1 2 3 111213 212223 >> inv(A) Warning: Matrix is close to singular or badly scaled. Results may be inaccurate. RCOND = 1.067522e-17. ans = 1.0e+15 * 0.3002 -0.60050.3002 -0.60051.2010 -0.6005 0.3002 -0.60050.3002 -- Python vs. Matlab: -- So Matlab at least warns about "Matrix is close to singular or badly scaled", which python (and I guess most other languages) does not... Which is the most accurate/best, even for such a bad matrix? Is it possible to say something about that? Looks like python has a lot more digits but maybe that's just a random result... I mean Element 1,1 = 2.81e14 in Python, but something like 3e14 in Matlab and so forth - there's a small difference in the results... With python, I would also kindly ask about how to avoid this problem in the future, I mean, this maybe means that I have to check the condition number at all times before doing anything at all ? How to do that? I hope you matlabticians like this topic, at least I myself find it interesting and many of you probably also program in some other language and then maybe you'll find this worthwhile to read about. -- http://mail.python.org/mailman/listinfo/python-list
Re: numpy (matrix solver) - python vs. matlab
On 04/30/2012 12:39 AM, Kiuhnm wrote: So Matlab at least warns about "Matrix is close to singular or badly scaled", which python (and I guess most other languages) does not... A is not just close to singular: it's singular! Ok. When do you define it to be singular, btw? Which is the most accurate/best, even for such a bad matrix? Is it possible to say something about that? Looks like python has a lot more digits but maybe that's just a random result... I mean Element 1,1 = 2.81e14 in Python, but something like 3e14 in Matlab and so forth - there's a small difference in the results... Both results are *wrong*: no inverse exists. What's the best solution of the two wrong ones? Best least-squares solution or whatever? With python, I would also kindly ask about how to avoid this problem in the future, I mean, this maybe means that I have to check the condition number at all times before doing anything at all ? How to do that? If cond(A) is high, you're trying to solve your problem the wrong way. So you're saying that in another language (python) I should check the condition number, before solving anything? You should try to avoid matrix inversion altogether if that's the case. For instance you shouldn't invert a matrix just to solve a linear system. What then? Cramer's rule? -- http://mail.python.org/mailman/listinfo/python-list
Re: numpy (matrix solver) - python vs. matlab
On 04/30/2012 02:38 AM, Nasser M. Abbasi wrote: On 04/29/2012 05:17 PM, someone wrote: I would also kindly ask about how to avoid this problem in the future, I mean, this maybe means that I have to check the condition number at all times before doing anything at all ? How to do that? I hope you'll check the condition number all the time. So how big can it (cond-number) be before I should do something else? And what to do then? Cramers rule or pseudoinverse or something else? You could be designing a building where people will live in it. If do not check the condition number, you'll end up with a building that will fall down when a small wind hits it and many people will die all because you did not bother to check the condition number when you solved the equations you used in your design. Also, as was said, do not use INV(A) directly to solve equations. In Matlab I used x=A\b. I used inv(A) in python. Should I use some kind of pseudo-inverse or what do you suggest? -- http://mail.python.org/mailman/listinfo/python-list
Re: numpy (matrix solver) - python vs. matlab
On 04/30/2012 02:57 AM, Paul Rubin wrote: someone writes: A is not just close to singular: it's singular! Ok. When do you define it to be singular, btw? Singular means the determinant is zero, i.e. the rows or columns are not linearly independent. Let's give names to the three rows: a = [1 2 3]; b = [11 12 13]; c = [21 22 23]. Then notice that c = 2*b - a. So c is linearly dependent on a and b. Geometrically this means the three vectors are in the same plane, so the matrix doesn't have an inverse. Oh, thak you very much for a good explanation. Which is the most accurate/best, even for such a bad matrix? What are you trying to do? If you are trying to calculate stuff with matrices, you really should know some basic linear algebra. Actually I know some... I just didn't think so much about, before writing the question this as I should, I know theres also something like singular value decomposition that I think can help solve otherwise illposed problems, although I'm not an expert like others in this forum, I know for sure :-) -- http://mail.python.org/mailman/listinfo/python-list
Re: numpy (matrix solver) - python vs. matlab
On 05/01/2012 08:56 AM, Russ P. wrote: On Apr 29, 5:17 pm, someone wrote: On 04/30/2012 12:39 AM, Kiuhnm wrote: You should try to avoid matrix inversion altogether if that's the case. For instance you shouldn't invert a matrix just to solve a linear system. What then? Cramer's rule? If you really want to know just about everything there is to know about a matrix, take a look at its Singular Value Decomposition (SVD). I know a bit about SVD - I used it for a short period of time in Matlab, though I'm definately not an expert in it and I don't understand the whole theory with orthogality behind making it work so elegant as it is/does work out. I've never used numpy, but I assume it can compute an SVD. I'm making my first steps now with numpy, so there's a lot I don't know and haven't tried with numpy... -- http://mail.python.org/mailman/listinfo/python-list
Re: numpy (matrix solver) - python vs. matlab
On 04/30/2012 03:35 AM, Nasser M. Abbasi wrote: On 04/29/2012 07:59 PM, someone wrote: I do not use python much myself, but a quick google showed that pyhton scipy has API for linalg, so use, which is from the documentation, the following code example X = scipy.linalg.solve(A, B) But you still need to check the cond(). If it is too large, not good. How large and all that, depends on the problem itself. But the rule of thumb, the lower the better. Less than 100 can be good in general, but I really can't give you a fixed number to use, as I am not an expert in this subjects, others who know more about it might have better recommendations. Ok, that's a number... Anyone wants to participate and do I hear something better than "less than 100 can be good in general" ? If I don't hear anything better, the limit is now 100... What's the limit in matlab (on the condition number of the matrices), by the way, before it comes up with a warning ??? -- http://mail.python.org/mailman/listinfo/python-list
Re: numpy (matrix solver) - python vs. matlab
On 05/01/2012 09:59 PM, Colin J. Williams wrote: On 01/05/2012 2:43 PM, someone wrote: [snip] a = [1 2 3]; b = [11 12 13]; c = [21 22 23]. Then notice that c = 2*b - a. So c is linearly dependent on a and b. Geometrically this means the three vectors are in the same plane, so the matrix doesn't have an inverse. Does it not mean that there are three parallel planes? Consider the example in two dimensional space. I actually drawed it and saw it... It means that you can construct a 2D plane and all 3 vectors are in this 2D-plane... -- http://mail.python.org/mailman/listinfo/python-list
Re: numpy (matrix solver) - python vs. matlab
On 05/01/2012 10:54 PM, Russ P. wrote: On May 1, 11:52 am, someone wrote: On 04/30/2012 03:35 AM, Nasser M. Abbasi wrote: What's the limit in matlab (on the condition number of the matrices), by the way, before it comes up with a warning ??? The threshold of acceptability really depends on the problem you are trying to solve. I haven't solved linear equations for a long time, but off hand, I would say that a condition number over 10 is questionable. Anyone knows the threshold for Matlab for warning when solving x=A\b ? I tried "edit slash" but this seems to be internal so I cannot see what criteria the warning is based upon... A high condition number suggests that the selection of independent variables for the linear function you are trying to fit is not quite right. For a poorly conditioned matrix, your modeling function will be very sensitive to measurement noise and other sources of error, if applicable. If the condition number is 100, then any input on one particular axis gets magnified 100 times more than other inputs. Unless your inputs are very precise, that is probably not what you want. Or something like that. Ok. So it's like a frequency-response-function, output divided by input... -- http://mail.python.org/mailman/listinfo/python-list
Re: numpy (matrix solver) - python vs. matlab
On 05/02/2012 01:05 AM, Paul Rubin wrote: someone writes: Actually I know some... I just didn't think so much about, before writing the question this as I should, I know theres also something like singular value decomposition that I think can help solve otherwise illposed problems, You will probably get better advice if you are able to describe what problem (ill-posed or otherwise) you are actually trying to solve. SVD I don't understand what else I should write. I gave the singular matrix and that's it. Nothing more is to say about this problem, except it would be nice to learn some things for future use (for instance understanding SVD more - perhaps someone geometrically can explain SVD, that'll be really nice, I hope)... just separates out the orthogonal and scaling parts of the transformation induced by a matrix. Whether that is of any use to you is unclear since you don't say what you're trying to do. Still, I dont think I completely understand SVD. SVD (at least in Matlab) returns 3 matrices, one is a diagonal matrix I think. I think I would better understand it with geometric examples, if one would be so kind to maybe write something about that... I can plot 3D vectors in matlab, very easily so maybe I better understand SVD if I hear/read the geometric explanation (references to textbook/similar is also appreciated). -- http://mail.python.org/mailman/listinfo/python-list
Re: numpy (matrix solver) - python vs. matlab
On 05/02/2012 01:38 AM, Russ P. wrote: On May 1, 4:05 pm, Paul Rubin wrote: someone writes: Actually I know some... I just didn't think so much about, before writing the question this as I should, I know theres also something like singular value decomposition that I think can help solve otherwise illposed problems, You will probably get better advice if you are able to describe what problem (ill-posed or otherwise) you are actually trying to solve. SVD just separates out the orthogonal and scaling parts of the transformation induced by a matrix. Whether that is of any use to you is unclear since you don't say what you're trying to do. I agree with the first sentence, but I take slight issue with the word "just" in the second. The "orthogonal" part of the transformation is non-distorting, but the "scaling" part essentially distorts the space. At least that's how I think about it. The larger the ratio between the largest and smallest singular value, the more distortion there is. SVD may or may not be the best choice for the final algorithm, but it is useful for visualizing the transformation you are applying. It can provide clues about the quality of the selection of independent variables, state variables, or inputs. Me would like to hear more! :-) It would really appreciate if anyone could maybe post a simple SVD example and tell what the vectors from the SVD represents geometrically / visually, because I don't understand it good enough and I'm sure it's very important, when it comes to solving matrix systems... -- http://mail.python.org/mailman/listinfo/python-list
Re: numpy (matrix solver) - python vs. matlab
On 05/02/2012 01:03 PM, Kiuhnm wrote: On 5/2/2012 8:00, someone wrote: Still, I dont think I completely understand SVD. SVD (at least in Matlab) returns 3 matrices, one is a diagonal matrix I think. I think I would better understand it with geometric examples, if one would be so kind to maybe write something about that... I can plot 3D vectors in matlab, very easily so maybe I better understand SVD if I hear/read the geometric explanation (references to textbook/similar is also appreciated). Russ's post is a very good starting point. I hope you read it. Ofcourse I do. -- http://mail.python.org/mailman/listinfo/python-list
Re: numpy (matrix solver) - python vs. matlab
On 05/02/2012 08:36 AM, Russ P. wrote: On May 1, 11:03 pm, someone wrote: On 05/02/2012 01:38 AM, Russ P. wrote: .. On May 1, 4:05 pm, Paul Rubinwrote: It would really appreciate if anyone could maybe post a simple SVD example and tell what the vectors from the SVD represents geometrically / visually, because I don't understand it good enough and I'm sure it's very important, when it comes to solving matrix systems... SVD is perhaps the ultimate matrix decomposition and the ultimate tool for linear analysis. Google it and take a look at the excellent Wikipedia page on it. I would be wasting my time if I tried to compete with that. Ok. To really appreciate the SVD, you need some background in linear algebra. In particular, you need to understand orthogonal transformations. Think about a standard 3D Cartesian coordinate frame. A rotation of the coordinate frame is an orthogonal transformation of coordinates. The original frame and the new frame are both orthogonal. Yep. A vector in one frame is converted to the other frame by multiplying by an orthogonal matrix. The main feature of an orthogonal matrix is that its transpose is its inverse (hence the inverse is trivial to compute). As far as i know, you have to replace orthogonal with: orthonormal. That much I at least think I know without even going to wikipedia first... The SVD can be thought of as factoring any linear transformation into a rotation, then a scaling, followed by another rotation. The scaling is represented by the middle matrix of the transformation, which is a diagonal matrix of the same dimensions as the original matrix. The singular values can be read off of the diagonal. If any of them are zero, then the original matrix is singular. If the ratio of the largest to smallest singular value is large, then the original matrix is said to be poorly conditioned. Aah, thank you very much. I can easily recognize some of this... Standard Cartesian coordinate frames are orthogonal. Imagine an x-y coordinate frame in which the axes are not orthogonal. Such a coordinate frame is possible, but they are rarely used. If the axes are parallel, the coordinate frame will be singular and will basically reduce to one-dimensional. If the x and y axes are nearly parallel, the coordinate frame could still be used in theory, but it will be poorly conditioned. You will need large numbers to represent points fairly close to the origin, and small deviations will translate into large changes in coordinate values. That can lead to problems due to numerical roundoff errors and other kinds of errors. Thank you very much for your time. It always helps to get the same explanation from different people with slightly different ways of explaining it. Thanks! -- http://mail.python.org/mailman/listinfo/python-list
Re: numpy (matrix solver) - python vs. matlab
On 05/02/2012 01:52 PM, Steven D'Aprano wrote: On Wed, 02 May 2012 08:00:44 +0200, someone wrote: On 05/02/2012 01:05 AM, Paul Rubin wrote: someone writes: Actually I know some... I just didn't think so much about, before writing the question this as I should, I know theres also something like singular value decomposition that I think can help solve otherwise illposed problems, You will probably get better advice if you are able to describe what problem (ill-posed or otherwise) you are actually trying to solve. SVD I don't understand what else I should write. I gave the singular matrix and that's it. You can't judge what an acceptable condition number is unless you know what your data is. http://mathworld.wolfram.com/ConditionNumber.html http://en.wikipedia.org/wiki/Condition_number If your condition number is ten, then you should expect to lose one digit of accuracy in your solution, over and above whatever loss of accuracy comes from the numeric algorithm. A condition number of 64 will lose six bits, or about 1.8 decimal digits, of accuracy. If your data starts off with only 1 or 2 digits of accuracy, as in your example, then the result is meaningless -- the accuracy will be 2-2 digits, or 0 -- *no* digits in the answer can be trusted to be accurate. I just solved a FEM eigenvalue problem where the condition number of the mass and stiffness matrices was something like 1e6... Result looked good to me... So I don't understand what you're saying about 10 = 1 or 2 digits. I think my problem was accurate enough, though I don't know what error with 1e6 in condition number, I should expect. How did you arrive at 1 or 2 digits for cond(A)=10, if I may ask ? -- http://mail.python.org/mailman/listinfo/python-list
Re: numpy (matrix solver) - python vs. matlab
On 05/02/2012 04:47 PM, Steven_Lord wrote: Russ, you and the OP (and others) may be interested in one of the books that Cleve Moler has written and made freely available on the MathWorks website: http://www.mathworks.com/moler/ The chapter Linear Equations in "Numerical Computing with MATLAB" includes a section (section 2.9, starting on page 14 if I remember correctly) that discusses norm and condition number and gives a more formal statement of what you described. The code included in the book is written in MATLAB, but even if you don't use MATLAB (since I know this has been cross-posted to comp.lang.python) there's plenty of good, crunchy mathematics in that section. I use matlab more than python. I just want to learn python, which seems extremely powerful and easy but yet, I'm a python beginner. Thank you very much for the reference, Mr. Lord. I'll look closely at that Moler-book, in about 1/2 hour or so Looking forward to learning more about this... -- http://mail.python.org/mailman/listinfo/python-list
Re: numpy (matrix solver) - python vs. matlab
On 05/02/2012 11:45 PM, Russ P. wrote: On May 2, 1:29 pm, someone wrote: If your data starts off with only 1 or 2 digits of accuracy, as in your example, then the result is meaningless -- the accuracy will be 2-2 digits, or 0 -- *no* digits in the answer can be trusted to be accurate. I just solved a FEM eigenvalue problem where the condition number of the mass and stiffness matrices was something like 1e6... Result looked good to me... So I don't understand what you're saying about 10 = 1 or 2 digits. I think my problem was accurate enough, though I don't know what error with 1e6 in condition number, I should expect. How did you arrive at 1 or 2 digits for cond(A)=10, if I may ask ? As Steven pointed out earlier, it all depends on the precision you are dealing with. If you are just doing pure mathematical or numerical work with no real-world measurement error, then a condition number of 1e6 may be fine. But you had better be using "double precision" (64- bit) floating point numbers (which are the default in Python, of course). Those have approximately 12 digits of precision, so you are in good shape. Single-precision floats only have 6 or 7 digits of precision, so you'd be in trouble there. For any practical engineering or scientific work, I'd say that a condition number of 1e6 is very likely to be completely unacceptable. So how do you explain that the natural frequencies from FEM (with condition number ~1e6) generally correlates really good with real measurements (within approx. 5%), at least for the first 3-4 natural frequencies? I would say that the problem lies with the highest natural frequencies, they for sure cannot be verified - there's too little energy in them. But the lowest frequencies (the most important ones) are good, I think - even for high cond number. -- http://mail.python.org/mailman/listinfo/python-list
Re: numpy (matrix solver) - python vs. matlab
On 05/03/2012 07:55 PM, Russ P. wrote: On May 3, 10:30 am, someone wrote: On 05/02/2012 11:45 PM, Russ P. wrote: For any practical engineering or scientific work, I'd say that a condition number of 1e6 is very likely to be completely unacceptable. So how do you explain that the natural frequencies from FEM (with condition number ~1e6) generally correlates really good with real measurements (within approx. 5%), at least for the first 3-4 natural frequencies? I would say that the problem lies with the highest natural frequencies, they for sure cannot be verified - there's too little energy in them. But the lowest frequencies (the most important ones) are good, I think - even for high cond number. Did you mention earlier what "FEM" stands for? If so, I missed it. Is it finite-element modeling? Whatever the case, note that I said, "If Sorry, yes: Finite Element Model. you are just doing pure mathematical or numerical work with no real- world measurement error, then a condition number of 1e6 may be fine." I forgot much more than I know about finite-element modeling, but isn't it a purely numerical method of analysis? If that I'm not sure exactly, what is the definition of a purely numerical method of analysis? I would guess that the answer is yes, it's a purely numerical method? But I also thing it's a practical engineering or scientific work... is the case, then my comment above is relevant. Uh, I just don't understand the difference: 1) "For any practical engineering or scientific work, I'd say that a condition number of 1e6 is very likely to be completely unacceptable." vs. 2) "If you are just doing pure mathematical or numerical work with no real-world measurement error, then a condition number of, 1e6 may be fine." I would think that FEM is a practical engineering work and also pure numerical work... Or something... By the way, I didn't mean to patronize you with my earlier explanation of orthogonal transformations. They are fundamental to understanding the SVD, and I thought it might be interesting to anyone who is not familiar with the concept. Don't worry, I think it was really good and I don't think anyone patronized me, on the contrary, people was/is very helpful. SVD isn't my strongest side and maybe I should've thought a bit more about this singular matrix and perhaps realized what some people here already explained, a bit earlier (maybe before I asked). Anyway, it's been good to hear/read what you've (and others) have written. Yesterday and earlier today I was at work during the day so answering/replying took a bit longer than I like, considering the huge flow of posts in the matlab group. But now I'm home most of the time, for the next 3 days and will check for followup posts quite frequent, I think... -- http://mail.python.org/mailman/listinfo/python-list
Re: numpy (matrix solver) - python vs. matlab
On 05/04/2012 12:58 AM, Russ P. wrote: Yeah, I realized that I should rephrase my previous statement to something like this: For any *empirical* engineering or scientific work, I'd say that a condition number of 1e6 is likely to be unacceptable. Still, I don't understand it. Do you have an example of this kind of work, if it's not FEM? I'd put finite elements into the category of theoretical and numerical rather than empirical. Still, a condition number of 1e6 would bother me, but maybe that's just me. Ok, but I just don't understand what's in the "empirical" category, sorry... Maybe the conclusion is just that if cond(A) > 1e15 or 1e16, then that problem shouldn't be solved and maybe this is also approx. where matlab has it's warning-threshold (maybe, I'm just guessing here)... So, maybe I could perhaps use that limit in my future python program (when I find out how to get the condition number etc, but I assume this can be googled for with no problems)... -- http://mail.python.org/mailman/listinfo/python-list
Re: numpy (matrix solver) - python vs. matlab
On 05/04/2012 05:52 AM, Steven D'Aprano wrote:
On Thu, 03 May 2012 19:30:35 +0200, someone wrote:
So how do you explain that the natural frequencies from FEM (with
condition number ~1e6) generally correlates really good with real
measurements (within approx. 5%), at least for the first 3-4 natural
frequencies?
I would counter your hand-waving ("correlates really good", "within
approx 5%" of *what*?) with hand-waving of my own:
Within 5% of experiments of course.
There is not much else to compare with.
"Sure, that's exactly what I would expect!"
*wink*
By the way, if I didn't say so earlier, I'll say so now: the
interpretation of "how bad the condition number is" will depend on the
underlying physics and/or mathematics of the situation. The
interpretation of loss of digits of precision is a general rule of thumb
that holds in many diverse situations, not a rule of physics that cannot
be broken in this universe.
If you have found a scenario where another interpretation of condition
number applies, good for you. That doesn't change the fact that, under
normal circumstances when trying to solve systems of linear equations, a
condition number of 1e6 is likely to blow away *all* the accuracy in your
measured data. (Very few physical measurements are accurate to more than
six digits.)
Not true, IMHO.
Eigenfrequencies (I think that is a very typical physical measurement
and I cannot think of something that is more typical) don't need to be
accurate with 6 digits. I'm happy with below 5% error. So if an
eigenfrequency is measured to 100 Hz, I'm happy if the numerical model
gives a result in the 5%-range of 95-105 Hz. This I got with a condition
number of approx. 1e6 and it's good enough for me. I don't think anyone
expects 6-digit accuracy with eigenfrequncies.
--
http://mail.python.org/mailman/listinfo/python-list
Re: numpy (matrix solver) - python vs. matlab
On 05/04/2012 06:15 AM, Russ P. wrote: On May 3, 4:59 pm, someone wrote: On 05/04/2012 12:58 AM, Russ P. wrote: Ok, but I just don't understand what's in the "empirical" category, sorry... I didn't look it up, but as far as I know, empirical just means based on experiment, which means based on measured data. Unless I am FEM based on measurement data? Still, I don't understand it, sorry. mistaken , a finite element analysis is not based on measured data. I'm probably a bit narrow-thinking because I just worked with this small FEM-program (in Matlab), but can you please give an example of a matrix-problem that is based on measurement data? Yes, the results can be *compared* with measured data and perhaps calibrated with measured data, but those are not the same thing. Exactly. That's why I don't understand what solving a matrix system using measurement/empirical data, could typically be an example of...? I agree with Steven D's comment above, and I will reiterate that a condition number of 1e6 would not inspire confidence in me. If I had a condition number like that, I would look for a better model. But that's just a gut reaction, not a hard scientific rule. I don't have any better model and don't know anything better. I still think that 5% accuracy is good enough and that nobody needs 6-digits precision for practical/engineering/empirical work... Maybe quantum physicists needs more than 6 digits of accuracy, but most practical/engineering problems are ok with an accuracy of 5%, I think, IMHO... Please tell me if I'm wrong. -- http://mail.python.org/mailman/listinfo/python-list
calling python (matplotlib) from c++
Hi all,
I want to call python (matplotlib) from C++. I use ubuntu linux and
compile with "-lpython2.7". Besides that, I have this line included:
#include
I can then do something like this:
Py_Initialize();
PyRun_SimpleString("print ' '");
PyRun_SimpleString("print ' '");
PyRun_SimpleString("print ' '");
PyRun_SimpleString("print 'PYTHON RUNNING =='");
PyRun_SimpleString("from time import time,ctime\n"
"print 'Today is',ctime(time())\n");
// Py_Main(argc, argv); // start up the python interpreter
Py_Finalize();
However - I cannot seem to pass two arrays of data, so I can plot the
data using matplotlib... I cannot even start matplotlib... Please
help/advice/come up with suggestions, someone who knows what to do :-)
--
http://mail.python.org/mailman/listinfo/python-list
python-noob - which container is appropriate for later exporting into mySql + matplotlib ?
Hi, Here's my data: --- 20130315T071500 39000. 10 26 48000. 1 40 20130315T071501 39000. 10 26 48000. 2 42 20130315T071501 39000. 10 26 47520. 15 69 20130315T071501 39000. 10 26 47160. 1 70 20130315T071501 39000. 10 26 47000. 1 72 20130315T071501 39000. 10 26 47000. 2 81 20130315T071501 39000. 10 26 47000. 6 85 20130315T071501 39000. 10 26 46520. 10 95 20130315T071501 43000. 10 36 46520. 10 95 20130315T071501 43200. 4 43 46520. 10 104 20130315T071501 44040. 1 45 46520. 10 108 20130315T071501 44080. 3 48 46520. 10 109 20130315T071501 44080. 3 48 46520. 11 113 20130315T071501 44080. 3 48 46400. 2 131 20130315T071501 45080. 1 51 46400. 2 145 20130315T071501 45080. 1 51 46200. 1 147 20130315T071501 45080. 1 60 46120. 1 182 20130315T071501 45520. 1 65 46120. 1 225 20130315T071501 45520. 1 73 46120. 2 247 20130315T08 45760. 1 133 46120. 2 378 20130315T080241 45760. 2 199 46120. 2 453 20130315T080945 45760. 3 217 46120. 2 456 20130315T081103 45760. 3 217 46080. 1 457 20130315T081105 45760. 3 218 46080. 2 458 20130315T081106 45760. 4 222 46080. 2 458 20130315T081107 45800. 1 229 46080. 2 458 20130315T082754 45800. 8 266 46080. 2 514 . ... etc. --- The first column is date + time. I know how to use simple tuples, list's and dict's. But I don't think it's a good idea to make each line an element in a list/dict ? I want to put this table into an appropriate container such that afterwards I want to: 1) Put the data into a mySql-table 2) Be able to easily plot column 1 vs. either of the other columns using matplotlib etc... I consider myself a python-newbie so I'm not that good with containers... I found something here: http://docs.python.org/dev/library/collections.html but I'm not really sure what is a good pythonic way of storing this data? I also think that maybe I ought to convert the first data+time column into a single number, because right now it's a string (because there's a "T" in the middle of column 1)... Letting the date+time-column (=column 1) become entirely numbers, makes it easier to plot in matplotlib afterwards... I'm not sure how to store this in an efficient manner... What (=how?) would you do it? Thanks... -- http://mail.python.org/mailman/listinfo/python-list
Re: python-noob - which container is appropriate for later exporting into mySql + matplotlib ?
On 2013-04-11 03:39, Cousin Stanley wrote:
for row in list_tuples :
print ' ' , row.date , row.time , row.col1 , row.col3 , row.col4
file_source.close()
Oh, that's great - thank you - I didn't know this named-tuple container
before... I'm still wondering whether or not it's the optimal container
type for me, because I just added a bit of matplotlib-code:
-
#!/usr/bin/env python
import csv
from collections import namedtuple as NT
file_source = open( 'someone.csv' )
# ---> individual column names ---
nt = NT( 'csv_data' , 'date time col1 col2 col3 col4 col5 col6' )
list_tuples = [ ]
for this_row in csv.reader( file_source ) :
# unpack the current row
zed , one , two , tre , fur , fiv , six = this_row
# split the date and time
d , t = zed.split( 'T' )
# convert individual columns in row to a named tuple
this_tuple = nt( d ,
t ,
float( one ) ,
int( two ) ,
int( tre ) ,
float( fur ) ,
int( fiv ) ,
int( six ) )
# save the current named tuple into a list
list_tuples.append( this_tuple )
# update_data_base( this_tuple )
# or
# update_data_base( choose individual columns )
file_source.close()
# individual elements of the named tuples
# can be accessed by name
#
# this might be convenient for settup up
# data for plots of diffeent columns
x=[]
y=[]
print
for row in list_tuples :
print ' ' , row.date , row.time , row.col1 , row.col3 , row.col4
x.append(row.col3)
y.append(row.col4)
import matplotlib.pyplot as plt
plt.plot(x,y)
plt.ylabel('some numbers')
plt.show()
-
As you can see, in order for me to make the x- and y-vectors, I need to
make a for-loop to access the individual rows in list_tuples and then I
append to the x- and y- lists...
Is there any clever way of avoiding this for loop, for either this
container or another clever container type?
If there isn't, then this is absolutely also an acceptable/good solution
for me... I also use Matlab and for matrices you can type e.g. plot(
matrix(:,3), matrix(:,4) ) to plot columns 3 against column 4. But
Matlab also has this problem, that it cannot store strings and numbers
in the same matrix - matrices must entirely be numeric, which my data
isn't (due to 1st column)...
Thanks for any input, if someone has any good ideas...
--
http://mail.python.org/mailman/listinfo/python-list
Re: python-noob - which container is appropriate for later exporting into mySql + matplotlib ?
On 2013-04-11 10:49, someone wrote: On 2013-04-11 03:39, Cousin Stanley wrote: Is there any clever way of avoiding this for loop, for either this container or another clever container type? Ah, I see - I can also just add a numpy array, i.e: -- import matplotlib.pyplot as plt test=numpy.random.rand(8,2) new_tuple = nt(d,t, float(one), int(two), int(tre), float(fur), int(fiv), test) #new_tuple is now: #csv_data(date='20130315', time='071500', col1=39000.0, col2=10, #col3=26, col4=48000.0, col5=1, col6=array([[ 0.77714064, 0.06729907], # [ 0.20418563, 0.97882722], # [ 0.39130897, 0.06611205], # [ 0.94938335, 0.50254674], # [ 0.82047434, 0.71624034], # [ 0.66618477, 0.92025612], # [ 0.2789423 , 0.19212809], # [ 0.7048946 , 0.79112071]])) x=new_tuple.col6[:,0] y=new_tuple.col6[:,1] plt.plot(x,y) plt.show() I get it - THANKS! -- http://mail.python.org/mailman/listinfo/python-list
Re: python-noob - which container is appropriate for later exporting into mySql + matplotlib ?
On 04/11/2013 07:58 PM, Cousin Stanley wrote: someone wrote: You can be creative with the data selections and pass them off to be plotted as needed If mysql is used instead of sqlite3 you should only have to monkey with the data type declarations in xdata_create.sql and the dbc.connect strings in the python code Uh, thank you very much for providing me with this (+ also the example in the other post)! Unfortunately, I'm struggling a bit with my code (I'm making some python-class'es), so it'll take a few days before I begin on the SQL-stuff... I'll get back, if the SQL-code you suggested causes any problems - thank you VERY much for both examples (in both posts)... I'll try it out ASAP, when I've made my code object-oriented and well-organized :-) Thanks! -- http://mail.python.org/mailman/listinfo/python-list
Re: python-noob - which container is appropriate for later exporting into mySql + matplotlib ?
On 2013-04-11 19:58, Cousin Stanley wrote:
someone wrote:
I want to put this table into an appropriate container
such that afterwards I want to:
1) Put the data into a mySql-table
You might consider using sqlite3 as a database manager
since it is "batteries included" with python
The stand-alone sqlite interpreter can first be used
to create an empty database named some.sql3
and create a table named xdata in that data base
sqlite3 some.sql3 '.read xdata_create.sql'
where the file xdata_create.sql contains
create table xdata
(
xdate integer ,
xtime integer ,
col1 real ,
col2 integer ,
col3 integer ,
col4 real ,
col5 integer ,
col6 integer
) ;
Oh, thank you very much! Now I understand this (I haven't really worked
much with sql before, so this was/is new to me, thanks!).
The csv data file can then be inserted into the xdata table
in the some.sql3 database via python
and ...
# python data selection example
# for column 4 between 8 and 9
I combined both code snippets into:
==
#!/usr/bin/python
import sqlite3 as DBM
import ipdb
fs = open( 'some.csv' )
ls = [ ]
dbc = DBM.connect( 'some.sql3' )
cur = dbc.cursor()
if 0:
sql = 'insert into xdata values( ? , ? , ? , ? , ? , ? , ? , ? )'
for row in fs :
dt, col1, col2, col3, col4,col5, col6 = row.strip().split(',' )
xdate , xtime = dt.split( 'T' )
xtuple = ( xdate, xtime, col1, col2, col3, col4, col5, col6 )
cur.execute( sql , xtuple )
dbc.commit()
else:
list_sql = [
'select xtime , col4' ,
'from xdata' ,
'where xtime >= 8 and xtime <= 9 ; ' ]
str_sql = '\n'.join( list_sql )
cur.execute( str_sql )
for row in cur :
#ipdb.set_trace()
# I get: TypeError: "tuple indices must be integers, not str"
# "ipdb> row" says: "(8, 46120.0)"
#print row[ 'xtime' ] , row[ 'col4' ]
print row[0] , row[1]
fs.close()
dbc.close()
==
I don't fully understand it yet, but it's nice to see that it works!
Thank you very much for that! Now I'll have to concentrate on figuring
out how/why it works :-)
You can be creative with the data selections
and pass them off to be plotted as needed
Yes, I understand. Thank you very much. As you can see, on my system I
had to use:
print row[0] , row[1]
instead of:
print row[ 'xtime' ] , row[ 'col4' ]
I'm not sure exactly why - is it because you have another version of
sqlite3 ? This is a bit strange, but anyway I can live with that - at
least for now...
If mysql is used instead of sqlite3
you should only have to monkey with
the data type declarations in xdata_create.sql
and the dbc.connect strings in the python code
Actually I didn't knew anything about this sqlite3 before now. This is
the first time I try it out, so I don't really know what's the
difference between sqlite3 and mysql...
But thank you very much for providing some code I can now study and
learn from !!! Much appreciated
--
http://mail.python.org/mailman/listinfo/python-list
Re: python-noob - which container is appropriate for later exporting into mySql + matplotlib ?
On 2013-04-11 20:44, Cousin Stanley wrote:
Cousin Stanley wrote:
The stand-alone sqlite interpreter can first be used
to create an empty database named some.sql3
and create a table named xdata in that data base
sqlite3 some.sql3 '.read xdata_create.sql'
This step can also be done in python
without using the stand-alone sqlite interpreter
Ah, that's great (and even better so I don't have to create the
xdata_create.sql file) - thank you!
I collected what you wrote and put together this script:
=
#!/usr/bin/python
import sqlite3 as DBM
import ipdb
# ls = [ ] # this seems to be un-used ?
dbc = DBM.connect( 'some.sql3' )
cur = dbc.cursor()
fs = open( 'some.csv' )
if 0: # select whether to create new database file or query from it?
if 0: # switch between "create table xdata" and "... if not exists"
list_sql = [
'create table xdata ',
'( ',
' xdate integer , ',
' xtime integer , ',
' col1 real , ',
' col2 integer , ',
' col3 integer , ',
' col4 real , ',
' col5 integer , ',
' col6 integer ',
') ;' ]
else:
list_sql = [
'create table if not exists xdata ' ,
'( ' ,
' xdate integer , ' ,
' xtime integer , ' ,
' col1 real, ' ,
' col2 integer , ' ,
' col3 integer , ' ,
' col4 real, ' ,
' col5 integer , ' ,
' col6 integer ' ,
') ; ' ]
# -
str_sql = '\n'.join( list_sql )
cur.execute( str_sql )
# -
# Insert data from input file fs ("some.csv")
sql = 'insert into xdata values( ? , ? , ? , ? , ? , ? , ? , ? )'
for row in fs :
dt, col1, col2, col3, col4,col5, col6 = row.strip().split(',' )
xdate , xtime = dt.split( 'T' )
xtuple = ( xdate, xtime, col1, col2, col3, col4, col5, col6 )
cur.execute( sql , xtuple )
dbc.commit()
else:
list_sql = [
'select xtime , col4' ,
'from xdata' ,
'where xtime >= 8 and xtime <= 81104 ; ' ]
str_sql = '\n'.join( list_sql )
cur.execute( str_sql )
for row in cur :
#ipdb.set_trace()
# I get: TypeError: "tuple indices must be integers, not str"
# "ipdb> row" says: "(8, 46120.0)"
#print row[ 'xtime' ] , row[ 'col4' ]
print row[0] , row[1]
fs.close()
dbc.close()
=
I think I can learn a lot from google, based on this code - I'm very
grateful for your help!
Now I just need to make a nice interface and couple it to matplotlib, so
it's easy to plot - I think there's a good chance that I can come up
with a good solution from here, based on the help I got from you people...
Thanks again!
--
http://mail.python.org/mailman/listinfo/python-list
Re: python-noob - which container is appropriate for later exporting into mySql + matplotlib ?
On 04/12/2013 06:58 PM, Cousin Stanley wrote: someone wrote: As you can see, on my system I had to use: print row[0] , row[1] instead of: print row[ 'xtime' ] , row[ 'col4' ] I'm not sure exactly why The magic there is setting up the row_factory after the database connection dbc = DBM.connect( 'some.sql3' ) dbc.row_factory = DBM.Row Ah, thanks a lot - now it works! This is much more "user-friendly"... I don't really know what's the difference between sqlite3 and mysql... MySQL is used through a client/server system where the db server is always running and client processes submit requests to it in the form of sql statements SQLite is used as a stand-alone single process with no external server involved Ok, I see... So SQLite is very good for "practicing"... I'll remember that, thank you. Both speak sql but there are some differences mostly in data base connection strings and data type declarations Basic sql selection is select these fields from these files where these conditions are met And that part of sql doesn't vary much among different data base managers Great, thank you very much... Looks like everything is on track now... I just have to sit and play with it and make a good interface with matplotlib, but I think I should be able to come up with something nice, based on the help I god in this thread... Thanks again... I just love this python language - makes it possible to do so much, in so little time and without being an expert at all... -- http://mail.python.org/mailman/listinfo/python-list
Re: python-noob - which container is appropriate for later exporting into mySql + matplotlib ?
On 04/13/2013 01:26 AM, Cousin Stanley wrote: someone wrote: So SQLite is very good for "practicing" Yes it is but it is also very good for much more than just practice Check the wikipedia info http://en.wikipedia.org/wiki/Sqlite Very interesting... "It is arguably the most widely deployed database engine, as it is used today by several widespread browsers, operating systems, and embedded systems, among others" The firefox browser keeps different sqlite database files for various uses I should remember to use this in the future for my small apps... If you use firefox check its default directory and you will see several files with .sqlite file type extensions Under debian debian linux ~/.mozilla/firefox/*.default You're right: /home/myUser/.mozilla/firefox/pv079lxv.default/addons.sqlite /home/myUser/.mozilla/firefox/pv079lxv.default/chromeappsstore.sqlite /home/myUser/.mozilla/firefox/pv079lxv.default/content-prefs.sqlite /home/myUser/.mozilla/firefox/pv079lxv.default/cookies.sqlite /home/myUser/.mozilla/firefox/pv079lxv.default/downloads.sqlite /home/myUser/.mozilla/firefox/pv079lxv.default/extensions.sqlite /home/myUser/.mozilla/firefox/pv079lxv.default/formhistory.sqlite /home/myUser/.mozilla/firefox/pv079lxv.default/permissions.sqlite /home/myUser/.mozilla/firefox/pv079lxv.default/places.sqlite /home/myUser/.mozilla/firefox/pv079lxv.default/search.sqlite /home/myUser/.mozilla/firefox/pv079lxv.default/signons.sqlite /home/myUser/.mozilla/firefox/pv079lxv.default/urlclassifier3.sqlite /home/myUser/.mozilla/firefox/pv079lxv.default/webappsstore.sqlite Very interesting, I didn't knew that :-) Many programmers, including pythonistas, use sqlite for a convenient and persistent data store where data can be stashed now and used later in many different ways through the diversity of sql selections I'll try to do this in the future also... I just have to practice a bit more with the SQL commands, but now I can create, update, delete, query and I guess that's the most important things to know :-) Thanks for your help and for providing interesting background info :-) -- http://mail.python.org/mailman/listinfo/python-list
Re: python-noob - which container is appropriate for later exporting into mySql + matplotlib ?
On 04/13/2013 03:44 AM, Steven D'Aprano wrote: On Fri, 12 Apr 2013 23:26:05 +, Cousin Stanley wrote: The firefox browser keeps different sqlite database files for various uses Yes, and I *really* wish they wouldn't. It's my number 1 cause of major problems with Firefox. E.g. http://kb.mozillazine.org/Bookmarks_history_and_toolbar_buttons_not_working_-_Firefox Oh, sorry to hear that... Actually I mostly use chromium (but I also have firefox installed). I just opened a few of my sqlite files from within sqlite3 - that was very interesting to see the contents of these files without being anything like a "hacker"... :-) Using a database for such lightweight data as bookmarks is, in my opinion, gross overkill and adds to the complexity of Firefox. More complexity leads to more bugs, e.g.: https://bugzilla.mozilla.org/show_bug.cgi?id=465684#c11 https://bugzilla.mozilla.org/show_bug.cgi?id=431558 On the other hand, I guess it's in the spirit of "open source" that it's easy for everyone to go in an see what's in the configuration files and (if one wants) modify and/or make own improvements/programs that tamper with these sql-files ? Please don't use a full-featured database if you don't need the overhead Ok, you're saying there's overhead I should think of... Most of my programs are rather small in comparison with commercial programs so I think I don't have to worry about overhead (I don't have any real speed-critical applications). of ACID compliance. And if you do, well, Sqlite is not fully ACID compliant. I just had to google what ACID compliance means and accordingly to this: http://en.wikipedia.org/wiki/SQLite "SQLite is ACID-compliant and implements most of the SQL standard, using a dynamically and weakly typed SQL syntax that does not guarantee the domain integrity." So you seem to disagree with wikipedia? I however don't understand what it means "to not guarantee domain integrity"... As I read this, I get the feeling that sqlite *IS* ACID compliant (wikipedia however doesn't use the wording: "fully ACID compliant", maybe this is the culprit) ? -- http://mail.python.org/mailman/listinfo/python-list
Re: python-noob - which container is appropriate for later exporting into mySql + matplotlib ?
On 04/13/2013 01:39 PM, Chris Angelico wrote: On Sat, Apr 13, 2013 at 9:08 PM, someone wrote: I just had to google what ACID compliance means and accordingly to this: http://en.wikipedia.org/wiki/SQLite "SQLite is ACID-compliant and implements most of the SQL standard, using a dynamically and weakly typed SQL syntax that does not guarantee the domain integrity." So you seem to disagree with wikipedia? Disagreeing with Wikipedia doesn't mean much, but try this: http://www.sqlite.org/atomiccommit.html Ok, thanks - I didn't read it all, but interesting. Note that there's a caveat: You have to tell SQLite to be ACID compliant, effectively. So, you're saying to me that by default SQLite isn't ACID compliant, if I begin to use it in my own small programs? I don't know so much about it - maybe it's a matter of definition... If I just google for the 3 words: "sqlite acid compliance" I get: Hit no. 1 is wikipedia. Hit no. 3 says: "SQLite is an ACID-compliant embedded relational database management system" Hit no. 4 says: "SQLite implements ACID-compliance by way of a transaction journal" Hit no. 5 says: "SQLite transactions are fully ACID-compliant, allowing safe access from.." Hit no. 6 says: "Techopedia explains SQLite. SQLite is atomicity, consistency, isolation, durability (ACID) compliant." Hit no. 7: "Tell me what you know about SQLite, the ACID-compliant embedded relational" Hit no. 9: "SQLite is superior to Jet for the major reason that SQLite is ACID-compliant whereas Jet, unfortunately, isn't..." Hit no. 10: "SQLite for Linux 3.6.17. An ACID-compliant relational database management system" I think maybe being it's a question of definitions, i.e. "well, Sqlite is not fully ACID compliant" vs. all the google hits that just tells that sqlite is "ACID compliant"... Do I understand you correct, that by "You have to tell SQLite to be ACID compliant, effectively", you're saying that by default SQLite isn't ACID compliant ? Next question: Is it something I should worry about in my own programs (I'm not sure, I'm an SQL noob)... ? Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: python-noob - which container is appropriate for later exporting into mySql + matplotlib ?
On 04/13/2013 04:03 PM, Chris Angelico wrote: On Sat, Apr 13, 2013 at 11:30 PM, someone wrote: On 04/13/2013 01:39 PM, Chris Angelico wrote: Note that there's a caveat: You have to tell SQLite to be ACID compliant, effectively. So, you're saying to me that by default SQLite isn't ACID compliant, if I begin to use it in my own small programs? ... Do I understand you correct, that by "You have to tell SQLite to be ACID compliant, effectively", you're saying that by default SQLite isn't ACID compliant ? First off: I am NOT inherently familiar with sqlite. I'm more familiar with PostgreSQL, DB2, and MySQL. I'm also not an expert at database engine design, so this discussion is from the point of view of an applications developer who has used databases from his apps. Ok, would be nice to hear the opinion from an sqlite expert then... True ACID compliance demands support at every level: 1) The application has to operate in logical units of work, which - apart from with DB2 - requires an explicit "BEGIN" query, or single-statement transactions. 2) The database engine must employ some form of write-ahead log. Different databases do this somewhat differently (according to the page I linked to, SQLite does this in reverse, maintaining a log that's sufficient to *undo* the transaction, while PostgreSQL does this forwards, maintaining a log that's sufficient to *redo* it as well - more effort, but it can be used for database replication), but one way or another, there must be a way to detect half-done transactions. 3) The operating system and filesystem must support a forced file synchronization (fsync/fdatasync), so the database engine can wait for the data to be written to disk. 4) The underlying media (hard disk, SSD, USB stick, etc) must respond to the fsync call by actually writing the content to persistent storage before returning. Ok. Failure at any level means the overall system is not ACID compliant. Roger... But google says sqlite is supposed to be ACID compliant (although maybe not "fully" as you indicate, I'm not sure about this)... PostgreSQL has a huge amount of code in it to try to deal with (or at least recognize) a level-3 failure, but nothing in the database engine can deal with level 1 or 4 issues. You'd have to actually test it. The easiest way is to get two computers, side by side, and run the database engine on one and a monitor on the other. To test some SSDs at work, I knocked together a little program that worked somewhat thus: * Connect to the database over TCP/IP (easy, as we were doing this with PostgreSQL) * Create a table with a number of rows with an ID and a counter, initialized to 0 * Repeatedly, in parallel, perform a transaction: - Increment the counter on one of the rows (at random) - Increment a "possible" in-memory counter for that row - Commit the database transaction - Increment a "confirmed" in-memory counter for that row * When an error of "database seems to be down" is detected, wait for it to come up again, then query the table. The counters must all be at least their corresponding "possible" value and at most the "confirmed". Ok, that doesn't sound to be so simple after all... With that running, I simply pulled the plug on the database computer. With a properly-configured hard disk, every one of the counters was within its correct range. With a lying SSD, though, they could be anywhere from "pretty close" (with a low workload - simulated by having only a single thread doing transactions and having it sleep for a few ms each iteration) to "pretty appalling" (with a bunch of threads spinning tightly, keeping the workload high). Once the SSD starts doing major write reordering, its throughput soars, but at the cost of trustworthiness. Ok, it would be nice to hear/read the opinion from another in here who've been working (a lot?) with sqlite... Next question: Is it something I should worry about in my own programs (I'm not sure, I'm an SQL noob)... ? Yes, it most certainly is. If you have any data that you care about, put together some kind of test that will allow you to literally pull the plug on the database, while still knowing whether or not your transaction was completed (so you'll most likely need some kind of "possible" / "confirmed" counter pair as I used above). I'm not so rich, so I prefer to go for a free database solution rather than an expensive license... I've heard good things about oracle and that's also what they used at my previous company, but it's not something I am willing to pay for, from my private/own money for my sparetime-projects... Maybe what you've written explains why somebody got corrupted firefox sqlite files... I'll just practice a bit more and remember your advice about testing - at least for "important" projects, I'll remember how you tested this with pulling out the plug and monitoring the data... -- http://mail.python.org/mailman/listinfo/python-list
Re: python-noob - which container is appropriate for later exporting into mySql + matplotlib ?
On 04/13/2013 04:36 PM, Roy Smith wrote: In article , Chris Angelico wrote: 2) The database engine must employ some form of write-ahead log. [...] one way or another, there must be a way to detect half-done transactions. 3) The operating system and filesystem must support a forced file synchronization (fsync/fdatasync), so the database engine can wait for the data to be written to disk. 4) The underlying media (hard disk, SSD, USB stick, etc) must respond to the fsync call by actually writing the content to persistent storage before returning. Some of the early Unix file systems were very fragile. One of the (often under-appreciated) major advances in BSD (it was certainly in 4.2, not sure how much earlier) was a new filesystem which was much more robust in the face of hardware failures and system crashes. Prior to Are you talking about (journaling?) filesystems such as ext3, ext4, JFS, ReiserFS and XFS ? http://en.wikipedia.org/wiki/Journaling_file_system BSD, the on-disk data could be left in an inconsistent state if the system crashed at the wrong time. In BSD, data was written to disk in such a way that every operation could either be backed out cleanly or had enough information to complete the transaction. Journaling filesystems? I myself use ext4... There's a comparison here: http://en.wikipedia.org/wiki/Comparison_of_file_systems ? -- http://mail.python.org/mailman/listinfo/python-list
Re: python-noob - which container is appropriate for later exporting into mySql + matplotlib ?
On 04/13/2013 04:56 PM, Walter Hurry wrote: On Sat, 13 Apr 2013 16:39:12 +0200, someone wrote: I'm not so rich, so I prefer to go for a free database solution rather than an expensive license ( but I do care about ACID compliance) Sounds to me that PostgreSQL is your man, then. Oh, ok. Thanks! BTW: I just read: "Yahoo runs a multi-petabyte modified PostgreSQL database that processes billions of events per day" - that's truely amazing, I think... I think maybe I'll experiment a bit with both mySql (small/medium sized databases) and for critical/important stuff I should go with PostgreSQL... Glad to hear this... Then I know what to look at... -- http://mail.python.org/mailman/listinfo/python-list
Re: python-noob - which container is appropriate for later exporting into mySql + matplotlib ?
On 04/13/2013 06:15 PM, Chris Angelico wrote: On Sun, Apr 14, 2013 at 12:39 AM, someone wrote: On 04/13/2013 04:03 PM, Chris Angelico wrote: Failure at any level means the overall system is not ACID compliant. Roger... But google says sqlite is supposed to be ACID compliant (although maybe not "fully" as you indicate, I'm not sure about this)... What your Google hits are telling you is that sqlite can (if configured correctly) pass level 2. But it doesn't guarantee anything about the other levels, so it's easy to have an, uhh, ACID leak. Ok, thank you very much, this is something I couldn't easily see in the first place... I think what I should do now is to play a bit with sqlite and then afterwards, when I'm happy I would begin to play with postgresql and be very happy with it, knowing that I can (hopefully) use that for all important projects in the rest of my life :-) I might also play a bit with mySql, because it's my impression that it also have a big user-group. But I read that postgresql is MUCH more "safe" to use (and a bit slower) than postgresql which on the other hand is VERY safe, being fully ACID-compliant... You'd have to actually test it. The easiest way is to get two computers, side by side, and run the database engine on one and a monitor on the other. Ok, that doesn't sound to be so simple after all... I gave a fairly wordy run-down of what I tested, but it's actually fairly simple in concept: Do a huge bunch of transactions, and keep a log of what's returned from the COMMIT query; then pull the power out. I'll try it (or something similar) out one day in the future and see what happens with the "corrupted" changes due to pulling out the network cable while transmitting data... Ok, it would be nice to hear/read the opinion from another in here who've been working (a lot?) with sqlite... Agreed. I'm sure someone will chime in. I'm not so rich, so I prefer to go for a free database solution rather than an expensive license... I've heard good things about oracle and that's also what they used at my previous company, but it's not something I am willing to pay for, from my private/own money for my sparetime-projects... I concur with Walter's assessment: You want PostgreSQL. It's free/open source software (highly permissive MIT-like license), massively trusted, and scales up beautifully. (That last one may not be significant to you, but it's still good to know your database can handle hundreds or thousands of tps on basic hardware.) I understand that scaling is VERY important and if I could choose between two "equally" opensource systems and one of them scales better than the other, I would definately work with the one that scales the most - that means that I don't have to learn how to use a whole new system, if I already learnt the system that scales best... And I just found on google that yahoo runs a HUGE PostgreSQL database... Very interesting - I'll definately try to play around with postgreSQL at some time in the future... -- http://mail.python.org/mailman/listinfo/python-list
Re: python-noob - which container is appropriate for later exporting into mySql + matplotlib ?
On 04/13/2013 07:02 PM, rusi wrote: On Apr 13, 9:15 pm, Chris Angelico wrote: On Sun, Apr 14, 2013 at 12:39 AM, someone wrote: On 04/13/2013 04:03 PM, Chris Angelico wrote: . . Failure at any level means the overall system is not ACID compliant. Ok, it would be nice to hear/read the opinion from another in here who've been working (a lot?) with sqlite... Agreed. I'm sure someone will chime in. I'm not so rich, so I prefer to go for a free database solution rather than an expensive license... I've heard good things about oracle and that's also what they used at my previous company, but it's not something I am willing to pay for, from my private/own money for my sparetime-projects... I concur with Walter's assessment: You want PostgreSQL. It's free/open source software (highly permissive MIT-like license), massively trusted, and scales up beautifully. (That last one may not be significant to you, but it's still good to know your database can handle hundreds or thousands of tps on basic hardware.) ChrisA Dunno why you guys are ACIDing a hapless python+SQL noob. That's ok - I'm very interested in hearing/reading this, so don't worry :-) As far as I can see he did not even know what ACID was... Just I think I know it know (maybe not all the details, but generally I know that it should be ACID-compliant for critical data to avoid corruption and bad data) :-) happened to start with mysql (without evidently knowing the DBMS area) and Cousin Stanley's recommendation to step a notch down from mysql to sqlite seems to me to be spot-on for his requirement. Agree - but after that I would like to play with a client/server-system, so that's also interesting to hear about... To the OP: Steven is welcome to his views about use of databases. Good to remember that everyone does not agree with him. This includes the firefox devs as well as python devs. Yes, I think I understand this discussion. I'm sorry to hear that the sqlite-database-files sometimes become corrupted. I haven't experienced this problem myself (AFAIR), because ~90% of the time I'm on chromium. In particular, sqlite in python is quite special. All the other databases have bridge modules to talk from python to the database. Which means that python runs and the database runs and the two talk asynchronously across the bridge using what is called a 'client-server model'. Now client-server is powerful and sophisticated and earlier it Yes, got it :-) was the only option. That is the noob database programmer had to grapple with sql (the basic stuff) along with the transaction/ACID advanced stuff. Yep, I understand your intentions... Sqlite changed the rules of the game. Sqlite allows programmers to play with sql without having to deal with client server headaches at the same time. Python amplified that change by bundling it with python. In short Python+Sqlite is a boon for beginners to programming+DBMS I completely agree with you that Python+Sqlite is really really great... But soon I'll also move on to using a client/server model and therefore I also appreciate the other comments/discussion related to e.g. failure or non-"fully-ACID compliance" of sqlite, which maybe can explain this firefox problem with corrupted database(s)... I think I learned a lot from this thread and know what I should be working on now... -- http://mail.python.org/mailman/listinfo/python-list
Re: python-noob - which container is appropriate for later exporting into mySql + matplotlib ?
On 04/13/2013 10:01 PM, Dennis Lee Bieber wrote:
On Sun, 14 Apr 2013 00:03:25 +1000, Chris Angelico
declaimed the following in gmane.comp.python.general:
[ ]
* Create a table with a number of rows with an ID and a counter,
initialized to 0
* Repeatedly, in parallel, perform a transaction:
- Increment the counter on one of the rows (at random)
- Increment a "possible" in-memory counter for that row
- Commit the database transaction
- Increment a "confirmed" in-memory counter for that row
* When an error of "database seems to be down" is detected, wait for
it to come up again, then query the table. The counters must all be at
least their corresponding "possible" value and at most the
"confirmed".
SQLite is a "file server" database (like M$ JET engine [aka:
"Access"]). It's locking system is multi-stage. It allows multiple
concurrent readers on a "shared" lock state. Only one connection can
perform write operations ("reserved" lock) alongside the readers. A
second connection attempting to perform a write will be rejected with a
database locked condition. Then it really gets nasty -- the writer
attempts to commit the update: The first step is to block other
connections from even entering the read state (the "pending" lock).
However, the writer itself is blocked until all remaining readers have
exited; only then does it have exclusive access to and SQLite makes
changes to the database file itself (prior to that, the writer
connection is changing page images in memory)
Ok, this makes sense... It's not something I'll bother about to begin
with, but maybe later (for critical apps) I can see that this is important.
[ ]
In the commit phase, SQLite first tries to ensure the rollback
journal is flushed to disk -- but that apparently is out of its control;
it can submit a sync command to the OS, but has to rely on what the OS
tells it about the state of the writes to disk (the book indicates that
some IDE drives would lie when queried about sync status, while still
having unwritten data in the on-board buffers). After the rollback
journal it submits the data to the database. I
I agree, this must be a problem, when the OS is lying...
Crash during journal write: restart finds no journal, that transaction
is lost but the database itself is clean
Crash after journal during database update, restart finds journal,
assumes database is suspect, and rolls back the pages, database is
restored to pre-transaction state
Crash after database sync during removal of journal, restart either
finds journal still there and rolls back the pages restoring to
pretransaction state, or the file was removed from the directory and
SQLite determines database file is good with the last transaction in
place.
Ok, this is a bit more advanced - I'll try to make my own experiments
now and then after some time I guess I can dig more into these details,
thanks.
--
http://mail.python.org/mailman/listinfo/python-list
Re: python-noob - which container is appropriate for later exporting into mySql + matplotlib ?
On 04/14/2013 12:22 AM, Walter Hurry wrote: On Sat, 13 Apr 2013 21:34:38 +0200, someone wrote: On 04/13/2013 04:56 PM, Walter Hurry wrote: On Sat, 13 Apr 2013 16:39:12 +0200, someone wrote: I'm not so rich, so I prefer to go for a free database solution rather than an expensive license ( but I do care about ACID compliance) Sounds to me that PostgreSQL is your man, then. Oh, ok. Thanks! BTW: I just read: "Yahoo runs a multi-petabyte modified PostgreSQL database that processes billions of events per day" - that's truely amazing, I think... I think maybe I'll experiment a bit with both mySql (small/medium sized databases) and for critical/important stuff I should go with PostgreSQL... Glad to hear this... Then I know what to look at... If it were me I wouldn't use MySQL for anything at all. I'd use sqlite for little non-critical local applications, and Postgres for the rest. Ok, thank you. I just came across a blog that said pytables is also a very good option? http://www.pytables.org/moin/PyTables?action=AttachFile&do=view&target=non-indexed.png Postgres is not difficult at all, provided you RTFM and follow the instructions (the documentation is superb). And whichever you use, you need to learn SQL anyway. Good to hear... I'll dig more into it, thank you... -- http://mail.python.org/mailman/listinfo/python-list
Re: python-noob - which container is appropriate for later exporting into mySql + matplotlib ?
On 04/14/2013 12:54 AM, Chris Angelico wrote: On Sun, Apr 14, 2013 at 8:31 AM, someone wrote: Ok, thank you. I just came across a blog that said pytables is also a very good option? http://www.pytables.org/moin/PyTables?action=AttachFile&do=view&target=non-indexed.png From what I gather, that's looking at performance of a non-indexable query on a 10,000,000-row table. That's going to suck whatever you do, and the exact level of suckitude doesn't really prove much. (Note that even the best options are taking half a second for this single query.) Interesting... Thank you very much for that information... A better test of a database is transactions per second of something that approximates to your real workload. For instance, English Wikipedia has roughly a hundred edits per minute (assessed by me just now by looking at the Recent Changes), and some ridiculous number of page reads per minute (not assessed, but believed to be somewhere between 11 and Graham's number); so a test of a proposed new database would have to mimic this ratio. Most of the queries involved should be able to be answered using indexes; in some cases, ONLY using the index (eg if you just want to know whether or not a row exists). PyTables may well outperform PostgreSQL in real usage, but that one graph doesn't tell me that. (Not to mention that it's measuring a somewhat old PG.) Ok, thank you very much... Sounds to me like PostgreSQL it is, then :-) -- http://mail.python.org/mailman/listinfo/python-list
Re: python-noob - which container is appropriate for later exporting into mySql + matplotlib ?
On 04/14/2013 12:34 AM, Chris Angelico wrote: On Sun, Apr 14, 2013 at 5:34 AM, someone wrote: I think maybe I'll experiment a bit with both mySql (small/medium sized databases) and for critical/important stuff I should go with PostgreSQL PostgreSQL isn't majorly slower than MySQL, and it's a lot more trustworthy in terms of database constraints and so on. MySQL is designed as a place for a single application to store its data, and it assumes that the application is king; PostgreSQL is designed as a database against which application(s) may execute queries, therefore it assumes that the database administrator is king. With heavy read/write workloads, I'd put my money on PostgreSQL every time; MySQL has a much greater problem with wide locks (eg table-level) and consequent loss of concurrency. Ok, thank you very much... Sounds like PostgreSQL is the best option for me to go on to from here, after I've played a bit my sqlite... -- http://mail.python.org/mailman/listinfo/python-list
anyone know pandas ? Don't understand error: NotImplementedError...
Hi, Here's my script (from http://brenda.moon.net.au/category/data-visualisation/: #!/usr/bin/python import pandas import datetime import numpy datesList = [datetime.date(2011,12,1), \ datetime.date(2011,12,2), \ datetime.date(2011,12,3), \ datetime.date(2011,12,10)] countsList = numpy.random.randn(len(datesList)) startData = datetime.datetime(2011,12,3) endData = datetime.datetime(2011,12,8) def convertListPairToTimeSeries(dList, cList): # my dateList had date objects, so convert back to datetime objects dListDT = [datetime.datetime.combine(x, datetime.time()) for x in dList] # found that NaN didn't work if the cList contained int data cListL = [float(x) for x in cList] # create the index from the datestimes list indx = pandas.Index(dListDT) # create the timeseries ts = pandas.Series(cListL, index=indx) # fill in missing days ts = ts.asfreq(pandas.datetools.DateOffset()) return ts print "\nOriginal datesList list:\n", datesList tSeries = convertListPairToTimeSeries(datesList, countsList) print "\nPandas timeseries:\n", tSeries # use slicing to change length of data tSeriesSlice = tSeries.ix[startData:endData] print "\nPandas timeseries sliced between", startData.date(), \ "and", endData.date(), ":\n", tSeriesSlice # use truncate instead of slicing to change length of data tSeriesTruncate = tSeries.truncate(before=startData, after=endData) print "\nPandas timeseries truncated between", startData.date(), \ "and", endData.date(), ":\n", tSeriesTruncate # my data had lots of gaps that were actually 0 values, not missing data # So I used this to fix the NaN outside the known outage startOutage = datetime.datetime(2011,12,7) endOutage = datetime.datetime(2011,12,8) tsFilled = tSeries.fillna(0) # set the known outage values back to NAN tsFilled.ix[startOutage:endOutage] = numpy.NAN print "\nPandas timeseries NaN reset to 0 outside known outage between", \ startOutage.date(), "and", endOutage.date(), ":\n", tsFilled print "\nPandas series.tail(1) and series.head(1) are handy for " +\ "checking ends of list:\n", tsFilled.head(1), tsFilled.tail(1) print tsFilled.plot(); # <== NotImplementedError...!!! If I run it, I get: ... ... .. 2011-12-090.00 2011-12-101.431665 Freq: <1 DateOffset> Pandas series.tail(1) and series.head(1) are handy for checking ends of list: 2011-12-01 -0.969533 Freq: <1 DateOffset> 2011-12-101.431665 Freq: <1 DateOffset> Traceback (most recent call last): File "./pandas_example.py", line 57, in tsFilled.plot(); File "/usr/lib/pymodules/python2.7/pandas/tools/plotting.py", line 985, in plot_series plot_obj.generate() File "/usr/lib/pymodules/python2.7/pandas/tools/plotting.py", line 376, in generate self._make_plot() File "/usr/lib/pymodules/python2.7/pandas/tools/plotting.py", line 623, in _make_plot if self.use_index and self._use_dynamic_x(): File "/usr/lib/pymodules/python2.7/pandas/tools/plotting.py", line 619, in _use_dynamic_x return (freq is not None) and self._is_dynamic_freq(freq) File "/usr/lib/pymodules/python2.7/pandas/tools/plotting.py", line 602, in _is_dynamic_freq freq = freq.rule_code File "/usr/lib/pymodules/python2.7/pandas/tseries/offsets.py", line 214, in rule_code raise NotImplementedError NotImplementedError Can anyone tell why this error appears and how to fix it? -- http://mail.python.org/mailman/listinfo/python-list
Re: anyone know pandas ? Don't understand error: NotImplementedError...
On 04/18/2013 03:44 PM, Wayne Werner wrote: On Wed, 17 Apr 2013, someone wrote: File "/usr/lib/pymodules/python2.7/pandas/tseries/offsets.py", line 214, in rule_code raise NotImplementedError NotImplementedError Can anyone tell why this error appears and how to fix it? I don't know anything about pandas, but my recommendation? $ vim /usr/lib/pymodules/python2.7/pandas/tseries/offsets.py (or nano or emacs - whatever editor you're comfortable with). Go to line 214, and take a look-see at what you find. My guess is it will be something like: def rule_code(): raise NotImplementedError() Which is terribly unhelpful. Oh, yes - you're completely right: # line 211 (empty line) @property # line 212 def rule_code(self): # line 213 raise NotImplementedError # line 214 # line 215 (empty line) Below and above this "rule_code" is code belonging to some other functions... hmmm... I also tried to look in: /usr/lib/pymodules/python2.7/pandas/tools/plotting.py But I'm very unfamiliar with pandas, so everything looks "correct" to me - because I don't understand the data structure, I think I cannot see what is wrong... -- http://mail.python.org/mailman/listinfo/python-list
Re: anyone know pandas ? Don't understand error: NotImplementedError...
On 04/18/2013 04:07 PM, Neil Cerutti wrote: On 2013-04-18, Wayne Werner wrote: On Wed, 17 Apr 2013, someone wrote: Go to line 214, and take a look-see at what you find. My guess is it will be something like: def rule_code(): raise NotImplementedError() Which is terribly unhelpful. It most likely means that the program is instantiating an abstract base class when it should be using one of its subclasses instead, e.g., BusinessDay, MonthEnd, MonthBegin, BusinessMonthEnd, etc. http://pandas.pydata.org/pandas-docs/dev/timeseries.html Hi Neil and Wayne, Thank you very much for your suggestions... I now found out something: In the function: def convertListPairToTimeSeries(dList, cList): ... ... # create the timeseries ts = pandas.Series(cListL, index=indx) # fill in missing days #ts = ts.asfreq(pandas.datetools.DateOffset()) return ts I had to out-comment the last line before the return-statement (not sure what that line is supposed to do, in the first case)... Now the program runs, but no plot is seen. Then I found out that I had to add: import matplotlib.pyplot as plt in the top of the program and add the following in the bottom of the program: plt.show() Final program: == #!/usr/bin/python import pandas import datetime import numpy import ipdb import matplotlib.pyplot as plt datesList = [datetime.date(2011,12,1), \ datetime.date(2011,12,2), \ datetime.date(2011,12,3), \ datetime.date(2011,12,10)] countsList = numpy.random.randn(len(datesList)) startData = datetime.datetime(2011,12,3) endData = datetime.datetime(2011,12,8) def convertListPairToTimeSeries(dList, cList): # my dateList had date objects, so convert back to datetime objects dListDT = [datetime.datetime.combine(x, datetime.time()) for x in dList] # found that NaN didn't work if the cList contained int data cListL = [float(x) for x in cList] # create the index from the datestimes list indx = pandas.Index(dListDT) # create the timeseries ts = pandas.Series(cListL, index=indx) # fill in missing days #ts = ts.asfreq(pandas.datetools.DateOffset()) return ts print "\nOriginal datesList list:\n", datesList tSeries = convertListPairToTimeSeries(datesList, countsList) print "\nPandas timeseries:\n", tSeries # use slicing to change length of data tSeriesSlice = tSeries.ix[startData:endData] print "\nPandas timeseries sliced between", startData.date(), \ "and", endData.date(), ":\n", tSeriesSlice # use truncate instead of slicing to change length of data tSeriesTruncate = tSeries.truncate(before=startData, after=endData) print "\nPandas timeseries truncated between", startData.date(), \ "and", endData.date(), ":\n", tSeriesTruncate # my data had lots of gaps that were actually 0 values, not missing data # So I used this to fix the NaN outside the known outage startOutage = datetime.datetime(2011,12,7) endOutage = datetime.datetime(2011,12,8) tsFilled = tSeries.fillna(0) # set the known outage values back to NAN tsFilled.ix[startOutage:endOutage] = numpy.NAN print "\nPandas timeseries NaN reset to 0 outside known outage between", \ startOutage.date(), "and", endOutage.date(), ":\n", tsFilled print "\nPandas series.tail(1) and series.head(1) are handy for " +\ "checking ends of list:\n", tsFilled.head(1), tsFilled.tail(1) print tsFilled.plot() plt.show() == This seem to work, although I don't fully understand it, as I'm pretty new to pandas... -- http://mail.python.org/mailman/listinfo/python-list
Re: Style formating of multiline query, advise
On Mar 18, 2:51 pm, John Machin wrote: > On Mar 18, 11:25 pm, someone wrote: > > > Hi, > > > what is good :) style for multiline queries to database? > > Is that one ok? > > query = """ SELECT * FROM ( > > SELECT a.columna, a.columnb, a.iso > > FROM all a > > WHERE (a.name = LOWER(%s)) ) AS c > > JOIN other as b on c.gid = b.id > > WHERE class = 'A' > > ORDER BY population DESC > > LIMIT %s;""" > > There's no tablet of stone, like PEP 8 :-) It seems to be a matter of > individual taste; clarity and consistency and not overdoing > parentheses count for me. > > IMO you have too much leading whitespace, you have "as" in upper, thanks for reply. So, no indentation? query is inside of function, of course. > lower and no case, and I'd set out the inner select differently: > > query = """ > SELECT * FROM ( > SELECT a.columna, a.columnb, a.iso > FROM all AS a > WHERE a.name = LOWER(%s) > ) AS c > JOIN other AS b ON c.gid = b.id > WHERE class = 'A' > ORDER BY population DESC > LIMIT %s; > """ > > Cheers, > John -- http://mail.python.org/mailman/listinfo/python-list
Re: Style formating of multiline query, advise
On Mar 18, 4:04 pm, Tino Wildenhain wrote: > John Machin wrote: > > On Mar 18, 11:25 pm, someone wrote: > >> Hi, > > >> what is good :) style for multiline queries to database? > >> Is that one ok? > >> query = """ SELECT * FROM ( > >> SELECT a.columna, a.columnb, a.iso > >> FROM all a > >> WHERE (a.name = LOWER(%s)) ) AS c > >> JOIN other as b on c.gid = b.id > >> WHERE class = 'A' > >> ORDER BY population DESC > >> LIMIT %s;""" > > > There's no tablet of stone, like PEP 8 :-) It seems to be a matter of > > individual taste; clarity and consistency and not overdoing > > parentheses count for me. > > > IMO you have too much leading whitespace, you have "as" in upper, > > lower and no case, and I'd set out the inner select differently: > > > query = """ > > SELECT * FROM ( > > SELECT a.columna, a.columnb, a.iso > > FROM all AS a > > WHERE a.name = LOWER(%s) > > ) AS c > > JOIN other AS b ON c.gid = b.id > > WHERE class = 'A' > > ORDER BY population DESC > > LIMIT %s; > > """ Agree, that looks nicer > > And I'd not recomment SELECT * for anything beside test queries in an > interactive session or if you are going to copy tables... yes, it I know. Thank you for responce! > > Regards > Tino > > smime.p7s > 4KViewDownload -- http://mail.python.org/mailman/listinfo/python-list
Re: Style formating of multiline query, advise
On Mar 18, 5:56 pm, Scott David Daniels wrote: > someone wrote: > > what is good :) style for multiline queries to database?... > > query = """ SELECT * FROM ( > > SELECT a.columna, a.columnb, a.iso > > FROM all a > > WHERE (a.name = LOWER(%s)) ) AS c > > JOIN other as b on c.gid = b.id > > WHERE class = 'A' > > ORDER BY population DESC > > LIMIT %s;""" > > Also, for SQL, (A) why are you using nested joins?, and inner select produce smaller set which is then joined with other table, kind a optimization > (B) always show what columns you want as output. SELECT * is a hack > to make interactive use easier, not a durable way to write queries. Yes, my original question was about formatting. It's not original query (only a part). But thank you anyway! > > query = """SELECT a.columna, a.columnb, a.iso, b.id, ... > FROM all AS a, other as b > WHERE a.name = LOWER(%s) > AND a.gid = b.id > AND b.class = 'A' > ORDER BY population DESC > LIMIT %s;""" > > --Scott David Daniels > [email protected] -- http://mail.python.org/mailman/listinfo/python-list
Re: Style formating of multiline query, advise
On Mar 19, 10:26 am, Marco Mariani wrote: > someone wrote: > >> Also, for SQL, (A) why are you using nested joins?, and > > > inner select produce smaller set which is then joined with other > > table, kind a optimization > > Did you time it? > I've done some "kind of a optimization" that slowed queries by tenfold, > because postgres didn't need my advice, and knew better. RDBMS Query itself is fast. I'll try to time it and will report my results here :) > performance is non-intuitive, really. If you're using mysql, YMMV, > because its optimizer is not as good. That's true, I don't trust optimizer > > > Yes, my original question was about formatting. It's not original > > query (only a part). > > Try this: > > http://www.dpriver.com/pp/sqlformat.htm > > My 2c: I use textwrap.dedent to strip leading spaces from every line. textwrap.dedent? What's that? I'm new in python. Will google it. How do your query formatting looks like then? -- http://mail.python.org/mailman/listinfo/python-list
Style formating of multiline query, advise
Hi, what is good :) style for multiline queries to database? Is that one ok? query = """ SELECT * FROM ( SELECT a.columna, a.columnb, a.iso FROM all a WHERE (a.name = LOWER(%s)) ) AS c JOIN other as b on c.gid = b.id WHERE class = 'A' ORDER BY population DESC LIMIT %s;""" Regards, Pet -- http://mail.python.org/mailman/listinfo/python-list
Escaping optional parameter in WHERE clause
Hi,
as you can see below I have some optional parameter for my query (mf,
age). They are in WHERE clause only if not empty.
In this function they are not escaped as, for example, 'search'
parameter, cause I can't pass them to execute function, which does
escaping automatically.
I could write another if's block like that
if mf and not age:
db.execute(query, search, mf, limit)
if age and not mf:
db.execute(query, search, age, limit)
if age and mf:
db.execute(query, search, mf, age, limit)
Is there a better way to deal with optional WHERE clause?
Pet
def getData(self, db, params):
search = params.get('search','')
age = params.get('age','')
mf = params.get('mf','')
limit = params.get('limit',1)
if mf:
mf = " AND mf = %s " % mf
if age:
age = " AND age = %s " % age
query = """
SELECT * FROM mytable
WHERE class = 'P'
AND name = %s
""" + mf + """
""" + age + """
ORDER BY id DESC
LIMIT %s;
"""
db.execute(query, search, limit)
result = db.fetchall()
return result
--
http://mail.python.org/mailman/listinfo/python-list
install pyPgSQL on Windows for python 2.5
Hi, does anyone know how to install pyPgSQL on Windows? There is no package for Python 2.5 on Homepage: http://sourceforge.net/project/showfiles.php?group_id=16528&package_id=20458&release_id=423036 I've tried to install running: python setup.py install But getting errors python setup.py install -> running install running build running build_py running build_ext error: Python was built with Visual Studio 2003; extensions must be built with a compiler than can generate compatible binaries. Visual Studio 2003 was not found on this system. If you have Cygwin installed, you can try compiling with MingW32, by passing "-c mingw32" to setup.py. I've installed newest Visual C++ Studio 2008 from Microsoft, but still no luck Pet -- http://mail.python.org/mailman/listinfo/python-list
formating query with empty parameter
Hello! if one of parameter in values is empty, I'm getting TypeError: not enough arguments for format string But how to handle such situation? It is ok for DB, that some of values are empty. def __insert(self, data): query = """ BEGIN; INSERT INTO table (a, b, c, d, e, f, g) VALUES (%s, %s, %s, %s, %s, %s, %s); COMMIT; """ values = [ data['a'], data['b'], data['c'], data['d'], data['e'], data['f'], data['g'] ] self.db.execute(query, *values) Thanks Pet -- http://mail.python.org/mailman/listinfo/python-list
unicode confusing
Hi, reading content of webpage (encoded in utf-8) with urllib2, I can't get parsed data into DB Exception: File "/usr/lib/python2.5/site-packages/pyPgSQL/PgSQL.py", line 3111, in execute raise OperationalError, msg libpq.OperationalError: ERROR: invalid UTF-8 byte sequence detected near byte 0xe4 I've already checked several python unicode tutorials, but I have no idea how to solve my problem. Appreciating your help Pet -- http://mail.python.org/mailman/listinfo/python-list
pyPgSql there is already a transaction in progres
Hi,
I'm using pyPgSQL for accessing Postgres and do some update and select
queries.
and getting WARNING: there is already a transaction in progress if I
run runUpdate more than once.
So, what happens is following:
1. SELECT address FROM address WHERE LOWER(address) = LOWER(%s); --
__existRecord
2. BEGIN;DELETE FROM failed WHERE uquery = %s;COMMIT; -- __delQuery
3. SELECT address FROM address WHERE LOWER(address) = LOWER(%s); --
again __existRecord
and here I'm getting the warning.
Can anyone explain me please what the problem is? I do select, then
delete transaction and then select again which doesn't work
Regards, Pet
class Bla:
def __init__:
pass
def runUpdate(self, number=5):
data = {}
data = self.__getUpdateItems(number)
for row in data:
try:
if self.__existRecord(row['q']):
self.__delQuery(row['q'])
except Exception, e:
print "Exception", e
def __delQuery(self, name):
query = """
BEGIN;DELETE FROM failed WHERE uquery = %s;COMMIT;
"""
try:
self.db.execute(query, name)
except Exception, e:
print "Exception: ", e
return True
def __existRecord(self, search):
query = """
SELECT address FROM address WHERE LOWER(address) = LOWER
(%s);
"""
try:
self.db.execute(query, search)
except Exception, e:
print "Exception: ", e
return self.db.fetchall()
def __getUpdateItems(self,number=5):
values = [number]
query = """
SELECT * FROM failed
WHERE id IS NULL
ORDER BY up DESC
LIMIT %s;
"""
result = []
try:
self.db.execute(query, *values)
result = self.db.fetchall()
except Exception, e:
print "Exception getUpdateItems: ", e
return result
--
http://mail.python.org/mailman/listinfo/python-list
How do I print to text browser with Qthread? (in PyQt)
Hello, I wrote Qthread that gets in the constructor a textBrowser from PyQt application and I tryed to print in this widget text that is updated each 5 seconds (Like application that counts numbers of the text brouser) Is anyone can explain me how to do this? I would glad if you can attach a code example. thanks someone -- http://mail.python.org/mailman/listinfo/python-list
print executed query
Hello! Is there a way to print a query for logging purpose as it was or will be sent to database, if I don't escape values of query by myself? cursor.execute(query, [id, somestring]) I could print query and values separate, but it would be great, if I could see how query is constructed and can the also copy it and execute in console. Im using psycopg2, btw Thanks! -- http://mail.python.org/mailman/listinfo/python-list
Re: print executed query
On Apr 29, 7:23 pm, Miki wrote: > > Is there a way to print a query for logging purpose as it was or will > > be sent to database, if I don't escape values of query by myself? > > ... > > Im using psycopg2, btw > > http://initd.org/psycopg/docs/advanced.html#connection-and-cursor-fac... > > HTH, > -- > Mikihttp://pythonwise.blogspot.com thanks, I'm already passing cursor_factory=RealDictCursor for results as dicts. Will try combine both. -- http://mail.python.org/mailman/listinfo/python-list
Re: print executed query
On 4 Mai, 07:01, Fred C wrote:
> On Apr 29, 2010, at 9:49 AM, Philip Semanchuk wrote:
>
>
>
>
>
> > On Apr 29, 2010, at 12:01 PM, someone wrote:
>
> >> Hello!
>
> >> Is there a way to print a query for logging purpose as it was or will
> >> be sent to database, if I don't escape values of query by myself?
>
> >> cursor.execute(query, [id, somestring])
>
> >> I could print query and values separate, but it would be great, if I
> >> could see how query is constructed and can the also copy it and
> >> execute in console.
>
> >> Im using psycopg2, btw
>
> > If you can fiddle with the Postgres server settings, the server has options
> > for logging lots of things, including the queries it executes.
>
> > Hope this helps
> > Philip
>
> > --
> >http://mail.python.org/mailman/listinfo/python-list
>
> smime.p7s
> 4KAnzeigenHerunterladen
Hello! Thanks for help! I've found much more simpler solution ;)
http://initd.org/psycopg/docs/cursor.html
query
Read-only attribute containing the body of the last query sent to the
backend (including bound arguments). None if no query has been
executed yet:
>>> cur.execute("INSERT INTO test (num, data) VALUES (%s, %s)", (42, 'bar'))
>>> cur.query
"INSERT INTO test (num, data) VALUES (42, E'bar')"
DB API extension The query attribute is a Psycopg extension to the DB
API 2.0.
--
http://mail.python.org/mailman/listinfo/python-list
variable variables
Hello, is it possible to make first attr variable? some_object.attr.attr so instead of attr I could use self.foo which has value "attr" Thanks! -- http://mail.python.org/mailman/listinfo/python-list
Re: variable variables
On Jun 18, 12:01 pm, "Gabriel Genellina" wrote: > En Fri, 18 Jun 2010 06:48:34 -0300, someone > escribió: > > > is it possible to make first attr variable? > > > some_object.attr.attr > > > so instead of attr I could use self.foo which has value "attr" > > I think you're looking for > getattr:http://docs.python.org/library/functions.html#getattr > > name = "spam" > getattr(some_object, name) == some_object.spam == getattr(some_object, > "spam") Thanks. I was looking for a "short way" to do it because I have a lot "some_object.attr.attr or some_object.other_attr.attr" in code. it looks like I cannot replace attr with just other variable and must type some_object.other_attr.attr or your solution which is however longer to type :) > > -- > Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: variable variables
On Jun 18, 12:49 pm, James Mills wrote: > On Fri, Jun 18, 2010 at 8:31 PM, someone wrote: > > I was looking for a "short way" to do it because I have a lot > > "some_object.attr.attr or some_object.other_attr.attr" in code. it > > looks like I cannot replace attr with just other variable and must > > type some_object.other_attr.attr or your solution which is however > > longer to type :) > > It would actually help to see some code. > here it is, In Foo I'd like to have instead of A self.type and the same in class B from some_module import some_object class Foo: def __init__(self): self.type = 'A' def printAttr(self): some_object.A.B some_object.A.C some_object.A.D some_object.A.E some_object.A.F some_object.A.G class Bar: def __init__(self): self.type = 'B' def printAttr(self): some_object.B.B some_object.B.C some_object.B.D some_object.B.E some_object.B.F some_object.B.G > --James -- http://mail.python.org/mailman/listinfo/python-list
Re: variable variables
On Jun 18, 2:05 pm, Bruno Desthuilliers wrote: > someone a crit : > > > > > > > On Jun 18, 12:49 pm, James Mills wrote: > >> On Fri, Jun 18, 2010 at 8:31 PM, someone wrote: > >>> I was looking for a "short way" to do it because I have a lot > >>> "some_object.attr.attr or some_object.other_attr.attr" in code. it > >>> looks like I cannot replace attr with just other variable and must > >>> type some_object.other_attr.attr or your solution which is however > >>> longer to type :) > >> It would actually help to see some code. > > > here it is, In Foo I'd like to have instead of A self.type and the > > same in class B > > > from some_module import some_object > > > class Foo: > > def __init__(self): > > self.type = 'A' > > > def printAttr(self): > > some_object.A.B > > some_object.A.C > > some_object.A.D > > some_object.A.E > > some_object.A.F > > some_object.A.G > > > class Bar: > > def __init__(self): > > self.type = 'B' > > > def printAttr(self): > > some_object.B.B > > some_object.B.C > > some_object.B.D > > some_object.B.E > > some_object.B.F > > some_object.B.G > > from some_module import some_object > > def print_attributes(obj, *attribs): > for attr in attribs: > print getattr(obj, attr, None) > > class Foo(object): > type = 'A' > > def print_attr(self): > print_attributes(getattr(someobject, self.type), *"BCDEFG") > > class Bar(Foo) > type = 'B' > > Still has a "code smell" thing to me, but hard to say not knowing the > real code and context. sorry, code is not about printing variables rather accessing, it's just example. -- http://mail.python.org/mailman/listinfo/python-list
Re: variable variables
On Jun 18, 2:37 pm, Bruno Desthuilliers wrote: > someone a crit : > > > On Jun 18, 2:05 pm, Bruno Desthuilliers > [email protected]> wrote: > (snip) > > >> Still has a "code smell" thing to me, but hard to say not knowing the > >> real code and context. > > > sorry, code is not about printing variables rather accessing, it's > > just example. > > Yeps, this I understood - hence the "but..." part of the sentence. What > I dislike the most here is the obj.otherobj.yetanother.attrib stuff - > might be ok in this case but it's an obvious violation of the law of > Demeter (=> AKA: only talk to your neighbour). > > But anyway - if Foo only needs to know about someobject.A and Bar only > needs to know about someobject.B, then resolve them once: > > from some_module import some_object > > class Foo: > def __init__(self): > self.type = 'A' > > def printAttr(self): > target = getattr(someobject, self.type) > target.B > target.C > target.D > # etc... > > This will also save a couple useless lookups. In fact, aliasing a > commonly accessed attribute (or method) to a local name is an old time > Python micro-optimization. Thanks, this looks good. I've missed point that self.type must be an instance of object and not just some string :) -- http://mail.python.org/mailman/listinfo/python-list
cannot get html content of tag with BeautifulSoup
Hello,
does anyone know how to get html contents of an tag with
BeautifulSoup? In example I'd like to get all html which is in first
tag, i.e. This is paragraph one. as
unicode object
p.contents gives me a list which I cannot join TypeError: sequence
item 0: expected string, Tag found
Thanks!
from BeautifulSoup import BeautifulSoup
import re
doc = ['Page title',
'This is
paragraph one.',
'This is paragraph two.',
'']
soup = BeautifulSoup(''.join(doc))
#print soup.prettify()
r = re.compile(r'<[^<]*?/?>')
for i, p in enumerate(soup.findAll('p')):
#print type(p) #
#print type(p.contents) #list
content = "".join(p.contents) #fails
p_without_html = r.sub(' ', content)
print p_without_html
--
http://mail.python.org/mailman/listinfo/python-list
Re: cannot get html content of tag with BeautifulSoup
On Jun 18, 5:41 pm, someone wrote:
> Hello,
>
> does anyone know how to get html contents of an tag with
> BeautifulSoup? In example I'd like to get all html which is in first
> tag, i.e. This is paragraph one. as
> unicode object
>
> p.contents gives me a list which I cannot join TypeError: sequence
> item 0: expected string, Tag found
>
> Thanks!
>
> from BeautifulSoup import BeautifulSoup
> import re
>
> doc = ['Page title',
> 'This is
> paragraph one.',
> 'This is paragraph two. p>',
> '']
> soup = BeautifulSoup(''.join(doc))
> #print soup.prettify()
> r = re.compile(r'<[^<]*?/?>')
> for i, p in enumerate(soup.findAll('p')):
> #print type(p) #
> #print type(p.contents) #list
> content = "".join(p.contents) #fails
>
> p_without_html = r.sub(' ', content)
> print p_without_html
p.renderContents() was what I've looked for
--
http://mail.python.org/mailman/listinfo/python-list
