problem with closures
Hi,
I have a problem with closures.
I am trying to implement yet another design by contract decorator which
would look like the following:
def contract(f):
def newf(*args, **kw):
import new
precondition = new.function(f.func_code.co_consts[1],
f.func_globals,'pre',
f.func_defaults,
f.func_closure)
precondition()
result=f(*args, **kw)
postcondition=new.function(f.func_code.co_consts[2],globals())
postcondition(result)
return result
return newf
@contract
def foo(x,y,g=2,z=1):
def pre():
assert x>1 and 00
print 'main'
return x+y+z*g
print foo(2,5,4,69)
The problem is that i get the following error message on line 7:
TypeError: arg 5 (closure) must be tuple
f.func_closure is indeed empty while
f.func_code.co_consts[1].co_freevars is logically equal to ('x','y').
Thanks for responding
Alain
--
http://mail.python.org/mailman/listinfo/python-list
challenge ?
I have a problem I wonder if it has been solved before. I have a dictionnary and I want the values in the dictionnary to be annotated with the rank that would be obtained by sorting the values def annotate_with_rank(my_dict): return my_annotated_dict In other words, any value a_value would become a 2-tuple (a_value,rank_of_a_value) I seek an elegant solution. Alain -- http://mail.python.org/mailman/listinfo/python-list
SPE question
Hi, Could someone tell me how to uninstall SPE under windows? Alain -- http://mail.python.org/mailman/listinfo/python-list
SNMP agent
Hi, I have a Python app and i would like to add some SNMP agent functionality to it. I know it's possible to write a sub-agent with netsnmp but it is in C and quite complicated. I know of YAPSNMP (a wrapper around NETSNMP) but it doesn't seem to support agent writing. Any suggestion ? Thanks to all in advance Alain -- http://mail.python.org/mailman/listinfo/python-list
Re: SNMP agent
On Apr 4, 1:30 pm, alf <[EMAIL PROTECTED]> wrote: > twistedmatrix.org? I already took a look at it but the agent functionality is somewhat primitive. I need something production-ready. Alain -- http://mail.python.org/mailman/listinfo/python-list
Re: SNMP agent
On Apr 5, 6:50 pm, [EMAIL PROTECTED] (Cameron Laird) wrote: > In article <[EMAIL PROTECTED]>, > > alain <[EMAIL PROTECTED]> wrote: > >On Apr 4, 1:30 pm, alf <[EMAIL PROTECTED]> wrote: > > >> twistedmatrix.org? > > >I already took a look at it but the agent functionality is somewhat > >primitive. I need something production-ready. > > >Alain > > 'Doesn't exist. > > I understand the sentiment; in principle, it shouldn't be hard > to write a library which supports construction of SNMP agents > in Python. I'm aware of no one who has done so publicly, though. > > Myself, I like using the Tcl-based Scotty. For severely-constrained > performance, though, you'll likely have to go to C, in some form. Hi Cameron, Thanks for the information. I still find it strange that, in all these years of existence, no one felt the need for a SNMP agent in Python. Do Pythoneers only write test tools and not real apps? Alain -- http://mail.python.org/mailman/listinfo/python-list
Re: too many values with string.split
Shawn Minisall a écrit : > I'm trying to unpack a list of 5 floats from a list read from a file and > python is telling me 5 variables are too many for the string.split > statement. Anyone have any other idea's? NOTE: the only reason I > convert it to a float instead of just leaving it as a string in the loop > is because I have to have it printed out as a float besides the names > and then the average displayed underneath > > thx > >#read in data line by line >for line in infile: >mylist = string.split(line) >firstName[counter] = mylist[0] >lastName[counter] = mylist[1] >grades[counter] = float(mylist[2]) >print firstName[counter], > lastName[counter],":","\t\t",grades[counter] >#increment counter >counter = counter + 1 > >#calculates and prints average score >grades = str(grades) >num1, num2, num3, num4, num5 = string.split(grades,",") >average = float(num1 + num2 + num3 + num4 + num5) / 5 >print >print "Average:" As I can see, grades is a string that looks like '[12.0,12.0, ...]' So you can't split it just with string.split () Rather than doing grades = str(grades) and split it, you have just to do : avarage = sum (grades) / len (grades) -- http://mail.python.org/mailman/listinfo/python-list
Re: I earn $36800 a month with google adsense
On 2 okt, 03:41, panguohua <[EMAIL PROTECTED]> wrote: > On 10 1 , 8 06 , panguohua <[EMAIL PROTECTED]> wrote: > > > On 10 1 , 1 13 , panguohua <[EMAIL PROTECTED]> wrote: > > > >www.space666.com > > > > a good website for making money with your blog.more information there > > > support! > > ! > > good Wow ! I have a question though. If you really make $38600 a month, how come you even bother to spam us? Alain -- http://mail.python.org/mailman/listinfo/python-list
Why does this not work?
I tried the following: myobj=object() myobj.newattr=5 results in: Traceback (most recent call last): File "", line 1, in ? AttributeError: 'object' object has no attribute 'newattr' Any idea? Alain -- http://mail.python.org/mailman/listinfo/python-list
Re: Why does this not work?
On Feb 2, 1:57 pm, Bart Van Loon <[EMAIL PROTECTED]> wrote: > It was 2 Feb 2007 04:41:48 -0800, when alain wrote: > > > I tried the following: > > > myobj=object() > > myobj.newattr=5 > > > results in: > > > Traceback (most recent call last): > > File "", line 1, in ? > > AttributeError: 'object' object has no attribute 'newattr' > > > Any idea? > > I think it's because... object has no attribute 'newattr' > > what else is there left to say? > > try: > > myobj=object() > print dir(myobj) > > does that contain 'myattr'? > > -- > groetjes, > BBBart > >"To make a bad day worse, spend it wishing for the impossible." -Calvin What about this: class Object(object):pass myobj=Object() myobj.newattr=5 and it works !!! Python allows the dynamic creation of attributes for an instance of an object. Alain -- http://mail.python.org/mailman/listinfo/python-list
Re: Python too complex ?!?!?!
On Nov 19, 9:44 am, Kay Schluehr <[EMAIL PROTECTED]> wrote: > On 17 Nov., 14:46, Brian <[EMAIL PROTECTED]> wrote: > > > Had a unsettling conversation with a CS instructor that > > teaches at local high schools and the community > > college. This person is a long-term Linux/C/Python > > programmer, but he claims that the install, config, and > > library models for C# have proved to be less > > problematic than Python. So both his courses (intro, > > data structs, algorithms) are taught in C#. > > I don't understand this complaint. How does your instructor installs > libraries for C#? When he uses Linux for teaching purposes I assume > his students have little problems typing some shell commands and when > he uses Windows his students might feel comfortable double clicking on > a Windows installer - most likely containing prepackaged binaries. > What have I missed? > > Kay I think i understand his complaint. Have you ever tried to install a package making use of easy_install? If you have, then you understand this is a real pain in the ass, especially if your internet access requires proxy authentication. The world was easy before easy_install ! Alain -- http://mail.python.org/mailman/listinfo/python-list
Re: Crackly sound in pygame
On Nov 30, 4:33 am, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote: > Hey guys I am running Windows XP and am having an issue with a game > that my team has created. Whenever an audio file is played it creates > a very distorted, crackly sound. Any ideas what could be the issue? > > Thanks No, and the more i look at your post, the more i can't see why. Sorry for not being able to help. Alain -- http://mail.python.org/mailman/listinfo/python-list
Re: Python's great, in a word
On Jan 7, 2:09 pm, [EMAIL PROTECTED] wrote: > I'm a Java guy who's been doing Python for a month now and I'm > convinced that > > 1) a multi-paradigm language is inherently better than a mono-paradigm > language > > 2) Python writes like a talented figure skater skates. > > Would you Python old-timers try to agree on a word or two that > completes: > > The best thing about Python is ___. > > Please, no laundry lists, just a word or two. I'm thinking "fluid" or > "grace" but I'm not sure I've done enough to choose. Paraphrasing Steve Jobs but in this context: PYTHON = a bycycle for the mind Best regards Alain -- http://mail.python.org/mailman/listinfo/python-list
Re: Python's great, in a word
On Jan 7, 6:27 pm, "Henry Chang" <[EMAIL PROTECTED]> wrote: > What exactly does it mean "a bycycle for the mind"?? > > (assuming s/bycycle/bicycle) Sorry for my bad spelling. The original quote from Steve Jobs reads "the computer is a bicycle for the mind". This is what i feel when programming in Python: a productivity boost compared to pedestrians and an immense pleasure. Alain -- http://mail.python.org/mailman/listinfo/python-list
Re: Python equivt of __FILE__ and __LINE__
On Feb 11, 10:58 am, "Bill Davy" <[EMAIL PROTECTED]> wrote: > Writing a quick and dirty assembler and want to give the user the location > of an error. The "assembly language" is Python. If the user wants to > generat some object code they write something like: > > Label(LoopLable) > Load(R4) > Dec() > JNZ(LoopLabel) > > I can use Python to do all the expression evalutaion, conversion from Python > FP to target FP, include files, macros (done as function definitions). The > functions like Load() generate the approproyte object code. > > So, for example, when a label is defined or referenced, I save the File,Line > so if there is not exactly one defintion or no references, I can report the > file location(s) to be considered. In the example, I would want to report > that LoopLable is not referenced, and LoopLabel is not defined. > > TIA, > Bill > > PSwww.SynectixLtd.comis not relevant def __LINE__(): try: raise Exception except: return sys.exc_info()[2].tb_frame.f_back.f_lineno def __FILE__(): return inspect.currentframe().f_code.co_filename Best regards Alain -- http://mail.python.org/mailman/listinfo/python-list
Re: Python equivt of __FILE__ and __LINE__
On Feb 12, 7:44 pm, Jeff Schwab <[EMAIL PROTECTED]> wrote: > It still would be nice to have syntax as clean as __FILE__ and __LINE__. There exists an undocumented builtin called __file__, but unfortunately no corresponding __line__ Alain -- http://mail.python.org/mailman/listinfo/python-list
Re: Python equivt of __FILE__ and __LINE__
On Feb 14, 1:50 am, "Gabriel Genellina" <[EMAIL PROTECTED]> wrote: > En Wed, 13 Feb 2008 13:07:31 -0200, alain <[EMAIL PROTECTED]> escribió: > > > There exists an undocumented builtin called __file__, but > > unfortunately no corresponding __line__ > > There is no __file__ builtin AFAIK; but there is __file__ module attribute > documented here:http://docs.python.org/ref/types.html#l2h-107 > > -- > Gabriel Genellina Sorry for the mistake, i should have double-checked. Alain -- http://mail.python.org/mailman/listinfo/python-list
import bug?
Hi all,
Running python 2.5, i experience a strange behaviour with the
following code:import imputil
def handle_pye(fullpath, fileinfo, name):
# Print a debugging message
print 'Importing "%s" from "%s"' % (name,fullpath)
data = open(fullpath).read()
return 0, compile(data,fullpath,'exec'),{}
im = imputil.ImportManager()
im.add_suffix('.pye',handle_pye)
im.install() # THIS SEEMS TO DO WEIRD THINGS TO SUBSEQUENT
IMPORTS
import functools
I then get the following traceback:
Traceback (most recent call last):
File "D:\deleteme\New1.py", line 12, in
import functools
File "C:\Python25\lib\imputil.py", line 103, in _import_hook
top_module = self._import_top_module(parts[0])
File "C:\Python25\lib\imputil.py", line 190, in _import_top_module
module = self.fs_imp.import_from_dir(item, name)
File "C:\Python25\lib\imputil.py", line 545, in import_from_dir
return self._process_result(result, fqname)
File "C:\Python25\lib\imputil.py", line 304, in _process_result
exec code in module.__dict__
File "C:\Python25\lib\functools.py", line 10, in
from _functools import partial
File "C:\Python25\lib\imputil.py", line 106, in _import_hook
raise ImportError, 'No module named ' + fqname
ImportError: No module named _functools
The mere fact of installing a custom importer seems to break the
import functionality.
Alain
--
http://mail.python.org/mailman/listinfo/python-list
Re: matrix algebra
On Sep 22, 11:32 am, "Tim Leslie" <[EMAIL PROTECTED]> wrote: > There is no need for a wrapper. Both numarray and Numeric have been > deprecated in favour of numpy, so numpy is the only one you need to > use. Numpy should have all the tools you need. If you find something > missing, there's a good chance it's included in scipy (which is built > on top of numpy). For full details seewww.scipy.org. I don't agree. I am myself a casual user of numpy without sophisticated needs. Numpy seems to be the only player in town and unfortunateely, though powerful, its syntax is not intuitive or easy to remember. I therefore welcome any attempt to simplify the api for a specific group of non-power users. Alain -- http://mail.python.org/mailman/listinfo/python-list
Re: Apache and suexec issue that wont let me run my python script
Νικόλαος Κούρας writes: [...] > [Thu May 30 15:29:33 2013] [error] [client 46.12.46.11] suexec failure: could > not open log file Here is a link to suexec documentation (at least some version of it, this is the second link provided by google): http://httpd.apache.org/docs/2.2/suexec.html Read this and check your setup to see if it matches. -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: Best Scripting Language for Embedded Work?
David T. Ashley writes: > We develop embedded software for 32-bit micros using Windows as the > development platform. I'll mostly ignore the "Windows" qualifier. If you're stuck with Windows CE or similar, then ask them what they suggest. If you're developing on Windows and deploy on something else (typically, some form of linux), then, well, think again. > We are seeking a general purpose scripting language to automate > certain tasks, like cleaning out certain directories of certain types > of files in preparation for ZIP'ing, generating certain source files > automatically, etc. That's exactly what "shells" are for. They are basically command interpreters providing a nice interface to various system utilities. Busybox is a variant where utilities are actually compiled in, and where you can compile your own stuff in as well. See http://en.wikipedia.org/wiki/Busybox > a)Should be able to compile the script interpreter as a monolithic > executable (no .DLL dependencies, etc.) for easy versioning and > distribution of the script interpreter. A fairly common requirement for shells. > b)Should be extensible, in that one could add commands or library > functions to the script interpreter in C (for efficiency), and the > whole script interpreter could again consist of a single executable > with no other dependencies. It looks like busybox is able to do this (the faq is fairly precise in implementation details---I've never done this but it looks kind of trivial). > c)Should be able to spawn compilers and capture the output, do file > I/O, and all the other minor expected stuff. Trivial for any shell, and probably for any scripting language. > d)Graphical capability would be nice. That's a large can of worms. I don't know any small, self-contained interpreter that includes a full-featured GUI framework. These things usually end up in shared libs, which you explicitely want to avoid... > I know that Tcl/Tk would do all of the above, but what about Python? > Any other alternatives? Most scripting languages have evolved way beyond "scripting" tasks, and usually rely on a fairly extensive set of libraries (either shared libraries or collection of modules). I haven't looked at tcl/tk since I know python, but they can probable be classified in the same category. Of course, expect python supporters to... support python (which, btw, interfaces with tk). Another contender is lua, which has a good reputation regarding embeddability. The answer really depends on your use case. I think you will be better off if you keep the GUI aspect separated from the rest. Here is stackoverflow entry discussing the use guis for (unix) shell scripts: http://stackoverflow.com/questions/928019/how-to-make-a-gui-for-bash-scripts (via google "linux shell script gui"). -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: Critic my module
Devyn Collier Johnson writes:
>I made a Python3 module that allows users to use certain Linux
> shell commands from Python3 more easily than using os.system(),
> subprocess.Popen(), or subprocess.getoutput(). This module (once
> placed with the other modules) can be used like this
Good, but I doubt it's really useful: I think nobody is going to add a
dependency on your module for, basically, one-line wrappers...
Here are a few comments:
> def ls():
> version = '0.3'
> print(subprocess.getoutput('ls'))
version is local here, so basically your first statement is useless
(search for "global" in python's language ref).
> def uname():
> version = '0.3'
> print(platform.uname())
I once learned: "never print anything in a library function". This is a
bad thing to do, for a variety of reasons. For instance, stdout may be
redirected during this call...
> def man(x):
> version = '0.3'
> print(subprocess.getoutput('man' + x))
getoutput is (essentially) Popen(...,shell=True), and the doc says:
"the use of shell=True is strongly discouraged in cases where the
command string is constructed from external input"
(for very good reasons)
> def clear_bash_history():
> version = '0.3'
> print(subprocess.getoutput('history -c'))
Who told you subprocess will use bash? Again, the doc:
"On Unix with shell=True, the shell defaults to /bin/sh."
All your uses of bash-isms may break (esp. "!!")
> def firefox():
> version = '0.3'
> print(subprocess.Popen('(firefox &)'))
See section "Replacing the os.spawn family" in... the doc.
> def go_back():
> version = '0.3'
> print(subprocess.Popen('cd !!:1'))
Hopeless. Have you tried this?
> def reboot():
> version = '0.3'
> print(subprocess.Popen('shutdown -r now'))
What do you expect this to print? I mean, after shutdown/reboot.
> version = '0.6b'
So, what's the version? 0.3 or 0.6b
(btw, are you sure this "version" is the same as the one you use in all
functions?).
-- Alain.
--
http://mail.python.org/mailman/listinfo/python-list
Re: code review
Thomas Jollans writes: >>>>> def is_valid_password(password): >>>>> return mud.minpass <= len(password) <= mud.maxpass > Which of the two comparisons is done first anyway? > "In the face of ambiguity, refuse the temptation to guess." There is no ambiguity. See the language reference: "Formally, if a, b, c, ..., y, z are expressions and op1, op2, ..., opN are comparison operators, then a op1 b op2 c ... y opN z is equivalent to a op1 b and b op2 c and ... y opN z, except that each expression is evaluated at most once." The last restriction (single evaluation of involved expressions) makes this a bit more than raw syntactic sugar. -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: code review
Thomas Jollans writes: > On 06/30/2012 11:47 PM, Terry Reedy wrote: >>>>>>>>> def is_valid_password(password): >>>>>>>>> return mud.minpass <= len(password) <= mud.maxpass >>>> >>>>> Which of the two comparisons is done first anyway? >>>>> "In the face of ambiguity, refuse the temptation to guess." >>>> >>>> There is no ambiguity. See the language reference: >>> Of course it's technically clearly defined, but the syntax isn't >>> explicit. >> Python pretty consistently evaluates expressions and equal precedence >> operators left to right. > > Yes. My sole point, really, is that "normally", one would expect these > two expressions to be equivalent: > > a < b < c > (a < b) < c > > This is clearly not true. That's the inconsistency here with the rest of > the language. No, comparison operators are different from arithmetic operators in that they always evaluate to a boolean. There are only rare cases where it makes sense to compare comparisons. > As soon as you read it as a ternary operator, the two comparisons are > logically simultaneous. There is no ternary operator, you can chain as many as you want, using whatever operators: if a <= b < c > d >= e: ... Once you view this as a conjonction of conditions, you find back the semantics of "and": short-circuit, left to right evaluation. I find this consistent. -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: the meaning of rï¾â¢.......ï¾
Henrik Faber writes: > On 23.07.2012 15:55, Henrik Faber wrote: > >> Dear Lord. >> >> Python 3.2 (r32:88445, Dec 8 2011, 15:26:58) >> [GCC 4.5.2] on linux2 >> Type "help", "copyright", "credits" or "license" for more information. >>>>> fööbär = 3 >>>>> fööbär >> 3 >> >> I didn't know this. How awful. > > Apparently, not all characters are fine with Python. Why can I not have > domino tiles are identifier characters? The answer is in the language reference: http://docs.python.org/py3k/reference/lexical_analysis.html#identifiers -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: print(....,file=sys.stderr) buffered?
Helmut Jarausch writes:
> On Mon, 13 Aug 2012 15:43:31 +, Grant Edwards wrote:
>
>> On 2012-08-13, Helmut Jarausch wrote:
>>> Hi,
>>>
>>> for tracing purposes I have added some print outs like
>>>
>>> print('+++ before calling foo',file=sys.stderr)
>>> x=foo(..)
>>> print('--- after calling foo',
>
> Sorry, this is a cut'n paste error. I did use
> print('--- after calling foo',file=sys.stderr)
>
>>>
>>> and within 'foo'
>>> print('>>> entering foo ...',file=sys.stderr)
>>>
>>> Now, when executing this, I always get
>>>
>>> +++ before calling foo
>>> --- after calling foo
>>>>>> entering foo ...
This can't happen with "normal" code. Are you playing with lambdas or
generators here? Or anything else that could introduce lazyness?
-- Alain.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Strange behavior
[email protected] writes: > However if you run the code you will notice only one of the strings > beginning with 'x' is removed from the startingList. > > def testFunc(startingList): > xOnlyList = []; > for str in startingList: > if (str[0] == 'x'): > print str; > xOnlyList.append(str) > startingList.remove(str) #this seems to be the problem > print xOnlyList; > print startingList > testFunc(['xasd', 'xjkl', 'sefwr', 'dfsews']) > > #Thanks for your help! Try with ['xasd', 'sefwr', 'xjkl', 'dfsews'] and you'll understand what happens. Also, have a look at: http://docs.python.org/reference/compound_stmts.html#the-for-statement You can't modify the list you're iterating on, better use another list to collect the result. -- Alain. P/S: str is a builtin, you'd better avoid assigning to it. -- http://mail.python.org/mailman/listinfo/python-list
Re: Strange behavior
Chris Angelico writes: > Other people have explained the problem with your code. I'll take this > example as a way of introducing you to one of Python's handy features > - it's an idea borrowed from functional languages, and is extremely > handy. It's called the "list comprehension", and can be looked up in > the docs under that name, > > def testFunc(startingList): > xOnlyList = [strng for strng in startingList if strng[0] == 'x'] > startingList = [strng for strng in startingList if strng[0] != 'x'] > print(xOnlyList) > print(startingList) > > It's a compact notation for building a list from another list. (Note > that I changed "str" to "strng" to avoid shadowing the built-in name > "str", as others suggested.) Fully agree with you: list comprehension is, imo, the most useful program construct ever. Extremely useful. But not when it makes the program traverse twice the same list, where one traversal is enough. -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: Strange behavior
[email protected] writes: > I got my answer by reading your posts and referring to: > http://docs.python.org/reference/compound_stmts.html#the-for-statement > (particularly the shaded grey box) Not that the problem is not specific to python (if you erase the current element when traversing a STL list in C++ you'll get a crash as well). > I guess I should have (obviously) looked at the doc's before posting > here; but im a noob. Python has several surprising features. I think it is a good idea to take some time to read the language reference, from cover to cover (before or after the various tutorials, depending on your background). -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: _mysql_exceptions.OperationalError: (2005, "Unknown MySQL server host
Hans Mulder writes: > On 15/08/12 15:30:26, nepaul wrote: >> The code: >> import MySQLDB >> strCmd = "user = 'root', passwd = '123456', db = 'test', host = 'localhost'" >> >> >> >> _mysql_exceptions.OperationalError: (2005, "Unknown MySQL server host 'user >> = 'root', >> passwd = '123456', db = 'test', host = 'localhost'' (11004)") > > This message means that the MySQL connector cannot find 'localhost'. No, it means that connect received a single string "user = 'root'..." instead of a set of individual keyword parameters, and took the whole string to be the name of the host (its first parameter). Of course, there is no host with such a name. The solution is to parse the string into individual values, and pass these in the correct order. -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: sqlalchemy.exc.ProgrammingError: (ProgrammingError) ('42000', "[42000] [Microsoft][ODBC SQL Server Driver][SQL Server]'f2f68'
nepaul writes:
> ===case1==:
> import sqlalchemy
> test1 = "631f2f68-8731-4561-889b-88ab1ae7c95a"
> cmdTest1 = "select * from analyseresult where uid = " + test1
> engine =
> sqlalchemy.create_engine("mssql+pyodbc://DumpResult:123456@localhost/DumpResult")
> c = engine.execute(cmdTest1)
> ==case2===:
> import sqlalchemy
> test2 = "123"
> cmdTest2 = "select * from analyseresult where uid = " + test2
> engine =
> sqlalchemy.create_engine("mssql+pyodbc://DumpResult:123456@localhost/DumpResult")
> c = engine.execute(cmdTest1)
>
>
> !
> case1 :wrong,(sqlalchemy.exc.ProgrammingError: (ProgrammingError) ('42000',
> "[42000] [Microsoft][ODBC SQL Server Driver][SQL Server]'f2f68') case2:work!
Print both cmdTest1 and cmdTest2. Look at them. Are they valid SQL
queries?
-- Alain.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Comparing strings from the back?
Roy Smith writes: > There's been a bunch of threads lately about string implementations, and > that got me thinking (which is often a dangerous thing). > > Let's assume you're testing two strings for equality. You've already > done the obvious quick tests (i.e they're the same length), and you're > down to the O(n) part of comparing every character. > > I'm wondering if it might be faster to start at the ends of the strings > instead of at the beginning? If the strings are indeed equal, it's the > same amount of work starting from either end. But, if it turns out that > for real-life situations, the ends of strings have more entropy than the > beginnings, the odds are you'll discover that they're unequal quicker by > starting at the end. I guess we all have examples with specific entropy distribution. Going backwards on long strings can be profitable with file paths. If that is the case, and if you spend a lot of time comparing strings, why not store them in reverse? > It doesn't seem un-plausible that this is the case. For example, most > of the filenames I work with begin with "/home/roy/". Most of the > strings I use as memcache keys have one of a small number of prefixes. > Most of the strings I use as logger names have common leading > substrings. In that case I would split the paths, and use some sort of string-id, or use intern() and then compare components with "is" instead of "==". (Basically, turning the string of chars into a strings of ids/ints/pointers.) > Things like credit card and telephone numbers tend to have much more > entropy in the trailing digits. On the other hand, hostnames (and thus > email addresses) exhibit the opposite pattern. Yes, real-world is a mess. > Anyway, it's just a thought. Has anybody studied this for real-life > usage patterns? I've tried the "intern()" trick several times with lists of strings, and was satisfied, but this was in specific cases (with a finite set of strings). For "short" strings of chars (up to, say a hundred of characters), I've never found anything significantly faster than the default sweep (that was in C/C++, not python). For longer and/or more structured strings/lists, making use of the structure is a good idea, but I see no general pattern here (as you note). > I'm also not sure how this work with all the possible UCS/UTF encodings. > With some of them, you may get the encoding semantics wrong if you don't > start from the front. If you're testing for equality, I can't see how this could matter, even with variable-length encodings. If you're comparing different encodings, then you need different access methods to random characters (but this doesn't affect the algorithm). If you're using variable-length encoding, e.g., UTF-8, accessing at a specific position is not possible. -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: which a is used?
Jayden writes: > # Begin > a = 1 > > def f(): > print a > > def g(): > a = 20 > f() > > g() > #End > > I think the results should be 20, but it is 1. Would you please tell me why? When python looks at g(), it sees that a variable a is assigned to, and decides it is a local variable. When it looks at f(), it sees a use of a but no assignment, so it decides it is a global variable and fetches the value from the outer scope. If you change f() to: def f(): print a a = 30 you change a into a local variable (and get another error). If you want to change the binding of a in g(), you can declare it global: def g(): global a a = 20 f() Very tricky, actually. -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: parse an environment file
Jason Friedman writes: [...] > I want my python 3.2.2 script, called via cron, to know what those > additional variables are. How? This is not a python question. Have a look at the crontab(5) man page, it's all explained there. -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: Combinations of lists
Steen Lysgaard writes:
> I am looking for a clever way to compute all combinations of two
> lists. Look at this example:
>
> h = ['A','A','B','B']
> m = ['a','b']
>
> the resulting combinations should be of the same length as h and each
> element in m can be used twice. The sought after result using h and m
> from above is:
>
> [['aA', 'aA', 'bB', 'bB'],
> ['aA', 'aB', 'bA', 'bB'],
> ['aB', 'aB', 'bA', 'bA']]
I can't make sense of your explanation, which doesn't seem to match your
example (the result is not of the same size as h).
Here is a way to compute { xh | x in m }, where xh is a list where x is
prepended to each element of h.
result = [ [ x+l for l in h ] for x in m ]
If there is no duplicate in the original lists, then there will be no
duplicate in the result. Is that what you are looking for?
-- Alain.
--
http://mail.python.org/mailman/listinfo/python-list
Re: pyw program not displaying unicode characters properly
jjmeric writes: > Our language lab at INALCO is using a nice language parsing and analysis > program written in Python. As you well know a lot of languages use > characters that can only be handled by unicode. > > Here is an example of the problem we have on some Windows computers. > In the attached screen-shot (DELETED), Usenet has no attachments. Place your document on some publicly accessible web-servers, if needed. > the bambara character (a sort of epsilon) is displayed as a square. > > The fact that it works fine on some computers and fails to display the > characters on others suggests that it is a user configuration issue: > Recent observations: it's OK on Windows 7 but not on Vista computers, > it's OK on some Windows XP computers, it's not on others Windows XP... You need a font that has glyphs for all unicode characters (at least the ones you use). See http://en.wikipedia.org/wiki/Unicode_font for a start. I don't know enough about Windows to give you a name. Anyone? -- Alain. P/S: and this has not much to do with python, which will happily send out any unicode char, and cannot know which ones your terminal/whatever will be able to display -- http://mail.python.org/mailman/listinfo/python-list
Re: pyw program not displaying unicode characters properly
Steven D'Aprano writes: > On Sun, 14 Oct 2012 19:19:33 +0200, Alain Ketterlin wrote: > >> Usenet has no attachments. > > *snarfle* > > You almost owed me a new monitor. I nearly sprayed my breakfast all over > it. [...] I owe you nothing, and you can do whatever you want with your breakfast. > "Usenet has no attachments" -- that's like saying that the Web has no > advertisements. Maybe the websites you visit have no advertisements, but > there's a *vast* (and often disturbing) part of the WWW that has > advertisements, some sites are nothing but advertisements.[...] I really don't know what you are ranting about here. See Dennis' response. Any idea about a reasonable complete unicode font on Windows? /That/ would be helpful. -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: ElementTree Issue - Search and remove elements
Tharanga Abeyseela writes:
> I need to remove the parent node, if a particular match found.
It looks like you can't get the parent of an Element with elementtree (I
would love to be proven wrong on this).
The solution is to find all nodes that have a Rating (grand-) child, and
then test explicitly for the value you're looking for.
>
> http://schemas..xx/xx/2011/06/13/xx";>
>
[...]
>
>
> M
> for child in
> root.findall(".//{http://schemas.CCC.com/CCC/2011/06/13/CC}Rating";):
>x = child.find('Rating').text
> if child[1].text == 'NC':
> print "found"
>root.remove('TVEpisode') ?
Your code doesn't work because findall() already returns Rating
elements, and these have no Rating child (so your first call to find()
fails, i.e., returns None). And list indexes starts at 0, btw.
Also, Rating is not a child of TVEpisode, it is a child of
ParentalControl.
Here is my suggestion:
# Find nodes having a ParentalControl child
for child in root.findall(".//*[ParentalControl]"):
x = child.find("ParentalControl/Rating").text
if x == "NC":
...
Note that a complete XPath implementation would make that simpler: your
query basically is //*[ParentalControl/Rating=="NC"]
-- Alain.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python does not take up available physical memory
Thomas Rachel writes: > Am 19.10.2012 21:03 schrieb Pradipto Banerjee: [...] >> Still got MemoryError, but at least this time python tried to use the >> physical memory. What I noticed is that before it gave me the error >> it used up to 1.5GB (of the 2.23 GB originally showed as available) - >> so in general, python takes up more memory than the size of the file >> itself. > > Of course - the file is not the only thing to be held by the process. > > I see several approaches here: > > * Process the file part by part - as the others already suggested, > line-wise, but if you have e.g. a binary file format, other partings > may be suitable as well - e.g. fixed block size, or parts given by the > file format. > > * If you absolutely have to keep the whole file data in memory, split > it up in several strings. Why? Well, the free space in virtual memory > is not necessarily contiguous. So even if you have 1.5G free, you > might not be able to read 1.5G at once, but you might succeed in > reading 3*0.5G. * try mmap, if you're lucky it will give you access to your data. (Note that it is completely unreasonable to load several Gs of data in a 32-bit address space, especially if this is text. So my real advice would be: * read the file line per line and pack the contents of every line into a list of objects; once you have all your stuff, process it -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: Split single file into multiple files based on patterns
satyam writes:
> I have a text file like this
>
> A1980JE3937 2732 4195 12.527000
> A1980JE3937 3465 9720 22.00
> A1980JE3937 2732 9720 18.00
> A1980KK18700010 130 303 4.985000
> A1980KK18700010 7 4915 0.435000
[...]
> I want to split the file and get multiple files like
> A1980JE3937.txt and A1980KK18700010.txt, where each file will
> contain column2, 3 and 4.
Sorry for being completely off-topic here, but awk has a very convenient
feature to deal with this. Simply use:
awk '{ print $2,$3,$4 > $1".txt"; }' /path/to/your/file
-- Alain.
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to pass class instance to a method?
ALeX inSide writes: > How to "statically type" an instance of class that I pass to a method > of other instance? Python does not do static typing. > I suppose there shall be some kind of method decorator to treat an > argument as an instance of class? Decorators are an option. Another is the use of the new parameter annotations (param : expr) in function/method parameters. (That's python 3, not 2). > Generally it is needed so IDE (PyCharm) can auto-complete instance's > methods and properties. You can't expect static info on the class of the object referenced by any name, unless you impose strong conventions on the code. > Pseudo-python-code example: > > i = MyClass() > > xxx(i, 1, 2); > > ... > def xxx(self, MyClass myclass, number, foobar): >myclass.classsmethod() #myclass - is an instance of known class Could: xxx(self,myclass : MyClass, ...) -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Vigil, the eternal morally vigilant programming language
I just came across Vigil, an extension to python for serious software engineers, at https://github.com/munificent/vigil and thought everybody in this group would be interested (sorry if it has been announced before). >From README: | Vigil is a very safe programming language, and an entry in the January | 2013 PLT Games competition. | | Many programming languages claim to take testing, contracts and safety | seriously, but only Vigil is truly vigilant about not allowing code | that fails to pass programmatic specifications. Enjoy. -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: Formate a number with commas
noydb writes:
> How do you format a number to print with commas?
>>>> import locale
>>>> locale.setlocale(locale.LC_ALL, "")
This sets the locale according to the environment (typically LANG---I'm
talking about linux, don't know others).
>>>> locale.format('%d', 2348721, True)
> '2,348,721'
This would not give the same result in environments with other locales
(typically de or fr or ...)
Anyway, it's probably the right thing to do: the user will get numbers
written according to its own locale.
If you really want commas whatever locale you're running in, you will
need to use setlocale to change number formatting to a locale that uses
commas. For instance:
locale.setlocale(LC_NUMERIC,"en_US.utf8")
locale.format('%d', 2348721, True)
-- Alain.
--
http://mail.python.org/mailman/listinfo/python-list
Re: subtraction of floating point numbers
Jaroslav Dobrek writes: > when I have Python subtract floating point numbers it yields weird > results. Example: > > 4822.40 - 4785.52 = 36.87992 We've had this discussion here one or two days ago... The usual answer is: please read "What Every Computer Scientist Should Know About Floating Point Arithmetic", at: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.22.6768 and check the answers posted these last days. In brief: you're working with floating point numbers, not reals (i.e., real "reals"). That's life. Deal with it, or move to specialized packages, like decimal. -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it technically possible to give Python option of naming process of running script?
Terry Reedy writes: > On 3/14/2012 12:02 PM, Grant Edwards wrote: > >> It seems like an excellent thing to add to the "os" module. > > If 'prctl' is a standard POSIX system call, then it should be a > candidate for inclusion in the os module if someone opens a tracker > enhancement issue and presents an argument in favor. It's not. The man page says "This call is Linux-specific." -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to do this snippet?
python writes: > tag23gr is a list of lists each with two items. > g23tag is an empty dictionary when I run the for loop below. > When is is complete each key is a graphic name who's values are a list > of tags. > > for item in tag23gr: > ... value, key = tuple(item) > ... if(g23tag.get(key)): > ... g23tag[key].append(value) > ... else: > ... g23tag[key] = [value] for item in tag23gr: g23tag.setdefault(item[0],[]).append(item[1]) -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to do this snippet?
nn writes: >> > for item in tag23gr: >> > ... value, key = tuple(item) >> > ... if(g23tag.get(key)): >> > ... g23tag[key].append(value) >> > ... else: >> > ... g23tag[key] = [value] >> >> for item in tag23gr: >> g23tag.setdefault(item[0],[]).append(item[1]) > Or alternatively: > > from collections import defaultdict > g23tag = defaultdict(list) > for item in tag23gr: > g23tag[item[0]].append(item[1]) Very handy in that case, but in general I dislike the idea of silently inserting a default value when the access is a read, e.g., in x=g23tag[wrung]. Explicit is better than implicit, as they say. YMMV. -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: No os.copy()? Why not?
Steven D'Aprano writes:
> On Tue, 03 Apr 2012 15:46:31 -0400, D'Arcy Cain wrote:
>
>> On 03/28/12 16:12, John Ladasky wrote:
>>> I'm looking for a Python (2.7) equivalent to the Unix "cp" command.
>>>>>open("outfile", "w").write(open("infile").read())
> Because your cp doesn't copy the FILE, it copies the file's CONTENTS,
> which are not the same thing.
> Consider:
> * permissions
> * access times
> * file ownership
> * other metadata
> * alternate streams and/or resource fork, on platforms that support them
> * sparse files
> By the time you finish supporting the concept of copying the file itself,
> rather than merely its content, you will have something similar to the
> shutil.copy command -- only less tested.
A minor point, but shutil.copy only "copies" contents and permissions
(no access times, etc.) You probably mean shutil.copy2.
And sparse files are really hard to reproduce, at least on Unix: on
Linux even the system's cp doesn't guarantee sparseness of the copy (the
manual mentions a "crude heuristic").
But of course shutil.copy is the best solution to mimic a raw cp.
-- Alain.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python Gotcha's?
Miki Tebeka writes: [...] > (Note that I want over http://wiki.python.org/moin/PythonWarts already). The "local variable and scoping" is, imho, something to be really careful about. Here is an example: class A(object): def __init__(self): self.x = 0 def r(self): return x # forgot self a = A() x = 1 print a.r() # prints 1 I know there is "no remedy". It's just really tricky. -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python randomly exits with Linux OS error -9 or -15
Janis writes: > I have this problem with my script exiting randomly with Linux OS > status code -9 (most often) or -15 (also sometimes, but much more > rarely). As far as I understand -9 corresponds to Bad file descriptor > and -15 Block device required. How do you get -9 and -15? Exit status is supposed to be between 0 and 127. With bash, 128+N is also used for processes that terminate on a signal. At the python level, subprocess.wait() uses negative numbers for signal-terminated processes. And 9 is SIGKILL and 15 is SIGTERM. My guess is that your script hits a limit, e.g., number of open files, or stack-size, or... But of course it's only a guess. -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: system call that is killed after n seconds if not finished
Jaroslav Dobrek writes:
> I would like to execute shell commands, but only if their execution
> time is not longer than n seconds. Like so:
>
> monitor(os.system("do_something"), 5)
>
> I.e. the command do_somthing should be executed by the operating
> system. If the call has not finished after 5 seconds, the process
> should be killed.
>
> How could this be done?
My system (linux) has a timeout command that does just this. Otherwise,
you may use subprocess.Popen to spawn the process, sleep for 5 seconds
in the parent, then use Popen.poll() to check whether the process has
finished, and finally Popen.kill() if it has not.
-- Alain.
--
http://mail.python.org/mailman/listinfo/python-list
Re: why () is () and [] is [] work in other way?
dmitrey writes: > I have spent some time searching for a bug in my code, it was due to > different work of "is" with () and []: >>>> () is () > True >>>> [] is [] > False > > (Python 2.7.2+ (default, Oct 4 2011, 20:03:08) > [GCC 4.6.1] ) > > Is this what it should be or maybe yielding unified result is better? Tuples are immutable, while lists are not. Because tuples are immutable, there is no point in having several empty tuples: one value is enough, and "is" recognizes this. Lists are different, they can change "value" (contents), and several names can be bound to the same list (in other words, you can have side-effects on a list). With: a = [] b = a a.append(42) print b you get [42]. If instead you do: a = [] b = [] a.append(42) print b you get the expected []. I.e., you need several distinct empty lists, to make sure they can change value independently. (In case you wonder: lists in Python are not linked lists of cons-cells, they are more monolithic structures, and two lists cannot share a common tail.) -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: tiny script has memory leak
gry writes: > sys.version --> '2.6 (r26:66714, Feb 21 2009, 02:16:04) \n[GCC 4.3.2 > [gcc-4_3-branch revision 141291]] > I thought this script would be very lean and fast, but with a large > value for n (like 15), it uses 26G of virtural memory, and things > start to crumble. > > #!/usr/bin/env python > '''write a file of random integers. args are: file-name how-many''' > import sys, random > > f = open(sys.argv[1], 'w') > n = int(sys.argv[2]) > for i in xrange(n): > print >>f, random.randint(0, sys.maxint) > f.close() sys.version is '2.6.6 (r266:84292, Sep 15 2010, 16:22:56) \n[GCC 4.4.5]' here, and your script works like a charm. BTW, I would use f.write() instead of print >> f (which I think is deprecated). -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: Dynamic comparison operators
[email protected] writes: > I would like to pass something like this into a function > test(val1,val2,'>=') > > and it should come back with True or False. def test(x,y,c): return c(x,y) Call with: test(v1,v2, lambda x,y:x<=y ). A bit noisy imho. If you have a finite number of comparison operators, put them in a dict: compares = dict([ ("<",lambda x,y:xhttp://mail.python.org/mailman/listinfo/python-list
Re: ./configure
Janet Heath writes: [...] > configure:3161: checking machine type as reported by uname -m > configure:3164: result: x86_64 > configure:3177: checking for --without-gcc > configure:3221: result: no > configure:3282: checking for gcc > configure:3312: result: no > configure:3375: checking for cc > configure:3422: result: no > configure:3478: checking for cl.exe > configure:3508: result: no > configure:3532: error: in `/usr/bin/Python-2.7.3': > configure:3534: error: no acceptable C compiler found in $PATH > See `config.log' for more details > > I can't seem to figure out the 2 errors at the bottom. Any suggestions? It looks like you have no C compiler (if you have one, configure can't find it in the obvious way). You need a C compiler to compile python. You should first install a C compiler. But are you sure you need to compile python? Isn't there a binary package available for your platform? -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: Help needed with nested parsing of file into objects
richard writes:
> Hi guys i am having a bit of dificulty finding the best approach /
> solution to parsing a file into a list of objects / nested objects any
> help would be greatly appreciated.
>
> #file format to parse .txt
> [code]
> An instance of TestArray
> a=a
> b=b
> c=c
> List of 2 A elements:
> Instance of A element
[...]
Below is a piece of code that seems to work on your data. It builds a
raw tree, i leave it to you to adapt and built the objects you want. The
assumption is that the number of leading blanks faithfully denotes
depth.
As noted in another message, you're probably better off using an
existing syntax (json, python literals, yaml, xml, ...)
-- Alain.
#!/usr/bin/env python
import sys
import re
RE = re.compile("( *)(.*)")
stack = [("-",[])] # tree nodes are: (head,[children])
for line in sys.stdin:
matches = RE.match(line)
if len(matches.group(2)) > 0:
depth = 1 + len(matches.group(1))
while len(stack) > depth:
stack[-2][1].append(stack[-1])
del stack[-1]
pass
stack.append( (matches.group(2),[]) )
pass
pass
while len(stack) > 1:
stack[-2][1].append(stack[-1])
del stack[-1]
pass
print(stack)
--
http://mail.python.org/mailman/listinfo/python-list
Re: Help needed with nested parsing of file into objects
richard writes:
[I'm leaving the data in the message in case anybody has troubles going
up-thread.]
> Hi guys still struggling to get the code that was posted to me on this
> forum to work in my favour and get the output in the format shown
> above. This is what I have so far. Any help will be greatly
> apprectiated.
>
> output trying to achieve
> parsed = [
> {
> "a":"a",
> "b":"b",
> "c":"c",
> "A_elements":[
> {
> "a":1,
> "b":2,
> "c":3
> },
> {
> "a":1,
> "b":2,
> "c":3
> }
>],
> "B_elements":[
> {
> "a":1,
> "b":2,
> "c":3,
> "C_elements":[
> {
> "a":1,
> "b":2,
> "c":3
> },
> {
> "a":1,
> "b":2,
> "c":3
> }
> ]
> }
>]
> },
>
> {
> "a":"1",
> "b":"2",
> "c":"3",
> }
>
> ]
>
> file format unchangeable
>
> An instance of TestArray
> a=a
> b=b
> c=c
> List of 2 A elements:
> Instance of A element
>a=1
>b=2
>c=3
> Instance of A element
>d=1
>e=2
>f=3
> List of 1 B elements
> Instance of B element
>a=1
>b=2
>c=3
>List of 2 C elements
> Instance of C element
> a=1
> b=2
> c=3
> Instance of C element
> a=1
> b=2
> c=3
>
> An instance of TestArray
> a=1
> b=2
> c=3
>
> def test_parser(filename):
> class Stanza:
> def __init__(self, values):
> for attr, val in values:
> setattr(self, attr, val)
>
> def build(couple):
> if "=" in couple[0]:
> attr, val = couple[0].split("=")
> return attr,val
> elif "Instance of" in couple[0]:
> match = re.search("Instance of (.+) element", couple[0])
> return ("attr_%s" % match.group(1),Stanza(couple[1]))
> elif "List of" in couple[0]:
> match = re.search("List of \d (.+) elements", couple[0])
> return ("%s_elements" % match.group(1),couple[1])
You forgot one case:
def build(couple):
if "=" in couple[0]:
attr, val = couple[0].split("=")
return attr,val
elif "Instance of" in couple[0]:
#match = re.search("Instance of (.+) element", couple[0])
#return ("attr_%s" % match.group(1),Stanza(couple[1]))
return dict(couple[1])
elif "An instance of" in couple[0]: # you forgot that case
return dict(couple[1])
elif "List of" in couple[0]:
match = re.search("List of \d (.+) elements", couple[0])
return ("%s_elements" % match.group(1),couple[1])
else:
pass # put a test here
> fo = open(filename, "r")
> RE = re.compile("( *)(.*)")
> stack = [("-",[])]
> for line in fo:
> matches = RE.match(line)
> if len(matches.group(2)) > 0:
> depth = 1 + len(matches.group(1))
> while len(stack) > depth:
> stack[-2][1].append(build(stack[-1]))
> del stack[-1]
> stack.append( (matches.group(2),[]) )
> while len(stack) > 1:
> stack[-2][1].append(stack[-1])
Change this to:
stack[-2][1].append(build(stack[-1])) # call build() here also
> del stack[-1]
> return stack
Actually the first and only element of stack is a container: all you
need is the second element of the only tuple in stack, so:
return stack[0][1]
and this is your list. If you need it pretty printed, you'll have to
work the hierarchy.
-- Alain.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Help needed with nested parsing of file into objects
richard writes:
>> > An instance of TestArray
>> > a=a
>> > b=b
>> > c=c
>> > List of 2 A elements:
>> > Instance of A element
>> > a=1
>> > b=2
>> > c=3
>> > Instance of A element
>> > d=1
>> > e=2
>> > f=3
>> > List of 1 B elements
>> > Instance of B element
>> > a=1
>> > b=2
>> > c=3
>> > List of 2 C elements
>> > Instance of C element
>> > a=1
>> > b=2
>> > c=3
>> > Instance of C element
>> > a=1
>> > b=2
>> > c=3
[...]
> Hi Alain thanks for the reply. With regards to the missing case "An
> Instance of" im not sure where/ how that is working as the case i put
> in originally "Instance of" is in the file and been handled in the
> previous case.
Both cases are different in your example above. Top level elements are
labeled "An instance ...", whereas "inner" instances are labeled
"Instance of ...".
> Also when running the final solution im getting a list of [None, None]
> as the final stack?
There's only one way this can happen: by falling through to the last
case of build(). Check the regexps etc. again.
> just busy debugging it to see whats going wrong. But sorry should have
> been clearer with regards to the format mentioned above. The objects
> are been printed out as dicts so where you put in
>
> elif "An Instance of" in couple[0]:
> return dict(couple[1])
>
> should still be ?
> elif "Instance of" in couple[0]:
> match = re.search("Instance of (.+) element", couple[0])
> return ("attr_%s" % match.group(1),Stanza(couple[1])) #
> instantiating new stanza object and setting attributes.
Your last "Instance of..." case is correct, but "An instance..." is
different, because there's no containing object, so it's probably more
like: return Stanza(couple[1]).
-- Alain.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Compare 2 times
Alain Ketterlin writes: > loial writes: > >> I have a requirement to test the creation time of a file with the >> current time and raise a message if the file is more than 15 minutes >> old. >> Platform is Unix. >> I have looked at using os.path.getctime for the file creation time and >> time.time() for the current time, but is this the best approach? > > No. getctime() returns the last "change" time. The creation time is not > kept anywhere. This may still match your requirement though. And os.path > is the right package to look at for such tasks. Sorry, it may happen that the filesystem you're working with provides that time, in which case it's called birthtime. This _may_ be available via os.stat(). -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: [newbie] Equivalent to PHP?
Gilles writes: > I notice that Python-based solutions are usually built as long-running > processes with their own web server (or can run in the back with eg. > Nginx and be reached through eg. FastCGI/WSGI ) while PHP is simply a > language to write scripts and requires a web server (short running > processes). It's an artefact of the server infrastructure, there is no rule here. Any solution used with one language could be used with the other. > Since web scripts are usually very short anyway (user sends query, > server handles request, sends response, and closes the port) because > the user is waiting and browsers usually give up after 30 seconds > anyway... why did Python solutions go for long-running processes while > PHP was built from the start as short-running processes? You misunderstand the problem here. It's not about the duration of the actions, it's about the latency it takes to read/parse/execute the script. HTTP is stateless anyway, so if the same "interpreter" handles several requests, what you save by keeping the interpreter alive is the load/parse phase. If you relaunch an interpreter for every HTTP request, you pay the same price again and again for something which is not even related to your scripts' execution. -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: Something is rotten in Denmark...
Steven D'Aprano writes: >> The part that I don't see much about in the docs (some books, that is) >> is that the lambda lookups occur late (the lambda is evaluated at the >> time it is called). The Python docs on-line *do say* this (I found too >> late) but its one quick phrase that can be missed. So, the i in >> range(10) is sitting there at '9' by the time *any* of the ten lambdas >> get called. This is not intuitive, nor good. IMHO > > I agree it's not intuitive. But where does it say that programming > language semantics must always be intuitive? Nowhere. But going against generally accepted semantics should at least be clearly indicated. Lambda is one of the oldest computing abstraction, and they are at the core of any functional programming language. Adding a quick hack to python and call it "lambda" is just abuse of terminology (I don't say python is the only abuser, implementing lambda is difficult; see, e.g., Apple C extension called "blocks" and their implementation of binding). > What do you expect this code to do? > > a = 42 > funcs = [(lambda x: x+a) for i in range(10)] > funcs[0](1) [...] > Do you agree that `a` should be late bound in this situation? No, absolutely, definitely not. But hey, I didn't learn programming with python. -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: Something is rotten in Denmark...
Gregory Ewing writes: > Alain Ketterlin wrote: >> But going against generally accepted semantics should at least >> be clearly indicated. Lambda is one of the oldest computing abstraction, >> and they are at the core of any functional programming language. > > Yes, and Python's lambdas behave exactly the *same* way as > every other language's lambdas in this area. Changing it to > do early binding would be "going against generally accepted > semantics". You must be kidding. Like many others, you seem to think that Scheme is a typical functional language, which it is not. Learn about ML (i.e., SML or CaML), Erlang, Haskell, etc. You can read, e.g., http://en.wikipedia.org/wiki/Closure_%28computer_science%29 The reason why we have the kind of lambdas we have in python (and scheme, and javascript, etc.) is just that it is way easier to implement. That's all I've said. And people have gotten used to it, without ever realizing they are using something completely different from what Church called the "lambda abstraction". Whether the python/... concept of lambda is useful or not is another, subjective question, that I'm not intersted in. If you're pleased with it, go ahead. (End of discussion for me.) -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: Standard Deviation One-liner
Billy Mays writes: > I'm trying to shorten a one-liner I have for calculating the standard > deviation of a list of numbers. I have something so far, but I was > wondering if it could be made any shorter (without imports). > a=lambda d:(sum((x-1.*sum(d)/len(d))**2 for x in d)/(1.*(len(d)-1)))**.5 You should make it two half-liners, because this one repeatedly computes sum(d). I would suggest: aux = lambda s1,s2,n: (s2 - s1*s1/n)/(n-1) sv = lambda d: aux(sum(d),sum(x*x for x in d),len(d)) (after some algebra). Completely untested, assumes data come in as floats. You get the idea. -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: Standard Deviation One-liner
Alain Ketterlin writes: > aux = lambda s1,s2,n: (s2 - s1*s1/n)/(n-1) > sv = lambda d: aux(sum(d),sum(x*x for x in d),len(d)) Err, sorry, the final square root is missing. -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: Lambda question
writes:
>>>> f = lambda x, n, acc=[]: f(x[n:], n, acc+[(x[:n])]) if x else acc
>>>> f("Hallo Welt", 3)
> ['Hal', 'lo ', 'Wel', 't']
>
> http://stackoverflow.com/questions/312443/how-do-you-split-a-list-into-evenly-s
> ized-chunks-in-python/312644
>
> It doesn't work with a huge list, but looks like it could be handy in certain
> circumstances. I'm trying to understand this code, but am totally lost.
With such dense code, it is a good idea to rewrite the code using some
more familiar (but equivalent) constructions. In that case:
f = x, n, acc=[]:
x
f(x[n:], n, acc+[(x[:n])])
acc
I've made one major change here: the " if else
" *expression* has been changed to an "if-the-else"
*instruction*. I use etc. to emphasize the fact that it is somehow
special, but it should run as the usual if-then-else construct. This
transformation is correct here only because 1) it has both "then" and
"else" branches, and 2) both branches evaluate an *expression* (i.e.,
they are of the form ..., and nothing else).
What now remains is to understand the logic of the computation. It is a
recursive definition of f, so it has a base case and a recursion case.
Note that the base case (my branch) does nothing except returning
what it gets as the third parameter. Wow, this code is in some sort able
to "anticipate": in some cases, f is called with a pre-cooked result
(it's often called an accumulator: some calling function has accumulated
data for f to use). Since f is calling f, it means that, even when f has
to call itself, it can still make some progress towards the final
result.
Now look at the recursive call: when we are in a situation where we
cannot make a final decision, we simply chop of (at most) n items
from the start of input list. If we do this, we're left with a (possibly
empty) list "tail" (x[n:]), and we've found a part of the result
(x[:n]).
How does the whole thing work. Imagine a sequence of calls to f, each
one contributing some part of the result (a n-letter chunk):
... -> f -> f -> f -> ... -> f (done)
In this chain of recursive calls, each call to f except the last
contributes one chunk, "accumulates" it in a partial result, and
computes the work that "remains" for the subsequent calls. The last call
"knows" it is the last, and simply acknowledges the fact that all
previous calls have done all the work. The acumulator gets filled along
this chain.
There are a few details that we need to make sure of:
1) what if the initial list has a lentgh that isn't a multiple of n?
This is taken care of by python's slicing (x[:n] will only go as far as
possible, maybe less than n items; and x[n:] will be empty if x has less
than n elements)
2) Where does the accumulator come from? The first call uses the default
value declared in the lambda parameters. Calling f("abcd",2) is like
calling f("abcd",2,[]).
We could have done this differently: for instance
f = lambda x,n: [x[:n]] + f(x[n:],n) if x else []
This has no accumulator, because the result is computed "the other way
round": subsequent calls are left with the tail of the list, return the
result, and then we put the starting chunk in front of the result. No
need for an accumulator, the result is built when "coming back" from
recursive calls (i.e., from right to left in the chain of calls pictured
as above). Somewhat surprisingly, this is usually less efficient than
the one you show. The reason is that here there is some work to do
before the recursive call (extracting a chunk) *and* after the call
(pasting together the chunk with the result coming back from the
recursive call). Therefore, all intermediate results have to be kept for
intermediate calls. This doesn't happen in your version: an intermediate
call was updating a "global" partial result (acc) and that was it. (This
remark has a lot of technical implications.)
-- Alain.
P/S: wikipedia has some info on "recursion" to start with if you want lo
learn more.
--
http://mail.python.org/mailman/listinfo/python-list
Re: International translation of docs - is it a scam?
Chris Gonnerman writes: > On the 30th of May, I received an email from a man (I'll leave out his > name, but it was properly male) offering to translate the docs for the > gdmodule (which I maintain) into Belorussian. [...] The same has happened on the gcc list, where it has been considered a scam. See, e.g., http://gcc.gnu.org/ml/gcc/2011-05/msg00046.html and messages referenced therein. -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: where the function has problem? n = 900 is OK , but n = 1000 is ERROR
Billy Mays <[email protected]> writes: > On 08/01/2011 06:06 PM, Steven D'Aprano wrote: >> Does your definition of "fixed" mean "gives wrong results for n>= 4 "? > Well, I don't know if you're trolling or just dumb: Steven is right, and you look dumb. >>> fibo(4) 3.0004 Even though the math is correct, your program is wrong. It doesn't even produce integers. And it will fail with overflow for big values. -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: Early binding as an option
Chris Angelico writes: > As I understand it, Python exclusively late-binds names; when you > define a function, nothing is ever pre-bound. This allows a huge > amount of flexibility (letting you "reach into" someone else's > function and change its behaviour), but it's flexibility that most > programs use seldom if at all. I agree with you on your last remark, but unfortunately it's part of the language. Therefore, there *are* programs that rely on the ability to rebind 'let' and others. Changing this would require changing the language, basically turning some builtins into keywords. (BTW, the dynamic binding also has implications for security.) [...] > Argument in favour: Efficiency is also a Good Thing, and with staples > like 'len', it's unlikely anyone will want to change them - yet the > interpreter still has to do a namespace lookup every time. Yes, and it can't do common subexpression elimination, code hoisting, etc. Basically, nothing can be optimized, and the interpreter has to execute bytecode that exactly represents source code. > I would want the end programmer to have the ultimate power here (not > the module designer). Something along the lines of: This global name > will never change, so don't bother looking it up every time. Maybe some module could provide specialized, "use-at-your-own-risk" versions of some functions/operators. An example is '+' which can mean so many things that any use of it probably spends more time finding the right version than actually doing the work. The problem with such pre-bound identifiers is that anybody with performance problems would start peppering his/her code with things like plus_float_float(x,y), leading to unreadable code, to all kinds of strange errors, etc. Nobody really wants this probably. [...] > Is this the realm of JIT compilation, or can it be done in regular > CPython? No, it's a matter of language definition. A JIT can't do much here (actually jitting is almost orthogonal to that question), at least it couldn't do much better than CPython. It just has to go through all the lookups. IIRC, unladden-swallow has tried the JIT route, using LLVM as the backend. It seems they gave up. -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: Early binding as an option
Chris Angelico writes: [...] > The only actual data I have on the subject is the perfect-numbers > search the other day; Pike managed the same task in a fraction of the > time that Python spent on it. Pike has a single integer type that > quietly switches from small to large at 2**32, with a noticeable > performance change. This is the same as Python 2, but could explain > some of the Python 3 penalty. There's only two obvious possibilities > (there may be others, of course): firstly, that the actual name lookup > is expensive; and secondly, that Pike is able to optimize integer > arithmetic by knowing that the value in question is an int, and it > will never be anything else. Pike is (at least partly) statically typed. Most of the lookups are solved at compile time, and have therefore zero overhead at run-time. So your second possibility is the good one, but probably not because of arithmetic optims, rather because of all the lookups Pike doesn't perform dynamically. -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: Java is killing me! (AKA: Java for Pythonheads?)
kj writes: [...] >def quant(xs, nlevels=MAXN, xlim=MAXX): [...] > My Java implementation of it already requires at least 8 method > definitions, with signatures: (BTW, your approach won't work if several optionals have the same type.) [...] - use Integer, Float, etc. everywhere. The compiler will box primitive types as needed at call sites - re. default values (on positional params), use null to indicate a use of the default, or use ellipsis, or use class-provided static fields at call sites, or use a special class to represent all the optional parameters - re. named parameters with default, use a map, or change the call sites to remove them -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: testing if a list contains a sublist
Roy Smith writes: >> what is the best way to check if a given list (lets call it l1) is >> totally contained in a second list (l2)? [...] > import re > > def sublist(l1, l2): > s1 = ''.join(map(str, l1)) > s2 = ''.join(map(str, l2)) > return re.search(s1, s2) This is complete nonsense (try it on [12] and [1,2]). The original problem is "string searching" (where strings here are sequences of numbers instead of characters). See http://en.wikipedia.org/wiki/String_searching_algorithm for any algorithm (Rabin-Karp seems appropriate to me). -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: testing if a list contains a sublist
Laszlo Nagy writes: >>>>> def sublist(lst1, lst2): >> s1 = ','.join(map(str, lst1)) >> s2 = ','.join(map(str, lst2)) >> return False if s2.find(s1)==-1 else True >> >> I don't know about best, but it works for the examples given. >> > For numbers, it will always work. I'm not even sure: if str() is locale-sensitive, it may introduce commas in numbers (I don't know whether str() is locale-sensitive, the doc doesn't mention anything.) > But what about > > lst1 = [",",",,"] > lst1 = [",",",",","] Yes. It will also fail on nested lists, for fundamental reasons which are impossible to handle with regexps. (Tough I'm not sure the OP had nested lists in mind.) The "brute-force" algorithm given somewhere else in this thread is probably the way to go, unless the lists are really long, in which case one of the "string searching" algorithm should be used (I would be surprised noone has implemented Boyer-Moore or Karp-Rabin). -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: pairwise combination of two lists
Yingjie Lin writes: > I have two lists: > > li1 = ['a', 'b'] > li2 = ['1', '2'] > > and I wish to obtain a list like this > > li3 = ['a1', 'a2', 'b1', 'b2'] > > Is there a handy and efficient function to do this, especially when > li1 and li2 are long lists. It's not difficult to write your own: def product(l1,l2): for x1 in l1: for x2 in l2: yield x1+x2 use it like: for p in product(l1,l2) ... The call to product() produces a generator, the cross product is never built in full, it should work on long lists. -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: List spam
"Jason Staudenmayer" writes: > I really like this list as part of my learning tools but the amount of > spam that I've been getting from it is CRAZY. Doesn't anything get > scanned before it sent to the list? I'm using nntp to read this newsgroup (through an academic server). No spam at all. -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: List spam
Ghodmode writes: [...] > Make an effort to curb the spam even if it means killing the newsgroup > availability. Choose mailman or Google Groups, or another single > solution. Make it members only, but allow anyone to register with an > automated confirmation email and a CAPTCHA. Appoint a list admin who > has a few minutes each day to scan subjects of emails for spammers and > remove them from the members list. Or save work and find a public nntp server (or setup one, or ask your provider), and use a news reader to follow the list (even thunderbird can do this). No spam, no need to store messages on your machine, auto-purge after a configurable delay, etc. Problem solved. -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: List spam
gene heskett writes: >> Or save work and find a public nntp server (or setup one, or ask your >> provider), and use a news reader to follow the list (even thunderbird >> can do this). No spam, no need to store messages on your machine, >> auto-purge after a configurable delay, etc. > That is asking the user to take considerable effort and resources to do > that. What is wrong with the mailing list only approach? Nothing really. Regarding effort and resources, once you've found a NNTP server there's very little effort (probably less than subscribing to a mailing list). I have 4 lines in my .emacs. And this lets me browse dozens of groups (or thousands if I had time for this). It might not be easy to find a server which will let you post, but that's because a few years back many internet providers decided that nntp was too much traffic. I guess it would now be considered ridiculous compared to the average web-site. But I'd like to return the question. What's wrong with nntp? It looks like everybody agrees that nntp brings spam. I just wanted to say that's not true, I use nntp extensively and haven't seen spam for months (I'm talking about 15-20 groups, not comp.lang.python only). The real problem here seems to be google groups, which in some way forwards spam to the mailing-list. How this happens is beyond my understanding. But let's try to fix the real problem. -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: Help with regular expression in python
Matt Funk writes:
> thanks for the suggestion. I guess i had found another way around the
> problem as well. But i really wanted to match the line exactly and i
> wanted to know why it doesn't work. That is less for the purpose of
> getting the thing to work but more because it greatly annoys me off that
> i can't figure out why it doesn't work. I.e. why the expression is not
> matches {32} times. I just don't get it.
Because a line is not 32 times a number, it is a number followed by 31
times "a space followed by a number". Using Jason's regexp, you can
build the regexp step by step:
number = r"\d\.\d+e\+\d+"
numbersequence = r"%s( %s){31}" % (number,number)
There are better ways to build your regexp, but I think this one is
convenient to answer your question. You still have to append what will
match the end of the line.
-- Alain.
P/S: please do not top-post
>> $ python
>> Python 2.6.5 (r265:79063, Apr 16 2010, 13:57:41)
>> [GCC 4.4.3] on linux2
>> Type "help", "copyright", "credits" or "license" for more information.
>>>>> data
>> '1.002000e+01 2.037000e+01 2.128000e+01 1.908000e+01 1.871000e+01
>> 1.914000e+01 2.007000e+01 1.664000e+01 2.204000e+01 2.109000e+01
>> 2.209000e+01 2.376000e+01 2.158000e+01 2.177000e+01 2.152000e+01
>> 2.267000e+01 1.084000e+01 1.671000e+01 1.888000e+01 1.854000e+01
>> 2.064000e+01 2.00e+01 2.20e+01 2.139000e+01 2.137000e+01
>> 2.178000e+01 2.179000e+01 2.123000e+01 2.201000e+01 2.15e+01
>> 2.15e+01 2.199000e+01 : (instance: 0) : some
>> description'
>>>>> import re
>>>>> re.findall(r"\d\.\d+e\+\d+", data)
>> ['1.002000e+01', '2.037000e+01', '2.128000e+01', '1.908000e+01',
>> '1.871000e+01', '1.914000e+01', '2.007000e+01', '1.664000e+01',
>> '2.204000e+01', '2.109000e+01', '2.209000e+01', '2.376000e+01',
>> '2.158000e+01', '2.177000e+01', '2.152000e+01', '2.267000e+01',
>> '1.084000e+01', '1.671000e+01', '1.888000e+01', '1.854000e+01',
>> '2.064000e+01', '2.00e+01', '2.20e+01', '2.139000e+01',
>> '2.137000e+01', '2.178000e+01', '2.179000e+01', '2.123000e+01',
>> '2.201000e+01', '2.15e+01', '2.15e+01', '2.199000e+01']
--
http://mail.python.org/mailman/listinfo/python-list
Re: Why doesn't threading.join() return a value?
Adam Skutt writes: > On Sep 2, 10:53 am, Roy Smith wrote: >> I have a function I want to run in a thread and return a value. It >> seems like the most obvious way to do this is to have my target >> function return the value, the Thread object stash that someplace, and >> return it as the return value for join(). >> > Yes, I know there's other ways for a thread to return values (pass the >> target a queue, for example), but making the return value of the >> target function available would have been the most convenient. I'm >> curious why threading wasn't implemented this way. > > I assume it is because the underlying operating system APIs do not > support it. Windows and POSIX threads only support returning an > integer when a thread exits, similar to the exit code of a process. Sorry, you're wrong, at least for POSIX threads: void pthread_exit(void *value_ptr); int pthread_join(pthread_t thread, void **value_ptr); pthread_exit can pass anything, and that value will be retrieved with pthread_join. Threads of a process share their address space, there is no reason to restrict their return value to an int. > More importantly, there's no way to tell whether the exit code of a > thread was set by user code or by the system. Even worse, some of > those integer values are reserved by some operating systems. I'm not sure what you are talking about here. Maybe you confuse threads with processes? Re. the original question: since you can define your own Thread subclass, with wathever attribute you want, I guess there was no need to use join() to communicate the result. The Thread's run() can store its result in an attribute, and the "client" can get it from the same attribute after a successful call to join(). -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why doesn't threading.join() return a value?
Adam Skutt writes: > On Sep 2, 2:23 pm, Alain Ketterlin > wrote: >> Sorry, you're wrong, at least for POSIX threads: >> >> void pthread_exit(void *value_ptr); >> int pthread_join(pthread_t thread, void **value_ptr); >> >> pthread_exit can pass anything, and that value will be retrieved with >> pthread_join. > > No, it can only pass a void*, which isn't much better than passing an > int. We'll have to disagree. A void* simply can point to anything you want. Since thread stacks disappear at end of thread, only dynamically allocated memory can be used to store the result. That's why you get a pointer. There is no restriction on that pointer provided it doesn't point to memory that has been deallocated. > Passing a void* is not equivalent to passing anything, not even in C. > Moreover, specific values are still reserved, like PTHREAD_CANCELLED. Thread cancellation is program logic (pthread_cancel), it doesn't mean you thread crashed, it means your program decided to cancel the thread. If you still care about the return value after having called pthread_cancel(), > Yes, it was strictly inappropriate for me to say both return solely > integers, but my error doesn't meaningful alter my description of the > situation. The interface provided by the underlying APIs is not > especially usable for arbitrary data transfer. Again, I may misunderstand your wording, but there is no "data transfer" at all, since memory is shared between threads. > Doubly so when we're discussing something like Python's threading > module. The OP was clearly discussing the case where a thread has a result, and how to get it back. POSIX threads let you do that. There are of course tons of other ways to do the same thing. Win32 will force you to use some other way. >> I'm not sure what you are talking about here. Maybe you confuse threads >> with processes? > > Windows threads have exit codes, just like processes. At least one > code is reserved and cannot be used by the programmer. Is that STILL_ACTIVE that we are talking about? That's an artefact of the design of GetExitCodeThread, which will return either the thread exit code or its own error code. The python lib could easily hide this, and use run()'s return value to store the (python) result somewhere. -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why doesn't threading.join() return a value?
Alain Ketterlin writes: >> Passing a void* is not equivalent to passing anything, not even in C. >> Moreover, specific values are still reserved, like PTHREAD_CANCELLED. > > Thread cancellation is program logic (pthread_cancel), it doesn't mean > you thread crashed, it means your program decided to cancel the thread. > If you still care about the return value after having called > pthread_cancel(), Sotry, forgot to end this sentence... What I mean is: If you still care about the return value after having called pthread_cancel(), your program logic is unnecessarily complex, and you should find some other way to handle this case. -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: Can't use subprocess.Popen() after os.chroot() - why?
Erik writes:
> import os
> from subprocess import Popen, PIPE
>
> os.chroot("/tmp/my_chroot")
> p = Popen("/bin/date", stdin=PIPE, stdout=PIPE, stderr=PIPE)
> stdout_val, stderr_val = p.communicate()
> print stdout_val
>
> but the Popen call is dying with the following exception:
>
> Traceback (most recent call last):
> File "./test.py", line 7, in
> p = Popen("/bin/date", stdin=PIPE, stdout=PIPE, stderr=PIPE)
> File "/home/erik/lib/python2.7/subprocess.py", line 679, in __init__
> File "/home/erik/lib/python2.7/subprocess.py", line 1224, in _execute_child
> File "/home/erik/lib/python2.7/pickle.py", line 1382, in loads
> File "/home/erik/lib/python2.7/pickle.py", line 858, in load
> File "/home/erik/lib/python2.7/pickle.py", line 971, in load_string
> LookupError: unknown encoding: string-escape
>
> Am I missing something here? does the chroot environment need to be
> populated with more than just the date executable in this case?
Yes, because /bin/date is probably dynamically linked: you need the libs
as well. (Try first with a statically linked executable, e.g.,
/bin/busybox, and everything should be ok).
I agree the message is strange. For some reason, subprocess is calling
pickle.loads, which calls rep.decode("string-escape"), which probably
needs to access some file and fails because of the chroot().
I woud advise to use the chroot command instead of chrooting your python
process, if that's possible for you.
-- Alain.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Parsing numeric ranges
Seldon writes:
> I have to convert integer ranges expressed in a popular "compact"
> notation (e.g. 2, 5-7, 20-22, 41) to a the actual set of numbers (i.e.
> 2,5,7,20,21,22,41).
What form does the input have? Are they strings, or some other
representation?
> Is there any library for doing such kind of things or I have to write
> it from scratch ?
What kind of result do you need? An explicit list? A generator?
Here is a naive solution where the input is a list of strings and the
result a generator:
def values(l):
for item in l:
bounds = item.split('-')
if len(bounds) == 1:
yield int(bounds[0])
elif len(bounds) == 2:
for v in range(int(bounds[0]),1+int(bounds[1])):
yield v
else:
pass # ignore, or throw, or...
# Use as in:
for x in values(["1","2-3","4-10"]):
print x
-- Alain.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Get Path of current Script
Alexander Schatten writes: > could someone help me with a small problem? I wrote a Python script > that does some RegEx... transformations. Now, this script loads some > configuration data from a file located in the same directory: sys.path[0] is the path to the directory containing the script that the interpreter started with. -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: DOCTYPE + SAX
jdownie writes: > I'm trying to get xml.sax to interpret a file that begins with… > > www.w3.org/TR/html4/loose.dtd"> > > After a while I get... > > http://www.w3.org/TR/html4/loose.dtd:31:2: error in processing > external entity reference > > …although… > > time curl http://www.w3.org/TR/html4/loose.dtd > [works] You're mistaken. There is no problem fetching the file, but there is a problem while parsing the file (at line 31, where you find a comment in an entity declaration, which is not acceptable in XML). You're trying to use HTML's SGML DTD in a XML document. Direct your doctype to XHTML's DTD, and everything will be fine (hopefully). BTW, your installation will probably let you use a locally cached copy of the DTD, instead of fetching a file at every parse. How this works depends somehow on the parser you use. -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Re: 3.1.4 release candidate 1
Raymond Hettinger writes: > On May 29, 3:44 pm, Benjamin Peterson wrote: >> On behalf of the Python development team, I'm happy as a swallow to announce >> a >> release candidate for the fourth bugfix release for the Python 3.1 >> series, Python >> 3.1.4. > > The Pi release of Python :-) And the same day we get the e release of Python... -- Alain. -- http://mail.python.org/mailman/listinfo/python-list
Typo-squatting PyPi
In case you haven't heard about this: https://developers.slashdot.org/story/17/09/16/2030229/pythons-official-repository-included-10-malicious-typo-squatting-modules Here is the Slashdot summary: | The Slovak National Security Office (NBU) has identified ten malicious | Python libraries uploaded on PyPI -- Python Package Index -- the | official third-party software repository for the Python programming | language. NBU experts say attackers used a technique known as | typosquatting to upload Python libraries with names similar to | legitimate packages -- e.g.: "urlib" instead of "urllib." The PyPI | repository does not perform any types of security checks or audits | when developers upload new libraries to its index, so attackers had no | difficulty in uploading the modules online. | | Developers who mistyped the package name loaded the malicious | libraries in their software's setup scripts. "These packages contain | the exact same code as their upstream package thus their functionality | is the same, but the installation script, setup.py, is modified to | include a malicious (but relatively benign) code," NBU explained. | Experts say the malicious code only collected information on infected | hosts, such as name and version of the fake package, the username of | the user who installed the package, and the user's computer hostname. | Collected data, which looked like "Y:urllib-1.21.1 admin testmachine", | was uploaded to a Chinese IP address. NBU officials contacted PyPI | administrators last week who removed the packages before officials | published a security advisory on Saturday." -- Alain. -- https://mail.python.org/mailman/listinfo/python-list
Re: integer copy
"ast" writes: > "ast" a écrit dans le message de > news:[email protected]... > > Neither works for large integers which is > even more disturbing > > a = 6555443 > b = copy.copy(a) > a is b > > True In copy.py: | [...] | def _copy_immutable(x): | return x | for t in (type(None), int, long, float, bool, str, tuple, | frozenset, type, xrange, types.ClassType, | types.BuiltinFunctionType, type(Ellipsis), | types.FunctionType, weakref.ref): | d[t] = _copy_immutable | [...] and d[t] is what decides how to copy, via _copy_dispatch. So none of the types listed above will be actually copied. It should be documented more precisely. You should file a documentation-improvement request. Also, the doc says: | This version does not copy types like module, class, function, method, | nor stack trace, stack frame, nor file, socket, window, nor array, nor | any similar types. But who knows what "similar types" are... -- Alain. -- https://mail.python.org/mailman/listinfo/python-list
Re: Tips or strategies to understanding how CPython works under the hood
ElChino writes: > Chris Angelico wrote: > >> CPython is a stack-based interpreter, which means it loads values onto >> an (invisible) internal stack, processes values at the top of the >> stack, and removes them when it's done. > > Is this similar to how Lua operates too? No. Lua uses a register-based (virtual) machine. See https://www.lua.org/doc/jucs05.pdf I think Lua was the first language in widespread use to move to a register-based machine. -- Alain. -- https://mail.python.org/mailman/listinfo/python-list
Re: CSV file edition
Manuel Rincon writes:
[...]
> Type=0 MarketTime=11:18:26.549 Price=112.8300
> Type=0 MarketTime=11:18:28.792 Price=112.8300
[...]
>
> I would need to filter only the numeric part of all the columns.
I assume that by "numeric" you mean the value after Price=
line.split()[2].split('=')[1]
line.rsplit('=',1)[1]
line.rpartition('=')[2]
line[line.rfind('=')+1:]
I prefer the last one. You can also use regexps, but it would be
overkill if your lines are that simple.
If your lines are really that simple, use line[39:] (check that 39 is
correct). Your data look like fixed-width fields.
-- Alain.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Doubt in line_profiler documentation
Abhiram R writes: [...] > https://github.com/rkern/line_profiler > > The definition for the time column says - > > "Time: The total amount of time spent executing the line in the timer's > units. In the header information before the tables, you will see a line > 'Timer unit:' giving the conversion factor to seconds. It may be different > on different systems." > For example, if the timer unit is :* 3.20802e-07 s* > and a particular instruction's time column says its value is 83.0, is the > time taken 83.0*3.20802e-07 s? Or is there more to it? That's it. > If my understanding is correct however, why would there be a need for > this? What could be the cause of this - " It may be different on > different systems "? Time is a complicated thing on a computer, and is only measured with a certain precision (or "resolution"). This precision may vary from system to system. It is customary to mention the resolution when profiling, because the resolution is usually coarse wrt processor frequency (typically 1 microsecond, around 3000 processor cycles at 3Ghz). So profiling very short running pieces of code is highly inaccurate. You can look at the code used in rkern, at https://github.com/rkern/line_profiler/blob/master/timers.c You'll see that on Windows it uses QueryPerformanceCounter() [*] and QueryPerformanceFrequency(). On Unix it uses gettimeofday(), which has a fixed/conventional resolution. By the way, the use of gettimeofday() is strange since this function is now deprecated... clock_gettime() should be used instead. It has an associated clock_getres() as well. -- Alain. [*] WTF is wrong with these microsoft developpers? Clocks and performance counters are totally different things. what's the need for confusing terms in the API? -- https://mail.python.org/mailman/listinfo/python-list
Re: "None" and "pass"
[email protected] (Stefan Ram) writes: > A participant of my Python course asked whether one could > also use "None" instead of "pass". What do you think? > > def f(): > pass > > can also be written as > > def f(): > None > > . Is there any place where "None" could not be used > instead of "pass"? No, an expression is always a valid statement: https://docs.python.org/3.6/reference/simple_stmts.html Use None, or 42+0, or 1+1+1+1+1+1+1+1+1, or whatever you want (that does not have any side-effect and/or throw an exception). And be fired right after your first code review. -- Alain. -- https://mail.python.org/mailman/listinfo/python-list
Re: Are the critiques in "All the things I hate about Python" valid?
Tim Delaney writes: [...] > As others have said, typing is about how the underlying memory is treated. No. It is much more than that. Typing is about everything you can say about a given statement. Some type systems are focusing on type labels only (like most statically typed programming languages), others are fairly elaborate (for instance, Eiffel's, if requires/ensures are considered part of the type system). There is a wide range of type systems, more or less practical depending on your needs (see https://www.cis.upenn.edu/~bcpierce/tapl/ if you want to see how far you can go with typing). [...] > C is statically and weakly typed. Variables know their types at compile > time (static typing). It is a feature of the language that you can cast any > pointer to any chunk of memory to be a pointer to any other type (normally > via void *). This is not coercion - it takes the bit pattern of memory of > one type and interprets it as the bit pattern for another type, and is weak > typing. No. C has much stronger rules, not on casting, but on accessing the pointees, which basically invalidates your argument. Refer to the C standard for details. > Python is strongly and dynamically typed. In Python, once you create an > object, it remains that type of object, no matter what you do to it*. That > makes it strongly typed. [...] But you can modify the class (not __class__) in whatever way you want. For instance: class X(object): def f(self): ... ... del X.f So, in Python, knowing that object x is an instance of class X tells you... essentially nothing. Call this strong typing if you want. In terms of type systems, it is (strong) simplistic-typing based on type labels, and labels carry no information whatsoever. Dynamic typing lets you avoid doing nonsense with bit-patterns, as you say, by gracefully crashing at run-time instead of performing "unexpected" actions. "Unexpected" is defined by the whole *execution* of the program up to the point where typing takes place. To understand whether a Python statement x.f() is meaningful (which is what typing is about), you have to know the full sequence of actions that have taken place on x *and* on its class (and on everything that that call may use) since the beginning of the program. For any non-trivial program, this is usually way too complex to capture, testing will be incomplete, and all you can do is run your program and see whether is goes through. As a general rule, if that's all you expect from typing, then, fine, call this "strong". I won't go as far, and just say that it is good enough for the programs I use Python for. -- Alain. -- https://mail.python.org/mailman/listinfo/python-list
Re: Are the critiques in "All the things I hate about Python" valid?
Chris Angelico writes:
> On Mon, Feb 19, 2018 at 7:40 PM, Alain Ketterlin
> wrote:
>> No. C has much stronger rules, not on casting, but on accessing the
>> pointees, which basically invalidates your argument. Refer to the C
>> standard for details.
>
> Really? What rules?
Look at the C11 standard, section 6.3.2.3 ("Pointers"), 6.5.§6-7
("effective types"), and 6.5.3.2 ("Address and indirection operators").
It is tiring to constantly correct misunderstandings about pointer
casting and dereferencing.
> $ cat demo.c; gcc -Wall demo.c; ./a.out
[...]
If you don't know what undefined behavior is, better avoid C. Your
program has UB, anything can happen, including a seemingly sensible
result.
>> But you can modify the class (not __class__) in whatever way you want.
>> For instance:
>>
>> class X(object):
>> def f(self): ...
>> ...
>> del X.f
>>
>> So, in Python, knowing that object x is an instance of class X tells
>> you... essentially nothing. Call this strong typing if you want. In
>> terms of type systems, it is (strong) simplistic-typing based on type
>> labels, and labels carry no information whatsoever.
>
> Sure you can. And you can do a lot of other things at run time, too.
> Monkey-patching doesn't change the fact that x really and truly is an
> instance of class X, it just changes what you can do with that object.
> So what you're saying is that Python's dynamism makes it possible to
> disrupt duck typing. Sure, I'll grant you that. But the type system
> itself isn't broken by that.
I didn't say it is broken. I said: it does very little.
-- Alain.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Are the critiques in "All the things I hate about Python" valid?
Chris Angelico writes:
> On Mon, Feb 19, 2018 at 9:04 PM, Alain Ketterlin
> wrote:
>> Look at the C11 standard, section 6.3.2.3 ("Pointers"), 6.5.§6-7
>> ("effective types"), and 6.5.3.2 ("Address and indirection operators").
>> It is tiring to constantly correct misunderstandings about pointer
>> casting and dereferencing.
>>
>>> $ cat demo.c; gcc -Wall demo.c; ./a.out
>> [...]
>>
>> If you don't know what undefined behavior is, better avoid C. Your
>> program has UB, anything can happen, including a seemingly sensible
>> result.
>
> Sure it can. But I don't know what you can mean by "stronger rules" if
> all it says is "that's undefined". Saying that behaviour is undefined
> does NOT mean that C has a stronger type system. I got no errors, not
> even a warning in -Wall mode, so there is no indication that my code
> did something wrong.
I used "stronger rules" in response to the OP claim ("C lets you
manipulate memory freely"). Undefined behavior is part of the semantics
of C, the same way infinite loops of signed-integer overflow are. What
compilers can do about it is a different matter. This is well
documented, those that want to ignore UB are on their own. Anyway, I
think it is irrelevant here. Search for "What every programmer should
know about undefined behavior" if you are interested.
-- Alain.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Are the critiques in "All the things I hate about Python" valid?
Steven D'Aprano writes:
> On Mon, 19 Feb 2018 09:40:09 +0100, Alain Ketterlin wrote:
>
>> Tim Delaney writes:
>>
>> [...]
>>> As others have said, typing is about how the underlying memory is
>>> treated.
>>
>> No. It is much more than that. Typing is about everything you can say
>> about a given statement.
>
> "Everything"? Truly *everything*?
Everything you can say.
> Given:
>
> # engage the type system somehow...
> # declare a, b, c: non-negative integers
> a = abs(int(input("Give me an integer")))
> b = a*a
> c = (a+1)*(a+1)
>
> can you give me an example of a type-system which is capable of telling
> me that:
>
> if (c - b) % 2 == 1:
> print("hello world")
> else:
> fire_missiles()
>
> will never fire the missiles?
Your example is ridiculously simple for all theorem provers I know (not
on Python code of course, since you can't even be sure int() etc. have
not been redefined).
Here is one that makes your point much better:
if a**n + b**n == c**n:
print("hello world")
else:
fire_missiles()
> I'd be impressed enough with a type system that knew that a%2 was always
> 0 or 1, although I suppose there could be some that already know that.
>
> Hell, I'd even be impressed if it could tell that c was not zero...
Your claim essentially is: since we cannot prove everything, let's not
even try to prove anything. Go on if you think this is the right way to
think about typing.
-- Alain.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Problem with difflib SequenceMatcher
Jay writes: > I am having an odd problem with difflib.SequenceMatcher. Sample code below: > > The strings "src" and "trg" differ only a little. How exactly? (Please be precise, it helps testing.) > The SequenceMatcher.ratio() for these strings 0.0. Many other similar > strings are working fine without problems (see below) with non-zero > ratios depending on how much difference there is between strings (as > expected). Calling SM(...,trg[1:],src[1:]) gives plausible result. See also the result of .get_matching_blocks() on your strings (it returns no matching blocks). It is all due to the "Autojunk" heuristics (see difflib's doc for details), which considers the first characters as junk. Call SM(...,autojunk=False). I have no idea why the maintainers made this stupid autojunk idea the default. Complain with them. -- Alain. > Tested on Python 2.7 on Ubuntu 14.04 > > Program follows: > --- > from difflib import SequenceMatcher as SM > > src = u"N KPT T HS KMNST KNFKXNS AS H KLT FR 0 ALMNXN AF PRFT PRPRT AN > RRL ARS T P RPLST P KMNS H ASTPLXT HS ANTSTRL KR0 PRKRM NN AS 0 KRT LP > FRRT 0S PRKRM KLT FR 0 RPT TRNSFRMXN AF XN FRM AN AKRRN AKNM T A SSLST > ANTSTRL SST" > trg = u"M KPT T HS KMNST KNFKXNS AS H KLT FR 0 ALMNXN AF PRFT PRPRT AN > RRL ARS T P RPLST P KMNS H ASTPLXT HS ANTSTRL KR0 PRKRM NN AS 0 KRT LP > FRRT 0S PRKRM KLT FR 0 RPT TRNSFRMXN AF XN FRM AN AKRRN AKNM T SSLST > ANTSTRL SST" > print src, '\n', trg, '\n', SM(None, trg, src).ratio() -- https://mail.python.org/mailman/listinfo/python-list
Re: how to refactor nested for loop into smaller for loop assume each of them independent?
meInvent bbird writes: > how to refactor nested for loop into smaller for loop assume each of them > independent? > > because memory is not enough > > for ii in range(1,2000): > for jj in range(1,2000): > for kk in range(1,2000): > print run(ii,jj,kk) n = 0 while n < 2000*2000*2000: ii = n // (2000*2000) jj = ( n % (2000*2000) ) // 2000 kk = n % 2000 run(ii,jj,kk) n += 1 or: for n in xrange(2000*2000*2000): ii,rr = divmod(n,2000*2000) jj,kk = divmod(rr,2000) run(ii,jj,kk) (I think CPython wll fold the constants, but I haven't checked). Are you sure this is your (real) problem? -- Alain. -- https://mail.python.org/mailman/listinfo/python-list
Re: How to pick out the same titles.
Seymore4Head writes: [...] > I have a long text file that has movie titles in it and I would like > to find dupes. > > The thing is that sometimes I have one called "The Killing Fields" and > it also could be listed as "Killing Fields" Sometimes the title will > have the date a year off. > > What I would like to do it output to another file that show those two > as a match. Try the difflib module (read the doc, its default behavior may be surprising). -- Alain. -- https://mail.python.org/mailman/listinfo/python-list
Re: Why does this list swap fail?
[email protected] writes: > L=[2,1] > L[0],L[L[0]-1]=L[L[0]-1],L[0] > > The L doesn't change. Can someone provide me the detail procedure of > this expression? From: https://docs.python.org/3/reference/simple_stmts.html#grammar-token-assignment_stmt | Although the definition of assignment implies that overlaps between the | left-hand side and the right-hand side are ‘simultaneous’ (for example | a, b = b, a swaps two variables), overlaps within the collection of | assigned-to variables occur left-to-right, sometimes resulting in | confusion. For instance, the following program prints [0, 2]: | | x = [0, 1] | i = 0 | i, x[i] = 1, 2 # i is updated, then x[i] is updated | print(x) In your case: - L[0] get 1 (from L[L[0]-1]) - L[L[0]-1] (from lhs) is now L[0], which gets 2 (from L[0] in rhs) Since both operations happen in sequence (not simultaneously), you just overwrite the first element twice. Congratulations, a very good corner case (you can't call it a bug, since it conforms to the definition of the language). -- Alain. -- https://mail.python.org/mailman/listinfo/python-list
Re: The hardest problem in computer science...
Steve D'Aprano writes: [...] > Fiction > ├─ Fantasy > │ ├─ Terry Pratchett > │ │ ├─ Discworld > │ │ │ ├─ Wyrd Sisters > │ │ │ └─ Carpe Jugulum > │ │ └─ Dodger > │ └─ JK Rowling [...] > what do we call the vertical and horizontal line elements? Box-drawing characters. At least that's how Unicode calls them. Even if you don't draw boxes... https://en.wikipedia.org/wiki/Box-drawing_character -- Alain. -- https://mail.python.org/mailman/listinfo/python-list
