pickling a subclass of tuple
Hi all, happy new year, I was trying to pickle a instance of a subclass of a tuple when I ran into a problem. Pickling doesn't work with HIGHEST_PROTOCOL. How should I rewrite my class so I can pickle it? Thanks , Fedor #!/usr/bin/env python import pickle class A(tuple): def __new__(klass, arg1,arg2): return super(A,klass).__new__(klass, (arg1,arg2)) a=A(1,2) print "no pickle",a print "normal pickle",pickle.loads(pickle.dumps(a)) print "highest protocol", pickle.loads(pickle.dumps(a,pickle.HIGHEST_PROTOCOL)) This is the output: ''' no pickle (1, 2) normal pickle (1, 2) highest protocol Traceback (most recent call last): File "./test.py", line 9, in ? print "highest protocol",pickle.loads(pickle.dumps(a,pickle.HIGHEST_PROTOCOL)) File "/usr/lib/python2.3/pickle.py", line 1394, in loads return Unpickler(file).load() File "/usr/lib/python2.3/pickle.py", line 872, in load dispatch[key](self) File "/usr/lib/python2.3/pickle.py", line 1097, in load_newobj obj = cls.__new__(cls, *args) TypeError: __new__() takes exactly 3 arguments (2 given) ''' -- http://mail.python.org/mailman/listinfo/python-list
Re: MySQLdb
Hi Daniel,
You should probably take a look at the executemany method of the cursor.
Your insert times might drop by a factor 20 . Here's an example.
Cheers,
Fedor
import time
import MySQLdb
db=MySQLdb.Connect(user="me",passwd="my password",db="test")
c=db.cursor()
n=0
tic=time.time()
for i in range(3000):
n+=c.execute('INSERT INTO testtable VALUES (%s)', (i,))
toc=time.time()
t1=toc-tic
print 'separate sql statements: %s, inserted %s records' % (t1,n)
tic=time.time()
n=c.executemany('INSERT INTO testtable VALUES (%s)', [(i,) for i in
range(3000)])
toc=time.time()
t2=toc-tic
print 'all at once %s inserted %s records' % (t2,n)
OUTPUT>>>
separate sql statements: 0.571248054504, inserted 3000 records
all at once 0.0253219604492 inserted 3000 records
--
http://mail.python.org/mailman/listinfo/python-list
mysqldb issue
Hi all, I have a problem with mysql connections. After about 28000-29000 connections, I get a "Can't connect to MySQL server on '127.0.0.1'" error. I have made a small program which generates the error """ import MySQLdb for i in range(3): if not i % 100: print i db = MySQLdb.connect(host='127.0.0.1', user='me',passwd='mypassword') c = db.cursor() c.close() db.close() """ This is the error after making about 28200 connections: ''' Traceback (most recent call last): File "", line 1, in ? File "/usr/tmp/python-12448vuu", line 7, in ? db = MySQLdb.connect(host='127.0.0.1', user='me', passwd='mypassword') File "/usr/local/lib/python2.3/site-packages/MySQLdb/__init__.py", line 64, in Connect return apply(Connection, args, kwargs) File "/usr/local/lib/python2.3/site-packages/MySQLdb/connections.py", line 116, in __init__ self._make_connection(args, kwargs2) File "/usr/local/lib/python2.3/site-packages/MySQLdb/connections.py", line 41, in _make_connection apply(super(ConnectionBase, self).__init__, args, kwargs) _mysql_exceptions.OperationalError: (2003, "Can't connect to MySQL server on '127.0.0.1' (99)") ''' Does anybody know how to solve this issue? System: Suse 8.1, mysql 4.0.14, mysqldb 1.0.1, python2.3 Thanks very much. Fedor -- http://mail.python.org/mailman/listinfo/python-list
Re: mysqldb issue
'127.0.0.1' (99)") ''' Does anybody know how to solve this issue? Off hand, I suspect you've run out of unused TCP sockets. Each connect results in your task allocating a port, and possibly MySQL allocating a fresh socket to handle the connection that it finds on its assigned port. That was it. Also changing '127.0.0.1' to 'localhost' solves the problem. Thanks. Fedor -- http://mail.python.org/mailman/listinfo/python-list
An "adapter", superset of an iterator
I've been using python as a tool to solve competitive programming problems for a while now and I've noticed a feature, python would benefit from having. Consider "reversed(enumerate(a))". This is a perfectly readable code, except it's wrong in the current version of python. That's because enumerate returns an iterator, but reversed can take only a sequence type. The feature I am describing (and proposing) solves this. Introducing an adapter type: this is an iterator, but it's items can be accessed out of order. More formally it has to: 1. Have __getitem__ to allow access by index 2. Have __len__ 3. Be immutable (It is a lot like the sequence protocol) An adapter can be converted to an iterator by accessing it from 0 to len(adapter). Which is done by iter(). (or by __iter__, I don't know which implementation would be "right") ``` iter(a) #is equivalent to (a[i] for i in range(len(a))) ``` For example any tuple is a valid adapter and any list can be easily converted to one. Built-in adapter-generators: "map" function should really return an adapter. ``` #possible implementation m=map(lambda x:x+1,[1,2,3,4,5]) #lambda is not called print(m[3])# gives 5 by calling the lambda on list's 3rd element, which is 4 #simplified implementation class map: def __init__(self,f,a): self.f=f self.a=a def __getitem__(self,idx): return self.f(self.a[idx]) def __len__(self): return len(self.a) ``` enumerate should really return an adapter ``` #simplified implementation class enumerate: def __init__(self,a): self.a = a def __getitem__(self,idx): return idx,self.a[idx] def __len__(self): return len(self.a) ``` reversed should really return an adapter ``` #simplified implementation class reversed: def __init__(self,a): self.a = a self.length=len(a) def __getitem__(self,idx): return self.a[self.length-idx-1] def __len__(self): return self.length ``` zip should really return an adapter range should really return an adapter filter should *not* return an adapter All of those functions return an adapter and take in an adapter. But some (excluding "reversed") should be able take an iterator and return an iterator. So the way I imagine a possible release version to work is that "reversed(enumerate(a))" works if a is an adapter and throws an exception if not Perhaps there should be even an adapter comprehension: ``` Add1 = (a+1 for a) #is equivalent to Add1 = lambda s: map((lambda a: a+1),s) transformed = Add1([1,2,3]) print(transformed[1])# should be 3 ``` This adapter feature also allows you to not use a key argument "key=" for certain functions (which could be non-existing). For example bisect.bisect functions before 3.10 didn't have a key= argument. Some third-party libraries could do this with it's functions too. (Subject to change) -- https://mail.python.org/mailman/listinfo/python-list
Newbie Q: modifying SQL statements
Hi all,
I'm in the process of learning Python by writing a job queue program.
Nothing fancy, mind you, just read from a table, shell out to a program,
write back to the table.
I'm working off of the tutorial listed here (amongst many places):
http://www.devx.com/dbzone/Article/22093
In my Jobs class I have:
def __iter__(self):
"creates a data set, and returns an iterator (self)"
q = "select * from %s" % (self.name)
self._query(q)
return self # an Iterator is an object
# with a next() method
def next(self):
"returns the next item in the data set,
or tells Python to stop"
r = self.dbc.fetchone()
if not r:
raise StopIteration
return r
which works well, but what if I want to modify the __iter__ query? I
want to be able to do something like this (and I know this is not the
right syntax but you'll get my drift):
for job in jobs: print job # which the above code does
for job in jobs("status = running"): print job
for job in jobs("jobid = 4"): print job
What's the pythonic way of doing this?
--
Regards,
Faber
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Newbie Q: modifying SQL statements
On 10/01/08 22:53 -0500, Mike Meyer wrote:
> Personally, I think it would be more pythonic to not try and use two
> different APIs to walk the list of jobs (... One Way To Do it):
>
> def __call__(self, where=None):
> q = "select * from %s" % (self.name,) + ("" if not where else (" where
> %s" % where))
Does this '("" if not where...' syntax actually work? I couldn't get it to
compile and I couldn't find any examples of such syntax (but you can't
expect googling for 'if not' to be too successful). I ended up
changing that line to:
q = "select * from %s" % (self.name,)
if where: q += "where %s" %where
> for r in self.dbc.iterresults() # I assume it has something like this
I don't think it does, if I read
http://dustman.net/andy/python/MySQLdb_obsolete/doc/MySQLdb-3.html
correctly.
--
Regards,
Faber Fedor
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Newbie Q: modifying SQL statements
On 11/01/08 18:29 -0500, Mike Meyer wrote: > It is a 2.5 feature, though. I was beginning to wonder of that was the case. > I converted all my clients to 2.5.1, > shortly after it was available, and haven't used anything older > since. Sorry 'bout that. No prob. -- Regards, Faber Fedor -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. -- http://mail.python.org/mailman/listinfo/python-list
Re: Newbie Q: modifying SQL statements
On 12/01/08 00:23 +0100, Hrvoje Niksic wrote:
> "Faber J. Fedor" <[EMAIL PROTECTED]> writes:
> > Does this '("" if not where...' syntax actually work?
>
> http://docs.python.org/whatsnew/pep-308.html
C'mon! I'm in Day Two of learning Python. You can't expect me to be
reading "What's New" docs already! :-)
I did find it interesting that the page mentioned said "Guido van Rossum
eventually chose a surprising syntax:". When I first saw the construct
I thought "Oh, they borrowed that from Perl". :-) (Although you can't do the
else part in Perl, it is a natural extension, IMO.)
--
Regards,
Faber Fedor
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
--
http://mail.python.org/mailman/listinfo/python-list
Re: where do my python files go in linux?
On 12/01/08 12:02 +0100, Jorgen Bodde wrote:
> Hi All,
>
> I am trying to make a debian package. I am following the tutorial by
> Horst Jens
> (http://showmedo.com/videos/video?name=linuxJensMakingDeb&fromSeriesID=37)
> and it is very informative. However one thing my app has and his
> doesn't, is multiple python files which need to be executed. For
> example
>
> {dir}/app
> app.py
>
> app.py calls a lot of modules in {dir}/app. Horst says the python file
> goes in /usr/bin/app.py which is ok with me, but I have multiple
> python files, and I decided to use an app.sh script to call my python
> files. In the /usr/bin I do not see subdirs so I assume that is not
> really desirable.
>
> Question 1. Where do I put the bulk of python scripts in a normal
> linux environment?
I would think you'd create a directory /usr/bin/app and then symlink
/usr/bin/app.py to /usr/bin/app/app.py.
> Question 2. Should I use *.pyc rather then *.py files to speed up
> executing as the user cannot write to /usr/bin or any other dir in the
> system and everytime my app runs it will recompile it
That would make sense.
> Thanks for any advice or maybe a good tutorial how to set up files in
> a linux environment
http://www.pathname.com/fhs/ (couldn't find a more
recent version, sorry.)
--
Regards,
Faber Fedor
President
Linux New Jersey, Inc.
908-320-0357
800-706-0701
http://www.linuxnj.com
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
--
http://mail.python.org/mailman/listinfo/python-list
