@staticmethods called more than once
Hi,
i'm somewhat confused working with @staticmethods. My logger and configuration
methods are called n times, but I have only one call.
n is number of classes which import the loger and configuration class
in the subfolder mymodule. What might be my mistake mistake?
Many thanks
Christian
### __init__.py ###
from mymodule.MyLogger import MyLogger
from mymodule.MyConfig import MyConfig
# my_test.py ##
from mymodule import MyConfig,MyLogger
#Both methods are static
key,logfile,loglevel = MyConfig().get_config('Logging')
log = MyLogger.set_logger(key,logfile,loglevel)
log.critical(time.time())
#Output
2013-05-21 17:20:37,192 - my_test - 17 - CRITICAL - 1369149637.19
2013-05-21 17:20:37,192 - my_test - 17 - CRITICAL - 1369149637.19
2013-05-21 17:20:37,192 - my_test - 17 - CRITICAL - 1369149637.19
2013-05-21 17:20:37,192 - my_test - 17 - CRITICAL - 1369149637.19
--
http://mail.python.org/mailman/listinfo/python-list
Re: @staticmethods called more than once
Am Dienstag, 21. Mai 2013 18:48:07 UTC+2 schrieb John Gordon: > In John Gordon writes: > > > > > You should only call addHandler() once. > > > > ...for each intended logging output destination, of course. If you want > > logging output to appear in a file and on-screen, then you would call > > addHandler() once with a file handler and once with a screen handler. > > > > But I think you may be calling addHandler multiple times for the same > > file handler, which is causing the duplicate logging output. > > > > -- > > John Gordon A is for Amy, who fell down the stairs > > [email protected] B is for Basil, assaulted by bears > > -- Edward Gorey, "The Gashlycrumb Tinies" Yes you're right. Many thanks! -- http://mail.python.org/mailman/listinfo/python-list
Pool.map mongodb cursor
Hi,
is it possible to avoid some memory overhead with a mongodb cursor and
multiprocessing? Regarding to the size of the cursor, Python consuming at
first a lot of memory. However the estimation is independend
among each other document (chunking?).
Maybe there is a better way using multiprocessing in place of
Pool?
score_proc_pool.map(scoring_wrapper,mongo_cursor,chunksize=1)
Inside the scoring_wrapper I'm writing estimated scores without a return
value.
def scoring_wrapper(doc):
profiles.update({'anyid':anyid},
{
'$set':{'profile':value}
},upsert=True)
Thanks in advance
Christian
--
http://mail.python.org/mailman/listinfo/python-list
re.finditer with lookahead and lookbehind
Hi, i have some trouble to split a pattern like s. Even have this problems with the first and last match. Some greedy problems? Thanks in advance Christian import re s='v1=pattern1&v2=pattern2&v3=pattern3&v4=pattern4&v5=pattern5&x1=patternx' pattern =r'(?=[a-z0-9]+=)(.*?)(?<=&)' regex = re.compile(pattern,re.IGNORECASE) for match in regex.finditer(s): print match.group(1) My intention: pattern1 pattern2 pattern3 pattern4 pattern5 patternx -- http://mail.python.org/mailman/listinfo/python-list
OrderedDict / DIctComprehension
Hi,
is there a way building an OrderedDict faster?
Thanks in advance
Christian
@timeit
def ordered(n=10):
d = OrderedDict()
for i in xrange(n):
d['key'+str(i)] = i
return d
@timeit
def comprehension(n=10):
d = { 'key'+str(i):i for i in xrange(n) }
return d
ordered()
comprehension()
'ordered' ((), {}) 0.724609 sec
'comprehension' ((), {}) 0.098318 sec
--
http://mail.python.org/mailman/listinfo/python-list
Re: SSH Connection with Python
Am Donnerstag, 25. Oktober 2012 12:31:46 UTC+2 schrieb Schneider: > Hi Folkz, > > how can i create a SSH-Connection with python? I have to send some > > commands to the remote host and parse their answers. > > greatz Johannes There is a module in chilkat. http://www.example-code.com/python/ssh_exec.asp Don't know if it is worth the money, never used it. Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: OrderedDict / DIctComprehension
Too bad that's not (using python2.7)
'ordered_dict_generator' ((), {}) 1.089588 sec
Anyway thanks for your hint!
> Hi,
>
>
>
> is there a way building an OrderedDict faster?
>
>
>
> Thanks in advance
>
> Christian
>
>
>
> @timeit
>
> def ordered(n=10):
>
> d = OrderedDict()
>
> for i in xrange(n):
>
> d['key'+str(i)] = i
>
> return d
>
>
>
>
>
> @timeit
>
> def comprehension(n=10):
>
> d = { 'key'+str(i):i for i in xrange(n) }
>
> return d
>
>
>
>
>
> ordered()
>
> comprehension()
>
>
>
> 'ordered' ((), {}) 0.724609 sec
>
> 'comprehension' ((), {}) 0.098318 sec
--
http://mail.python.org/mailman/listinfo/python-list
mysql insert with tuple
Hi ,
my purpose is a generic insert via tuple , because the number of fields and
can differ. But I'm stucking .
ilist=['hello',None,7,None,None]
#This version works, but all varchar fields are in extra '' enclosed.
con.execute(""" INSERT INTO {} VALUES %r; """.format(table) , (tuple(ilist),))
#This produce (1054, "Unknown column 'None' in 'field list'"),
#but without None values it works.
con.execute(""" INSERT INTO {} VALUES %r; """.format(table) % (tuple(ilist),))
Many thanks,
Christian
--
http://mail.python.org/mailman/listinfo/python-list
Re: mysql insert with tuple
Am Mittwoch, 21. November 2012 20:49:14 UTC+1 schrieb Hans Mulder:
> On 21/11/12 18:19:15, Christian wrote:
>
> > Hi ,
>
> >
>
> > my purpose is a generic insert via tuple , because the number of fields
> > and can differ. But I'm stucking .
>
> >
>
> > ilist=['hello',None,7,None,None]
>
> >
>
> > #This version works, but all varchar fields are in extra '' enclosed.
>
> > con.execute(""" INSERT INTO {} VALUES %r; """.format(table) ,
> > (tuple(ilist),))
>
> >
>
> > #This produce (1054, "Unknown column 'None' in 'field list'"),
>
> > #but without None values it works.
>
> > con.execute(""" INSERT INTO {} VALUES %r; """.format(table) %
> > (tuple(ilist),))
>
>
>
> How about:
>
>
>
> con.execute("""INSERT INTO {} VALUES ({})"""
>
> .format(table, ",".join("%s" for _ in ilist)), ilist)
>
>
>
> Or perhaps break it down into smaller steps:
>
>
>
> bind_variables = ",".join("%s" for _ in ilist))
>
> query = "INSERT INTO {} VALUES ({})".format(table, bind_variables)
>
> con.execute(query, ilist)
>
>
>
>
>
> Hope this helps,
>
>
>
> -- HansM
Thank you both!. However, for future issues I'll take Chris advise about a ORM
into account . It's only a sort of data crunching (offline), so SQL Injection
isn't a problem.
--
http://mail.python.org/mailman/listinfo/python-list
[email protected]
I play around with redis. Isn't it possible to handle BitSet with
Python "as" in Java?
BitSet users = BitSet.valueOf(redis.get(key.getBytes()));
all.or(users);
System.out.println(all.cardinality())
I try something with the struct and bitstring libs , but haven't any
success. Even the follow snippet didn't work, beacause
bitset[0] isn't approriate.
bitset = r.get('bytestringFromRedis')
x = "{0:b}".format(ord(bitset[0]))
Thanks in advance
Christian
--
http://mail.python.org/mailman/listinfo/python-list
filter max from iterable for grouped element
Hi,
as beginner in python , I struggle somewhat to filter out only the
maximum in the values for and get hmax.
Maybe it easier when i change the structure of h?
Many thanks in advance
Christian
h = {'abvjv': ('asyak', 0.9014230420411024),
'afqes': ('jarbm', 0.9327883839839753),
'aikdj': ('jarbm', 0.9503941616408824),
'ajbhn': ('jarbm', 0.9323583083061541),
'ajrje': ('jbhdj', 0.9825125732711598),
'anbrw': ('jarbm', 0.950801828672098)}
hmax = {'abvjv': ('asyak', 0.9014230420411024),
'ajrje': ('jbhdj', 0.9825125732711598),
'anbrw': ('jarbm', 0.950801828672098)}
--
http://mail.python.org/mailman/listinfo/python-list
Re: filter max from iterable for grouped element
On 19 Mrz., 09:45, Peter Otten <[email protected]> wrote: > Christian wrote: > > as beginner in python , I struggle somewhat to filter out only the > > maximum in the values for and get hmax. > > h = {'abvjv': ('asyak', 0.9014230420411024), > > 'afqes': ('jarbm', 0.9327883839839753), > > 'aikdj': ('jarbm', 0.9503941616408824), > > 'ajbhn': ('jarbm', 0.9323583083061541), > > 'ajrje': ('jbhdj', 0.9825125732711598), > > 'anbrw': ('jarbm', 0.950801828672098)} > > > hmax = {'abvjv': ('asyak', 0.9014230420411024), > > 'ajrje': ('jbhdj', 0.9825125732711598), > > 'anbrw': ('jarbm', 0.950801828672098)} > > You can create an intermediate dict: > > >>> d = {} > >>> for k, (k2, v) in h.items(): > > ... d.setdefault(k2, []).append((v, k)) > ...>>> import pprint > >>> pprint.pprint(d) > > {'asyak': [(0.9014230420411024, 'abvjv')], > 'jarbm': [(0.9323583083061541, 'ajbhn'), > (0.950801828672098, 'anbrw'), > (0.9327883839839753, 'afqes'), > (0.9503941616408824, 'aikdj')], > 'jbhdj': [(0.9825125732711598, 'ajrje')]} > > Now find the maximum values: > > >>> for k, pairs in d.items(): > > ... v, k2 = max(pairs) > ... assert k2 not in hmax > ... hmax[k2] = k, v > ...>>> pprint.pprint(hmax) > > {'abvjv': ('asyak', 0.9014230420411024), > 'ajrje': ('jbhdj', 0.9825125732711598), > 'anbrw': ('jarbm', 0.950801828672098)} > > > Maybe it easier when i change the structure of h? > > Maybe. Here's one option: > > >>> pprint.pprint(data) > > [('jarbm', 0.9323583083061541, 'ajbhn'), > ('jarbm', 0.950801828672098, 'anbrw'), > ('jarbm', 0.9327883839839753, 'afqes'), > ('asyak', 0.9014230420411024, 'abvjv'), > ('jbhdj', 0.9825125732711598, 'ajrje'), > ('jarbm', 0.9503941616408824, 'aikdj')]>>> data.sort() > >>> from itertools import groupby > >>> def last(items): > > ... for item in items: pass > ... return item > ...>>> dmax = [last(group) for key, group in groupby(data, key=lambda item: > item[0])] > >>> pprint.pprint(dmax) > > [('asyak', 0.9014230420411024, 'abvjv'), > ('jarbm', 0.950801828672098, 'anbrw'), > ('jbhdj', 0.9825125732711598, 'ajrje')] Thanks a lot. -- http://mail.python.org/mailman/listinfo/python-list
multiprocessing & itertools.product Iterator
Hey,
I struggle to "extend" a multiprocessing example to my problem with a
itertools.product result iterator.
How I have to assign the combos.next() elements approriate to
Pool.imap/calc functions?
Thanks in advance
Christian
from multiprocessing import Process,Queue,Pool
import Calculation
import DataSimulation
from itertools import product
def produce_example_combos(size=6,na=1000,nb=10):
data = DataSimulation.DataSimulation()
a = [data.generate_simple(size) for x in xrange(na)]
b = [data.generate_simple(size) for x in xrange(nb)]
it = product(a,b)
return it
def calc(elements):
calc.q.put("Doing:" + elements[0] + elements[1])
ratio = Calculation.ratio(elements[0],elements[1])
return ratio
def calc_init(q):
calc.q = q
if __name__ == '__main__':
combos = produce_example_combos()
print "tesdata generated"
q = Queue()
p = Pool(10, calc_init, [q])
results = p.imap(calc,combos.next())
p.close()
for i in combos:
print q.get()
print results.next()
--
http://mail.python.org/mailman/listinfo/python-list
String formatting - mysql insert
Hi,
I get some problem when i like to set the table name dynamic.
I'm appreciate for any help.
Christian
### works
newcur.execute ( """ INSERT INTO events (id1,id2) VALUES (%s,%s);
""" , (rs[1],rs[2]))
### works not
newcur.execute ( """ INSERT INTO %s_events (id1,id2) VALUES (%s,
%s); """ , (table_name,rs[1],rs[2]))
### works but is not really perfect: None from rs list result in
"None" instead of NULL.
newcur.execute ( """ INSERT INTO %s_events (id1,id2) VALUES
('%s','%s'); """ % (table_name,rs[1],rs[2]))
--
http://mail.python.org/mailman/listinfo/python-list
Re: String formatting - mysql insert
On 14 Jul., 17:31, Billy Mays wrote:
> On 07/14/2011 11:00 AM, Christian wrote:
>
>
>
>
>
>
>
>
>
> > Hi,
>
> > I get some problem when i like to set the table name dynamic.
> > I'm appreciate for any help.
>
> > Christian
>
> > ### works
> > newcur.execute ( """ INSERT INTO events (id1,id2) VALUES (%s,%s);
> > """ , (rs[1],rs[2]))
>
> > ### works not
> > newcur.execute ( """ INSERT INTO %s_events (id1,id2) VALUES (%s,
> > %s); """ , (table_name,rs[1],rs[2]))
>
> > ### works but is not really perfect: None from rs list result in
> > "None" instead of NULL.
> > newcur.execute ( """ INSERT INTO %s_events (id1,id2) VALUES
> > ('%s','%s'); """ % (table_name,rs[1],rs[2]))
>
> You shouldn't use The bottom form at all since that is how injection
> attacks occur.
>
> The reason the second version doesn't work is because the the execute
> command escapes all of the arguments before replacing them. Example:
>
> sql = """SELECT * FROM table WHERE col = %s;"""
> cur.execute(sql, ('name',))
> # The actual sql statement that gets executed is:
> # SELECT * FROM table WHERE col = 'name';
> # Notice the single quotes.
>
> --
> Bill
thanks you guys!
--
http://mail.python.org/mailman/listinfo/python-list
Approximate comparison of two lists of floats
Hello, i have e little performance problem with my code... i have to compare many lists of very much floats. at moment i have nested for-loops for a in range( len(lists) ): for b in range( a+1 , len(lists) ): for valuea in lists[a]: equal=False for valueb in lists[b]: if inTolerance( valuea , valueb , 1.0): # inTolerance is an own function, which checks if the difference of valuea and valueb is not more then 1.0% equal=True break if equal: print a , "and" , b , "are equal" i found a version with set which is faster, but i cannot assign an tolerance (%) for a in range( len(lists) ): for b in range( a+1 , len(lists) ): if len( lists[a] ) == len( set( lists[a] ).intersection( set( lists[b] ) ) ): print a , "and" , b , "are equal" have you an idea how i can change my code, that i can compare many lists of floats with a tolerance in percentage very fast? (sorry for my bad englisch ;-) ) thanks christian -- http://mail.python.org/mailman/listinfo/python-list
memory usage multi value hash
Hello, i'm not very experienced in python. Is there a way doing below more memory efficient and maybe faster. I import a 2-column file and then concat for every unique value in the first column ( key) the value from the second columns. So The ouptut is something like that. A,1,2,3 B,3,4 C,9,10,11,12,90,34,322,21 Thanks for advance & regards, Christian import csv import random import sys from itertools import groupby from operator import itemgetter f=csv.reader(open(sys.argv[1]),delimiter=';') z=[[i[0],i[1]] for i in f] z.sort(key=itemgetter(0)) mydict = dict((k,','.join(map(itemgetter(1), it))) for k, it in groupby(z, itemgetter(0))) del(z) f = open(sys.argv[2], 'w') for k,v in mydict.iteritems(): f.write(v + "\n") f.close() -- http://mail.python.org/mailman/listinfo/python-list
build the "sql where cause" dynamic
Hi,
from some radio buttons in a django app i concat string like that:
But have no idea how i get the or when there different values
for a specified p column (# is currently my intended splitter maybe i
can concat a better input?).
input: "p2=1#p2=2#p1=3#p1=1#p1=5#pc=1#py=1"
output: "p2 in ('1','2') and p1 in ('3','1','5') and pc in ('1') and
py in ('1')"
Many thanks for any starting point
Christian
--
http://mail.python.org/mailman/listinfo/python-list
Re: build the "sql where cause" dynamic
On 2 Mai, 10:13, Chris Angelico wrote:
> On Mon, May 2, 2011 at 5:48 PM, christian wrote:
> > Hi,
>
> > from some radio buttons in a django app i concat string like that:
> > But have no idea how i get the or when there different values
> > for a specified p column (# is currently my intended splitter maybe i
> > can concat a better input?).
>
> > input: "p2=1#p2=2#p1=3#p1=1#p1=5#pc=1#py=1"
>
> > output: "p2 in ('1','2') and p1 in ('3','1','5') and pc in ('1') and
> > py in ('1')"
>
> Start by splitting the string and iterating:
> for el in input.split('#'):
>
> (obviously your variable won't want to be called 'input' but you knew
> that already)
>
> Then split on the = sign and add to a list or dictionary. Be wary of
> elements that don't have an equals sign in them.
>
> That looks like the querystring straight from the GET form, where
> you're using tickboxes. You could quite happily use '&' as your
> primary delimiter, if that's the case.
>
> Chris Angelico
Many Thanks
Christian
--
http://mail.python.org/mailman/listinfo/python-list
memory utilization blow up with dict structure
Hi,
I'm wondering why python blow up a dictionary structure so much.
The ids and cat substructure could have 0..n entries but in the most cases they
are <= 10,t is limited by <= 6.
Thanks for any advice to save memory.
Christian
Example:
{'0a0f7a3a0e09826caef1bff707785662': {'ids':
{'aa316b86-8169-11e6-bab9-0050563e2d7c',
'aa3174f0-8169-11e6-bab9-0050563e2d7c',
'aa319408-8169-11e6-bab9-0050563e2d7c',
'aa3195e8-8169-11e6-bab9-0050563e2d7c',
'aa319732-8169-11e6-bab9-0050563e2d7c',
'aa319868-8169-11e6-bab9-0050563e2d7c',
'aa31999e-8169-11e6-bab9-0050563e2d7c',
'aa319b06-8169-11e6-bab9-0050563e2d7c'},
't': {'type1', 'type2'},
'dt': datetime.datetime(2016, 9, 11, 15, 15, 54, 343000),
'nids': 8,
'ntypes': 2,
'cat': [('ABC', 'aa316b86-8169-11e6-bab9-0050563e2d7c', '74', ''),
('ABC','aa3174f0-8169-11e6-bab9-0050563e2d7c', '3', 'type1'),
('ABC','aa319408-8169-11e6-bab9-0050563e2d7c','3', 'type1'),
('ABC','aa3195e8-8169-11e6-bab9-0050563e2d7c', '3', 'type2'),
('ABC','aa319732-8169-11e6-bab9-0050563e2d7c', '3', 'type1'),
('ABC','aa319868-8169-11e6-bab9-0050563e2d7c', '3', 'type1'),
('ABC','aa31999e-8169-11e6-bab9-0050563e2d7c', '3', 'type1'),
('ABC','aa319b06-8169-11e6-bab9-0050563e2d7c', '3', 'type2')]},
I did a fresh read from pickled object to have a "clean" env.
linux-64bit:
sys.getsizeof(superdict)
50331744
len(superdict)
941272
VmPeak: 2981364 kB
VmSize: 2850288 kB
VmLck: 0 kB
VmPin: 0 kB
VmHWM: 2108936 kB
VmRSS: 1978076 kB
VmData: 2541724 kB
VmStk: 140 kB
VmExe: 4 kB
VmLib:36 kB
VmPTE: 4380 kB
VmSwap:0 kB
--
https://mail.python.org/mailman/listinfo/python-list
Re: memory utilization blow up with dict structure
Am Freitag, 23. September 2016 12:02:47 UTC+2 schrieb Chris Angelico:
> On Fri, Sep 23, 2016 at 7:05 PM, Christian wrote:
> > I'm wondering why python blow up a dictionary structure so much.
> >
> > The ids and cat substructure could have 0..n entries but in the most cases
> > they are <= 10,t is limited by <= 6.
> >
> > Example:
> >
> > {'0a0f7a3a0e09826caef1bff707785662': {'ids':
> > {'aa316b86-8169-11e6-bab9-0050563e2d7c',
> > 'aa3174f0-8169-11e6-bab9-0050563e2d7c',
> > 'aa319408-8169-11e6-bab9-0050563e2d7c',
> > 'aa3195e8-8169-11e6-bab9-0050563e2d7c',
> > 'aa319732-8169-11e6-bab9-0050563e2d7c',
> > 'aa319868-8169-11e6-bab9-0050563e2d7c',
> > 'aa31999e-8169-11e6-bab9-0050563e2d7c',
> > 'aa319b06-8169-11e6-bab9-0050563e2d7c'},
> > 't': {'type1', 'type2'},
> > 'dt': datetime.datetime(2016, 9, 11, 15, 15, 54, 343000),
> > 'nids': 8,
> > 'ntypes': 2,
> > 'cat': [('ABC', 'aa316b86-8169-11e6-bab9-0050563e2d7c', '74', ''),
> >('ABC','aa3174f0-8169-11e6-bab9-0050563e2d7c', '3', 'type1'),
> >('ABC','aa319408-8169-11e6-bab9-0050563e2d7c','3', 'type1'),
> >('ABC','aa3195e8-8169-11e6-bab9-0050563e2d7c', '3', 'type2'),
> >('ABC','aa319732-8169-11e6-bab9-0050563e2d7c', '3', 'type1'),
> >('ABC','aa319868-8169-11e6-bab9-0050563e2d7c', '3', 'type1'),
> >('ABC','aa31999e-8169-11e6-bab9-0050563e2d7c', '3', 'type1'),
> >('ABC','aa319b06-8169-11e6-bab9-0050563e2d7c', '3', 'type2')]},
> >
> >
> > sys.getsizeof(superdict)
> > 50331744
> > len(superdict)
> > 941272
>
> So... you have a million entries in the master dictionary, each of
> which has an associated collection of data, consisting of half a dozen
> things, some of which have subthings. The very smallest an object will
> ever be on a 64-bit Linux system is 16 bytes:
>
> >>> sys.getsizeof(object())
> 16
>
> and most of these will be much larger:
>
> >>> sys.getsizeof(8)
> 28
> >>> sys.getsizeof(datetime.datetime(2016, 9, 11, 15, 15, 54, 343000))
> 48
> >>> sys.getsizeof([])
> 64
> >>> sys.getsizeof(('ABC', 'aa316b86-8169-11e6-bab9-0050563e2d7c', '74', ''))
> 80
> >>> sys.getsizeof('aa316b86-8169-11e6-bab9-0050563e2d7c')
> 85
> >>> sys.getsizeof({})
> 240
>
> (Bear in mind that sys.getsizeof counts only the object itself, not
> the things it references - that's why the tuple can take up less space
> than one of its members.)
Thanks for this clarification!
>
> I don't think your collections can average less than about 1KB (even
> the textual representation of your example data is about that big),
> and you have a million of them. That's a gigabyte of memory, right
> there. Your peak memory usage is showing 3GB, so most likely, my
> conservative estimates have put an absolute lower bound on this. Try
> doing everything exactly the same as you did, only without actually
> loading the pickle - then see what memory usage is. I think you'll
> find that the usage is fully legitimate.
>
> > Thanks for any advice to save memory.
>
> Use a database. I suggest PostgreSQL. You won't have to load
> everything into memory all at once that way, and (bonus!) you can even
> update stuff on disk without rewriting everything.
Yes it seems I haven't a chance to avoid that, especially because the dict
example isn't smaller then it will be in real. I'm in a trade-off between
performance and scalability , so the dict construction should be fast as
possible and having reads+writes (using mongodb) is a performance drawback.
Christian
> ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
How to call a script from another?
From a not even newbie: Without knowing much about Python (yet) I'm trying to install the CMS Zope via FTP (with the well documented changes to make it work on an Apache server). By birth Zope is started from a shell script. And not having the permissions to execute such ones I'll try writing a .py script (with the shebang that I allready knows will do the job) to call another .py script like the original shell script does. So my question is: How do I call a .py script from another .py script? (Writing the rest - I hope - is piece of cake ;-) ). (And yes, I know that there are a lot of other problems but for starters I just would like to try to call a .py script from another .py script.) Thanks Chris -- http://mail.python.org/mailman/listinfo/python-list
Re: How to call a script from another?
Thanks guy's, you have opened my eyes and made my next step a whole lot easier. Chris -- http://mail.python.org/mailman/listinfo/python-list
Set an environment variable
Another question from a not even newbie: In Unix you can set an environment variable with the command export PYTHONPATH but I would like to set the variable from at .py script. So my question is: How do I export an environment variable in a .py script? Thanks Chris -- http://mail.python.org/mailman/listinfo/python-list
Re: Set an environment variable
Thanks Jeff and the crazy 88. -- http://mail.python.org/mailman/listinfo/python-list
Re: Set an environment variable
> > The closest thing you can do is that: > > -myScript.py-- > print 'export MY_VARIABLE=value' > -- > > -myScript.sh-- > python myScript.py > /tmp/chgvars.sh > . /tmp/chgvars.sh > -- Can I write a .py script that calls a .sh script that executes the export command and then calls another .py script (and how would the first .py script look)? That would be much more what is my basic problem. Thanks Chris -- http://mail.python.org/mailman/listinfo/python-list
Re: Set an environment variable
Erik Max Francis wrote: > Christian wrote: > >> Can I write a .py script that calls a .sh script that executes the >> export command and then calls another .py script (and how would the >> first .py script look)? > > No, the shell script that the Python program would invoke would be a > different process and so commands executed in it would have no effect on > the state of another. > So executing an .sh script that calls a .py script works different when executed from a command promt than when executed from a starter .py script? -- http://mail.python.org/mailman/listinfo/python-list
Re: Set an environment variable
Steve Holden wrote:
> ::
> one.py
> ::
> import os
> os.environ['STEVE'] = "You are the man"
> os.system("python two.py")
> print "Ran one"
> ::
> two.py
> ::
> import os
> print "STEVE is", os.environ['STEVE']
> print "Ran two"
> [EMAIL PROTECTED] tmp]$ python one.py
> STEVE is You are the man
> Ran two
> Ran one
> [EMAIL PROTECTED] tmp]$
>
> Hope this helps.
>
> regards
> Steve
Thanks Steve, you're quite right, you are the man. And thanks to all the
rest of you for your kind help and patient understanding. I have learned
quite a lot and is about to consider my self advanced to the status of
Python newbie.
So here is my final question:
Do I call the .sh script with a .py script like this:
os.system("/path/to/the/script/startupscript.sh")
--
http://mail.python.org/mailman/listinfo/python-list
Re: Set an environment variable
Steve Holden wrote: > Time you answered your own questions by trying things at the interactive > interpreter prompt! > > regards > Steve Right again, Steve. Thanks -- http://mail.python.org/mailman/listinfo/python-list
Apache & Python 500 Error
Hello, i have an apache 1.3 server with python on debian. Python works fine but the scripts wont´t work. This easy script i want to do on apache: #!/usr/bin/python import os os.getcwd() in apache config i have done this: AddHandler python-program .py PythonHandler python Order allow,deny Allow from all #PythonDebug On the script is in this directory /var/www/python but i get an 500 error everytime, with "every" script - why that - i´m newbie - sorry for that :) greetings from germany to newsgroup christian -- http://mail.python.org/mailman/listinfo/python-list
Re: Apache & Python 500 Error
Jeremy Bowers schrieb: But first, check your apache error log. Then you will know what your error is, too. Thanks for help, here is my error log: [Thu Feb 3 13:52:49 2005] [error] PythonHandler python: Traceback (most recent call last): [Thu Feb 3 13:52:49 2005] [error] PythonHandler python: File "/usr/lib/python2.3/site-packages/mod_python/apache.py", line 181, in Dispatch\nmodule = import_module(module_name, _req) [Thu Feb 3 13:52:49 2005] [error] PythonHandler python: File "/usr/lib/python2.3/site-packages/mod_python/apache.py", line 335, in import_module\nmodule = imp.load_module(mname, f, p, d) [Thu Feb 3 13:52:49 2005] [error] PythonHandler python: File "/var/www/python/python.py", line 1 [Thu Feb 3 13:52:49 2005] [error] PythonHandler python: from mod_python import apache [Thu Feb 3 13:52:49 2005] [error] PythonHandler python: ^ [Thu Feb 3 13:52:49 2005] [error] PythonHandler python: SyntaxError: invalid syntax My test script: #!/usr/bin/python print 'Content-Type: text/plain\r' print '\r' import os print os.getcwd() thanks to Fuzzyman for test script -- http://mail.python.org/mailman/listinfo/python-list
Python CGI
Hi, I'd like to use Python for CGI-Scripts. Is there a manual how to setup Python with Fast-CGI? I'd like to make sure that Python scripts aren't executed by www-user, but the user who wrote the script. -- Gruß, Christian -- https://mail.python.org/mailman/listinfo/python-list
WSGI (was: Re: Python CGI)
On 05/20/2014 03:52 AM, Tim Chase wrote:
> While Burak addressed your (Fast-)CGI issues, once you have a
> test-script successfully giving you output, you can use the
> standard-library's getpass.getuser() function to tell who your script
> is running as.
LoadModule wsgi_module modules/mod_wsgi.so
AddHandler wsgi-script .wsgi
WSGIDaemonProcess myproj user=chris threads=3
[root@t-centos1 ~]# ps -ef|grep chris
chris 1201 1199 0 08:47 ?00:00:00 /usr/sbin/httpd
---8<---
#!/usr/bin/python
import getpass
def application(environ, start_response):
status = '200 OK'
output = 'Hello World!'
output += getpass.getuser()
response_headers = [('Content-type', 'text/plain'),
('Content-Length', str(len(output)))]
start_response(status, response_headers)
return [output]
--->8---
Hello World!root
Hmm, why is it root?
I'm using Apache and mod_userdir. Can I define WSGIDaemonProcess for
each user?
- Chris
--
https://mail.python.org/mailman/listinfo/python-list
Daemon with n subscriber
Hi, experimenting with zmq. I like to start/stop/restart n independent subscriber with one deamon service in Python. How I should adapt a common daemon class in python? Thanks for a starting point Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: does anybody earn a living programming in python?
walterbyrd wrote: > If so, I doubt there are many. > > I wonder why that is? Previously I used Python while earning a living working in IT at a college. Currently it is putting food on the table via contract jobs. I imagine there are "many" out there like me, doing just that. Christian http://www.dowski.com -- http://mail.python.org/mailman/listinfo/python-list
Re: does anybody earn a living programming in python?
I said: > > Previously I used Python while earning a living working in IT at a > > college. Currently it is putting food on the table via contract jobs. > > I imagine there are "many" out there like me, doing just that. faulkner wrote: > where do you find these "contract jobs", if you don't mind my asking? Both of my current contracts came by way of my involvement with the CherryPy open source project. Christian http://www.dowski.com -- http://mail.python.org/mailman/listinfo/python-list
Re: ANN: CherryPy 3.0 RC1
> "Christian Wyglendowski" <[EMAIL PROTECTED]> writes: > > > I'm happy to announce the first release candidate for CherryPy 3.0. Ben Finney wrote: > > Congratulations, I'm glad to see an announcement for CherryPy. > > Please, in future, don't send HTML message bodies to public forums; > plain text is far better for such a wide audience. My apologies. I wrongly assumed that anyone using an email client that only supported plain text would simply use the plain text part of the email. Thanks for bringing this to my attention. Christian http://www.dowski.com -- http://mail.python.org/mailman/listinfo/python-list
build python on Celeron D
Hi, I'm having problems with python 2.4.4 built on a Celeron D. I had problems earlier with that processor which I think has understands some 64 bit instructions (whatever that means) but is still 32 bit. Thus when e.g. trying to build numpy with the self built python I get that error: Traceback (most recent call last): File "setup.py", line 89, in ? setup_package() File "setup.py", line 59, in setup_package from numpy.distutils.core import setup File "/mnt/home/ck/prog/scipy/numpy-1.0.1/numpy/__init__.py", line 36, in ? File "numpy/core/__init__.py", line 5, in ? import multiarray ImportError: /media/hda6/home/ck/prog/scipy/numpy-1.0.1/numpy/core/multiarray.so: undefined symbol: Py_InitModule4 which is related to 32<->64 issues, right? So my question is how do I force python to be built with 32bit? Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: build python on Celeron D
Chris Lasher wrote: > On Mar 26, 10:48 pm, Christian <[EMAIL PROTECTED]> wrote: >> Traceback (most recent call last): >> File "setup.py", line 89, in ? >> setup_package() >> File "setup.py", line 59, in setup_package >> from numpy.distutils.core import setup >> File "/mnt/home/ck/prog/scipy/numpy-1.0.1/numpy/__init__.py", line 36, in ? >> File "numpy/core/__init__.py", line 5, in ? >> import multiarray >> ImportError: >> /media/hda6/home/ck/prog/scipy/numpy-1.0.1/numpy/core/multiarray.so: >> undefined >> symbol: Py_InitModule4 >> >> which is related to 32<->64 issues, right? > > <http://www.thescripts.com/forum/thread35987.html> > > Do you have more than one Python installation on this machine? The > link above indicates that if you build NumPy with a different build of > Python than you will run it from, you will get this same error. Yes, but I took care to not confuse them. > Granted, I'm sure there are many ways to get said error, but that was > the most obvious from a five minute search. I found that: "To prevent loading extension modules that assume a 32-bit size type into an interpreter that has a 64-bit size type, Py_InitModule4 is renamed to Py_InitModule4_64." (http://www.python.org/dev/peps/pep-0353/) So as numpy's setup.py cannot call 'Py_InitModule4' this means, that the interpreter is built for 64bit systems. Maybe numpy is more accurate in testing wether it run's on 64 or on 32 bit than the python build script itself and therefore expect's Py_InitModule4 to be present. Isn't there perhaps a configure option to force 32 bit? Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: wxPython, Syntax highlighting
Eric von Horst wrote: > Hi, > > I am looking for wx widget that has the ability to show text, edit the > text (like a TextCtrl) but also does syntax highlighting (if the text > is e.g. XML or HTML). > I have been looking in the latest wxPython version but I could not > really find anything. Have a look at StyledTextControl (wx.stc) in the demo. Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: XML Parsing
[EMAIL PROTECTED] wrote: > I want to parse this XML file: > > > > > > > filename > > Hello > > > > > filename2 > > Hello2 > > > > > > This XML will be in a file called filecreate.xml > > As you might have guessed, I want to create files from this XML file > contents, so how can I do this? > What modules should I use? What options do I have? Where can I find > tutorials? Will I be able to put > this on the internet (on a googlepages server)? > > Thanks in advance to everyone who helps me. > And yes I have used Google but I am unsure what to use. > Try this: http://www.python.org/doc/2.4.1/lib/expat-example.html Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: Kill thread
On Apr 9, 5:14 am, "Teja" <[EMAIL PROTECTED]> wrote: > Hi all, > > Can any on help me out in killing a thread (i.e deleteing the reources > like, stack ,memory etc) which is started with > win32process.beginthreadex()??? > > Rite now, I am suspending the thread. But any pointers as to how to > delete the thread permanently? > > Its pretty urgent... Please... > > Teja.P Well, the answer with Python threads is that you don't kill them - you ask them to go away. But since you are using something in the pywin32 package, that rule might not apply. Perhaps you would have better luck asking on the python-win32 list: http://mail.python.org/mailman/listinfo/python-win32 Christian http://www.dowski.com -- http://mail.python.org/mailman/listinfo/python-list
Re: database wrapper ?
On Feb 1, 5:56 pm, Stef Mientki wrote: > Is SQLalchemy the best / most popular database wrapper ? > Are there any alternatives ? As others have confirmed, SQLAlchemy is far and away the most popular Python ORM. Another one to have a look at though is Dejavu (http://www.aminus.net/ dejavu). The user community isn't nearly as large but the software itself is quite good. One of its distinctives is that you can treat multiple disparate data stores as one cohesive relational structure. It also isn't bound to SQL databases. There are storage implementations included for file system, Shelve and in-memory storage. The 2.0 alpha code in trunk also has a memcached storage implementation. HTH, Christian http://www.dowski.com -- http://mail.python.org/mailman/listinfo/python-list
Re: database wrapper ?
On Feb 6, 10:17 am, Christian wrote: > One of its distinctives is that ... Not sure how I forgot this, but Dejavu also lets you write your datastore queries in a LINQ-like syntax. Robert Brewer, the author, is giving a talk [1] about it at this year's PyCon in the US. Christian http://www.dowski.com [1] http://pycon.org/2009/conference/talks/#proposal_link_91 -- http://mail.python.org/mailman/listinfo/python-list
Re: Ban Xah Lee
Xah Lee schrieb: Of interest: ⢠Why Can't You Be Normal? http://xahlee.org/Netiquette_dir/why_cant_you_be_normal.html IMHO the point that you never reply to responds is what makes it problematic. I have seen 10 or more threads started by you and in not a single one of those I have seen any sort of second post by you. Also the other thing that makes you appear like a troll is that the only posts where you are visible on the usenet are your own! Usenet is there for discussion. What you do seems to be mostly doing a often highly intelligent monologue and awaiting comment on it. Its not the purpose of Usenet. Simply calling you a troll is wrong. You are after all better than that. Though I think you are misusing the Usenet. For what you do you should rather write a weblog so people interested in your monologues could follow them in a place where they are by definition on topic. Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: Ban Xah Lee
Xah Lee schrieb: Christian wrote: On Mar 9, 1:22 pm, Christian wrote: XahLeeschrieb:> Of interest: ⢠Why Can't You Be Normal? http://xahlee.org/Netiquette_dir/why_cant_you_be_normal.html IMHO the point that you never reply to responds is what makes it problematic. I have seen 10 or more threads started by you and in not a single one of those I have seen any sort of second post by you. Also the other thing that makes you appear like a troll is that the only posts where you are visible on the usenet are your own! Usenet is there for discussion. What you do seems to be mostly doing a often highly intelligent monologue and awaiting comment on it. Its not the purpose of Usenet. Simply calling you a troll is wrong. You are after all better than that. Though I think you are misusing the Usenet. For what you do you should rather write a weblog so people interested in your monologues could follow them in a place where they are by definition on topic. Christian In the article you quoted: http://xahlee.org/Netiquette_dir/why_cant_you_be_normal.html contains this passage: « Some people says that i don't participate in discussion, and this is part of the reason they think i'm a so-called âtrollâ. Actually i do, and read every reply to my post, as well have replied to technical questions other posted. Most replies to my posts are attacks or trivial (of few sentences) i don't consider worthy to reply. A few, maybe 10% replies to my unconventional posts, i consider having some value. But if i don't have sufficiently remarkable opinion on what they remarked, i don't reply. Also, if all i wanted to say is âthanksâ, i tend to avoid posting such trivial posts too. (i used to reply by personal email in such cases, I still do sometimes now, but today that can be considered intrusive.) » I have read the passage though the 10% replies does not reflect my own experience with your posts. Thats why I pointed out that you never reply to the posts, at least not to the ones I have seen. Truly Your Superior, Do you really think that of yourself? Now I really am disappointed of you. Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: Feeding differeent data types to a class instance?
On Mar 14, 2:16 pm, kuru wrote: > Hi > > Thank you so much for all these great suggestions. I will have time > today to try all these and see which one works best for me Answers to your question have spread onto the Python blogosphere as well. http://pythonconquerstheuniverse.wordpress.com/2010/03/17/multiple-constructors-in-a-python-class/ http://blog.dowski.com/2010/03/17/my-take-on-multiple-constructors/ That second one is my response. Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: object.enable() anti-pattern
Am 08.05.2013 10:52, schrieb Steven D'Aprano: > Basically, any time you have two steps required for using an object, e.g. > like this: > > obj = someobject(arg1, arg2) > obj.enable() > > you should move the make-it-work functionality out of the enable method > and into __init__ so that creating the object creates it in a state ready > to work. In general I agree that an object.enable() function is ugly. ;) But it's not necessarily the best solution for all problems. If the resource needs some kind of cleanup, the context api (__enter__() / __exit__()) is perfectly fine way to enable and disable the object. For example: class MyFile: def __init__(self, filename): self.filename = filename def __enter__(self): self.open() return self def __exit__(self, exc_type, exc_val, exc_tb): self.close() I suggest that you mention the context API in your blog post, too. Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: Unicode humor
Am 08.05.13 15:19, schrieb Roy Smith: Apropos to any of the myriad unicode threads that have been going on recently: http://xkcd.com/1209/ http://xkcd.com/1137/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Differences of "!=" operator behavior in python3 and python2 [ bug? ]
Am 13.05.2013 01:23, schrieb Mr. Joe: > I seem to stumble upon a situation where "!=" operator misbehaves in > python2.x. Not sure if it's my misunderstanding or a bug in python > implementation. Here's a demo code to reproduce the behavior - > """ Python 2.7 doesn't use the negation of __eq__ when your class doesn't provide a __ne__ function. Just add a print() to your __eq__ method and you'll notice the different. You have to provide both: class DemoClass(object): def __init__(self, val): self.val = val def __eq__(self, other): if not isinstance(other, DemoClass): return NotImplemented return self.val == other.val def __ne__(self, other): if not isinstance(other, DemoClass): return NotImplemented return self.val != other.val or def __ne__(self, other): result = self.__eq__(other) if result is NotImplemented: return NotImplemented return not result -- http://mail.python.org/mailman/listinfo/python-list
PDF generator decision
Hi folks, This questions may be asked several times already, but the development of relevant software continues day-for-day. For some time now I've been using xhtml2pdf [1] to generate PDF documents from HTML templates (which are rendered through my Django-based web application. This have been working for some time now but I'm constantly adding new templates and they are not looking like I want it (sometimes bold text is bold, sometimes not, layout issues, etc). I'd like to use something else than xhtml2pdf. So far I'd like to ask which is the (probably) best way to create PDFs in Python (3)? It is important for me that I am able to specify not only background graphics, paragaphs, tables and so on but also to specify page headers/footers. The reason is that I have a bunch of documents to be generated (including Invoice templates, Quotes - stuff like that). Any advice is welcome. Thanks. [1] https://github.com/chrisglass/xhtml2pdf -- http://mail.python.org/mailman/listinfo/python-list
Re: Harmonic distortion of a input signal
Am 20.05.13 19:23, schrieb jmfauth:
Non sense.
Dito.
The discrete fft algorithm is valid only if the number of data
points you transform does correspond to a power of 2 (2**n).
Where did you get this? The DFT is defined for any integer point number
the same way.
Just if you want to get it fast, you need to worry about the length. For
powers of two, there is the classic Cooley-Tukey. But there do exist FFT
algorithms for any other length. For example, there is the Winograd
transform for a set of small numbers, there is "mixed-radix" to reduce
any length which can be factored, and there is finally Bluestein which
works for any size, even for a prime. All of the aforementioned
algorithms are O(log n) and are implemented in typical FFT packages. All
of them should result (up to rounding differences) in the same thing as
the naive DFT sum. Therefore, today
Keywords to the problem: apodization, zero filling, convolution
product, ...
Not for a periodic signal of integer length.
eg. http://en.wikipedia.org/wiki/Convolution
How long do you read this group?
Christian
--
http://mail.python.org/mailman/listinfo/python-list
Re: Harmonic distortion of a input signal
Oops, I thought we were posting to comp.dsp. Nevertheless, I think numpy.fft does mixed-radix (can't check it now) Am 20.05.13 19:50, schrieb Christian Gollwitzer: Am 20.05.13 19:23, schrieb jmfauth: Non sense. Dito. The discrete fft algorithm is valid only if the number of data points you transform does correspond to a power of 2 (2**n). Where did you get this? The DFT is defined for any integer point number the same way. Just if you want to get it fast, you need to worry about the length. For powers of two, there is the classic Cooley-Tukey. But there do exist FFT algorithms for any other length. For example, there is the Winograd transform for a set of small numbers, there is "mixed-radix" to reduce any length which can be factored, and there is finally Bluestein which works for any size, even for a prime. All of the aforementioned algorithms are O(log n) and are implemented in typical FFT packages. All of them should result (up to rounding differences) in the same thing as the naive DFT sum. Therefore, today Keywords to the problem: apodization, zero filling, convolution product, ... Not for a periodic signal of integer length. eg. http://en.wikipedia.org/wiki/Convolution How long do you read this group? Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: Utility to locate errors in regular expressions
Am 24.05.13 14:58, schrieb Malte Forkel:
Finding out why a regular expression does not match a given string can
very tedious. I would like to write a utility that identifies the
sub-expression causing the non-match.
Try
http://laurent.riesterer.free.fr/regexp/
it shows the subexpressions which cause the match by coloring the parts.
Not exacty what you want, but very intuitive and powerful. Beware this
is Tcl and there might be subtle differences in RE syntax, but largely
it's the same.
Christian
--
http://mail.python.org/mailman/listinfo/python-list
Re: Beginner Question: 3D Models
Am 19.06.13 04:47, schrieb [email protected]: However, for one part of the program I'd like to be able to create a 3D model based on the user input. The model would be very basic consisting of a number of lines and objects. We have 3D models of each component within our CAD system so it would be great if we could utilize those models. Have a look at vtk http://www.vtk.org/ Using VTK you can import CAD models and visualize them, combine to scenes and export. VTK has Python bindings. It is a real big library, but focused on polygonal models, i.e. it will happily import STL and OBJ, but not IGES and the like ith real curves. Then the question is how you'd want to export your model. VTK can export to VRML and X3D, but if you want to CREATE a real model by CSG of the exisiting parts, you would need a true CAD system. There is not much useful free stuff out there, you could try BRL-CAD or OpenCascade. The latter also has python bindings. http://www.pythonocc.org/ Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: tkinter
Ahoj, Am 23.06.13 18:06, schrieb JK: Nazdar mládenci, this is an English (only) speaking group. Therefore you will not get much response by posting in Czech. měl bych zájem dokončit překlad Tkinteru (http://tkinter.programujte.com/index.htm), na kterém před šesti lety pracovali zejména Pavel Kosina a Jakub Vojáček. Poslal jsem jim mejla ale nehlásí se mi. Poradíte mi? Před několika dny jsem přeložil pěkné texty o Tkinteru (http://www.python-course.eu/python_tkinter.php). Překlad jsem zatím odložil na https://bitbucket.org/tovim/tkinter-klein-cs, odkud si jej případně můžete stáhnout a prohlédnout. Budu jej ještě se svolením autora editovat. Then you should ask the primary authors of http://www.python-course.eu/python_tkinter.php for their permission. BTW, I've not read this tutorial in detail, but IMHO it is a sin these days to not use ttk widgets. That's one main reason why people think that Tk sucks, because the non-themed widgets look odd and oldfashioned. Using Ttk it's possible to have applications practically indistinguishable from native code on Win&Mac. Concerning http://tkinter.programujte.com/index.htm, I don't know if somebody has better contact to the authors here. I'd like to throw in yet another page: http://www.tkdocs.com/tutorial/index.html This is a cross-language tutorial on Tk. It uses ttk from the beginning and is placed under creative commons. In any case, thanks for your willingness to help. Other users will probably find your work very helpful. Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: io module and pdf question
Am 25.06.13 08:33, schrieb rusi: On Tuesday, June 25, 2013 9:48:44 AM UTC+5:30, [email protected] wrote: 1. Is there another way to get metadata out of a pdf without having to install another module? 2. Is it safe to assume pdf files should always be encoded as latin-1 (when trying to read it this way)? Is there a chance they could be something else? If your code is binary open in binary mode (mode="rb") rather than choosing a bogus encoding. You then have to make your strings also binary (b-prefix) Also I am surprised that it works at all. Most pdfs are compressed I thought?? PDFs are a binary format, yes. But they are not, as a whole, compressed. They are made up of "objects", which can be referred to and crosslinked, and these objects are represented as ASCII strings. Some of these can specify a "decoding filter". Examples for the filter include zlib compression (/FlateDecode) and jpeg compression (/DCTDecode). Most of the PDF objects are therefore not encoded. It is, however, possible to include a PDF into another PDF and to encode it, but that's a rare case. Therefore the metadata can usually be read in text mode. However, to correctly find all objects, the xref-table indexes offsets into the PDF. It must be treated binary in any case, and that's the funny reason for the first 3 characters of the PDF - they must include characters with the 8th bit set, such that FTP applications treat it as binary. Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: Best Scripting Language for Embedded Work?
Hi David, you have multi-posted this to comp.lang.tcl. Please don't do that - use crossposting and a proper follow-up (as I did now) Am 10.07.13 03:29, schrieb David T. Ashley: We develop embedded software for 32-bit micros using Windows as the development platform. Robert's answer made me hesitate - what exactly is your platform? Are you writing the scripts for the embedded platform, or for Windows, or does the embedded controller run Windows RT or something like this? We are seeking a general purpose scripting language to automate certain tasks, like cleaning out certain directories of certain types of files in preparation for ZIP'ing, generating certain source files automatically, etc. Selection criteria: a)Should be able to compile the script interpreter as a monolithic executable (no .DLL dependencies, etc.) for easy versioning and distribution of the script interpreter. (Note that I'm not asking that the script be a single executable, just the interpreter. To run a script you'd need both the script and the interpreter. The script would be a text file, and the interpreter would be a single .EXE.) You are referring to tclkits. Yes, it's indeed possible to compile Tcl into a statically linked binary, and C extension packages can be statically linked, too. But tclkits are cheating: There are some files like the standard library (i.e. init.tcl, the clock command etc., unicode encondings...) which are packed into a database and attached to the tclkit. The tclkit then opens itself via the file system to read these files. I don't know if this is possible in a typical embedded system. If you are really talking about Windows, no issue. Python has similar capabilities, look for pyinstaller or py2exe. b)Should be extensible, in that one could add commands or library functions to the script interpreter in C (for efficiency), and the whole script interpreter could again consist of a single executable with no other dependencies. (Note that I'm not asking that the script be a single executable, just the interpreter. To run a script you'd need both the script and the interpreter. The script would be a text file, and the interpreter would be a single .EXE.) That is possible in Tcl using static packages. In Python I don't know, but I think it should be possible. c)Should be able to spawn compilers and capture the output, do file I/O, and all the other minor expected stuff. no real issue d)Graphical capability would be nice. For GUI Python relies on either Tcl/Tk, wxwidgets, QT, GTK... I think it is possible to assemble these into a packaged binary, too. However it will be a rather large thing in the end. I know that Tcl/Tk would do all of the above, but what about Python? Any other alternatives? I think Tcl/Tk is a really good match, especially if you are trying to do GUI, which is very easy there. Also most scripting stuff is available out of the box. Another option might be Lua http://www.lua.org/ Very compact (a static binary is about ~200K), clean synatx, relatively fast. OTOH, the standard library is of course not so extensive as for Tcl or Python. Bash is also an option, because it is some kind of standard for scripting. But on Windows it seems alien and you'd need something like MobaXTerm to get it into a single file. Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: Casting classes WAS: Documenting builtin methods
Am 12.07.2013 02:23, schrieb Mark Janssen: > A user was wondering why they can't change a docstring in a module's class. For CPython builtin types (classes) and function have read-only doc strings for multiple reasons. Internally the doc strings are stored as constant C string literals. The __doc__ attribute is implemented as descriptor that turns the const char *tp_doc member into a Python string. const char* are ... constant. :) All types and builtins are shared across all subinterpreters of Python. The types and their doc strings are identical. Most people are not aware of the subinterpreter feature. Subinterpreters run in different threads of the same process but are isolated from each other. Mutable type and builtin doc strings would break the isolation. You could share information between subinterpreters by e.g. setting the doc string of the int type. We don't want that. Heap types (classes defined with Python code) are not shared. The same goes to modules, even the sys module. Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: How do I get the OS System Font Directory(Cross-Platform) in python?
Am 11.07.2013 19:19, schrieb Metallicow: > @ Chris “Kwpolska” Warrick > Thanks, that is a start anyway. > a Pure-Python way was what I was wanting, not win32api stuff. > > "C:\Windows\Fonts" > The windows path proves valid. Works on XP, Vista, 7. Not sure about win8? That's the wrong way to do it. You have to use the proper Windows API to get to the font directory. It's SHGetKnownFolderPath() with FOLDERID_Font or SHGetFolderPath() with CSIDL_FONTS. See http://bugs.python.org/issue1763 Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: Best Scripting Language for Embedded Work?
Hi David,
Am 12.07.13 03:18, schrieb David T. Ashley:
On Wed, 10 Jul 2013 09:03:54 +0200, Christian Gollwitzer
wrote:
>
Robert's answer made me hesitate - what exactly is your platform? Are
you writing the scripts for the embedded platform, or for Windows, or
does the embedded controller run Windows RT or something like this?
Writing the scripts to run on Windows 7, but the scripts assist in
building the software for an embedded system. Because the embedded
system may be safety-related, it makes sense to be careful about how
the script interpreter is versioned, and a single monolithic
executable makes this easier.
I see. In your situation, I'd also keep the sources to the interpreter
in your backup. Who knows if you might need them to get it running on
some future Windows system.
Python has similar capabilities, look for pyinstaller or py2exe.
Sorry, I miscommunicated. I don't need to embed the script in the
.EXE. I just want the interpreter to be a single .EXE, so it is
easier to ensure that every software developer has the same version of
the interpreter or that we are using the correct earlier version if an
older product needs to rebuilt in the future.
I understand. But nothing prevents you to wrap a simple script loader
main script; something like
execfile(argv[1])
would do it in Python. pyinstaller has a single-file wrapping mode,
where you get the analogue of a starpack in Tcl.
But I still think Tcl/Tk is a much better fit. For one thing,
pyinstaller makes huge executables. I did this on Linux to wrap a
program with matplotlib and Tkinter; I ended up with ~100MB in size.
That's because pyinstaller pulled in a lot of system libraries, among
them QT, wxWidgets etc. Without them, the program did not run despite it
used only Tkinter. In contrast, a starpack wish with just Tcl&Tk is
around 2MB in size.
Second, Tcl/Tk is much nicer to use interactively. Consider an
interactive environment, which woud not include the "ls" command. In
Tcl, you can define
proc ls {args} {
if {[length $args] == 0} { set args * }
puts [join [glob -nocomplain {*}$args] \n]
}
and then, the following commands all work as expected
ls
ls *.jpg
ls *.txt *.jpg
The analogue in Python would look like this
import glob
def ls(*args):
for pat in args if len(args)>0 else ('*'):
print '\n'.join(glob.glob(pat))
but using it is ugly and inconvenient compared to the shellish way in Tcl:
ls()
ls('*.txt')
ls('*.jpg', '*.txt')
Of course this is a contrived example, because all interactive
environments already provide ls. But I usually extend my Tcl with
similar commands and then just use tkcon to enter them on-the-fly.
I also think that for your use case, you will not need to write C
extensions. A standard starpack gives you many things out of the box,
like looking into ZIP files. Even creating them is just two pages of
code. I would suggest that you put together a couple of packages, put
them in a lib/ folder, have your main.tcl something like
lappend auto_path [file join [file dirname [info script]] lib]
package require tkcon
tkcon show
and bake that together using sdx into a starpack. Then you have a single
interpreter, with an interactive console, and you can extend this by Tcl
modules and still have a single executable.
For example, you can get some inspiration from kbs.tcl, which contains a
rather complete make system. It usually builds tclkits, but you can rip
the build system off and write your own dependencies with it. Make a
package out of it, and then your scripts would just package require
make, to get things done which are best describe by a dependency graph.
Christian
--
http://mail.python.org/mailman/listinfo/python-list
Re: Bluetooth Sockets
Am 13.07.2013 10:53, schrieb Simfake Fake: > Hi. I'm trying to connect to a bluetooth serial adaptor using python > 3.x. However, in python 3.3.2 win x32, I get "AttributeError: module has > no attribute AF_..." when trying to use socket.AF_BLUETOOTH, despite the > docs http://docs.python.org/3.3/library/socket.html . The code I have is > very similar to that > found > http://kevindoran1.blogspot.com.au/2013/04/bluetooth-programming-with-python-3.html > I need to use python 3.x, as this is eventually going to run within > Blender.. It's going to run within blender so I can use blender to > visualize and solve inverse-kinematic stuff for a robot arm. AF_BLUETOOTH is only available on Linux and some BSD variants. Windows has different bluetooth stacks. -- http://mail.python.org/mailman/listinfo/python-list
Re: Floating point minimum and maximum exponent values
Am 16.07.2013 14:04, schrieb Chris Angelico: > Piece of extreme oddity, this. > help(sys.float_info) > lots of other info ... > | max_exp > | DBL_MAX_EXP -- maximum int e such that radix**(e-1) is representable > | > | min_exp > | DBL_MIN_EXP -- minimum int e such that radix**(e-1) is a > normalized float > > > So it's technically correct. Followup question: Why is it off by one? Because float.h defines DBL_MAX_EXP like that. Why? I can't tell, too. -- http://mail.python.org/mailman/listinfo/python-list
Re: tkinter redraw rates
Am 18.07.13 06:38, schrieb [email protected]: On Thursday, July 18, 2013 9:07:24 AM UTC+8, Dave Angel wrote: Nope - don't use that. Instead, post an event on the queue, and return to the mainloop() from whence we came. def test_thread(self): if self.loader_thread.isAlive(): self.root_window.after(100, self.test_thread) return I see, though it should be noted that your method doesn't actually block the rest of the even handler code from running, had to fiddle with it a bit to get that to work. May I ask what exactly is the rationale behind implementing it like this, though? Exactly this is the goal of it. Event handlers are supposed to run in as short time as possible, and should never block. The reason is that you want the events to be processed in the order they come in, such that he user can still move the window, resize it, iconify/maximize etc. That said, the code still looks odd to me. I have used the Tk with multithreading in the past, but directly from Tcl, and not from Python. The basic idea is to have the background thread (which does the work) signal the main thread about its status, i.e. in the worker thread: for i in range(50): some_odd_computation() signal('progress', i) signal('finished') and in the main thread you bind() to the events fired from the worker thread. That way you don't run any periodic polling. I fear that Tkinter has a shortcoming which does not allow this pattern to be implemented. The tricky thing is to implement this signal() function, which must post an event to another thread. From the C level, there is Tcl_ThreadQueueEvent() which does this. It arranges for a C function to be run from the event loop of another thread. From Tcl, thread::send does this. To use it from Tkinter, it would be necessary to create a Tcl interpreter in the worker thread *without* loading Tk. Some day I should dive into the innards of Tkinter to see if this is possible. Then you could implement signal() simply by def signal(sig, data=''): tclinterp.eval('thread::send -async $mainthread {event generate . <<%s>> -data {%s}'%sig%data) and in the main thread bind() to the virtual events. Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: How can I make this piece of code even faster?
How about using numpy? Am 20.07.13 22:22, schrieb [email protected]: Ok, I'm working on a predator/prey simulation, which evolve using genetic algorithms. At the moment, they use a quite simple feed-forward neural network, which can change size over time. Each brain "tick" is performed by the following function (inside the Brain class): count = -1 for x in range(hidden_num): temp = 0 for y in range(input_num): count += 1 temp += inputs[y] * h_weight[count] hidden[x] = 1/(1+e**(-temp)) I don't really understand this loop, but it looks to me like a matrix-vector multiplication of the matrix of weights (indexed with a single continous index) with the inputs. Given that you reshape the weights() array correctly into a hidden_num-by-input_num array, this would result in import numpy as np hidden = 1.0/(1.0 + np.exp(-np.dot(h_weights, inputs))) The matrix-vector product is then executed by a compiled loop. You can reshape your existing lists into that form by inputs = np.array(inputs) h_weight = np.reshape(h_weight, (hidden_num, input_num)) ... but of course you should use this only to check whether you get the correct result. You don't want to do that in the loop, instead, store the weights always in matrix form. Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: tkinter progress bar
Am 23.07.13 08:52, schrieb [email protected]: Hi, How can I add a tkinter progress bar in python 3.2 to start before a loop and end after it. I am looking for a very simple solution. def MyFunc(): Start progress bar for fileName in fileList: … End progress bar 1. There is a progress bar widget in ttk. At the beginning, you set maximum to the number of files in your list 2. In the loop, you set "value" of the progressbar to the current file number. You can also attach a variable 3. The bar is only redrawn when you process events. The simplest way to do this is by calling update() on the progress bar, which processes all pending events. Despite of it looking like a method, update() is really a global function within Tcl and updates all widgets in your interface. You must make sure, therefore, that the user does not trigger another event which interferes with your download, such as pressing the button for starting it again. The easiest way is to disable the button at the begin and reenable it at the end. 4. If processing of a single file takes a long time, the only way to have the GUI responsive is to put the work in a background thread. That seems to be more involved. Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 3: dict & dict.keys()
Am 24.07.2013 18:34, schrieb Chris Angelico: > Side point: Why is iterating over a dict equivalent to .keys() rather > than .items()? It feels odd that, with both options viable, the > implicit version iterates over half the dict instead of all of it. > Obviously it can't be changed now, even if .items() were the better > choice, but I'm curious as to the reason for the decision. Consider this: if key in dict: ... for key in dict: ... It would be rather surprising if "in" as containment checks operates on keys and "in" as iterator returns (key, value) tuples. Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: Read STDIN as bytes rather than a string
Am 19.06.2012 01:13, schrieb Jason Friedman: > I tried this: sys.stdin wraps a buffered reader which itself wraps a raw file reader. >>> sys.stdin <_io.TextIOWrapper name='' mode='r' encoding='UTF-8'> >>> sys.stdin.buffer <_io.BufferedReader name=''> >>> sys.stdin.buffer.raw <_io.FileIO name='' mode='rb'> You should read from sys.stdin.buffer unless you really need the bare metal. Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: Finding absolute path of imported module?
Am 19.06.2012 19:55, schrieb Roy Smith: > So, the question is, is there any way to dump all the *absolute* > pathnames of all the imported modules? I can iterate over > sys.modules.values(), but that doesn't give me absolute pathnames, so > I can't tell which version of the symlink existed when the module was > imported. You can use os.path.abspath(module.__file__) to get the absolute path of a module. This works reliable unless you use os.chdir() in your code. abspath() may not normalize symlinks (not sure about it) but you can check for symlink with os.path.islink() (uses os.lstat) and resolve the link with os.readlink(). Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: Py3.3 unicode literal and input()
Am 18.06.2012 20:45, schrieb Terry Reedy: > The simultaneous reintroduction of 'ur', but with a different meaning > than in 2.7, *was* a problem and it should be removed in the next release. FYI: http://hg.python.org/cpython/rev/8e47e9af826e Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: Is that safe to use ramdom.random() for key to encrypt?
Am 20.06.2012 17:25, schrieb D'Arcy Cain: > As "they" say, random number generation is too important to be left > to chance. :-) Hilarious! You made my day! :) -- http://mail.python.org/mailman/listinfo/python-list
Re: Introspect imports from module
Am 21.06.2012 10:03, schrieb Bastian Ballmann: > Any suggestions how I could just get the import of module.to.inspect? > Thanks && have a nice day! You could try a completely different approach and use the compiler package to inspect the abstract syntrax tree of a compiled module. The approach has the nice side effect that you don't have to import a module to inspect its imports. This script may serve as an example for you: http://svn.zope.org/Zope3/trunk/utilities/importchecker.py?rev=113742&view=auto Christian -- http://mail.python.org/mailman/listinfo/python-list
tiffany 0.5 released
Tiffany - Read/Write Multipage-Tiff with PIL without PIL Tiffany stands for any tiff. The tiny module solves a large set of problems, has no dependencies and just works wherever Python works. Tiffany was developed in the course of the *DiDoCa* project and will always appear on PyPi. Version 0.5 --- This is a compatibility update for Python 3.2. Tiffany now works on Python 2.6, 2.7 and 3.2. This version also reduces the needed PIL files to four * 2. The final reduction to a single source is almost ready and will be 0.6 . Please let me know if this stuff works for you, and send requests to or use the links in the bitbucket website: https://bitbucket.org/didoca/tiffany cheers -- Chris -- Christian Tismer :^)<mailto:[email protected]> tismerysoft GmbH : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de work +49 173 24 18 776 mobile +49 173 24 18 776 fax n.a. PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ -- http://mail.python.org/mailman/listinfo/python-list
tiffany 0.5 released
Tiffany - Read/Write Multipage-Tiff with PIL without PIL Tiffany stands for any tiff. The tiny module solves a large set of problems, has no dependencies and just works wherever Python works. Tiffany was developed in the course of the *DiDoCa* project and will always appear on PyPi. Version 0.6 --- This is a compatibility update for Python 3.2. Tiffany now works on Python 2.6, 2.7 and 3.2. This version also reduces the needed PIL files to only four. The same files work for python2 and python3. You can help me by sending some tiff files with different encoding and larger file size for testing. I'm also thinking of - an interface to Qt (without adding a dependency) - a command line interface, to make tiffany into a new tiff tool, - support for other toolkits that need to handle tiff files. Ideas about this are most welcome. Please let me know if this stuff works for you, and send requests to or use the links in the bitbucket website: https://bitbucket.org/didoca/tiffany cheers -- Chris -- Christian Tismer :^)<mailto:[email protected]> tismerysoft GmbH : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de work +49 173 24 18 776 mobile +49 173 24 18 776 fax n.a. PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
tiffany 0.6 released
Tiffany - Read/Write Multipage-Tiff with PIL without PIL Tiffany stands for any tiff. The tiny module solves a large set of problems, has no dependencies and just works wherever Python works. Tiffany was developed in the course of the *DiDoCa* project and will always appear on PyPi. Version 0.6 --- This is a compatibility update for Python 3.2. Tiffany now works on Python 2.6, 2.7 and 3.2. This version also reduces the needed PIL files to only four. The same files work for python2 and python3. You can help me by sending some tiff files with different encoding and larger file size for testing. I'm also thinking of - an interface to Qt (without adding a dependency) - a command line interface, to make tiffany into a new tiff tool, - support for other toolkits that need to handle tiff files. Ideas about this are most welcome. Please let me know if this stuff works for you, and send requests to or use the links in the bitbucket website: https://bitbucket.org/didoca/tiffany cheers -- Chris -- Christian Tismer :^)<mailto:[email protected]> tismerysoft GmbH : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de work +49 173 24 18 776 mobile +49 173 24 18 776 fax n.a. PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: tiffany 0.6 released
Abour tiffany... On 6/25/12 3:46 AM, Christian Tismer wrote: Tiffany - Read/Write Multipage-Tiff with PIL without PIL Tiffany stands for any tiff. The tiny module solves a large set of problems, has no dependencies and just works wherever Python works. Tiffany was developed in the course of the *DiDoCa* project and will always appear on PyPi. Version 0.6 --- This is a compatibility update for Python 3.2. Tiffany now works on Python 2.6, 2.7 and 3.2. This version also reduces the needed PIL files to only four. The same files work for python2 and python3. You can help me by sending some tiff files with different encoding and larger file size for testing. I'm also thinking of - an interface to Qt (without adding a dependency) - a command line interface, to make tiffany into a new tiff tool, - support for other toolkits that need to handle tiff files. Ideas about this are most welcome. Please let me know if this stuff works for you, and send requests to or use the links in the bitbucket website: https://bitbucket.org/didoca/tiffany cheers -- Chris Howdy. I saw quite a lot of downloads of this package now, but not a single reaction or any feedback. How can I get more than just down-loaders? I would like to know, if - this module is of use for you - if there are wishes to get more functionality - if it makes sense at all - if there are feature requests. I could stop right now, freeze development as the stuff does what is supposed to do in the relevant Python versions, name it 1.0 and forget about it. But I don't want to. Can somebody please give me some feedback? Criticism? Helpful, useless, boring, whatsoever? thanks -- cheers - Chris -- Christian Tismer :^)<mailto:[email protected]> tismerysoft GmbH : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship*http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de work +49 173 24 18 776 mobile +49 173 24 18 776 fax n.a. PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today?http://www.stackless.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: tiffany 0.6 released
Hi Steven, On 26.06.12 04:18, Steven D'Aprano wrote: On Mon, 25 Jun 2012 23:36:59 +0200, Christian Tismer wrote: I saw quite a lot of downloads of this package now, but not a single reaction or any feedback. Feel fortunate that you are getting any downloads at all :) In my experience, if you are really lucky, perhaps one in a hundred people who download a package will comment about it. Ok, good to know. This is my first little package (funny, eh? Did much larger things), and this is new to me. How can I get more than just down-loaders? I would like to know, if - this module is of use for you - if there are wishes to get more functionality - if it makes sense at all - if there are feature requests. You can't force people to comment. Do you have a blog where you can report new functionality or bug fixes? That might help drive interest. Good input. I was trying to blog several times, but somehow lost it, again. Thanks a lot, I will force myself to blog. (but I'm more into email conversations, maybe doing stuff for too long ? ) ... If you are writing software for fame and attention, you're in the wrong industry :) I assume not, at least I think. I want to do things right, and to see if they are right or if I'm on the wrong track, that's why I want the feedback. And I got quite some feedback on Stackless Python, but tiffany is a small niche project for probably orders of magnitude less people. BTW, I have no need for your software, but that doesn't mean it isn't a good and valuable piece of work. Thank you for sharing it with the community, even if only a few people find it useful. Thanks for the adjustment. Now I'm feeling fine and will move on to other targets ;-) cheers -- Chris -- Christian Tismer :^) <mailto:[email protected]> tismerysoft GmbH : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de work +49 173 24 18 776 mobile +49 173 24 18 776 fax n.a. PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Why has python3 been created as a seperate language where there is still python2.7 ?
On 26.06.12 08:34, Stefan Behnel wrote: Devin Jeanpierre, 26.06.2012 08:15: On Mon, Jun 25, 2012 at 11:35 PM, Steven D'Aprano Making print a statement in the first place was a mistake, but fortunately it was a simple enough mistake to rectify once the need for backward compatibility was relaxed. Hmmm, why is the function so much better than the statement? You like using it in expressions? Or is it that you like passing it in as a callback? First of all, the statement has a rather special syntax that is not obvious and practically non-extensible. It also has hidden semantics that are hard to explain and mixes formatting with output - soft-space, anyone? The function is straight forward, configurable, does one thing, works with help() and doesn't get in the way. And something as rarely[1] used as a print simply doesn't deserve special syntax. Oh, and, yes, you can even pass it into some code as callback, although I rarely had a need for that. I agree, and I don't want to revive an old discussion of the print statement. I just still don't see the point why the transition is made so uni-directional? With python2.7, it is great that "from __future__ import print_function" exists. But porting old code (PIL for instance) imposes a lot of changes which don't make sense, but produce overhead. Some are simple things like the print statement, which is used only in the debugging code. Enforcing the syntax change enforces changing many modules, which could otherwise work just fine as they are. I think, for the small importance of the print statement in code, it would have made the transition easier, if python 3 was as flexible as python 2.7, with a symmetric "from __past__ import print_statement" construct. That would have at least my acceptance much quicker, because the necessity of modifying stuff would reduce to the few changes which are important in a few modules. So right now, I try to use python 3, but the flexibility is right now in python2.7 . cheers - Chris -- Christian Tismer :^) <mailto:[email protected]> tismerysoft GmbH : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de work +49 173 24 18 776 mobile +49 173 24 18 776 fax n.a. PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Why has python3 been created as a seperate language where there is still python2.7 ?
On 27.06.12 13:02, Chris Angelico wrote: On Wed, Jun 27, 2012 at 8:25 PM, Christian Tismer wrote: I think, for the small importance of the print statement in code, it would have made the transition easier, if python 3 was as flexible as python 2.7, with a symmetric "from __past__ import print_statement" construct. For how long? Will Python require, in perpetuity, the code to support this? Must the print statement be enhanced when the print function is? What about bug fixes? How much dev time is required to enable backward compatibility past a boundary across which backward compatibility was not promised? And if there's a limit to the duration of this __past__ directive, when should it be and what should happen after that point? Much easier to simply say no. Just as a note: It is not that I'm lazy or against python3 or anything. The opposite is true, as I'm a long-term developer and python evangelist. My argument simply addresses to get as much acceptance of python3 as quickly as possible, and some hard work put into a backward feature would IMHO have been better for python3. I would even like much more drastic changes that python3 actually does, to make the move really worth moving. What happened was a bit the opposite: huge effort for making a really useful python 2.7. My strategy would have put less effort into that, and more to make python3 clearly the thing that people want and need. Right now I think python 2.7 is simply too good. - And especially for the print statement: It is a bad idea that the print statement _must_ create a syntax error. It is IMHO not important to support it really and could just work more or less, with no new features, because as said: print, function or not, is not important enough to enforce a rewrite everywhere because of syntax error. That hides the real semantic changes which _are_ important. So what I would have done is to let it work in an imperfect way. People then have the chance to rewrite with the print function, where it makes sense. But old packages which need to be changed, only because they have lots of if DEBUG: print xxx, yyy, ... could just stay unchanged or migrated later after the real meat has been properly tested etc. Well, I missed the right time to discuss that, so this is just a useless note about history. cheers -- Chris -- Christian Tismer :^) <mailto:[email protected]> tismerysoft GmbH : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de work +49 173 24 18 776 mobile +49 173 24 18 776 fax n.a. PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: tiffany 0.6 released
... Thanks for the adjustment. Now I'm feeling fine and will move on to other targets ;-) By the way: Our conversation seems to have a real effect on downloads. :-) It has been quite a boost since 20 hours from some 25-40 to now over 200. cheers -- chris -- Christian Tismer :^) <mailto:[email protected]> tismerysoft GmbH : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de work +49 173 24 18 776 mobile +49 173 24 18 776 fax n.a. PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: tiffany 0.6 released
On 27.06.12 15:24, Christian Tismer wrote: ... Thanks for the adjustment. Now I'm feeling fine and will move on to other targets ;-) By the way: Our conversation seems to have a real effect on downloads. :-) It has been quite a boost since 20 hours from some 25-40 to now over 200. but it _may_ be that this is also an effect of supporting python 3.2, which probably made quite much sense, after all. -- Christian Tismer :^) <mailto:[email protected]> tismerysoft GmbH : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de work +49 173 24 18 776 mobile +49 173 24 18 776 fax n.a. PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Why has python3 been created as a seperate language where there is still python2.7 ?
On 27.06.12 15:44, Stefan Behnel wrote: Christian Tismer, 27.06.2012 15:15: print, function or not, is not important enough to enforce a rewrite everywhere because of syntax error. That hides the real semantic changes which _are_ important. So what I would have done is to let it work in an imperfect way. People then have the chance to rewrite with the print function, where it makes sense. But old packages which need to be changed, only because they have lots of if DEBUG: print xxx, yyy, ... could just stay unchanged or migrated later after the real meat has been properly tested etc. Well, at least a SyntaxError makes it easy to find. And even if you don't intend to use 2to3, you can still run it once to generate a patch for you that only fixes up "print". In many cases, that will also make it work in Py2 in such an "imperfect" way, either by doing the right thing already or by printing out a tuple instead of a space separated string. If it's only for debug output (which, as I said, would better be served by the logging module), I can't see why that would be all that unacceptable. Ok, here comes the real story: I extracted a few files from the old PIL package and made the stuff that was needed for my own little utility with a couple of monkey-patches, import hooks etc. After quite some trouble, this worked unter python 2.6 and 2.7, with the unchanged original PIL files. And I was after that: abuse PIL without maintaining it! Then I got real problems when trying to make it run under python 3.2, and from then on I needed two sets of source files, mostly because of the print mess. Most other things were adjustable by monkey-patches, moving to the io module etc. etc. In the end, the only thing that required a real change of source code that I could not circumvent was the indexing of bytes, which is giving chars under python 2.x, but integers under 3.x. At this point my hope to keep the unmodified PIL files died, although without print, this would have been an isolated single 20 lines block in the TiffImagePlugin only, instead of much more changes criss-cross over several sources. Anyway, I ended up with a port of the relevant PIL files to python 3 with some backward-compatible additions (for heaven's sake, at least python 3 accepts the __future__ imports), so now I'm down to single source files, but unfortunately changed files, and I have to track changes in the future. That battle was lost: I wanted the PIL files simply to be drop-ins, without going to work on PIL, because then I would do a complete rewrite after some discussion with the author. That's why I was unhappy with py3's missing flexibility. ciao -- Chris -- Christian Tismer :^) <mailto:[email protected]> tismerysoft GmbH : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de work +49 173 24 18 776 mobile +49 173 24 18 776 fax n.a. PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Why has python3 been created as a seperate language where there is still python2.7 ?
On 6/27/12 8:58 PM, Serhiy Storchaka wrote: On 27.06.12 17:34, Christian Tismer wrote: That's why I was unhappy with py3's missing flexibility. Excessive flexibility is amorphism. Random notes without context and reasoning are no better than spam. My answer as well, of course, so let's stop here. -- Christian Tismer :^) <mailto:[email protected]> tismerysoft GmbH : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de work +49 173 24 18 776 mobile +49 173 24 18 776 fax n.a. PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: how can I implement "cd" like shell in Python?
Am 28.06.2012 13:09, schrieb Sergi Pasoev: > Do you mean to implement the cd command ? To what extent do you want to > implement it ? if what you want is just to have a script to change the > current working directory, it is as easy as this: Please note that you can't change the working directory of another process. Your snipplet won't alter the current working directory of the shell. "cd" is a builtin shell command, not a binary. Christian -- http://mail.python.org/mailman/listinfo/python-list
tiffany 0.6.1 released
Tiffany - Read/Write Multipage-Tiff with PIL without PIL Tiffany stands for any tiff. The tiny module solves a large set of problems, has no dependencies and just works wherever Python works. Tiffany was developed in the course of the *DiDoCa* project and will always appear on PyPi. Version 0.6.1 - This version uses the new int.from_bytes/to_bytes methods from python3.2 and emulates them on python2.6/2.7 . This migration was tested using pytest. Tiffany is quite unlikely to change anymore until user requests come, or I get better test data: Testing with larger tiff files -- The implementation right now copies data in one big chunk. I would like to make that better/not limited by memory. For that, I need a tiff file that is a few megabytes big. Can somebody please send me one? Extending Tiffany? -- I'm also thinking of - an interface to Qt (without adding a dependency) - a command line interface, to make tiffany into a new tiff tool, - support for other toolkits that need to handle tiff files. Ideas about this are most welcome. Please let me know if this stuff works for you, and send requests to or use the links in the bitbucket website: https://bitbucket.org/didoca/tiffany cheers -- Chris -- Christian Tismer :^)<mailto:[email protected]> tismerysoft GmbH : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de work +49 173 24 18 776 mobile +49 173 24 18 776 fax n.a. PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: tiffany 0.6.1 released
Am 30.06.2012 18:25, schrieb Paul Rubin: > Christian Tismer writes: >> Tiffany stands for any tiff. The tiny module solves a large set of >> problems, has no dependencies and just works wherever Python works. >> Tiffany was developed in the course of the *DiDoCa* project and will >> always appear on PyPi. > > This sounds pretty neat. I didn't comment on it earlier because I > haven't tried it out, since I haven't had occasion to deal with tiff > files anytime recently. But I've had to process them for some projects > in the past, and tiffany would have been useful then. It's good to know > that it's out there. I've developed smc.freeimage exclusively to process large amounts of TIFF images. I estimate that we have processed more than twelve million unique TIFF images with about half a petabyte of data. The packages uses Cython to wrap FreeImage (containing libtiff, libpng, libjpeg, libraw and more) and LittleCMS2. The package is mostly fitted to our needs, a bit limited (e.g. no conversion of CMYK to RGB with color management) and doesn't follow recent best practices for Cython code, but it does it job well. I need to clean up the code base some day when more people get interested in the lib. Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: locals().update(...)
Am 04.07.2012 13:56, schrieb [email protected]: > I expected this to work: It doesn't work and that's documented: http://docs.python.org/library/functions.html?highlight=locals#locals -- http://mail.python.org/mailman/listinfo/python-list
Re: How to safely maintain a status file
Am 08.07.2012 13:29, schrieb Richard Baron Penman: > My initial solution was a thread that writes status to a tmp file > first and then renames: > > open(tmp_file, 'w').write(status) > os.rename(tmp_file, status_file) You algorithm may not write and flush all data to disk. You need to do additional work. You must also store the tmpfile on the same partition (better: same directory) as the status file with open(tmp_file, "w") as f: f.write(status) # flush buffer and write data/metadata to disk f.flush() os.fsync(f.fileno()) # now rename the file os.rename(tmp_file, status_file) # finally flush metadata of directory to disk dirfd = os.open(os.path.dirname(status_file), os.O_RDONLY) try: os.fsync(dirfd) finally: os.close(dirfd) > This works well on Linux but Windows raises an error when status_file > already exists. > http://docs.python.org/library/os.html#os.rename Windows doesn't suppport atomic renames if the right side exists. I suggest that you implement two code paths: if os.name == "posix": rename = os.rename else: def rename(a, b): try: os.rename(a, b) except OSError, e: if e.errno != 183: raise os.unlink(b) os.rename(a, b) Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: How to safely maintain a status file
Am 08.07.2012 22:57, schrieb Laszlo Nagy: > But even if the rename operation is atomic, there is still a race > condition. Your program can be terminated after the original status file > has been deleted, and before the temp file was renamed. In this case, > you will be missing the status file (although your program already did > something just it could not write out the new status). You are contradicting yourself. Either the OS is providing a fully atomic rename or it doesn't. All POSIX compatible OS provide an atomic rename functionality that renames the file atomically or fails without loosing the target side. On POSIX OS it doesn't matter if the target exists. You don't need locks or any other fancy stuff. You just need to make sure that you flush the data and metadata correctly to the disk and force a re-write of the directory inode, too. It's a standard pattern on POSIX platforms and well documented in e.g. the maildir RFC. You can use the same pattern on Windows but it doesn't work as good and doesn't guaranteed file integrity for two reasons: 1) Windows's rename isn't atomic if the right side exists. 2) Windows locks file when a program opens a file. Other programs can't rename or overwrite the file. (You can get around the issue with some extra work, though.) Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: How to safely maintain a status file
Am 09.07.2012 07:50, schrieb Plumo: >> Windows doesn't suppport atomic renames if the right side exists. I >> suggest that you implement two code paths: > > Problem is if the process is stopped between unlink and rename there > would no status file. Yeah, you have to suffer all of Windows' design flaws. You could add a backup status file or use a completely different approach. -- http://mail.python.org/mailman/listinfo/python-list
Re: tiffany 0.6.1 released
On 30.06.12 18:25, Paul Rubin wrote: Christian Tismer writes: Tiffany stands for any tiff. The tiny module solves a large set of problems, has no dependencies and just works wherever Python works. Tiffany was developed in the course of the *DiDoCa* project and will always appear on PyPi. This sounds pretty neat. I didn't comment on it earlier because I haven't tried it out, since I haven't had occasion to deal with tiff files anytime recently. But I've had to process them for some projects in the past, and tiffany would have been useful then. It's good to know that it's out there. Meanwhile I got some feedback and test data. (Thanks to Christian and Anthon) It turns out to be a problem with multiple strips in a tiff file. PIL does not support that. Maybe I can find an easy solution, maybe I'm better off using smc.freeimage as suggested by Christian Heimes, we will see. Right now I'm pretty exhaused after EuroPython... cheers - chris -- Christian Tismer :^) <mailto:[email protected]> tismerysoft GmbH : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/ 14482 Potsdam: PGP key -> http://pgp.uni-mainz.de work +49 173 24 18 776 mobile +49 173 24 18 776 fax n.a. PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Interview Questions
Am 09.07.2012 23:22, schrieb Peter: > One of my favourite questions when interviewing - and it was 100% reliable > :-) - "what are your hobbies?" > > If the answer included programming then they were hired, if not, then they > went to the "B" list. on the contrary! When a potential candidate has computer stuff as her main hobby then she goes on the no-hire list. I prefer people that can cope with stress and pressure as well as people who can think outside the box. When you work with computers all day at work *and* at home then you are unable to shut off mentally. Gardening is great hobbies for a developer. You need to be patient, reliable and provide constantly good work to grow your own vegetables. Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: How to safely maintain a status file
Am 09.07.2012 22:24, schrieb John Nagle: > Rename on some file system types (particularly NFS) may not be atomic. The actual operation is always atomic but the NFS server may not notify you about success or failure atomically. See http://linux.die.net/man/2/rename, section BUGS. > That's because you're using the wrong approach. See how to use > ReplaceFile under Win32: > > http://msdn.microsoft.com/en-us/library/aa365512%28VS.85%29.aspx The page doesn't say that ReplaceFile is an atomic op. Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Interview Questions
Am 10.07.2012 01:40, schrieb Roy Smith: > Do you really want to make hire/no-hire decisions based on somebody's > ability to second-guess what you probably wanted to hear when you asked > a pointless question? I don't want her/him to second-guess at all. I expect a straight and honest answer. Second-guessing leads to cheating and lying which doesn't work in the long run. "I don't like to disclose my personal life" is also a good answer as it shows that the candidate self-confidence and doesn't sell privacy for a job. -- http://mail.python.org/mailman/listinfo/python-list
Re: tiffany 0.6.1 released
Am 09.07.2012 17:51, schrieb Christian Tismer: > It turns out to be a problem with multiple strips in a tiff file. > PIL does not support that. Maybe I can find an easy solution, > maybe I'm better off using > > smc.freeimage > > as suggested by Christian Heimes, > > we will see. Right now I'm pretty exhaused after EuroPython... Yes, TIFF is a complex format. Even baseline TIFF has several ways to store data (e.g. chunky or planar configuration, different settings for stripes). PIL didn't support several features as well as bitonal compression (G3, G4). I'll clean up the code of smc.freeimage and release it on bitbucket over the next couple of days. Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Interview Questions
Am 10.07.2012 09:33, schrieb Steven D'Aprano: > This is why I hate job interviews. You have like 30 minutes, or even as > little as 30 seconds, to make a good impression on somebody who may or > may not be capable of telling the difference between a cheese sandwich > and a box of hair -- and even the *good* interviewers are probably making > their judgement on the basis of subjective factors with no right or wrong > answers. IMHO one category of answers is always wrong: lies. You may oversell yourself a bit, you can (and should) keep private matters to yourself but don't lie. > And live in a house in the suburbs with enough room for a garden, good > soil, and not in the shadow of buildings. And work hours where you are > home during daylight hours. Almost everybody can garden under ideal conditions. I grow about 15 herbs, strawberries, tomatoes, chillies and flowers on a small balcony in the middle of the city. This year I'm going to harvest at least 200 tomatoes from two plants in a 1m * 40cm * 40cm box of soil. I even have a calabash plant that grows like crazy. See? :) Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: How to safely maintain a status file
Am 12.07.2012 14:30, schrieb Laszlo Nagy: > This is not a contradiction. Although the rename operation is atomic, > the whole "change status" process is not. It is because there are two > operations: #1 delete old status file and #2. rename the new status > file. And because there are two operations, there is still a race > condition. I see no contradiction here. Sorry, but you are wrong. It's just one operation that boils down to "point name to a different inode". After the rename op the file name either points to a different inode or still to the old name in case of an error. The OS guarantees that all processes either see the first or second state (in other words: atomic). POSIX has no operation that actually deletes a file. It just has an unlink() syscall that removes an associated name from an inode. As soon as an inode has no names and is not references by a file descriptor, the file content and inode is removed by the operating system. rename() is more like a link() followed by an unlink() wrapped in a system wide global lock. > It is not entirely true. We are talking about two processes. One is > reading a file, another one is writting it. They can run at the same > time, so flushing disk cache forcedly won't help. You need to flush the data to disk as well as the metadata of the file and its directory in order to survive a system crash. The close() syscall already makes sure that all data is flushed into the IO layer of the operating system. With POSIX semantics the reading process will either see the full content before the rename op or the full content after the rename op. The writing process can replace the name (rename op) while the reading process reads the status file because its file descriptor still points to the old status file. Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: How to safely maintain a status file
Am 12.07.2012 19:43, schrieb Laszlo Nagy: > Well, I didn't know that this is going to work. At least it does not > work on Windows 7 (which should be POSIX compatible?) Nope, Windows's file system layer is not POSIX compatible. For example you can't remove or replace a file while it is opened by a process. Lot's of small things work slightly differently on Windows or not at all. Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: [Python] RE: How to safely maintain a status file
Am 13.07.2012 21:57, schrieb MRAB: > It's possible to create a temporary file even in Windows. Windows has a open() flag named O_TEMPORARY for temporary files. With O_TEMPORARY the file is removed from disk as soon as the file handle is closed. On POSIX OS it's common practice to unlink temporary files immediately after the open() call. -- http://mail.python.org/mailman/listinfo/python-list
Re: How to safely maintain a status file
Am 13.07.2012 03:52, schrieb Steven D'Aprano: > And some storage devices (e.g. hard drives, USB sticks) don't actually > write data permanently even when you sync the device. They just write to > a temporary cache, then report that they are done (liar liar pants on > fire). Only when the cache is full, or at some random time at the > device's choosing, do they actually write data to the physical media. > > The result of this is that even when the device tells you that the data > is synched, it may not be. Yes, that's another issue. Either you have to buy expensive enterprise hardware with UPS batteries or you need to compensate for failures on software level (e.g. Hadoop cluster). We have big storage devices with double redundant controllers, on board buffer batteries, triple redundant power supplies, special RAID disks, multipath IO fiber channel links and external backup solution to keep our data reasonable safe. Christian -- http://mail.python.org/mailman/listinfo/python-list
