managing multiple subprocesses
Hi guys,
I realise this question has been answered in one form or another many
times before but I can't quite find the solution I need. I am trying to
run multiple subprocesses from a python script and then wait until all
subprocesses have completed before continuing. The subprocesses run on
other machines ie this is a coarse grained parallel computation task.
So the commands I am using are:
for x in range(0, limit):
os.system("xgrid ./someprogram %d &" % x)
and then to check whether they are all finished I do a very crude:
joblist = commands.getoutput("xgrid -job list")
while joblist != "{jobArray = (); }":
time.sleep(1)
joblist = commands.getoutput("xgrid -job list")
Xgrid is a system for parallel computation on Mac OS X. It is quite
good, check it out at www.apple.com/acg/xgrid. But anyway the method I
am using above, while it works, it quite often hangs. I'd like to just
run all the subprocesses at once and then check within the script,
rather than an auxilliary, to see if everything has returned. The
output of the commands is not important as that gets redirected to
files outside the script. I have tried all sorts of popens / excevs /
os.systems / commands etc etc. I realise the subprocess module may have
what I need but I can't get python 2.4 on the Mac so I need a 2.3 based
solution. Any help is much appreciated. Cheers.
--
http://mail.python.org/mailman/listinfo/python-list
managing multiple subprocesses
Hi guys,
I realise this question has been answered in one form or another many
times before but I can't quite find the solution I need. I am trying to
run multiple subprocesses from a python script and then wait until all
subprocesses have completed before continuing. The subprocesses run on
other machines ie this is a coarse grained parallel computation task.
So the commands I am using are:
for x in range(0, limit):
os.system("xgrid ./someprogram %d &" % x)
and then to check whether they are all finished I do a very crude:
joblist = commands.getoutput("xgrid -job list")
while joblist != "{jobArray = (); }":
time.sleep(1)
joblist = commands.getoutput("xgrid -job list")
Xgrid is a system for parallel computation on Mac OS X. It is quite
good, check it out at www.apple.com/acg/xgrid. But anyway the method I
am using above, while it works, it quite often hangs. I'd like to just
run all the subprocesses at once and then check within the script,
rather than an auxilliary, to see if everything has returned. The
output of the commands is not important as that gets redirected to
files outside the script. I have tried all sorts of popens / excevs /
os.systems / commands etc etc. I realise the subprocess module may have
what I need but I can't get python 2.4 on the Mac so I need a 2.3 based
solution. Any help is much appreciated. Cheers.
--
http://mail.python.org/mailman/listinfo/python-list
Improve usage of Python 3
Hi! Just like many, I want the projects in which I work on to move to Python 3. And incredibly, there are a few users on the same project who refuse to use python 3 simply because of the print statement. That has probably already been discussed, but since I actually couldn't find anything relevant about, I decided to ask here anyway. What are the changes of something like from __past__ import print_statement or to make both print and print() work on Python 3 ? Sorry for the dumb question, but although it's not a very pure solution, it might be one that would be very well accepted. Some reference: http://stackoverflow.com/questions/20250130/from-past-import-print-statement https://news.ycombinator.com/item?id=6990481 https://news.ycombinator.com/item?id=6987309 https://news.ycombinator.com/item?id=6985792 http://code.activestate.com/lists/python-dev/10/ http://code.activestate.com/lists/python-dev/101115/ Thanks Marcos -- https://mail.python.org/mailman/listinfo/python-list
Dynamic list name from a string
Hi, I need to create a list with a dynamic name, something like this: '%s_list' %my_dynamic list = [] It's this possible? -- http://mail.python.org/mailman/listinfo/python-list
Re: Dynamic list name from a string
Ok, thanks Steven and Andre for yours reply. This is my situation: I've a list of objects of differents classes, with a common attribute name Something like this: class Object1: obj1_name other fields date class Object2: obj2_name other fields date ... class ObjectN: objN_name other fields date the lists are created dynamically at runtime, depending or not if there one or more correspondig Objects, ex. list_object1 = [object1_1, object1_2 ... object1_n] ... list_objectN = [object2_1, object2_2 ... object2_n] Then i need to include all into one global list sorting the various objects by date ex global_list = [object1_1, object2_n, object1_2 ... etc] I think that Andre solution it's the best way to solve my dynamics lists problem, now I'll verify how to create the global list and sort it -- http://mail.python.org/mailman/listinfo/python-list
Heuristic
I have a store, so I want to maximize the profit. I have all the suppliers with diferent prices, some providers can send products to a client an others not, this has a plus price. Some providers has a discount over the tansport if a quantity is reached. Sometimes its better to me receive the order and resend to the client if I have a transport discount. Not all the suppliers has all products for a order. So I want to create a function which I can pass the data, and generates all the possibilities so I can find the maximum profit. Have I to use heuristics? Do you know some examples?. Thanks in advance. -- http://mail.python.org/mailman/listinfo/python-list
Re: Heuristic
On 25 jun, 03:28, Terry Reedy wrote: > On 6/24/2010 9:13 PM, Marcos wrote: > > > I have a store, so I want to maximize the profit. I have all the > > suppliers with diferent prices, some providers can send products to a > > client an others not, this has a plus price. Some providers has a > > discount over the tansport if a quantity is reached. > > > Sometimes its better to me receive the order and resend to the client > > if I have a transport discount. > > > Not all the suppliers has all products for a order. > > > So I want to create a function which I can pass the data, and > > generates all the possibilities so I can find the maximum profit. > > > Have I to use heuristics? Do you know some examples?. > > You would not use a heuristic to generate all possibilities. You might > use one to *avoid* doing that, and still get a good, not necessarily > optimal, answer. Wikipedia has an general entry on 'heuristic'. > Algorithm books often specifically discuss heuristic algorithms. > > -- > Terry Jan Reedy Thanks terry I am reading about Knapsack problem, but the main problem I have its the repeated elements, so If i am itering all elements some can be repeated, I dont know how to avoid this, I thinked in neural networks too. Do you have any ideas?. -- http://mail.python.org/mailman/listinfo/python-list
Re: Heuristic
On 25 jun, 03:36, geremy condra wrote: > On Thu, Jun 24, 2010 at 9:13 PM, Marcos wrote: > > I have a store, so I want to maximize the profit. I have all the > > suppliers with diferent prices, some providers can send products to a > > client an others not, this has a plus price. Some providers has a > > discount over the tansport if a quantity is reached. > > > Sometimes its better to me receive the order and resend to the client > > if I have a transport discount. > > > Not all the suppliers has all products for a order. > > > So I want to create a function which I can pass the data, and > > generates all the possibilities so I can find the maximum profit. > > > Have I to use heuristics? Do you know some examples?. > > > Thanks in advance. > > Depending on the exact constraints this may be a linear programming > problem (very simple) and even if it isn't there are likely to be good > approximation methods. With more information will come better > advice. > > Geremy Condra I geremy Im goint to put you some example: Providers: Number 1- Product 1: 56$ Product 4: 45$ Number 2- Product 2: 24$ Product 4: 34$ Number 3- Product 1: 45$ Product 2: 65$ Product 3: 65$ Product 4: 76$ Transport Provider number 3 can transport direct to customer with a plus of 5$, It can deply to a distribution center. Provider number 2 and 1 only transport to a distribution center. Poriver number 2 gives free transport over 500$, and number 1 over 1000$. Each provider has a transport cost, that we have in tables. If we deply to the distribution center we have to add the transport to the consumer. Order: 10 units product 1 6 units product 2 20 units product 3 40 units product 4 So which is the best way to deploy the order optimizing profit?. This example has random quantities of product and prices, but I think now its clear. Regards. -- http://mail.python.org/mailman/listinfo/python-list
Re: Heuristic
On 25 jun, 04:00, MRAB wrote: > Terry Reedy wrote: > > On 6/24/2010 9:13 PM, Marcos wrote: > >> I have a store, so I want to maximize the profit. I have all the > >> suppliers with diferent prices, some providers can send products to a > >> client an others not, this has a plus price. Some providers has a > >> discount over the tansport if a quantity is reached. > > >> Sometimes its better to me receive the order and resend to the client > >> if I have a transport discount. > > >> Not all the suppliers has all products for a order. > > >> So I want to create a function which I can pass the data, and > >> generates all the possibilities so I can find the maximum profit. > > >> Have I to use heuristics? Do you know some examples?. > > > You would not use a heuristic to generate all possibilities. You might > > use one to *avoid* doing that, and still get a good, not necessarily > > optimal, answer. > > True. Basically there are two ways of approaching the problem. One is to > try all the combinations, which will guarantee that you'll find the best > solution, but if there are an enormous number of combinations then that > could take a very long time. The other way is to use a heuristic to get > an reasonable solution in an reasonable time. It's a trade-off. > > > Wikipedia has an general entry on 'heuristic'. > > Algorithm books often specifically discuss heuristic algorithms. Do you know it there is some way to generate all the scenario possibilities?. So I Can put an array an data an generate all. I have the lack of repeated elements that I cant solve. Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: Heuristic
What do you suggest Terry? I think this problem its similar to traveler salesman. Some people solve it with Heuristic and some with Linear programming or Stochastic programming. On 25 jun, 03:28, Terry Reedy wrote: > On 6/24/2010 9:13 PM, Marcos wrote: > > > I have a store, so I want to maximize the profit. I have all the > > suppliers with diferent prices, some providers can send products to a > > client an others not, this has a plus price. Some providers has a > > discount over the tansport if a quantity is reached. > > > Sometimes its better to me receive the order and resend to the client > > if I have a transport discount. > > > Not all the suppliers has all products for a order. > > > So I want to create a function which I can pass the data, and > > generates all the possibilities so I can find the maximum profit. > > > Have I to use heuristics? Do you know some examples?. > > You would not use a heuristic to generate all possibilities. You might > use one to *avoid* doing that, and still get a good, not necessarily > optimal, answer. Wikipedia has an general entry on 'heuristic'. > Algorithm books often specifically discuss heuristic algorithms. > > -- > Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: debian python2.4
On Fri, Dec 03, 2004 at 08:57:46PM +0530, km wrote: > > Hi all, > is there a debian binary of python2.4 ? well, 2.4 cvs is in since a couple of weeks in unstable. release will come up soon I guess. -- (Not so) Random fortune: sos fanático del fútbol? añorás el prode? http://ega.zidev.com/ -- http://mail.python.org/mailman/listinfo/python-list
[[email protected]: modifying locals]
I repost this here, as somebody in python-help told it was probably out of their league. - Forwarded message from Marcos Dione - Date: Sun, 11 Oct 2015 15:30:05 +0200 From: Marcos Dione Subject: modifying locals To: [email protected] Message-ID: <[email protected]> Content-Type: text/plain; charset=us-ascii First of all, I'm not subscribed to the list, so please CC me. I'm writing a language[1] heavily based on Python. The way I'm implementing it is by having my own parser (based/stealed from pypy), obtaining an AST, modifying it and running the result with exec(). One of its particular features is that I intend to support semi-transparent remote execution using paramiko. For that, there is a construction that looks like this: [local code 1] with remote(...): [remote code] [local code 2] The idea is that [remote code] is effectively executed in a remote machine specified in remote()'s parameters. This is implemented[3] by remote() being a context manager. Part of the transparency is that globals and locals at the remote() point will be accesible to the remote code (so far only variables), and that modifications to locals will be visible back to [local code 2]. So far I managed to do the first part, but not the second. I managed to transfer the locals back from the remote. My problem is modifying the locals in remote()'s __exit__() method. As I'm running the code using exec(), it's not just a matter of modifying locals()[2]. What I tried was the following: * When I execute the code, I pass an empty dictionary to hold the locals. I save a reference to it, and I try to use it to modify the locals [local code 2] sees, but as locals are related to stack frame, it doesn't work. * This leads to the second attempt: use the stack to modify remote()'s caller's locals by[4]: inception_locals= sys._getframe().f_back.f_locals inception_locals.update (locals) Somehow this doesn't work either, and it's driving me crazy. I run out of ideas, so maybe you could throw some? Thanks in advance. -- [1] if you're interested, check https://github.com/StyXman/ayrton/ [2] isn't it? Just by writing that I got my doubts... [3] https://github.com/StyXman/ayrton/blob/develop/ayrton/functions.py#L112 [4] https://github.com/StyXman/ayrton/blob/develop/ayrton/functions.py#L254 -- (Not so) Random fortune: They called me mad, and I called them mad, and damn them, they outvoted me. -- Nathaniel Lee - End forwarded message - -- (Not so) Random fortune: Cruickshank's Law of Committees: If a committee is allowed to discuss a bad idea long enough, it will inevitably decide to implement the idea simply because so much work has already been done on it. -- https://mail.python.org/mailman/listinfo/python-list
Show Image with WebKit
Hello to everybody, I'm from Argentina.
My problem is the next. I receive in the request one image that i've
upload, and I want to show that again in another page, how can i do that? I
try this:
from WebKit.Page import Page
class gestorControl(Page):
def actions(self):
return Page.actions(self) + ["upload"]
def writeContent(self):
tweb = self.transaction()
file = self.request().field('archivo')
data = file.value
res = self.response()
wr = self.write
res.setHeader("Content-type", "image/jpeg")
wr(data)
res.flush()
I know that the header is setted right (I checked that with a firefox
plugin, I don't remember now its name), but the "data" that I'm sending is
showed as chars, not as an image.
My problem persists if I change Page for HTTPContent.
What I'm doing wrong? Is what I want possible?
Thanks, and sorry for my english :)
Marcos Alcazar
--
http://mail.python.org/mailman/listinfo/python-list
Re: Show Image with WebKit
Gabriel, I already tried In ower list in Argentina... I reveived a lot of answers of people who wants to help me, but we can't find the solution to my problem, because of that I'm asking here. Anyway, thank's to try to aproach me to ower community in PyAr. Ah, the thread is "Guardar y recuperar imagen desde Postgres". That was my first problem, and then it become to the one that I'm trying to solve now. Thanks a lot, Marcos Alcazar -- http://mail.python.org/mailman/listinfo/python-list
PyInt_FromLong gives segfault on small numbers (<257)
Hi, I'm trying to call python (python 2.6) functions from C++ using MS VC+ + 6.0 and any calls to PyInt_FromLong with numbers below 257 give me exceptions, no problems with bigger numbers PyObject *pValue; pValue = PyInt_FromLong(1L); (or pValue = PyInt_FromLong(1); Any ideas of what can I be doing wrong? Thanks, Marcos. -- http://mail.python.org/mailman/listinfo/python-list
Can anyone send me the last version of psycopg2 by email?
I have a minor trouble here with my Internet connection. Can anyone send me the last version of psycopg2 to this email? Thanks a lot for the help. Regards and best wishes -- Marcos Luis Ortíz Valmaseda Sr. Software Engineer (UCI) http://marcosluis2186.posterous.com http://www.linkedin.com/in/marcosluis2186 Twitter: @marcosluis2186 Fin a la injusticia, LIBERTAD AHORA A NUESTROS CINCO COMPATRIOTAS QUE SE ENCUENTRAN INJUSTAMENTE EN PRISIONES DE LOS EEUU! http://www.antiterroristas.cu http://justiciaparaloscinco.wordpress.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Newbie question about text encoding
On Wed, Feb 25, 2015 at 9:19 AM, Steven D'Aprano < [email protected]> wrote: > Laura Creighton wrote: > > > Dave Angel > > are you another Native English speaker living in a world where ASCII > > is enough? > > ASCII was never enough. Not even for Americans, who couldn't write things > like "I bought a comic book for 10¢ yesterday", let alone interesting > things from maths and science. > > ASCII was a necessity back then because RAM and storage are too small. > I missed the whole 7-bit ASCII period, my first computer (Mac 128K) already > had an extended character set beyond ASCII. But even that never covered the > I miss the days when I was coding with my XT computer (640kb RAM) too. Things were so simple back then. > full range of characters I wanted to write, and then there was the horrible > mess that you got whenever you copied text files from a Mac to a DOS or > Windows PC or visa versa. Yes, even in 1984 we were transferring files and > running into encoding issues. > > > > -- > Steven > > -- > https://mail.python.org/mailman/listinfo/python-list > -- Marcos | I love PHP, Linux, and Java <http://javadevnotes.com/java-integer-to-string-examples> -- https://mail.python.org/mailman/listinfo/python-list
Re: Are threads bad? - was: Future of Pypy?
On Wed, Feb 25, 2015 at 1:46 PM, Marko Rauhamaa wrote: > Marcos Almeida Azevedo : > > > Synchronized methods in Java really makes programming life simpler. > > But I think it is standard practice to avoid this if there is a > > lighter alternative as synchronized methods are slow. Worse case I > > used double checked locking. > > I have yet to see code whose performance suffers from too much locking. > However, I have seen plenty of code that suffers from anomalies caused > by incorrect locking. > > > Of course code with locking is slower than one without. The locking mechanism itself is the overhead. So use it only when necessary. There is a reason why double checked locking was invented by clever programmers. > Marko > -- > https://mail.python.org/mailman/listinfo/python-list > -- Marcos | I love PHP, Linux, and Java <http://javadevnotes.com/java-integer-to-string-examples> -- https://mail.python.org/mailman/listinfo/python-list
Re: tarfile vs zipfile
On Tue, Mar 3, 2015 at 6:39 PM, Gregory Ewing wrote: > Mark Lawrence wrote: > >> On 03/03/2015 02:29, Terry Reedy wrote: >> >> Plus tartfiles come from unix world, whereas zip was used instead in >>> Windows world. >>> >>> Is the tart bit the thing that you can eat, a loose woman or something >> else, such as a typo? :) >> > > Tartfiles would be what you get from xxx sites, no? > tartfiles is the compressed air that comes from behind and flies around unwanted by the public > > -- > Greg > -- > https://mail.python.org/mailman/listinfo/python-list > -- Marcos | I love PHP, Linux, and Java <http://javadevnotes.com/java-long-to-string-examples> -- https://mail.python.org/mailman/listinfo/python-list
Re: HELP!! How to ask a girl out with a simple witty Python code??
Why does it need to be nerdy. Why not just buy her a good book on Python? On Thu, Mar 5, 2015 at 9:34 AM, Xrrific wrote: > Guys, please Help!!! > > I am trying to impress a girl who is learning python and want ask her out > at the same time. > > Could you please come up with something witty incorporating a simple > python line like If...then... but..etc. > > You will make me a very happy man!!! > > Thank you very much!!! > -- > https://mail.python.org/mailman/listinfo/python-list > -- Marcos | I love PHP, Linux, and Java <http://javadevnotes.com/java-long-to-string-examples> -- https://mail.python.org/mailman/listinfo/python-list
