Copy-on-write when forking a python process
Hi all, Long time reader, first time poster. I am wondering if anything can be done about the COW (copy-on-write) problem when forking a python process. I have found several discussions of this problem, but I have seen no proposed solutions or workarounds. My understanding of the problem is that an object's reference count is stored in the "ob_refcnt" field of the PyObject structure itself. When a process forks, its memory is initially not copied. However, if any references to an object are made or destroyed in the child process, the page in which the objects "ob_refcnt" field is located in will be copied. My first thought was the obvious one: make the ob_refcnt field a pointer into an array of all object refcounts stored elsewhere. However, I do not think that there would be a way of doing this without adding a lot of complexity. So my current thinking is that it should be possible to disable refcounting for an object. This could be done by adding a field to PyObject named "ob_optout". If ob_optout is true then py_INCREF and py_DECREF will have no effect on the object: from refcount import optin, optout class Foo: pass mylist = [Foo() for _ in range(10)] optout(mylist) # Sets ob_optout to true for element in mylist: optout(element) # Sets ob_optout to true Fork_and_block_while_doing_stuff(mylist) optin(mylist) # Sets ob_optout to false for element in mylist: optin(element) # Sets ob_optout to false Has anyone else looked into the COW problem? Are there workarounds and/or other plans to fix it? Does the solution I am proposing sound reasonable, or does it seem like overkill? Does anyone foresee any problems with it? Thanks, --jac -- http://mail.python.org/mailman/listinfo/python-list
Re: Retrieving Python Keywords
Actually this is all it takes:
import keywords
print keywords.kwlist
--jac
On Sat, Apr 9, 2011 at 8:57 PM, Chris Angelico wrote:
> On Sun, Apr 10, 2011 at 11:28 AM, candide wrote:
>> Python is very good at introspection, so I was wondering if Python (2.7)
>> provides any feature to retrieve the list of its keywords (and, as, assert,
>> break, ...).
>
> I don't know about any other way, but here's a really REALLY stupid
> method. For every possible alphabetic string, attempt to eval() it; if
> you get NameError or no error at all, then it's not a keyword.
> SyntaxError means it's a keyword.
>
eval("foo")
>
> Traceback (most recent call last):
> File "", line 1, in
> eval("foo")
> File "", line 1, in
> NameError: name 'foo' is not defined
eval("lambda")
>
> Traceback (most recent call last):
> File "", line 1, in
> eval("lambda")
> File "", line 1
> lambda
> ^
> SyntaxError: unexpected EOF while parsing
eval("eval")
>
>
> Yes, it's stupid. But I'm feeling rather mischievous today. :)
>
> Chris Angelico
> --
> http://mail.python.org/mailman/listinfo/python-list
>
--
http://mail.python.org/mailman/listinfo/python-list
Re: Pythonic infinite for loop?
If I understand your question correctly, what you want is probably something like: i = 0 lst=[] while True: try: lst.append(parse_kwdlist(dct["Keyword%d"%i])) i += 1 except KeyError: break --jac On Thu, Apr 14, 2011 at 9:10 PM, Chris Angelico wrote: > Apologies for interrupting the vital off-topic discussion, but I have > a real Python question to ask. > > I'm doing something that needs to scan a dictionary for elements that > have a particular beginning and a numeric tail, and turn them into a > single list with some processing. I have a function parse_kwdlist() > which takes a string (the dictionary's value) and returns the content > I want out of it, so I'm wondering what the most efficient and > Pythonic way to do this is. > > My first draft looks something like this. The input dictionary is > called dct, the output list is lst. > > lst=[] > for i in xrange(1,1000): # arbitrary top, don't like this > try: > lst.append(parse_kwdlist(dct["Keyword%d"%i])) > except KeyError: > break > > I'm wondering two things. One, is there a way to make an xrange object > and leave the top off? (Sounds like I'm risking the numbers > evaporating or something.) And two, can the entire thing be turned > into a list comprehension or something? Generally any construct with a > for loop that appends to a list is begging to become a list comp, but > I can't see how to do that when the input comes from a dictionary. > > In the words of Adam Savage: "Am I about to feel really, really stupid?" > > Thanks in advance for help... even if it is just "hey you idiot, you > forgot about X"! > > Chris Angelico > -- > http://mail.python.org/mailman/listinfo/python-list > -- http://mail.python.org/mailman/listinfo/python-list
Re: De-tupleizing a list
itertools can help you do this too:
import itertools
tl = [('0A',), ('1B',), ('2C',), ('3D',)]
itertools.chain.from_iterable(tl)
list(itertools.chain.from_iterable(tl))
['0A', '1B', '2C', '3D']
Checkout http://docs.python.org/library/itertools.html#itertools.chain
for more info.
On Mon, Apr 25, 2011 at 11:08 PM, Paul Rubin wrote:
> Gnarlodious writes:
>> I have an SQLite query that returns a list of tuples:
>> [('0A',), ('1B',), ('2C',), ('3D',),...
>> What is the most Pythonic way to loop through the list returning a
>> list like this?:
>> ['0A', '1B', '2C', '3D',...
>
> Try:
>
> tlist = [('0A',), ('1B',), ('2C',), ('3D',)]
> alist = [x for (x,) in tlist]
> --
> http://mail.python.org/mailman/listinfo/python-list
>
--
http://mail.python.org/mailman/listinfo/python-list
