[Python-Dev] PyCFunction_* Missing
Hi, Just noticed the new Python 2.6.2 docs now dont have any reference to * PyCFunction_New * PyCFunction_NewEx * PyCFunction_Check * PyCFunction_Call Ofcourse these are still in the source code but Im wondering if this is intentional that these functions should be for internal use only? -- - Campbell ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] C/Python API Index removed?
This page used to give an index of the C/Python API functions too http://docs.python.org/genindex-all.html But a week or so ago I noticed all these functions are now missing (I remember they existed in 2.6.1 docs) Was this intentional? Quite a while ago, ~2.5 the C/API docs had their own index which personally I prefer. http://docs.python.org/c-api/index.html This page is called an index but Im looking for a page like http://docs.python.org/genindex-all.html which includes all C/API function names. Is this the right place to mail such problems? Thanks -- - Campbell ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Warnings when no file exists.
Hi, there has been a problem in blender3d for 6~ years or so thats eluded me, I decided to look into today. - Whenever the a script raises a warnings python prints out binary garbage in the console. Some users complain when they run python games in blender they get beeps coming from the PC speaker. It turns out that _warning.c's setup_context() is taking the first value of argv (line 534 in 2.6.2), which in our case is the blender binary. then some part of the binary is printed to the console. Apart from the beeps and not being helpful this also can mess up the console's state - a like "cat /dev/random" might. But the real problem is that warnings expect a file to exist, in blender we have our own internal text's that dont have a corresponding file on disk, so setting __file__ in the global dict will just point to a location that doesn't exist. It surprises me that warnings do this since exceptions work as expected, printing useful stack traces from our built in texts. Incase this helps, the scripts are converted into a buffer and run like this... text->compiled = Py_CompileString( buf, text->id.name+2, Py_file_input ); PyEval_EvalCode( text->compiled, globaldict, globaldict ); Does anyone know of a workaround for this? Im sure there are other cases where you may want to run compiled code that isnt related to a file. -- - Campbell ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Suggested PySys_SetArgv use with a (char **) argv ?
In Python3 PySys_SetArgv takes (wchar_t **) for the argv, I looked into converting a (char **) into a (wchar_t **) and while its possible its lengthy enough not to be trivial, see python.c:18 - char2wcharm(), its 102 lines with ifdef's and goto's, not including the loop lower down that loops over the argv. Looking further python has a very similar function for ./PC/bdist_wininst/install.c, for do_run_installscript - widen_string(). Since many C applications take argv as a (char **) it seems reasonable for python to keep a function for the C api that accepts a (char **) argument for argv. otherwise each C applications that embeds python will need to write their own conversion function. also noticed Demo/embed/demo.c is out of date, giving (char **) to PySys_SetArgv(). -- - Campbell ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Suggested PySys_SetArgv use with a (char **) argv ?
The reason I was a big vague is that I'm not really bothered HOW its done, I'd just like there to be some way to use PySys_SetArgv with a (char **) Definitely not suggesting that PySys_SetArgv be reverted to the way it worked in py2.x, if there was 2 versions of PySys_SetArgv that would be acceptable to me. PySys_SetArgvASCII() maybe? Alternately a function to convert the char **argv to wchar_t **argv could be ok too. PyOS_char2wchar would be useful though it still means you need to manually copy the argv, convert and free it. On Tue, Jul 7, 2009 at 3:18 PM, Lisandro Dalcin wrote: > Sorry... > > -- Forwarded message -- > From: Lisandro Dalcin > Date: Tue, Jul 7, 2009 at 7:16 PM > Subject: Re: [Python-Dev] Suggested PySys_SetArgv use with a (char **) argv ? > To: "Martin v. Löwis" > > > On Tue, Jul 7, 2009 at 6:46 PM, "Martin v. Löwis" wrote: >>> Since many C applications take argv as a (char **) it seems reasonable >>> for python to keep a function for the C api that accepts a (char **) >>> argument for argv. >> > > I've run in the same issue, and basically had to copy&paste from > Python sources... > >> >> I'm not quite sure what you are suggesting: either that there is a >> function in the C API that accepts a (char**), or that PySys_SetArgv >> be that function. >> >> The latter alternative is out of question, as it breaks the Windows >> port of Python - there is simply no way of representing argv with >> char** in a meaningful way. It also breaks backwards compatibility. >> > > Indeed... > >> As for creating another function: please provide a patch to >> bugs.python.org. I believe this patch is *very* difficult to >> implement, unless the function can accept some severe limitations. >> > > What about the Python lib exposing a public PyOS_char2wchar() ? > > No idea how to write a Windows implementation, though.. > > > -- > Lisandro Dalcín > --- > Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC) > Instituto de Desarrollo Tecnológico para la Industria Química (INTEC) > Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET) > PTLC - Güemes 3450, (3000) Santa Fe, Argentina > Tel/Fax: +54-(0)342-451.1594 > > > > -- > Lisandro Dalcín > --- > Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC) > Instituto de Desarrollo Tecnológico para la Industria Química (INTEC) > Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET) > PTLC - Güemes 3450, (3000) Santa Fe, Argentina > Tel/Fax: +54-(0)342-451.1594 > -- - Campbell ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Py_TPFLAGS_HEAPTYPE too overloaded
On Sun, Jul 26, 2009 at 11:01 AM, Joshua Haberman wrote:
> I'm writing a C Python extension that needs to generate PyTypeObjects
> dynamically. Unfortunately, the Py_TPFLAGS_HEAPTYPE flag is overloaded
> in a way that makes it difficult to achieve this goal.
>
> The documentation for Pt_TPFLAGS_HEAPTYPE says:
>
> Py_TPFLAGS_HEAPTYPE
>
> This bit is set when the type object itself is allocated
> on the heap. In this case, the ob_type field of its
> instances is considered a reference to the type, and the
> type object is INCREF’ed when a new instance is created,
> and DECREF’ed when an instance is destroyed (this does not
> apply to instances of subtypes; only the type referenced
> by the instance’s ob_type gets INCREF’ed or DECREF’ed).
>
> This sounds like exactly what I want. I want my type object INCREF'd
> and DECREF'd by its instances so it doesn't leak or get deleted
> prematurely. If this were all that Py_TPFLAGS_HEAPTYPE did, it would
> work great for me.
>
> Unfortunately, Py_TPFLAGS_HEAPTYPE is also overloaded to mean
> "user-defined type" (as opposed to a built-in type). It controls
> numerous subtle behaviors such as:
>
> - whether the type's name is module.type or just type.
> - whether you're allowed to set __name__, __module__, or __bases__ on the
> type.
> - whether you're allowed to set __class__ on instances of this type.
> - whether the module name comes from the type name or the __module__
> attribute.
> - whether it will use type->tp_doc as the docstring
> - whether its repr() calls it a "class" or a "type".
> - whether you can set attributes of the type.
> - whether someone is attempting the Carlo Verre hack.
>
> So I'm stuck with an unenviable choice. I think the lesser of two evils
> is to *not* specify Py_TPFLAGS_HEAPTYPE, because the worst that will
> happen is that my types will leak. This is not as bad as having someone
> set __class__ on one of my instances, or set attributes on my type, etc.
>
> Ideally the interpreter would have a separate flag like
> Py_TPFLAGS_BUILTIN that would trigger all of the above behaviors, but
> still make it possible to have dynamically generated built-in types get
> garbage collected appropriately.
>
> At the very least, the documentation I cited above should make it clear
> that Py_TPFLAGS_HEAPTYPE controls more than just whether the type gets
> INCREF'd and DECREF'd. Based on the list of behaviors I discovered
> above, it is almost certainly not correct for a C exension type to be
> declared with Py_TPFLAGS_HEAPTYPE.
>
> Josh
Hi Joshua, recently I also needed to dynamically make subtypes from C,
I tried 2 ways of doing this, one is to do the C equivalent of calling
type("name",(bases,...), dict) and the other is to malloc()
PyTypeObject's, fill in the slots and run PyType_Ready on them to
initialize them.
It seems the first is the expected way to make your own types so I
assume thats what your doing?, Just wondering because if you do it the
second way I think youll have more control and the types will be more
limited (like internal types).
I'm not expert enough in this area to know if malloc'ing PyTypeObject
and initializing has some other problems.
- Campbell
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Py_CmpToRich removed, alternatives for comparison?
For blender we have a number of types defined in the C/API like meshes, lamps, metaballs, nurbs etc that dont make sense with some of richcmp's operations. A problem I have is that in python 3.1 the Py_CmpToRich function is removed. Should we copy Py_CmpToRich into our source tree? Otherwise we have fairly long cmp functions that have Py_CmpToRich inline. For C extension writers what is the suggested method for comparing types where Py_LT, Py_GT Py_GE etc are not useful? Since people sometimes ask why use py3.1, we're doing a rewrite that wont be ready for quite some time. -- - Campbell ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Best practice for new namespace (from C/API)
Hi, I'm writing because I'm working on a project where the user can
run scripts that don't reference a file but use internal application
text data.
Otherwise we are not doing anything tricky, only that the scripts
should each run independently (no cruft left from the previous scripts
namespace, sharing sys.modules etc is fine).
Something which is unclear to me even after looking over pythonrun.c
is the correct way to setup a namespace.
For years we have been doing this and it seemed to work fine...
PyObject *d = PyDict_New(); // new namespace
PyDict_SetItemString(d, "__builtins__", PyEval_GetBuiltins());
PyDict_SetItemString(d, "__file__", PyString_FromString(filename));
// fake, avoids sys.argv[0] being used for warnings.
/* --- snip ---*/
PyEval_EvalCode(compiled_text, d, d);
Recently a developer reported a bug where pickle wasn't working, it
turns out that in a few places python expects the __main__ modules
namespace to match that if the running script: _pickle.c's
save_global() in this case
eg:
>>> spam = 10
>>> print(__import__("__main__").__dict__["spam"])
... 10
Once I found this was the problem it was simple to use __main__'s
namespace however there are still things that are not clear about
exactly how this should be done.
Simplified code...
PyObject *item, *dict= PyModule_GetDict(PyImport_AddModule("__main__"));
PyDict_Clear(dict);
PyDict_SetItemString(dict, "__builtins__", PyImport_AddModule("builtins"));
item = PyUnicode_FromString( "__main__" );
PyDict_SetItemString( dict, "__name__", item );
Py_DECREF(item);
PyDict_SetItemString(d, "__file__", PyString_FromString(filename));
// fake, avoids sys.argv[0] being used for warnings.
/* --- snip ---*/
PyEval_EvalCode(compiled_text, dict, dict);
Still this leaves me with the following questions...
- Whats the best way to manage the namespace for running multiple
scripts one after another? clear and initialize __main__'s dict each
time?, keep a copy and restore it after each run?
- should the original __main__ namespace be restored after running a script?
- pickle expects: __import__("__main__").__dict__ == ***the current
namespace***, is this apart of the python spec or just something
specific to pickle?
- PyDict_SetItemString(d, "__builtins__", PyEval_GetBuiltins()) acts
differently to PyDict_SetItemString(dict, "__builtins__",
PyImport_AddModule("builtins")), using the PyEval_GetBuiltins() dict
adds every member of __builtins__ when running:
print(__import__("__main__").__dict__.keys()), rather then just
showing __builtins__.
--
- Campbell
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Calling Methods from Pythons C API with Keywords
Hey Guys, My first post on this list so I hope this is the right place to post and relevant. Im rewriting parts of the Blender3D python API that has got a bit old and needs an update. Im making a PyList subtype with the C/Python API, this involves intercepting calls to standard list methods to make sure Blenders array data is in Sync with the list's data. Iv got it working for tp_as_sequence, tp_as_mapping, iter and dealloc etc but methods are a problem. I want to add my own call's before and after PyLists standard functions but have a proplem with functons that use keywords and have no API equivalent. For example, I cant use the API's PyList_Sort because that dosnt support keywords like... ls.sort(key=lambda a: a.foo)) And the Problem with PyObject_CallMethod is that it dosnt accept keywords. PyObject_CallMethod((PyObject *)mylist, "sort", "O", args); Looking at abstract.c, PyObject_CallMethod uses call_function_tail, which calls "PyObject_Call(callable, args, NULL);" - so Its not currently possible with PyObject_CallMethod. But I cant find any way to do this in a few lines. I could use PyEval_CallObjectWithKeywords but that would mean Id need to get the method from the list manually which Ill look into, but unless Im missing something here, it seems PyObject_CallMethodWithKeywords would be a nice addition to the Python API that cant be done in a straight forward way at the moment. - Thanks ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Calling Methods from Pythons C API with Keywords
Hrvoje Nikšić wrote:
> On Wed, 2007-06-20 at 00:21 +1000, Campbell Barton wrote:
>> I want to add my own call's before and after PyLists standard functions
>> but have a proplem with functons that use keywords and have no API
>> equivalent.
>> For example, I cant use the API's PyList_Sort because that dosnt support
>> keywords like...
>>
>> ls.sort(key=lambda a: a.foo))
>>
>> And the Problem with PyObject_CallMethod is that it dosnt accept keywords.
>
> Note that you can always simply call PyObject_Call on the bound method
> object retrieved using PyObject_GetAttrString. The hardest part is
> usually constructing the keywords dictionary, a job best left to
> Py_BuildValue and friends. When I need that kind of thing in more than
> one place, I end up with a utility function like this one:
>
> /* Equivalent to PyObject_CallMethod but accepts keyword args. The
>format... arguments should produce a dictionary that will be passed
>as keyword arguments to obj.method.
>
>Usage example:
> PyObject *res = call_method(lst, "sort", "{s:O}", "key", keyfun));
> */
>
> PyObject *
> call_method(PyObject *obj, const char *methname, char *format, ...)
> {
> va_list va;
> PyObject *meth = NULL, *args = NULL, *kwds = NULL, *ret = NULL;
>
> args = PyTuple_New(0);
> if (!args)
> goto out;
> meth = PyObject_GetAttrString(obj, methname);
> if (!meth)
> goto out;
>
> va_start(va, format);
> kwds = Py_VaBuildValue(format, va);
> va_end(va);
> if (!kwds)
> goto out;
>
> ret = PyObject_Call(meth, args, kwds);
> out:
> Py_XDECREF(meth);
> Py_XDECREF(args);
> Py_XDECREF(kwds);
> return ret;
> }
>
> It would be nice for the Python C API to support a more convenient way
> of calling objects and methods with keyword arguments.
Thanks for the hint, I ended up using PyObject_Call.
This seems to work, EXPP_PyTuple_New_Prepend - is a utility function
that returns a new tuple with self at the start (needed so args starts
with self)
I dont think I can use PyObject_GetAttrString because the subtype would
return a reference to this function - rather then the lists original
function, Id need an instance of a list and dont have one at that point.
__
static PyObject * MaterialList_sort(BPy_MaterialList *self, PyObject
*args, PyObject *keywds )
{
PyObject *ret;
PyObject *newargs = EXPP_PyTuple_New_Prepend(args, (PyObject *)self);
sync_list_from_materials__internal(self); # makes sure the list matches
blenders materials
ret = PyObject_Call(PyDict_GetItemString(PyList_Type.tp_dict, "sort"),
newargs, keywds);
Py_DECREF(newargs);
if (ret)
sync_materials_from_list__internal(self); # makes blenders
materials
match the lists
return ret;
}
_
Later on Ill probably avoid using PyDict_GetItemString on
PyList_Type.tp_dict all the time since the methods for lists does not
change during python running. - Can probably be assigned to a constant.
--
Campbell J Barton (ideasman42)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Calling Methods from Pythons C API with Keywords
Hrvoje NikÅ¡iÄ wrote: > [ Note that this discussion, except maybe for the suggestion to add a > simpler way to call a method with keyword args, is off-topic to > python-dev. ] Is there a list for this kind of discussion? Iv tried asking questions on the freenode python chat room but almost very few people there do C/Python api development. -- Campbell J Barton (ideasman42) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Calling Methods from Pythons C API with Keywords
The reason I asked on this in the first place is I had looked through the python source to make sure PyObject_Call had no equivalent that supported keywords, and since I needed to do this I figured it might be worth considering for Pythons API. Im sure everyone could write their own PyObject_Call, if they had to but thats what the API's for. Hrvoje Nikšić wrote: > On Wed, 2007-06-20 at 19:26 +0200, "Martin v. Löwis" wrote: >> As Hrvoje says: try python-list (aka comp.lang.python). If you don't >> get an answer, you didn't phrase your question interestingly enough, >> or nobody knows the answer, or nobody has the time to tell you. > > The thing with comp.lang.python is that it is followed by a large number > of Python users, but a much smaller number of the C API users -- which > is only natural, since the group is about Python, not about C. For most > users the Python/C API is an implementation detail which they never have > to worry about. > > Futrhermore, questions about the C API often concern CPython > implementation details and so they don't feel like they would belong in > comp.lang.python. As an experiment, it might make sense to open a > mailing list dedicated to the Python C API. It could become a useful > support forum for extension writers (a group very useful to Python) and > maybe even a melting pot for new ideas regarding CPython, much like > comp.lang.python historically provided ideas for Python the language. Agree a Python/C API List would be great, in fact I cant see any reasons not to have it- likely the pure python users dont want to know about refcounting problems.. etc anyway. http://mail.python.org/mailman/listinfo http://www.python.org/community/sigs/ There are lists/newsgroup for py2exe and pyrex, Python-ObjectiveC etc, Python/C API seems much more generic, and its also fairly tricky to use at times - when doing more advanced stuff (subtyping has been tricky for me anyway). I expect the dev's of pyrex, pygame etc might also need to discuss C API spesific issues as well. Iv had roughly this conversation in IRC... Q. Hi, Id like to know how wrap python subtype methods in the C API A. C dosnt have classes, use C++ Q. no I want to use pythons C API, A. Subtypes are easy to do in python.. .. you get the idea... Quite a few "python only" users dont understand where the Python/C API fits in and its annoying to have to explain the question each time (yes, Iv had these conversations more then once) - Cam ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Py/C API sig is here! --- (Was "Calling Methods from Pythons C API with Keywords")
Hrvoje NikÅ¡iÄ wrote: > On Thu, 2007-06-21 at 19:25 +0200, "Martin v. Löwis" wrote: >> In the past, we created special-interest groups for such discussion. >> Would you like to coordinate a C sig? See >> >> http://www.python.org/community/sigs/ > > A SIG sounds like an excellent idea. If created, a newcomer with a C > API question could then be redirected to the SIG's mailing list, where > (hopefully, in time) there would be enough knowledgable people to answer > his question. > > As for me coordinating the SIG, I'm not sure if that would be a good > idea. For one, I don't know what a coordinator really does and how much > time the job takes from one's daily activities. But more importantly, > my interest in Python's C API is related to my current needs at work. > If the situation at work changes, I will probably have much less time > (if any) to devote to the C API discussions. This mailing list is now running, if your interested in asking/answering questions about the Py/C api sign up here. http://mail.python.org/mailman/listinfo/capi-sig -- Campbell J Barton (ideasman42) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] python 3.0, tp_compare not used for == test?
Hi, I have been writing a new C api that can build with both python 2.6 and 3.0
I found that when building with python 2.6, doing "a==b" between 2
different pyrna_struct_Type's would run tp_compare
But with python 3.0, "a==b" will always be false and tp_compare
function would not even run.
The only way to get it to run the tp_compare function was to do
"cmp(a, b)" however Id like to be able to use == still.
The only way I could get "a==b" to work in python 3.0 was to add a
tp_richcompare function which runs tp_compare with Py_CmpToRich()
anyway
Is this the intended behavior? - if so its important for external C
API's to be updated to support this.
Included the PyType below with some relevant details. I looked at
other Python 3.0 types but could not see why this would happen.
// snip,
// initialized with PyType_Ready
if( PyType_Ready( &pyrna_struct_Type ) < 0 )
return NULL;
// header definition for BPy_StructRNA
typedef struct {
PyObject_HEAD /* required python macro */
PointerRNA ptr;
int freeptr; /* needed in some cases if ptr.data is created on the
fly, free when deallocing */
} BPy_StructRNA;
// PyType
/*---BPy_StructRNA method
def--*/
PyTypeObject pyrna_struct_Type = {
PyVarObject_HEAD_INIT(NULL, 0)
"StructRNA",/* tp_name */
sizeof( BPy_StructRNA ),/* tp_basicsize */
0, /* tp_itemsize */
/* methods */
( destructor ) pyrna_struct_dealloc,/* tp_dealloc */
NULL, /* printfunc tp_print; */
NULL, /* getattrfunc
tp_getattr; */
NULL, /* setattrfunc
tp_setattr; */
( cmpfunc ) pyrna_struct_compare, /* tp_compare */
( reprfunc ) pyrna_struct_repr, /* tp_repr */
/* Method suites for standard classes */
NULL, /* PyNumberMethods *tp_as_number; */
NULL, /* PySequenceMethods
*tp_as_sequence; */
NULL, /* PyMappingMethods
*tp_as_mapping; */
/* More standard operations (here for binary compatibility) */
( hashfunc )pyrna_struct_hash, /* hashfunc tp_hash; */
NULL, /* ternaryfunc tp_call;
*/
NULL, /* reprfunc tp_str; */
( getattrofunc ) pyrna_struct_getattro, /* getattrofunc tp_getattro; */
( setattrofunc ) pyrna_struct_setattro, /* setattrofunc tp_setattro; */
/* Functions to access object as input/output buffer */
NULL, /* PyBufferProcs *tp_as_buffer; */
/*** Flags to define presence of optional/expanded features ***/
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, /* long tp_flags; */
NULL, /* char *tp_doc;
Documentation string */
/*** Assigned meaning in release 2.0 ***/
/* call function for all accessible objects */
NULL, /* traverseproc tp_traverse; */
/* delete references to contained objects */
NULL, /* inquiry tp_clear; */
/*** Assigned meaning in release 2.1 ***/
/*** rich comparisons ***/
(richcmpfunc)pyrna_struct_richcmp, /* richcmpfunc tp_richcompare;
*/
/*** weak reference enabler ***/
0, /* long tp_weaklistoffset; */
/*** Added in release 2.2 ***/
/* Iterators */
NULL, /* getiterfunc tp_iter; */
NULL, /* iternextfunc tp_iternext; */
/*** Attribute descriptor and subclassing stuff ***/
pyrna_struct_methods, /* struct PyMethodDef *tp_methods; */
NULL, /* struct PyMemberDef *tp_members; */
NULL, /* struct PyGetSetDef
*tp_getset; */
NULL, /* struct _typeobject *tp_base; */
NULL, /* PyObject *tp_dict; */
NULL, /* descrgetfunc tp_descr_get; */
NULL, /* descrsetfunc tp_descr_set; */
0, /* long tp_dictoffset; */
NULL, /* initproc tp_init; */
NULL, /* allocfunc tp_alloc; */
pyrna_struct_new, /* newfunc tp_new; */
/* Low-level free-memory routine */
NULL, /* freefunc tp_free; */
/* For PyObject_IS_GC */
NULL, /* inquiry tp_is_gc; */
NULL, /* PyObject *tp_bases; */
/* method resolution order */
NULL, /* PyObject *tp
[Python-Dev] Using environment variables problematic for embedded python.
Hi,
We've run into an issue recently with blender3d on ms-windows where we
want to enforce the encoding is UTF-8 with the embedded python
interpreter.
(the encoding defaults to cp437).
I naively thought setting the environment variable before calling
Py_Initialize() would work, but the way python DLL loads, it gets its
own environment variables that cant be modified directly [1].
eg, _putenv("PYTHONIOENCODING=utf-8:surrogateescape");
We had bug reports by windows users not able to export files because
the stdout errors on printing paths with unsupported encoding. [2],[3]
---
Of course we could distribute blender with a bat file launcher that
sets env variables, or ask the user to set their env variable - but I
dont think this is really a good option.
I tried overriding the stderr & stdout, but this caused another bug in
a part of out code that catches exception messages from the stderr.
[4]
import sys, io
sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='utf-8',
errors='surrogateescape', line_buffering=True)
sys.stderr = io.TextIOWrapper(sys.stderr.buffer, encoding='utf-8',
errors='surrogateescape', line_buffering=True)
IMHO either of these solutions would be fine.
* have a PyOS_PutEnv() function, gettext has gettext_putenv() to
workaround this problem.
* manage this the same as Py_GetPythonHome(), which can be defined by
the embedding application to override the default.
Id like to know if there is some known solution to workaround this
issue, if not - would either of these would be acceptable in python
(can write up a patch if it helps)
Regards,
Campbell
---
[1]
http://stackoverflow.com/questions/5153547/environment-variables-are-different-for-dll-than-exe
[2] http://projects.blender.org/tracker/index.php?func=detail&aid=32750
[3] http://projects.blender.org/tracker/index.php?func=detail&aid=31555
[4] http://projects.blender.org/tracker/?func=detail&aid=32720
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using environment variables problematic for embedded python.
On Thu, Oct 4, 2012 at 1:35 PM, Campbell Barton wrote:
> Hi,
>
> We've run into an issue recently with blender3d on ms-windows where we
> want to enforce the encoding is UTF-8 with the embedded python
> interpreter.
> (the encoding defaults to cp437).
>
> I naively thought setting the environment variable before calling
> Py_Initialize() would work, but the way python DLL loads, it gets its
> own environment variables that cant be modified directly [1].
> eg, _putenv("PYTHONIOENCODING=utf-8:surrogateescape");
>
> We had bug reports by windows users not able to export files because
> the stdout errors on printing paths with unsupported encoding. [2],[3]
>
> ---
>
> Of course we could distribute blender with a bat file launcher that
> sets env variables, or ask the user to set their env variable - but I
> dont think this is really a good option.
>
> I tried overriding the stderr & stdout, but this caused another bug in
> a part of out code that catches exception messages from the stderr.
> [4]
> import sys, io
> sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='utf-8',
> errors='surrogateescape', line_buffering=True)
> sys.stderr = io.TextIOWrapper(sys.stderr.buffer, encoding='utf-8',
> errors='surrogateescape', line_buffering=True)
>
>
>
> IMHO either of these solutions would be fine.
>
> * have a PyOS_PutEnv() function, gettext has gettext_putenv() to
> workaround this problem.
>
> * manage this the same as Py_GetPythonHome(), which can be defined by
> the embedding application to override the default.
>
>
> Id like to know if there is some known solution to workaround this
> issue, if not - would either of these would be acceptable in python
> (can write up a patch if it helps)
>
> Regards,
> Campbell
>
> ---
>
> [1]
> http://stackoverflow.com/questions/5153547/environment-variables-are-different-for-dll-than-exe
> [2] http://projects.blender.org/tracker/index.php?func=detail&aid=32750
> [3] http://projects.blender.org/tracker/index.php?func=detail&aid=31555
> [4] http://projects.blender.org/tracker/?func=detail&aid=32720
To follow up and give a correction to overwriting sys.stdout/stderr,
The issue seemed to be that __stderr__/__stdout__ was later
overwritten, loosing the original reference to the buffer (maybe
refcount issue here?), either way it would silence the output.
import sys, io
sys.__stdout__ = sys.stdout =
io.TextIOWrapper(io.open(sys.stdout.fileno(), "wb", -1),
encoding='utf-8', errors='surrogateescape', newline="\n",
line_buffering=True)
sys.__stderr__ = sys.stderr =
io.TextIOWrapper(io.open(sys.stderr.fileno(), "wb", -1),
encoding='utf-8', errors='surrogateescape', newline="\n",
line_buffering=True)
This all works as expected without bug [4] (above), however on exit I
get an assert in MSVCR90.DLL's write.c (called from python32_d.dll):
_VALIDATE_CLEAR_OSSERR_RETURN((_osfile(fh) & FOPEN), EBADF, -1);
I'd rather not loose more time debugging why this assert happens,
IMHO this is too low-level a way to change the encoing of stdio/stderr
and error-prone too, so some way to reliably set PYTHONIOENCODING from
a program embedding python is still needed.
--
- Campbell
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using environment variables problematic for embedded python.
Checked Python 3.4 (latest hg checkout) and this area of the code
hasn't changed: pythonrun,c initstdio()
reported the issue:
http://bugs.python.org/issue16129
On Thu, Oct 4, 2012 at 2:51 PM, Guido van Rossum wrote:
> This seems more fit for the tracker; can you file there? (Then post the
> issue link here.) I do think you have a legitimate use case to set the
> default encoding to utf-8. (Though there may be a way already.) Does Python
> 3.3 have te same bug?
>
>
> On Wednesday, October 3, 2012, Campbell Barton wrote:
>>
>> On Thu, Oct 4, 2012 at 1:35 PM, Campbell Barton
>> wrote:
>> > Hi,
>> >
>> > We've run into an issue recently with blender3d on ms-windows where we
>> > want to enforce the encoding is UTF-8 with the embedded python
>> > interpreter.
>> > (the encoding defaults to cp437).
>> >
>> > I naively thought setting the environment variable before calling
>> > Py_Initialize() would work, but the way python DLL loads, it gets its
>> > own environment variables that cant be modified directly [1].
>> > eg, _putenv("PYTHONIOENCODING=utf-8:surrogateescape");
>> >
>> > We had bug reports by windows users not able to export files because
>> > the stdout errors on printing paths with unsupported encoding. [2],[3]
>> >
>> > ---
>> >
>> > Of course we could distribute blender with a bat file launcher that
>> > sets env variables, or ask the user to set their env variable - but I
>> > dont think this is really a good option.
>> >
>> > I tried overriding the stderr & stdout, but this caused another bug in
>> > a part of out code that catches exception messages from the stderr.
>> > [4]
>> > import sys, io
>> > sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='utf-8',
>> > errors='surrogateescape', line_buffering=True)
>> > sys.stderr = io.TextIOWrapper(sys.stderr.buffer, encoding='utf-8',
>> > errors='surrogateescape', line_buffering=True)
>> >
>> >
>> >
>> > IMHO either of these solutions would be fine.
>> >
>> > * have a PyOS_PutEnv() function, gettext has gettext_putenv() to
>> > workaround this problem.
>> >
>> > * manage this the same as Py_GetPythonHome(), which can be defined by
>> > the embedding application to override the default.
>> >
>> >
>> > Id like to know if there is some known solution to workaround this
>> > issue, if not - would either of these would be acceptable in python
>> > (can write up a patch if it helps)
>> >
>> > Regards,
>> > Campbell
>> >
>> > ---
>> >
>> > [1]
>> > http://stackoverflow.com/questions/5153547/environment-variables-are-different-for-dll-than-exe
>> > [2] http://projects.blender.org/tracker/index.php?func=detail&aid=32750
>> > [3] http://projects.blender.org/tracker/index.php?func=detail&aid=31555
>> > [4] http://projects.blender.org/tracker/?func=detail&aid=32720
>>
>> To follow up and give a correction to overwriting sys.stdout/stderr,
>> The issue seemed to be that __stderr__/__stdout__ was later
>> overwritten, loosing the original reference to the buffer (maybe
>> refcount issue here?), either way it would silence the output.
>>
>>
>> import sys, io
>> sys.__stdout__ = sys.stdout =
>> io.TextIOWrapper(io.open(sys.stdout.fileno(), "wb", -1),
>> encoding='utf-8', errors='surrogateescape', newline="\n",
>> line_buffering=True)
>> sys.__stderr__ = sys.stderr =
>> io.TextIOWrapper(io.open(sys.stderr.fileno(), "wb", -1),
>> encoding='utf-8', errors='surrogateescape', newline="\n",
>> line_buffering=True)
>>
>> This all works as expected without bug [4] (above), however on exit I
>> get an assert in MSVCR90.DLL's write.c (called from python32_d.dll):
>> _VALIDATE_CLEAR_OSSERR_RETURN((_osfile(fh) & FOPEN), EBADF, -1);
>>
>> I'd rather not loose more time debugging why this assert happens,
>> IMHO this is too low-level a way to change the encoing of stdio/stderr
>> and error-prone too, so some way to reliably set PYTHONIOENCODING from
>> a program embedding python is still needed.
>>
>> --
>> - Campbell
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>
>
> --
> --Guido van Rossum (python.org/~guido)
--
- Campbell
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
