Re: [Python-Dev] Dataclasses, frozen and __post_init__
On 20 February 2018 at 15:11, Steven D'Aprano wrote: > So in principle, we could have a mutable class, and a immutable one, and > when you flick the switch, the instance.__class__ changes from mutable > to frozen. > > If you don't hate this, we can think about the details needed to get > it work in practice. This doesn't technically require any additional changes, as you can already do it with __post_init__: @dataclass class MyRecord: a: int b: str c: float def __post_init__(self): # self is still mutable here self.__class__ = _LockedMyRecord @dataclass(frozen=True) class _LockedMyRecord(MyRecord): pass This is also the kind of runtime behaviour modification where hiding how it's implemented is likely to create more problems than it solves, as even though having "type(obj)" and "obj.__class__" refer to different types is a formally supported state for instances, there are also lots of APIs where it isn't well defined whether an attribute lookup will use the nominal class or the true underlying type. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dataclasses, frozen and __post_init__
On Mon, Feb 19, 2018 at 8:16 PM, Glenn Linderman wrote: > On 2/19/2018 7:02 PM, Guido van Rossum wrote: > > But how? > > On Mon, Feb 19, 2018 at 5:06 PM, Chris Barker - NOAA Federal < > chris.bar...@noaa.gov> wrote: > >> ... maybe it would be helpful to be able to >> freeze an instance after creation for multiple use-cases? >> > > And there's the crux of the issue... if the creator of Python can't figure > out how to make mutable objects immutable by fiat after creation, then it > is unlikely anyone else can! > Um, the question was meant in the Socratic way. :-) (Snipped the rest.) TBH, I don't hate Nick's solution that assigns to __class__ in __post_init__. You can probably come up with a class decorator that hides construction of the frozen subclass. I do think that it should be used very sparingly, as the existence of the subclass is hard to hide: instances of MyRecord will show as instances of _LockedMyRecord when printed or otherwise inspected, which could be confusing. There's another very simple low-tech solution: @dataclass class MyRecord: a: int # etc. frozen: bool = False def freeze(self): self.frozen = True def __setattr__(self, name, value): if self.frozen: raise AttributeError super().__setattr__(name, value) def __hash__(self): assert self.frozen return super().__hash__() It can easily be weaponized via an extra decorator, and it's simpler in that it doesn't use a magical second class. I'm just not keen on adding the extra attribute to all frozen instances created via @dataclass(frozen=True), because it's extra space overhead. (In fact you can conclude that I'm not keen on exposing *any* of these "solutions" as a flag on the @dataclass() decorator. They all feel pretty fragile to me.) -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dataclasses, frozen and __post_init__
On Tue, Feb 20, 2018 at 8:45 AM, Guido van Rossum wrote: > TBH, I don't hate Nick's solution that assigns to __class__ in > __post_init__. You can probably come up with a class decorator that hides > construction of the frozen subclass. I do think that it should be used very > sparingly, as the existence of the subclass is hard to hide: instances of > MyRecord will show as instances of _LockedMyRecord when printed or > otherwise inspected, which could be confusing. > > There's another very simple low-tech solution: > > @dataclass > class MyRecord: > a: int > # etc. > frozen: bool = False > def freeze(self): > self.frozen = True > def __setattr__(self, name, value): > if self.frozen: raise AttributeError > super().__setattr__(name, value) > def __hash__(self): > assert self.frozen > return super().__hash__() > This sure seems far more straightforward and les magic than __class__ swapping. > It can easily be weaponized via an extra decorator, and it's simpler in > that it doesn't use a magical second class. I'm just not keen on adding the > extra attribute to all frozen instances created via > @dataclass(frozen=True), because it's extra space overhead. > Is that one attribute that big a deal? I suppose that simple dataclasses with only two or so attributes would see a significant change, but if you're really worried about space, wouldn't you use a namedtuple or simply a tuple anyway? (In fact you can conclude that I'm not keen on exposing *any* of these > "solutions" as a flag on the @dataclass() decorator. They all feel pretty > fragile to me.) > so provide a: @freezable_dataclass Additional decorator that folks that want the ability to freeze and unfreeze instances can use? and if it overrides __set__attr, it could be used an virtually any class to make a frozen version, yes? I can't say I have enough need to bother writing that myself, but if others do -- it seems like a good solution. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dataclasses, frozen and __post_init__
On 2/17/2018 2:35 PM, Guido van Rossum wrote: PS. I have to ponder why frozen dataclasses don't use `__new__`. As I'm sure everyone is now aware after the rest of this discussion: it's because the returned object isn't really immutable. That said, I have threatened to create a decorator similar to typing.NamedTuple that has the @dataclass features (except maybe __post_init__) and returns a class that does inherit from tuple, in which case it really would use `__new__`. I'll save that for 3.8, if it ever happens. There's a lot there to think about, first. For now, typing.NamedTuple is the way to go if you want something based on namedtuple yet using type hints. Eric ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dataclasses, frozen and __post_init__
On Tue, Feb 20, 2018 at 12:50 PM, Chris Barker wrote: > Is that one attribute that big a deal? I suppose that simple dataclasses > with only two or so attributes would see a significant change, but if > you're really worried about space, wouldn't you use a namedtuple or simply > a tuple anyway? > It's not just the space I worry about, it's about stomping on the user's namespace for instance variables. So far dataclasses works by adding things to the class only -- I want to place the bar pretty high before we change that. Anyway, there are several solutions now that people can implement as separate class decorators and distribute via PyPI (or copy into their own codebase) so I don't think now's the time to consider adding this to dataclasses. Maybe for Python 3.8. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dataclasses, frozen and __post_init__
On Tue, Feb 20, 2018 at 1:37 PM, Eric V. Smith wrote: > On 2/17/2018 2:35 PM, Guido van Rossum wrote: > >> PS. I have to ponder why frozen dataclasses don't use `__new__`. >> > > As I'm sure everyone is now aware after the rest of this discussion: it's > because the returned object isn't really immutable. > > That said, I have threatened to create a decorator similar to > typing.NamedTuple that has the @dataclass features (except maybe > __post_init__) and returns a class that does inherit from tuple, in which > case it really would use `__new__`. I'll save that for 3.8, if it ever > happens. There's a lot there to think about, first. For now, > typing.NamedTuple is the way to go if you want something based on > namedtuple yet using type hints. > But then the class would also inherit a bunch of misfeatures from tuple (like being indexable and having a length). It would be nicer if it used __slots__ instead. (Also, the problem with __slots__ is the same as the problem with inheriting from tuple, and it should just be solved right, somehow.) -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dataclasses, frozen and __post_init__
> On Feb 20, 2018, at 5:38 PM, Guido van Rossum wrote: > >> On Tue, Feb 20, 2018 at 1:37 PM, Eric V. Smith wrote: >>> On 2/17/2018 2:35 PM, Guido van Rossum wrote: >>> PS. I have to ponder why frozen dataclasses don't use `__new__`. >> >> As I'm sure everyone is now aware after the rest of this discussion: it's >> because the returned object isn't really immutable. >> >> That said, I have threatened to create a decorator similar to >> typing.NamedTuple that has the @dataclass features (except maybe >> __post_init__) and returns a class that does inherit from tuple, in which >> case it really would use `__new__`. I'll save that for 3.8, if it ever >> happens. There's a lot there to think about, first. For now, >> typing.NamedTuple is the way to go if you want something based on namedtuple >> yet using type hints. > > But then the class would also inherit a bunch of misfeatures from tuple (like > being indexable and having a length). Right. That’s the “lots to think about” part. It might just be unworkable. > It would be nicer if it used __slots__ instead. (Also, the problem with > __slots__ is the same as the problem with inheriting from tuple, and it > should just be solved right, somehow.) Agreed. Eric___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dataclasses, frozen and __post_init__
> On Feb 20, 2018, at 2:38 PM, Guido van Rossum wrote: > > But then the class would also inherit a bunch of misfeatures from tuple (like > being indexable and having a length). It would be nicer if it used __slots__ > instead. FWIW, George Sakkis made a tool like this about nine years ago. https://code.activestate.com/recipes/576555-records It would need to be modernized to include default arguments, types annotations and whatnot, but otherwise it has great performance and low API complexity. > (Also, the problem with __slots__ is the same as the problem with inheriting > from tuple, and it should just be solved right, somehow.) Perhaps a new variant of __init_subclass__ would work. Raymond ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dataclasses, frozen and __post_init__
Steven D'Aprano wrote: So in principle, we could have a mutable class, and a immutable one, and when you flick the switch, the instance.__class__ changes from mutable to frozen. That seems unfriendly to subclasses as well. To extend a class you now need to subclass both the mutable and immutable versions of the base class, ensure they share the behaviour they should have in common (possibly by using a third mixin class) and arrange for the freezing switch to install the immutable subclass instead of the base one. All of which seems like a huge amount of hassle just to save an instance attribute. And I don't even want to think what multiple inheritance would do to all this. -- Greg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] pycapsule:PyObject * is NULL pointer
i have question:call,c-->python-->c.
1.the c pointer void* abc="123" by pycapsule in the c code.
..
void* lpContext = "abc";
PyObject * lpPyContext = PyCapsule_New(lpContext, "Context",NULL);
..
PyTuple_SetItem(pArgs, 1, lpPyContext);
PyObject* pyResult = PyObject_CallObject(pFunc, pArgs);
2.c call python:
in the python code:
import ctypes
pycapsule = ctypes.windll.LoadLibrary("C:\Users\zhaoya16975\Documents\Visual
Studio 2017\Projects\cpython\Release\pycapsule.dll")
def test( lpContext,lpRequest,lpAnswer):
print lpContext
pycapsule.hello()
pycapsule.GetContext(lpContext)
.
the lpContest is ""
but,i can't lpContext in the pycapsule.GetContest:
the capsule is null poniter,the GetContext no execute!!
void* FUNCTION_CALL_MODE GetContext(PyObject *capsule) {
printf(" GetContext..\n");
//if (!PyCapsule_IsValid((PyObject *)capsule, "Context")) {
// printf(" the Capsule Context is no Valid\n");
// return NULL;
//}
//return PyCapsule_GetPointer((PyObject *)capsule, "Context");
return NULL;
}
FUNCTION_CALL_MODE is __stdcall i think no problem.because, if
GetContext(PyObject *capsule) -->GetContext() is ok. PyObject *capsule from
python to c unknown error occurred.
i hope c call python pass void* ,and python call c pass void* ,but the capsule
call no success?___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] How is the GitHub workflow working for people?
It's been a year and 10 days since we moved to GitHub, so I figured now is as good a time as any to ask people if they are generally happy with the workflow and if there is a particular sticking point to please bring it up on the core-workflow mailing list so we can potentially address it. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dataclasses, frozen and __post_init__
Chris Barker - NOAA Federal wrote: If I have this right, on the discussion about frozen and hash, a use case was brought up for taking a few steps to create an instance (and thus wanting it not frozen) and then wanting it hashable. Which pointed to the idea of a “ freeze this from now on” method. The problem with a "freeze" method is that it doesn't play well with subclasses. The object ends up frozen by the base class's __new__ or __init__, preventing the subclass from doing its own initialisation unless it uses some form of back-door access. And if there is a back door, it might as well be used for all initialisation, maing the freeze method unnecessary. -- Greg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] pycapsule:PyObject * is NULL pointer
On 2018-02-21 02:28, 赵亚 wrote:
This "python-dev" list is for development of the Python language itself.
I think that this message should instead have been sent to "python-list".
i have question:call,c-->python-->c.
1.the c pointer void* abc="123" by pycapsule in the c code.
..
void* lpContext = "abc";
PyObject * lpPyContext = PyCapsule_New(lpContext, "Context",NULL);
..
Is this C code calling the function 'test' in the Python code?
If yes, are you sure that lpPyContext should be in pArgs at index 1 and
not at index 0?
The function 'test' expects argument 0 => lpContext, argument 1 =>
lpRequest, argument 2 => lpAnswer.
PyTuple_SetItem(pArgs, 1, lpPyContext);
PyObject* pyResult = PyObject_CallObject(pFunc, pArgs);
2.c call python:
in the python code:
import ctypes
pycapsule = ctypes.windll.LoadLibrary("C:\Users\zhaoya16975\Documents\Visual Studio
2017\Projects\cpython\Release\pycapsule.dll")
def test( lpContext,lpRequest,lpAnswer):
print lpContext
pycapsule.hello()
pycapsule.GetContext(lpContext)
.
the lpContest is ""
but,i can't lpContext in the pycapsule.GetContest:
the capsule is null poniter,the GetContext no execute!!
void* FUNCTION_CALL_MODE GetContext(PyObject *capsule) {
printf(" GetContext..\n");
//if (!PyCapsule_IsValid((PyObject *)capsule, "Context")) {
// printf(" the Capsule Context is no Valid\n");
// return NULL;
//}
//return PyCapsule_GetPointer((PyObject *)capsule, "Context");
return NULL;
}
FUNCTION_CALL_MODE is __stdcall i think no problem.because, if GetContext(PyObject
*capsule) -->GetContext() is ok. PyObject *capsule from python to c unknown
error occurred.
i hope c call python pass void* ,and python call c pass void* ,but the capsule
call no success?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] void* from c to python and to c？
I have a demand: c call python ,have a void* pointer pass to python and python call c,the void* pointer need pass to c. i don't know how to do，have any good idea ？ thanks！___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] pycapsule:PyObject * is NULL pointer
hi @MRAB
i try again，it not ok. test files:
1.cpython.cpp ---c call python maintenance.py
2.maintenance.py python maintenance.py call pycapsule.cpp(dll)
the code file,see attachment.
At 2018-02-21 11:18:18, "MRAB" wrote:
>On 2018-02-21 02:28, 赵亚 wrote:
>
>This "python-dev" list is for development of the Python language itself.
>I think that this message should instead have been sent to "python-list".
>
>> i have question:call,c-->python-->c.
>> 1.the c pointer void* abc="123" by pycapsule in the c code.
>> ..
>> void* lpContext = "abc";
>> PyObject * lpPyContext = PyCapsule_New(lpContext, "Context",NULL);
>> ..
>
>Is this C code calling the function 'test' in the Python code?
>
>If yes, are you sure that lpPyContext should be in pArgs at index 1 and
>not at index 0?
>
>The function 'test' expects argument 0 => lpContext, argument 1 =>
>lpRequest, argument 2 => lpAnswer.
>
>> PyTuple_SetItem(pArgs, 1, lpPyContext);
>> PyObject* pyResult = PyObject_CallObject(pFunc, pArgs);
>>
>> 2.c call python:
>>
>> in the python code:
>> import ctypes
>> pycapsule = ctypes.windll.LoadLibrary("C:\Users\zhaoya16975\Documents\Visual
>> Studio 2017\Projects\cpython\Release\pycapsule.dll")
>> def test( lpContext,lpRequest,lpAnswer):
>> print lpContext
>> pycapsule.hello()
>> pycapsule.GetContext(lpContext)
>> .
>> the lpContest is ""
>> but,i can't lpContext in the pycapsule.GetContest:
>> the capsule is null poniter,the GetContext no execute!!
>>
>> void* FUNCTION_CALL_MODE GetContext(PyObject *capsule) {
>> printf(" GetContext..\n");
>> //if (!PyCapsule_IsValid((PyObject *)capsule, "Context")) {
>> // printf(" the Capsule Context is no Valid\n");
>> // return NULL;
>> //}
>> //return PyCapsule_GetPointer((PyObject *)capsule, "Context");
>> return NULL;
>>
>> }
>>
>> FUNCTION_CALL_MODE is __stdcall i think no problem.because, if
>> GetContext(PyObject *capsule) -->GetContext() is ok. PyObject *capsule
>> from python to c unknown error occurred.
>>
>> i hope c call python pass void* ,and python call c pass void* ,but the
>> capsule call no success?
>>
>___
>Python-Dev mailing list
>Python-Dev@python.org
>https://mail.python.org/mailman/listinfo/python-dev
>Unsubscribe:
>https://mail.python.org/mailman/options/python-dev/zhaoya881010%40163.com
maintenance.py
Description: Binary data
// cpython.cpp : ¶¨Òå¿ØÖÆ̨ӦÓóÌÐòµÄÈë¿Úµã¡£
//
#include "stdafx.h"
#include "cpython.h"
PyObject *g_pName= NULL;
PyObject *g_pModule = NULL;
int FUNCTION_CALL_MODE ReProc(void* lpContext,void* lpRequest,void* lpAnswer) {
PyObject *pDict = NULL;
PyObject *pFunc = NULL;
PyObject *pArgs = NULL;
PyObject * lpPyContext = PyCapsule_New(lpContext, "Context",NULL);
PyObject * lpPyRequest = PyCapsule_New(lpRequest, "Request", NULL);
PyObject * lpPyAnswer = PyCapsule_New(lpAnswer, "Answer", NULL);
pDict = PyModule_GetDict(g_pModule);
if (!pDict)
{
printf("Can't find dict in py_add!\n");
return -1;
}
pFunc = PyDict_GetItemString(pDict, "test");
if (!pFunc || !PyCallable_Check(pFunc))
{
printf("Can't find function!\n");
getchar();
return -1;
}
pArgs = PyTuple_New(3);
//PyTuple_SetItem(pArgs, 0, Py_BuildValue("i", 2006));
PyTuple_SetItem(pArgs, 0, lpPyContext);
PyTuple_SetItem(pArgs, 1, lpPyRequest);
PyTuple_SetItem(pArgs, 2, lpPyAnswer);
//µ÷ÓÃpythonµÄReqProcº¯Êý
PyObject* pyResult = PyObject_CallObject(pFunc, pArgs);
if (pArgs)
{
Py_DECREF(pArgs);
}
}
int FUNCTION_CALL_MODE OnInit()
{
//³õʼ»¯£¬ÔØÈëpythonµÄÀ©Õ¹Ä£¿é
Py_Initialize();
//Åжϳõʼ»¯ÊÇ·ñ³É¹¦
if (!Py_IsInitialized())
{
printf("Python init failed!\n");
return -1;
}
//PyRun_SimpleString Ϊºê£¬Ö´ÐÐÒ»¶Îpython´úÂë
//µ¼È뵱ǰ·¾¶
//C:\Users\zhaoya16975\PycharmProjects\complugin\coreplugin.py
PyRun_SimpleString("import sys");
PyRun_SimpleString("sys.path.append(\"C:\\Users\\zhaoya16975\\go\\src\\GoProxy\\plugin\\py\")");
//¼ÓÔØÃûΪpy_addµÄpython½Å±¾
g_pName = PyString_FromString("maintenance");
g_pModule = PyImport_Import(g_pName);
if (!g_pModule)
{
printf("Load coreplugin.py failed!\n");
getchar();
return -1;
}
//¹Ø±Õpythonµ÷ÓÃ
//Py_Finalize();
return 0;
}
void FUNCTION_CALL_MODE OnClose() {
}
void main() {
//¼ÓÔسõʼ»¯py²å¼þ
OnInit();
//
void* lpContext;
void* lpRequest;
void*
[Python-Dev] Making "Provisional" an explicit PEP state
Hi folks, In the current PEP workflow, provisionally accepted PEPs are marked as "Accepted", and remain in that state until they're declared stable and moved to Final. This makes it hard to identify which APIs and file formats have been approved to have their reference implementations released (whether in CPython or in other tools), so I filed https://github.com/python/peps/issues/270 some time ago to propose making Provisional an explicit state in its own right, distinct from Accepted. https://github.com/python/peps/pull/577 is a PR with a concrete proposal along those lines: * "Provisional" is a new state that parallels Accepted, with the same basic state flows * PEP 0 gains a new section, listing the provisionally accepted PEPs separately from the accepted ones * PEP 484 (type hinting), and the PyPA static build metadata (pyproject.toml) PEPs are marked as provisionally accepted Regards, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
