Re: Defining a Python enum in a C extension - am I doing this right?

[Bartosz Golaszewski]

On Sat, Jul 31, 2021 at 3:01 PM Bartosz Golaszewski  wrote:
>
> On Fri, Jul 30, 2021 at 2:41 PM Serhiy Storchaka  wrote:
> >
> > 23.07.21 11:20, Bartosz Golaszewski пише:
> > > I'm working on a Python C extension and I would like to expose a
> > > custom enum (as in: a class inheriting from enum.Enum) that would be
> > > entirely defined in C.
> >
> > I think that it would be much easier to define it in Python, and then
> > either import a Python module in your C code, or exec a Python code as a
> > string.
> >
>
> You mean: evaluate a string like this:
>
> '''
> import enum
>
> class FooBar(enum.Enum):
> FOO = 1
> BAR = 2
> BAZ = 3
> '''
>
> And then pull in the FooBar type from the resulting dictionary into my
> C code? Sounds good actually. I think I'll be able to add the FooBar
> type to another type's tp_dict too - because some enums I want to
> create will be nested in other classes.
>
> Bart

Just a follow-up: this is how I did it eventually:

```
#include 

typedef struct {
PyObject_HEAD;
} dummy_object;

PyDoc_STRVAR(dummy_type_doc, "Dummy type in which the enum will be nested.");

static PyTypeObject dummy_type = {
PyVarObject_HEAD_INIT(NULL, 0)
.tp_name = "pycenum.DummyType",
.tp_basicsize = sizeof(dummy_object),
.tp_flags = Py_TPFLAGS_DEFAULT,
.tp_doc = dummy_type_doc,
.tp_new = PyType_GenericNew,
.tp_dealloc = (destructor)PyObject_Del,
};

PyDoc_STRVAR(module_doc,
"C extension module defining a class inheriting from enum.Enum.");

static PyModuleDef module_def = {
PyModuleDef_HEAD_INIT,
.m_name = "pycenum",
.m_doc = module_doc,
.m_size = -1,
};

static int add_foobar_enum(PyObject *module)
{
static const char *foobar_src =
"class FooBar(enum.Enum):\n"
"FOO = 1\n"
"BAR = 2\n"
"BAZ = 3\n";

PyObject *main_mod, *main_dict, *enum_mod, *result, *foobar_type;
int ret;

main_mod = PyImport_AddModule("__main__");
if (!main_mod)
return -1;

main_dict = PyModule_GetDict(main_mod);
if (!main_dict) {
Py_DECREF(main_mod);
return -1;
}

enum_mod = PyImport_ImportModule("enum");
if (!enum_mod) {
Py_DECREF(main_mod);
return -1;
}

ret = PyDict_SetItemString(main_dict, "enum", enum_mod);
Py_DECREF(enum_mod);
if (ret) {
Py_DECREF(main_mod);
return -1;
}

result = PyRun_String(foobar_src, Py_single_input,
  main_dict, main_dict);
if (!result) {
Py_DECREF(main_mod);
return -1;
}

foobar_type = PyDict_GetItemString(main_dict, "FooBar");
if (!foobar_type) {
Py_DECREF(main_mod);
return -1;
}

ret = PyDict_SetItemString(dummy_type.tp_dict, "FooBar", foobar_type);
Py_DECREF(foobar_type);
Py_DECREF(main_mod);
if (ret)
return -1;

PyType_Modified(&dummy_type);

return ret;
}

PyMODINIT_FUNC PyInit_pycenum(void)
{
PyObject *module;
int ret;

module = PyModule_Create(&module_def);
if (!module)
return NULL;

ret = PyModule_AddStringConstant(module, "__version__", "0.0.1");
if (ret) {
Py_DECREF(module);
return NULL;
}

ret = PyType_Ready(&dummy_type);
if (ret) {
Py_DECREF(module);
return NULL;
}

ret = add_foobar_enum(module);
if (ret) {
Py_DECREF(module);
return NULL;
}

Py_INCREF(&dummy_type);
ret = PyModule_AddObject(module, "DummyType", (PyObject *)&dummy_type);
if (ret) {
Py_DECREF(&dummy_type);
Py_DECREF(module);
return NULL;
}

return module;
}
```

Bart
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: argparse: delimiter for argparse list arguments

[Michael Torrie]

On 8/2/21 1:43 PM, Sven R. Kunze wrote:
> Hi everyone,
> 
> maybe, I am missing something here but is it possible to specify a 
> delimiter for list arguments in argparse:
> 
> https://docs.python.org/3/library/argparse.html
> 
> Usually, '--' is used to separate two lists (cf. git).

I've not seen this syntax in git. Are you referring the the double and
triple dot notation git uses?  Can you give me an example of how git
uses -- as a list separator?

Typically -- on a command line means that's the end of the any special
switches and anything else, even if it looks like a command-line switch,
should not be parsed, and passed straight through as a normal parameter.

-- 
https://mail.python.org/mailman/listinfo/python-list

[RELEASE] Python 3.10.0rc1 is available

[Pablo Galindo Salgado]

Python 3.10.0 is almost ready. This release, 3.10.0rc1, is the penultimate
release preview. You can get it here:

https://www.python.org/downloads/release/python-3100rc1/


*This is the first release candidate of Python 3.10*
This release, **3.10.0rc1**, is the penultimate release preview.  Entering
the release candidate phase, only reviewed code changes which are
clear bug fixes are allowed between this release candidate and the final
release. The second candidate and the last planned release preview is
currently planned for 2021-09-06 while the official release is planned for
2021-10-04.

There will be no ABI changes from this point forward in the 3.10 series and
the goal is that there will be as few code changes as possible.

*Call to action*
Core developers: all eyes on the docs now

   - Are all your changes properly documented?
   - Did you notice other changes you know of to have insufficient
   documentation?

Community members

We strongly encourage maintainers of third-party Python projects to prepare
their projects for 3.10 compatibilities during this phase. As always,
report any issues to the Python bug tracker .
Please keep in mind that this is a preview release and its use is **not**
recommended for production environments.

*And now for something completely different*

In theoretical physics, quantum chromodynamics (QCD) is the theory of the
strong interaction between quarks and gluons, the fundamental particles
that make up composite hadrons such as the proton, neutron, and pion. The
QCD analog of electric charge is a property called color. Gluons are the
force carrier of the theory, just as photons are for the electromagnetic
force in quantum electrodynamics. There are three kinds of charge in QCD
(as opposed to one in quantum electrodynamics or QED) that are usually
referred to as "color charge" by loose analogy to the three kinds of color
(red, green and blue) perceived by humans. Other than this nomenclature,
the quantum parameter "color" is completely unrelated to the everyday,
familiar phenomenon of color.


*We hope you enjoy those new releases!*
Thanks to all of the many volunteers who help make Python Development and
these releases possible! Please consider supporting our efforts by
volunteering yourself or through organization contributions to the Python
Software Foundation.

Regards from cloudy London,

Your friendly release team,
Pablo Galindo @pablogsal
Ned Deily @nad
Steve Dower @steve.dower
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: argparse: delimiter for argparse list arguments

[Jon Ribbens via Python-list]

On 2021-08-03, Michael Torrie  wrote:
> On 8/2/21 1:43 PM, Sven R. Kunze wrote:
>> maybe, I am missing something here but is it possible to specify a 
>> delimiter for list arguments in argparse:
>> 
>> https://docs.python.org/3/library/argparse.html
>> 
>> Usually, '--' is used to separate two lists (cf. git).
>
> I've not seen this syntax in git. Are you referring the the double and
> triple dot notation git uses?  Can you give me an example of how git
> uses -- as a list separator?

Loads of git commands do this. e.g. commit, diff, log, status, etc.
It's not completely unlike what you're describing above, which is
already supported automatically by argparse.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: argparse: delimiter for argparse list arguments

[Roel Schroeven]

Jon Ribbens via Python-list schreef op 3/08/2021 om 17:48:

On 2021-08-03, Michael Torrie  wrote:
> On 8/2/21 1:43 PM, Sven R. Kunze wrote:
>> maybe, I am missing something here but is it possible to specify a 
>> delimiter for list arguments in argparse:
>> 
>> https://docs.python.org/3/library/argparse.html
>> 
>> Usually, '--' is used to separate two lists (cf. git).

>
> I've not seen this syntax in git. Are you referring the the double and
> triple dot notation git uses?  Can you give me an example of how git
> uses -- as a list separator?

Loads of git commands do this. e.g. commit, diff, log, status, etc.
It's not completely unlike what you're describing above, which is
already supported automatically by argparse.
Commands like git commit do not use '--' to separate two lists, but as 
Dan and Sven said to separate options from arguments, even if the 
arguments start with '-' (like many other programs -- I think this is 
standard in GNU). If you're 100% certain that none of the filenames 
start with '-', you can leave out '--' without any change in behavior. 
Especially when scripting and/or using wildcards it's best always to use 
that '--' to avoid nasty surprises.


$ git commit -m "hello" -- hello.py

is exactly the same as

$ git commit -m "hello" hello.py

It's just that the former is safer to use, because it properly works 
even in cases like


$ git commit -m "hello" -- -hello.py

whereas

$ git commit -m "hello" -hello.py

will likely not do what you expect.

As https://git-scm.com/docs/git-commit says:

> --
>   Do not interpret any more arguments as options.

--
"The saddest aspect of life right now is that science gathers knowledge
faster than society gathers wisdom."
-- Isaac Asimov

--
https://mail.python.org/mailman/listinfo/python-list

Re: argparse: delimiter for argparse list arguments

[Sven R. Kunze]

It could be but I've seen them used somewhere else.

I wouldn't bikeshed on this yet, as I haven't found a way to do this so 
far. Let's imagine the following parser:


parser.add_argument('things',action='append')
parser.add_argument('stuff',action='append')

At least from my point of view, I don't any way to separate both lists 
on this command call:



cool-script.py thing1 thing2 stuff1 stuff2


Do I miss something here?


Best
Sven

On 03.08.21 01:49, Dan Stromberg wrote:


Isn't -- usually used to signal the end of options?

On Mon, Aug 2, 2021 at 12:52 PM Sven R. Kunze > wrote:


Hi everyone,

maybe, I am missing something here but is it possible to specify a
delimiter for list arguments in argparse:

https://docs.python.org/3/library/argparse.html


Usually, '--' is used to separate two lists (cf. git).

Cheers,
Sven

-- 
https://mail.python.org/mailman/listinfo/python-list




--
https://mail.python.org/mailman/listinfo/python-list

Re: argparse: delimiter for argparse list arguments

[Chris Angelico]

On Wed, Aug 4, 2021 at 7:07 AM Sven R. Kunze  wrote:
>
> It could be but I've seen them used somewhere else.
>
> I wouldn't bikeshed on this yet, as I haven't found a way to do this so
> far. Let's imagine the following parser:
>
> parser.add_argument('things',action='append')
> parser.add_argument('stuff',action='append')
>
> At least from my point of view, I don't any way to separate both lists
> on this command call:
>
>
> cool-script.py thing1 thing2 stuff1 stuff2
>
>
> Do I miss something here?
>

I'd probably design it like this:

cool-script.py --things thing1 thing2 --stuff stuff1 stuff2

which argparse already handles.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: argparse: delimiter for argparse list arguments

[Jon Ribbens via Python-list]

On 2021-08-03, Roel Schroeven  wrote:
> Jon Ribbens via Python-list schreef op 3/08/2021 om 17:48:
>> On 2021-08-03, Michael Torrie  wrote:
>> > On 8/2/21 1:43 PM, Sven R. Kunze wrote:
>> >> maybe, I am missing something here but is it possible to specify a 
>> >> delimiter for list arguments in argparse:
>> >> 
>> >> https://docs.python.org/3/library/argparse.html
>> >> 
>> >> Usually, '--' is used to separate two lists (cf. git).
>> >
>> > I've not seen this syntax in git. Are you referring the the double and
>> > triple dot notation git uses?  Can you give me an example of how git
>> > uses -- as a list separator?
>>
>> Loads of git commands do this. e.g. commit, diff, log, status, etc.
>> It's not completely unlike what you're describing above, which is
>> already supported automatically by argparse.
>
> Commands like git commit do not use '--' to separate two lists, but as 
> Dan and Sven said to separate options from arguments,

That is not quite correct. Check out 'git diff' for example - it takes
as position arguments zero, one, or two commit hashes followed by zero
or more pathnames.  You can use '--' to separate these two lists,
because of course sometimes a pathname can be indistinguishable from
a commit hash.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: argparse: delimiter for argparse list arguments

[Roel Schroeven]

Jon Ribbens via Python-list schreef op 3/08/2021 om 22:55:

>> Loads of git commands do this. e.g. commit, diff, log, status, etc.
>> It's not completely unlike what you're describing above, which is
>> already supported automatically by argparse.
>
> Commands like git commit do not use '--' to separate two lists, but as 
> Dan and Sven said to separate options from arguments,


That is not quite correct. Check out 'git diff' for example - it takes
as position arguments zero, one, or two commit hashes followed by zero
or more pathnames.  You can use '--' to separate these two lists,
because of course sometimes a pathname can be indistinguishable from
a commit hash.
Ah yes, I stand corrected, git diff indeed uses -- to separate two lists 
(commits and paths).


--
"Too often we hold fast to the cliches of our forebears. We subject all
facts to a prefabricated set of interpretations. Too often we enjoy the
comfort of opinion without the discomfort of thought."
-- John F Kennedy

--
https://mail.python.org/mailman/listinfo/python-list

SQLALchemy: update with in clause from kwargs

[Larry Martell]

I am trying to write a function that takes kwargs as a param and
generates an update statement where the rows to be updated are
specified in an in clause.

Something like this:

def update_by_in(self, **kwargs):
filter_group = []
for col in kwargs['query_params']:
# obviously this line does not work as col is a string,
but this is the intent
filter_group.append(col.in_(tuple(kwargs['query_params'][col])))


self._session.query(self.model_class).filter(*filter_group).update(kwargs['values'])

self.update_by_in(
**{'query_params': {'companyCode': ['A', 'B', 'C']},
'values': {'portfolioName': 'test'}}
 )

Is there a way to do this? I think I need to use setattr in building
up the filter_group list, but I'm not quite sure how to do it.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: SQLALchemy: update with in clause from kwargs

[MRAB]

On 2021-08-04 02:08, Larry Martell wrote:

I am trying to write a function that takes kwargs as a param and
generates an update statement where the rows to be updated are
specified in an in clause.

Something like this:

 def update_by_in(self, **kwargs):
 filter_group = []
 for col in kwargs['query_params']:
 # obviously this line does not work as col is a string,
but this is the intent
 filter_group.append(col.in_(tuple(kwargs['query_params'][col])))

 
self._session.query(self.model_class).filter(*filter_group).update(kwargs['values'])

self.update_by_in(
 **{'query_params': {'companyCode': ['A', 'B', 'C']},
 'values': {'portfolioName': 'test'}}
  )

Is there a way to do this? I think I need to use setattr in building
up the filter_group list, but I'm not quite sure how to do it.


If it's any help, on this page:

https://docs.sqlalchemy.org/en/14/core/metadata.html

it has this:

# access the column "employee_id":
employees.columns.employee_id

# or just
employees.c.employee_id

# via string
employees.c['employee_id']
--
https://mail.python.org/mailman/listinfo/python-list

Re: SQLALchemy: update with in clause from kwargs

[dn via Python-list]

On 04/08/2021 13.08, Larry Martell wrote:
> I am trying to write a function that takes kwargs as a param and
> generates an update statement where the rows to be updated are
> specified in an in clause.
> 
> Something like this:
> 
> def update_by_in(self, **kwargs):
> filter_group = []
> for col in kwargs['query_params']:
> # obviously this line does not work as col is a string,
> but this is the intent
> filter_group.append(col.in_(tuple(kwargs['query_params'][col])))
> 
> 
> self._session.query(self.model_class).filter(*filter_group).update(kwargs['values'])
> 
> self.update_by_in(
> **{'query_params': {'companyCode': ['A', 'B', 'C']},
> 'values': {'portfolioName': 'test'}}
>  )
> 
> Is there a way to do this? I think I need to use setattr in building
> up the filter_group list, but I'm not quite sure how to do it.


When feeling bamboozled by a problem, particularly when using
sophisticated tools such as SQLAlchemy, the trick is often to simplify
the problem.

Step 1 (using the sample data provided)
Write the query on paper - and as constant-values.

Step 2
Compare the two input dicts with that requirement.

Step 3
Work-out the transformation(s) required...


One complexity is that the parameter to update_by_in() is formed by
joining two dicts. However, the function later tries to treat them in
distinct fashions. Why the join/why not two parameters?

companyCode = ['A', 'B', 'C']
values = {'portfolioName': 'test'}

leading to:

self.update_by_in( companyCode, values )

and:

def update_by_in(self, company_code, portfolio_type ):



As to the core of the question-asked, I'm a little confused (which may
be my fuzzy head). Do you want the update(s) - portrayed as a list -
like this:

[('A', 'test'), ('B', 'test'), ('C', 'test')]

like this:

[('A', 'portfolioName'), ('B', None), ('C', None)]

or only:

[('A', 'portfolioName')]


You will find a friend in the itertools (PSL) library:

import itertools as it

list( it.product( companyCode, values.values() ) )
[('A', 'test'), ('B', 'test'), ('C', 'test')]

list( it.zip_longest( companyCode, values.values() ) )
[('A', 'test'), ('B', None), ('C', None)]

list( zip( companyCode, values ) )
[('A', 'portfolioName')]


Now, have we simplified things to the point of being able to more-easily
code the update and filter?

PS I fear even that step is/those steps are more complicated than needed
- but you know your data and schema better than I!


Critique:
1 never, never, never(!) use a name which will "shadow" a Python keyword
or frequently-used function-name (in this case "values" ) - if you don't
confuse Python you will confuse simple-boys like me! Plus, when you come
back in six-month's time, does "values" tell you what kind of value it
is/holds?
2 Python != C | Java
Thus: company_code, portfolio_name


Web.Refs:
https://www.dictionary.com/browse/bamboozled
https://docs.python.org/3/library/itertools.html
-- 
Regards,
=dn
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Defining a Python enum in a C extension - am I doing this right?

[Sean DiZazzo]

On Tuesday, August 3, 2021 at 3:04:19 AM UTC-7, Bartosz Golaszewski wrote:
> On Sat, Jul 31, 2021 at 3:01 PM Bartosz Golaszewski  wrote: 
> > 
> > On Fri, Jul 30, 2021 at 2:41 PM Serhiy Storchaka  wrote: 
> > > 
> > > 23.07.21 11:20, Bartosz Golaszewski пише: 
> > > > I'm working on a Python C extension and I would like to expose a 
> > > > custom enum (as in: a class inheriting from enum.Enum) that would be 
> > > > entirely defined in C. 
> > > 
> > > I think that it would be much easier to define it in Python, and then 
> > > either import a Python module in your C code, or exec a Python code as a 
> > > string. 
> > > 
> > 
> > You mean: evaluate a string like this: 
> > 
> > ''' 
> > import enum 
> > 
> > class FooBar(enum.Enum): 
> > FOO = 1 
> > BAR = 2 
> > BAZ = 3 
> > ''' 
> > 
> > And then pull in the FooBar type from the resulting dictionary into my 
> > C code? Sounds good actually. I think I'll be able to add the FooBar 
> > type to another type's tp_dict too - because some enums I want to 
> > create will be nested in other classes. 
> > 
> > Bart
> Just a follow-up: this is how I did it eventually: 
> 
> ``` 
> #include  
> 
> typedef struct { 
> PyObject_HEAD; 
> } dummy_object; 
> 
> PyDoc_STRVAR(dummy_type_doc, "Dummy type in which the enum will be nested."); 
> 
> static PyTypeObject dummy_type = { 
> PyVarObject_HEAD_INIT(NULL, 0) 
> .tp_name = "pycenum.DummyType", 
> .tp_basicsize = sizeof(dummy_object), 
> .tp_flags = Py_TPFLAGS_DEFAULT, 
> .tp_doc = dummy_type_doc, 
> .tp_new = PyType_GenericNew, 
> .tp_dealloc = (destructor)PyObject_Del, 
> };
> PyDoc_STRVAR(module_doc, 
> "C extension module defining a class inheriting from enum.Enum."); 
> 
> static PyModuleDef module_def = { 
> PyModuleDef_HEAD_INIT, 
> .m_name = "pycenum", 
> .m_doc = module_doc, 
> .m_size = -1, 
> };
> static int add_foobar_enum(PyObject *module) 
> { 
> static const char *foobar_src = 
> "class FooBar(enum.Enum):\n" 
> " FOO = 1\n" 
> " BAR = 2\n" 
> " BAZ = 3\n"; 
> 
> PyObject *main_mod, *main_dict, *enum_mod, *result, *foobar_type; 
> int ret; 
> 
> main_mod = PyImport_AddModule("__main__"); 
> if (!main_mod) 
> return -1; 
> 
> main_dict = PyModule_GetDict(main_mod); 
> if (!main_dict) { 
> Py_DECREF(main_mod); 
> return -1;
> } 
> 
> enum_mod = PyImport_ImportModule("enum");
> if (!enum_mod) { 
> Py_DECREF(main_mod); 
> return -1; 
> } 
> 
> ret = PyDict_SetItemString(main_dict, "enum", enum_mod); 
> Py_DECREF(enum_mod); 
> if (ret) { 
> Py_DECREF(main_mod); 
> return -1; 
> } 
> 
> result = PyRun_String(foobar_src, Py_single_input, 
> main_dict, main_dict); 
> if (!result) { 
> Py_DECREF(main_mod); 
> return -1; 
> } 
> 
> foobar_type = PyDict_GetItemString(main_dict, "FooBar"); 
> if (!foobar_type) { 
> Py_DECREF(main_mod); 
> return -1; 
> } 
> 
> ret = PyDict_SetItemString(dummy_type.tp_dict, "FooBar", foobar_type); 
> Py_DECREF(foobar_type); 
> Py_DECREF(main_mod); 
> if (ret) 
> return -1; 
> 
> PyType_Modified(&dummy_type); 
> 
> return ret; 
> } 
> 
> PyMODINIT_FUNC PyInit_pycenum(void) 
> { 
> PyObject *module;
> int ret; 
> 
> module = PyModule_Create(&module_def); 
> if (!module) 
> return NULL; 
> 
> ret = PyModule_AddStringConstant(module, "__version__", "0.0.1");
> if (ret) { 
> Py_DECREF(module); 
> return NULL; 
> } 
> 
> ret = PyType_Ready(&dummy_type); 
> if (ret) { 
> Py_DECREF(module); 
> return NULL; 
> } 
> 
> ret = add_foobar_enum(module); 
> if (ret) { 
> Py_DECREF(module); 
> return NULL; 
> } 
> 
> Py_INCREF(&dummy_type); 
> ret = PyModule_AddObject(module, "DummyType", (PyObject *)&dummy_type); 
> if (ret) { 
> Py_DECREF(&dummy_type);
> Py_DECREF(module); 
> return NULL; 
> } 
> 
> return module; 
> }
> ``` 
> 
> Bart
No
-- 
https://mail.python.org/mailman/listinfo/python-list