[Python-Dev] Re: Improvement to SimpleNamespace

[Serhiy Storchaka]

15.04.20 05:59, Raymond Hettinger пише:

SimpleNamespace() is really good at giving attribute style-access. I would like 
to make that functionality available to the JSON module (or just about anything 
else that accepts a custom dict) by adding the magic methods for mappings so 
that this works:

  catalog = json.load(f, object_hook=SimpleNamespace)


AFAIK, the only thing required for this is to make SimpleNamespace 
accepting a single positional argument: a mapping or a sequence of 
key-value pairs. It is easy to do. No new magic methods are needed.


As a workaround you can use

object_hook=lambda x: SimpleNamespace(**x)


  print(catalog['clothing']['mens']['shoes']['extra_wide']['quantity']) 
 # currently possible with dict()
  print(catalog.clothing.mens.shoes.extra_wide.quantity]) # 
proposed with SimpleNamespace()


It is already possible with SimpleNamespace.


  print(catalog.clothing.boys['3t'].tops.quantity   
# would also be supported


I would prefer to have a separate class if you want to have attribute 
and indexing access in the same object.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TYIXRGOLWGV5TNJ3XMV443O3RWLEYB65/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Improvement to SimpleNamespace

[Raymond Hettinger]

[Serhiy]
> As a workaround you can use
> 
> object_hook=lambda x: SimpleNamespace(**x)

That doesn't suffice because some valid JSON keys are not valid identifiers.  
You still need a way to get past those when they arise:  
catalog.books.fiction['Paradise Lost'].isbn  Also, it still leaves you with 
using setattr(ns, attrname, attrvalue) or tricks with vars() when doing 
updates.  The AttrDict recipe is popular for a reason.


Raymond
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/MNVWBEJI465QUODJEYPMAXPXOX3UDJ6Q/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Improvement to SimpleNamespace

[Serhiy Storchaka]

15.04.20 10:06, Raymond Hettinger пише:

[Serhiy]

As a workaround you can use

object_hook=lambda x: SimpleNamespace(**x)


That doesn't suffice because some valid JSON keys are not valid identifiers.  
You still need a way to get past those when they arise:  
catalog.books.fiction['Paradise Lost'].isbn  Also, it still leaves you with 
using setattr(ns, attrname, attrvalue) or tricks with vars() when doing 
updates.  The AttrDict recipe is popular for a reason.


Then it obviously should be different class than SimpleNamespace. There 
are too much differences between them, and they are used in different 
circumstances.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/A6BLAOIMI64IKNVUU2YRSQHPVLNYGMFL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Modify the C API to hide implementation details

[Ronald Oussoren via Python-Dev]

> On 15 Apr 2020, at 03:39, Victor Stinner  wrote:
> 
> Hi Ronald,
> 
> Le mar. 14 avr. 2020 à 18:25, Ronald Oussoren  a 
> écrit :
>> Making “PyObject” opaque will also affect the stable ABI because even types 
>> defined using the PyTypeSpec API embed a “PyObject” value in the structure 
>> defining the instance layout. It is easy enough to change this in a way that 
>> preserves source-code compatibility, but I’m  not sure it is possible to 
>> avoid breaking the stable ABI.
> 
> Oh, that's a good point. I tracked this issue at:
> https://bugs.python.org/issue39573#msg366473
> 
>> BTW. This will require growing the PyTypeSpec ABI a little, there are 
>> features you cannot implement using that API for example the buffer protocol.
> 
> I tracked this feature request at:
> https://bugs.python.org/issue40170#msg366474

Another issue with making structures opaque is that this makes it at best 
harder to subclass builtin types in an extension while adding additional data 
fields to the subclass. This is a similar issue as the fragile base class issue 
that was fixed in Objective-C 2.0 by adding a level of indirection, and could 
probably be fixed in a similar way in Python.

Ronald
—

Twitter / micro.blog: @ronaldoussoren
Blog: https://blog.ronaldoussoren.net/ 


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HBOKY4WKIV7Z4Y5RSYISEU7D5ABI2B2I/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Improvement to SimpleNamespace

[Cameron Simpson]

On 14Apr2020 23:08, Glenn Linderman  wrote:

On 4/14/2020 10:09 PM, Cameron Simpson wrote:
Like many others, I recently implemented one of these 
__getattr__+__getitem__ SimpleNamespaces.  I'm hacking on some 
mappings which map dotted-names to values.  So the natural 
implementation is dicts or dict subclasses.  But I'm also feeding 
these to format strings, so I want to write:


 "Hi {dotted.name}"

so I've made a nesting of SimpleNamespace so that I can use nice 
dotted a.b.c.d type names in the format string. And because I'm 
using .format_map, the top level namespace also supports 
__getitem__.


Now, my dict subclass has a .ns() method returning one of the above 
SimpleNamespace subclasses, but I can readily imagine a utility 
function in collection that made such a thing. [...]


The below is something of a digression from the main topic, so before 
digressing let me prefix with:


I'm with Raymond here. I think my position is that unlike most classes, 
SimpleNamespace has very simple semantics, and no __getitem__ facility 
at all, so making __getitem__ map to __getattr__ seems low impact.


Now the digression:

Interesting comments. So .ns allows you to convert a nested dict to a 
dotted one "on demand" but the initial conversion only needs to do the 
top level object?


In principle, yes. As it happens all of the subnamespaces are this class 
as well, so one can __getitem__ on them too. I just... don't need to in 
a format string.


As you might imagine this has sort of evolved from my initial problem of 
accessing the dotted.name values in a format string, and it still has 
some iteration to go there.  str.format() requires completely filled out 
keyword arguments, but str.format_map() accepts any mapping, so 
__getitem__ can do whatever one wants. In my base class it is a thin 
wrapper around getattr().


Because I'm starting with a flat dict like:

   { 'a': 1
 'b': 2,
 'b.c': 3,
 'b.d': 4,
 'b.d.e': 5,
   }

the .ns() utility method constructs a complete nested tree of 
SimpleNamespace objects.


That sounds similar to the glom that Nathaniel brought to the attention 
of this thread, which I hadn't found before.  But glom requires bulkier 
syntax. When I read his example, I thought to myself "What if one could 
write his 'glom(json_obj, "a.b.c")' example as  json_obj['a.b.c']


Hah, I've got one of those, too, almost :-)

I've got a cs.sqlalchemy_utils module with a 
get_json_field(column_value, field_name) function (it has a "set" 
friend).  It was intended to access a JSON blob database column, and the 
field_name it accepts is a dotted name used to traverse a nesting of 
dicts. I see its signature looks like glom().


  or better as non_json_obj.a.b.c ?  I can imagine an implementation of 
my first "what-if" fully in the top-level object, but not of the second 
(but my internals imagination is probably limited).


Nested dicts are a natural thing, and what you get from nested JSON 
structures, particularly since JavaScript objects _are_ dicts, and the 
attribute stuff is almost syntactic sugar.


Indeed, many of my use-cases are non-json, although a bunch are json 
also.


My use case for the .ns() stuff is nonJSON.

One of the reasons my dict-subclass has a .ns() method to create a set 
of nested namespaces is that an ordinary class has plenty of method 
names which will overlap dict element names. So making the dict's 
__getitem__ available via __getattr__ leads to trouble as soon as a key 
matches a dict method name.


For this reason I gave it a separate .ns() method to construct the 
SimpleNamespaces, which are a _transcription_ of the original values.


Cheers,
Cameron Simpson 
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NDGACBBC5TLFSBYJYKIOB2IDZQELMC7O/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Providing fix for modulefinder.py regression on Windows

[Brett Cannon]

Barry Scott wrote:
> > On 12 Apr 2020, at 21:49, Eric V. Smith e...@trueblade.com wrote:
> > On 4/12/2020 4:08 PM, Barry Scott wrote:
> > On 11 Apr
> > 2020, at 16:28, Barry Scott  > mailto:ba...@barrys-emacs.org> wrote:
> > modulefinder.py does not open source files in "rb" which
> > prevents compile() from applying the encoding rules.
> > snip
> > I have created the bpo, the pr and have the checks passing except that I 
> > need a
> > NEWS.d entry
> > to make bedevere/news happy. I see the files have a special filename is 
> > that from an
> > automatic
> > process of a tool I need to run?
> > Barry
> > The tool is blurb (or the online blurb-it): 
> > https://devguide.python.org/committing/#what-s-new-and-news-entries
> > https://devguide.python.org/committing/#what-s-new-and-news-entries
> > I was reading this https://devguide.python.org/#contributing
> > https://devguide.python.org/#contributing
> > that does not mention blurb.
> > I would not have though to look in the section on Accepting Pull Requests, 
> > as
> that not what I'm doing.

It's also available through the "Details" link of the failing status check on 
the PR.

-Brett

> Barry
> > Eric
> >
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LGXEXD7U3CXPQEBHIJ7664KXQ3JGDSRT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Modify the C API to hide implementation details

[Brett Cannon]

It seems a little odd to be dictating website updates about other VMs in this 
PEP. I'm not arguing that we shouldn't update the site, I just think requiring 
it as part of this PEP seems tangential to what the PEP is focusing on.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TKHNENOXP6H34E73XGFOL2KKXSM4Z6T2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Improvement to SimpleNamespace

[Victor Stinner]

Le mer. 15 avr. 2020 à 05:02, Raymond Hettinger
 a écrit :
> I would like to make that functionality available to the JSON module (or just 
> about anything else that accepts a custom dict) by adding the magic methods 
> for mappings so that this works:
>
>  catalog = json.load(f, object_hook=SimpleNamespace)
>  print(catalog['clothing']['mens']['shoes']['extra_wide']['quantity'])
>   # currently possible with dict()
>  print(catalog.clothing.mens.shoes.extra_wide.quantity]) 
> # proposed with SimpleNamespace()
>  print(catalog.clothing.boys['3t'].tops.quantity  
>  # would also be supported

It seems like there are many projects on PyPI which provide such
features. A few examples:

* https://pypi.org/project/attrdict/ "mapping objects that allow their
elements to be accessed both as keys and as attributes"
* https://pypi.org/project/dictlib/ "Dictionary keys as object attributes"
* https://pypi.org/project/objdict/ "An ObjDict supports all normal
dict operations, but adds support for accessing and setting entries as
attributes"
* https://pypi.org/project/mydict/ "dict subclass which tries to act
like JavaScript objects, so you can use the dot notation (d.foo) to
access members of the object"
* https://pypi.org/project/dpath/ "accessing and searching
dictionaries via /slashed/paths ala xpath". Example:
dpath.util.search(x, "a/b/[cd]")
* etc.

There are many variants:

* Support nested dictionaries or not
* "Path" as a string ala XPath using "/" or "." separator -versus-
don't give a direct access into nested dictionaries
* Optional default value when a key/attribute is missing
* etc.

Is there a project on PyPI which already already been battle-tested
and gained enough popularity to consider that it's now mature enough
to enter the stdlib?

The problem is that so far, I see no clear "winner" of a "standard"
API. So maybe leaving such feature on PyPI is better for now. It seems
like these projects are opinionated about what should be the "right"
behavior :-)

And yeah, there are even more advanced tools like https://glom.readthedocs.io/

As I wrote in the issue, I would prefer to leave SimpleNamespace
unchanged. If such feature lands into the stdlib, I would prefer it to
be a new class.

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XDFTMWK5727ETLVNEANWYN6A3HQN6STT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Improvement to SimpleNamespace

[Brett Cannon]

The "hey I'm a dict, but look over hear and now you can treat my like like any 
other object" duality doesn't sit well with me. I do understand the idea of 
wanting to convert something to convert JSON data into a more object-like 
access pattern, though.

So if something were to be brought in, for me the question would be whether 
it's an object that wraps a dict internally w/o re-exposing __getitem__() to 
the dict like a new veneer, or the creation of a new object that deep-copies 
the dict so it's more of an object transformation.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/UU7WWQ7IYUIALUOTQNRZMRIG6CJPCBGY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Improvement to SimpleNamespace

[Paul Moore]

On Wed, 15 Apr 2020 at 15:41, Brett Cannon  wrote:
>
> The "hey I'm a dict, but look over hear and now you can treat my like like 
> any other object" duality doesn't sit well with me. I do understand the idea 
> of wanting to convert something to convert JSON data into a more object-like 
> access pattern, though.

I don't like the duality, but I *do* like (in some situations) the
object-with-dynamic-attributes style of access. Basically, when access
is predominantly via literal strings and heavily nested,

obj["element"]["sub"]["another"]

is *far* more punctuation-heavy and (IMO) difficult to read as a
consequence, than

obj.element.sub.another

Paul
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RZQG5BIY4KJA5NIONSXOY4OG26ALZ57G/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Improvement to SimpleNamespace

[Brett Cannon]

I will also say that when I do want such a shift from JSON or dict data 
structures to a more concrete one I write my own classmethod to provide some 
error checking, so I haven't personally needed a package to do this for me.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SNHAH3MSFHYT7CSW6HIRML5YT7VNDS6L/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Improvement to SimpleNamespace

[David Mertz]

A very long time ago, I wrote an XML library (Gnosis Utilities
xml_objectify) that had this same issue, and adopted the "duality" approach
(where possible, with both dictionary and other styles also available).
JSON is sort of the new XML, and it feels like the same concern.  FWIW,
LXML explicitly adopted my approach for an optional (mostly) compatible
mode, and unlike me, they maintained that thing during the last 15 years
since mine was touched.

I think the way to think about parsing JSON needn't be "this is just a
Python dictionary once we read it", but rather that we have a custom
dict-like object that borrows JSON/Javascript semantics.  I've mentioned
that I think a new name emphasizes this.  And living in `collections`
rather than `types` also fits this purpose better.

However we get there, writing `obj.element.sub.another` is SO MUCH nicer.

On Wed, Apr 15, 2020 at 11:01 AM Paul Moore  wrote:

> On Wed, 15 Apr 2020 at 15:41, Brett Cannon  wrote:
> >
> > The "hey I'm a dict, but look over hear and now you can treat my like
> like any other object" duality doesn't sit well with me. I do understand
> the idea of wanting to convert something to convert JSON data into a more
> object-like access pattern, though.
>
> I don't like the duality, but I *do* like (in some situations) the
> object-with-dynamic-attributes style of access. Basically, when access
> is predominantly via literal strings and heavily nested,
>
> obj["element"]["sub"]["another"]
>
> is *far* more punctuation-heavy and (IMO) difficult to read as a
> consequence, than
>
> obj.element.sub.another
>
> Paul
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/RZQG5BIY4KJA5NIONSXOY4OG26ALZ57G/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NVDGT6JEDYXVYXHWGWYAE42BUQCPI5PP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Improvement to SimpleNamespace

[Ivan Pozdeev via Python-Dev]

First of all, be aware of the limitations of this approach (which will need to 
be clearly documented if we go this way):

 * It only allows valid Python identifiers -- while JSON accepts any sequence 
of Unicode characters for keys
   (http://www.ecma-international.org/publications/files/ECMA-ST/ECMA-404.pdf, 
p.5)
 * It will clash with any regular attributes of whatever type implements this 
senantic
 * It's not very scalable and will upset static code checkers and IDEs as it's 
impossible to determine if the attribute name is valid or
   provide typing hints


Now, I think it's possible to make a class that would /wrap/ an existing 
mapping.
This would completely decouple the convenience syntax from the storage 
semantics and any specific modules and types.

When writing a proof-of-concept implementation, however, I bumped into the need to distinguish which of the child objects are containers 
(thus need to be wrapped as well) and which are the leaves (thus need to be returned as is). I guess this is going to be 
application-specific. And the same problem will arise with any implementation, not only a wrapper.


On 15.04.2020 5:59, Raymond Hettinger wrote:

SimpleNamespace() is really good at giving attribute style-access. I would like 
to make that functionality available to the JSON module (or just about anything 
else that accepts a custom dict) by adding the magic methods for mappings so 
that this works:

  catalog = json.load(f, object_hook=SimpleNamespace)
  print(catalog['clothing']['mens']['shoes']['extra_wide']['quantity']) 
 # currently possible with dict()
  print(catalog.clothing.mens.shoes.extra_wide.quantity]) # 
proposed with SimpleNamespace()
  print(catalog.clothing.boys['3t'].tops.quantity   
# would also be supported

I've already seen something like this in production; however, people are having 
to write custom subclasses to do it.  This is kind of bummer because the custom 
subclasses are a pain to write, are non-standard, and are generally somewhat 
slow.  I would like to see a high-quality version this made more broadly 
available.

The core idea is keep the simple attribute access but make it easier to load 
data programmatically:

 >>> ns = SimpleNamespace(roses='red', violets='blue')
 >>> thing = input()
 sugar
 >>> quality = input()
 sweet
 >>> setattr(ns, thing, quality)# current
 >>> ns['sugar'] = 'sweet'   # proposed

If the PEP 584 __ior__ method were supported, updating a SimpleNamespace would 
be much cleaner:

   ns |= some_dict

I posted an issue on the tracker: https://bugs.python.org/issue40284 .  There 
was a suggestion to create a different type for this, but I don't see the point 
in substantially duplicating everything SimpleNamespace already does just so we 
can add some supporting dunder methods.   Please add more commentary so we can 
figure-out the best way to offer this powerful functionality.


Raymond


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JOMND56PJGRN7FQQLLCWONE5Z7R2EKXW/
Code of Conduct: http://python.org/psf/codeofconduct/
--
Regards,
Ivan
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/IEKXDY7HUUCGF4BI327URFAWV67WQJZJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Improvement to SimpleNamespace

[Gregory P. Smith]

On Wed, Apr 15, 2020 at 12:49 PM Ivan Pozdeev via Python-Dev <
python-dev@python.org> wrote:

> First of all, be aware of the limitations of this approach (which will
> need to be clearly documented if we go this way):
>
>- It only allows valid Python identifiers -- while JSON accepts any
>sequence of Unicode characters for keys (
>http://www.ecma-international.org/publications/files/ECMA-ST/ECMA-404.pdf,
>p.5)
>- It will clash with any regular attributes of whatever type
>implements this senantic
>
> Agreed, these are all problems that lead to long term code maintenance
headaches (add a new method and a bunch of existing code mysteriously
breaks).  The dict ['key'] lookup / get('key') syntax must always remain
for access to those as a result.  this is mere syntactic sugar convenience.

Unfortunately it's a slippery slope; do this just for JSON and people will
ask why it can't be done for any dict and why it isn't the default
available everywhere.

Similar to -> in Perl.

>
>-
>- It's not very scalable and will upset static code checkers and IDEs
>as it's impossible to determine if the attribute name is valid or provide
>typing hints
>
>
Not necessarily.  If you've given the JSON a schema, you can use that to
teach tools attr names and types.  otherwise anything doing such checking
should see that this is an instance of a type with a __getattr__
implemented and thus all attributes must be deemed valid.


>-
>
> --
> Now, I think it's possible to make a class that would *wrap* an existing
> mapping.
> This would completely decouple the convenience syntax from the storage
> semantics and any specific modules and types.
>
> When writing a proof-of-concept implementation, however, I bumped into the
> need to distinguish which of the child objects are containers (thus need to
> be wrapped as well) and which are the leaves (thus need to be returned as
> is). I guess this is going to be application-specific. And the same problem
> will arise with any implementation, not only a wrapper.
> On 15.04.2020 5:59, Raymond Hettinger wrote:
>
> SimpleNamespace() is really good at giving attribute style-access. I would 
> like to make that functionality available to the JSON module (or just about 
> anything else that accepts a custom dict) by adding the magic methods for 
> mappings so that this works:
>
>  catalog = json.load(f, object_hook=SimpleNamespace)
>  print(catalog['clothing']['mens']['shoes']['extra_wide']['quantity'])
>   # currently possible with dict()
>  print(catalog.clothing.mens.shoes.extra_wide.quantity]) 
> # proposed with SimpleNamespace()
>  print(catalog.clothing.boys['3t'].tops.quantity  
>  # would also be supported
>
> I've already seen something like this in production; however, people are 
> having to write custom subclasses to do it.  This is kind of bummer because 
> the custom subclasses are a pain to write, are non-standard, and are 
> generally somewhat slow.  I would like to see a high-quality version this 
> made more broadly available.
>
> The core idea is keep the simple attribute access but make it easier to load 
> data programmatically:
>
> >>> ns = SimpleNamespace(roses='red', violets='blue')
> >>> thing = input()
> sugar
> >>> quality = input()
> sweet
> >>> setattr(ns, thing, quality)# current
> >>> ns['sugar'] = 'sweet'   # proposed
>
> If the PEP 584 __ior__ method were supported, updating a SimpleNamespace 
> would be much cleaner:
>
>   ns |= some_dict
>
> I posted an issue on the tracker: https://bugs.python.org/issue40284 .  There 
> was a suggestion to create a different type for this, but I don't see the 
> point in substantially duplicating everything SimpleNamespace already does 
> just so we can add some supporting dunder methods.   Please add more 
> commentary so we can figure-out the best way to offer this powerful 
> functionality.
>
>
> Raymond
>
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to 
> python-dev-leave@python.orghttps://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/JOMND56PJGRN7FQQLLCWONE5Z7R2EKXW/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
> --
> Regards,
> Ivan
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/IEKXDY7HUUCGF4BI327URFAWV67WQJZJ/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@pyt

[Python-Dev] Re: Improvement to SimpleNamespace

[Gregory P. Smith]

On Wed, Apr 15, 2020 at 7:37 AM Victor Stinner  wrote:

> Le mer. 15 avr. 2020 à 05:02, Raymond Hettinger
>  a écrit :
> > I would like to make that functionality available to the JSON module (or
> just about anything else that accepts a custom dict) by adding the magic
> methods for mappings so that this works:
> >
> >  catalog = json.load(f, object_hook=SimpleNamespace)
> >
> print(catalog['clothing']['mens']['shoes']['extra_wide']['quantity'])
> # currently possible with dict()
> >  print(catalog.clothing.mens.shoes.extra_wide.quantity])
>  # proposed with SimpleNamespace()
> >  print(catalog.clothing.boys['3t'].tops.quantity
># would also be supported
>
> It seems like there are many projects on PyPI which provide such
> features. A few examples:
>
> * https://pypi.org/project/attrdict/ "mapping objects that allow their
> elements to be accessed both as keys and as attributes"
> * https://pypi.org/project/dictlib/ "Dictionary keys as object attributes"
> * https://pypi.org/project/objdict/ "An ObjDict supports all normal
> dict operations, but adds support for accessing and setting entries as
> attributes"
> * https://pypi.org/project/mydict/ "dict subclass which tries to act
> like JavaScript objects, so you can use the dot notation (d.foo) to
> access members of the object"
> * https://pypi.org/project/dpath/ "accessing and searching
> dictionaries via /slashed/paths ala xpath". Example:
> dpath.util.search(x, "a/b/[cd]")
> * etc.
>
> There are many variants:
>
> * Support nested dictionaries or not
> * "Path" as a string ala XPath using "/" or "." separator -versus-
> don't give a direct access into nested dictionaries
> * Optional default value when a key/attribute is missing
> * etc.
>
> Is there a project on PyPI which already already been battle-tested
> and gained enough popularity to consider that it's now mature enough
> to enter the stdlib?


+1  that was my first reaction as well.  We don't need this battery.

If you wanted to make it a more general thing, introduce a special operator
for it instead of overloading ".".  Ex: copy Perl and adopt -> as the
"lookup the name in a dictionary without typing quotes around the constant
value" operator.  (please don't!)

by the time we're saying "JSON should decode into a structured format" we
can also say very true things like "JSON data should be validated against a
schema" at which point we're ultimately going to wind up reinventing
https://googleapis.dev/python/protobuf/latest/google/protobuf/json_format.html
and equivalent other things.

-gps

The problem is that so far, I see no clear "winner" of a "standard"
> API. So maybe leaving such feature on PyPI is better for now. It seems
> like these projects are opinionated about what should be the "right"
> behavior :-)
>
> And yeah, there are even more advanced tools like
> https://glom.readthedocs.io/
>
> As I wrote in the issue, I would prefer to leave SimpleNamespace
> unchanged. If such feature lands into the stdlib, I would prefer it to
> be a new class.
>
> Victor
> --
> Night gathers, and now my watch begins. It shall not end until my death.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/XDFTMWK5727ETLVNEANWYN6A3HQN6STT/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/C5AVB5Z3SQ2LLUYKU65YLSVTLCL2XFOM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Improvement to SimpleNamespace

[Glenn Linderman]

On 4/15/2020 12:47 PM, Ivan Pozdeev via Python-Dev wrote:
When writing a proof-of-concept implementation, however, I bumped into 
the need to distinguish which of the child objects are containers 
(thus need to be wrapped as well) and which are the leaves (thus need 
to be returned as is). I guess this is going to be 
application-specific. And the same problem will arise with any 
implementation, not only a wrapper.


Do the child objects truly need to be wrapped, or just accessed?

Thanks for your comments though, they inspired a thought.

The problem with the glom syntax versus the dotted syntax is that the 
minimal syntax is bulky.


obj.abc.def.ghi versus   glom( obj, 'abc.def.ghi')

The problem with attribute syntax is the conflict with regular 
attributes, and the limitation of valid identifier characters.  Glom 
probably doesn't have these problems.


So my new thought is that we need a new syntax.  The minimal existing 
syntax might be


obj._('abc.def.ghi')     or maybe   obj['abc.def.ghi']

although the latter would conflict with regular dict lookups, but obj 
isn't a regular dict, so that might not matter.



A new syntax might be:

obj.'abc.def.ghi'


In any of these syntaxes, one could enhance it to use variables in some 
spots similarly to f-strings (but from the beginning, so no 'f' would be 
required)


foo = 'def'
obj.'abc.{foo}.ghi'

would access the same item as previous examples.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3A5V62LY3DAPAREZMB7MVRYX4432NR7F/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Improvement to SimpleNamespace

[Ivan Pozdeev via Python-Dev]

On 16.04.2020 0:34, Glenn Linderman wrote:

On 4/15/2020 12:47 PM, Ivan Pozdeev via Python-Dev wrote:
When writing a proof-of-concept implementation, however, I bumped into the need to distinguish which of the child objects are containers 
(thus need to be wrapped as well) and which are the leaves (thus need to be returned as is). I guess this is going to be 
application-specific. And the same problem will arise with any implementation, not only a wrapper.


Do the child objects truly need to be wrapped, or just accessed?


They need to be wrapped on the fly, to support nested  ".key" accesses.



Thanks for your comments though, they inspired a thought.

The problem with the glom syntax versus the dotted syntax is that the minimal 
syntax is bulky.

obj.abc.def.ghi versus   glom( obj, 'abc.def.ghi')

The problem with attribute syntax is the conflict with regular attributes, and the limitation of valid identifier characters. Glom 
probably doesn't have these problems.


"Glom syntax" still excludes the delimiter, whatever it is, from use in keys. 
So it's still a further limitation compared to the JSON spec.



So my new thought is that we need a new syntax.  The minimal existing syntax 
might be

obj._('abc.def.ghi')     or maybe   obj['abc.def.ghi']

although the latter would conflict with regular dict lookups, but obj isn't a 
regular dict, so that might not matter.


A new syntax might be:

obj.'abc.def.ghi'


In any of these syntaxes, one could enhance it to use variables in some spots similarly to f-strings (but from the beginning, so no 'f' 
would be required)


foo = 'def'
obj.'abc.{foo}.ghi'

would access the same item as previous examples.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3A5V62LY3DAPAREZMB7MVRYX4432NR7F/
Code of Conduct: http://python.org/psf/codeofconduct/
--
Regards,
Ivan

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/76BY73UZRU2QUTEMDQDD6J4TIVSOWKJ7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Improvement to SimpleNamespace

[Glenn Linderman]

On 4/15/2020 2:57 PM, Ivan Pozdeev via Python-Dev wrote:


On 16.04.2020 0:34, Glenn Linderman wrote:

On 4/15/2020 12:47 PM, Ivan Pozdeev via Python-Dev wrote:
When writing a proof-of-concept implementation, however, I bumped 
into the need to distinguish which of the child objects are 
containers (thus need to be wrapped as well) and which are the 
leaves (thus need to be returned as is). I guess this is going to be 
application-specific. And the same problem will arise with any 
implementation, not only a wrapper.


Do the child objects truly need to be wrapped, or just accessed?


They need to be wrapped on the fly, to support nested  ".key" accesses.


That sounds like an implementation detail... that is one possible 
implementation.

The parent object could alternately look inside, it seems.




Thanks for your comments though, they inspired a thought.

The problem with the glom syntax versus the dotted syntax is that the 
minimal syntax is bulky.


obj.abc.def.ghi versus   glom( obj, 'abc.def.ghi')

The problem with attribute syntax is the conflict with regular 
attributes, and the limitation of valid identifier characters. Glom 
probably doesn't have these problems.


"Glom syntax" still excludes the delimiter, whatever it is, from use 
in keys. So it's still a further limitation compared to the JSON spec.


That could be handled with escaping, when necessary.





So my new thought is that we need a new syntax.  The minimal existing 
syntax might be


obj._('abc.def.ghi')     or maybe   obj['abc.def.ghi']

although the latter would conflict with regular dict lookups, but obj 
isn't a regular dict, so that might not matter.



A new syntax might be:

obj.'abc.def.ghi'


In any of these syntaxes, one could enhance it to use variables in 
some spots similarly to f-strings (but from the beginning, so no 'f' 
would be required)


foo = 'def'
obj.'abc.{foo}.ghi'

would access the same item as previous examples.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3A5V62LY3DAPAREZMB7MVRYX4432NR7F/

Code of Conduct: http://python.org/psf/codeofconduct/
--
Regards,
Ivan

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/76BY73UZRU2QUTEMDQDD6J4TIVSOWKJ7/

Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BG3MJYLHGS4E4QDRRPSOJHXBEBPSHRQC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Improvement to SimpleNamespace

[Nathaniel Smith]

On Wed, Apr 15, 2020 at 2:59 PM Ivan Pozdeev via Python-Dev
 wrote:
> "Glom syntax" still excludes the delimiter, whatever it is, from use in keys. 
> So it's still a further limitation compared to the JSON spec.

Glom does let you be specific about the exact lookup keys if you want,
to handle keys that contain embedded periods, or non-string keys. The
syntax looks like:

from glom import glom, Path
glom(obj, Path("a", "b.c", 2))

https://glom.readthedocs.io/en/latest/api.html#specifier-types

For a simple case like this t's a bit wordier than obj["a"]["b.c"][2],
but OTOH you get better error message on failed lookups,
null-coalescing support by using default=, etc.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/32Q7H5LLES3C2WUIYWTPICXGWWWP4UQU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Improvement to SimpleNamespace

[Eric Snow]

On Wed, Apr 15, 2020 at 2:12 AM Serhiy Storchaka  wrote:
> Then it obviously should be different class than SimpleNamespace. There
> are too much differences between them, and they are used in different
> circumstances.

+1

Keep in mind that I added SimpleNamespace when implementing PEP 421,
to use for the new "sys.implementation".  The whole point was to keep
it simple, as the docs [1] suggest.

-eric

[1] https://docs.python.org/3/library/types.html#types.SimpleNamespace
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/WOBJYHGZGN2IPFTJ2IVCTGJILQVFEFC5/
Code of Conduct: http://python.org/psf/codeofconduct/