Yaml serializer output

[Sergey Fursov]

Hi there,

Before opening a ticket I would like to discuss my problem here.

Django docs say:

```
YAML¶ <https://docs.djangoproject.com/en/4.0/topics/serialization/#yaml> 

YAML serialization looks quite similar to JSON. The object list is 
serialized as a sequence mappings with the keys “pk”, “model” and “fields”. 
Each field is again a mapping with the key being name of the field and the 
value the value:
- fields: {expire_date: !!timestamp '2013-01-16 08:16:59.844560+00:00'} 
model: sessions.session pk: 4b678b301dfd8a4e0dad910de3ae245b 
```
https://docs.djangoproject.com/en/4.0/topics/serialization/#yaml

But I cannot find a way to produce the same output. Both available to me 
environments (MacOS and Debian linux inside a Docker container) product a 
simpler output, like

```
>> obj = core.ModelA.objects.get(id=1)
>> print(serializers.serialize('yaml', [obj]))
- model: core.modela
  pk: 1
  fields:
created: 2017-02-05 16:11:44.310613+00:00
 other fields ...
```

I've found this after running full Django test suite when 6 test from 
django/tests/timezones/tests.py failed. Example failure:

```
FAIL: test_naive_datetime_with_microsecond 
(timezones.tests.SerializationTests)
--
Traceback (most recent call last):
  File "/Users/geyser/coding/opensource/django/tests/timezones/tests.py", 
line 671, in test_naive_datetime_with_microsecond
self.assert_yaml_contains_datetime(data, "2011-09-01 13:20:30.405060")
  File "/Users/geyser/coding/opensource/django/tests/timezones/tests.py", 
line 625, in assert_yaml_contains_datetime
self.assertRegex(yaml, r"\n  fields: {dt: !(!timestamp)? '%s'}" % 
re.escape(dt))
AssertionError: Regex didn't match: "\\n  fields: {dt: !(!timestamp)? 
'2011\\-09\\-01\\ 13:20:30\\.405060'}" not found in '- model: 
timezones.event\n  pk: null\n  fields:\ndt: 2011-09-01 
13:20:30.405060\n'
```

I've tried to install several different pyyaml versions (from 6.0 down to 
5.1.2).

Do I miss something or it is a valid issue for opening a ticket?

Thanks,
Sergey Fursov

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1dfa6764-a63d-4233-be5d-2f0a6bdeecdan%40googlegroups.com.

Re: Yaml serializer output

[Sergey Fursov]

Please, disregard the part about failing tests, I had an old version 
locally without this commit applied 
https://github.com/django/django/commit/a57c783dd4e6dc73847081221827a1902eede88b
Anyway, I think it might be confusing (at least it is confusing to me) that 
by default yaml serializer produces a different output than provided in docs

воскресенье, 26 декабря 2021 г. в 22:21:42 UTC+3, Sergey Fursov: 

> Hi there,
>
> Before opening a ticket I would like to discuss my problem here.
>
> Django docs say:
>
> ```
> YAML¶ <https://docs.djangoproject.com/en/4.0/topics/serialization/#yaml> 
>
> YAML serialization looks quite similar to JSON. The object list is 
> serialized as a sequence mappings with the keys “pk”, “model” and “fields”. 
> Each field is again a mapping with the key being name of the field and the 
> value the value:
> - fields: {expire_date: !!timestamp '2013-01-16 08:16:59.844560+00:00'} 
> model: sessions.session pk: 4b678b301dfd8a4e0dad910de3ae245b 
> ```
> https://docs.djangoproject.com/en/4.0/topics/serialization/#yaml
>
> But I cannot find a way to produce the same output. Both available to me 
> environments (MacOS and Debian linux inside a Docker container) product a 
> simpler output, like
>
> ```
> >> obj = core.ModelA.objects.get(id=1)
> >> print(serializers.serialize('yaml', [obj]))
> - model: core.modela
>   pk: 1
>   fields:
> created: 2017-02-05 16:11:44.310613+00:00
>  other fields ...
> ```
>
> I've found this after running full Django test suite when 6 test from 
> django/tests/timezones/tests.py failed. Example failure:
>
> ```
> FAIL: test_naive_datetime_with_microsecond 
> (timezones.tests.SerializationTests)
> --
> Traceback (most recent call last):
>   File "/Users/geyser/coding/opensource/django/tests/timezones/tests.py", 
> line 671, in test_naive_datetime_with_microsecond
> self.assert_yaml_contains_datetime(data, "2011-09-01 13:20:30.405060")
>   File "/Users/geyser/coding/opensource/django/tests/timezones/tests.py", 
> line 625, in assert_yaml_contains_datetime
> self.assertRegex(yaml, r"\n  fields: {dt: !(!timestamp)? '%s'}" % 
> re.escape(dt))
> AssertionError: Regex didn't match: "\\n  fields: {dt: !(!timestamp)? 
> '2011\\-09\\-01\\ 13:20:30\\.405060'}" not found in '- model: 
> timezones.event\n  pk: null\n  fields:\ndt: 2011-09-01 
> 13:20:30.405060\n'
> ```
>
> I've tried to install several different pyyaml versions (from 6.0 down to 
> 5.1.2).
>
> Do I miss something or it is a valid issue for opening a ticket?
>
> Thanks,
> Sergey Fursov
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/6793d234-92f5-46fa-bc19-3996626dc456n%40googlegroups.com.

Re: Yaml serializer output

[Sergey Fursov]

Sorry, I forgot to mention here the PR I made -
https://github.com/django/django/pull/15242

On Tue, Dec 28, 2021, 14:13 'Adam Johnson' via Django developers
(Contributions to Django itself)  wrote:

> I've made a PR to update the docs:
> https://github.com/django/django/pull/15250
>
> On Mon, 27 Dec 2021 at 08:13, Sergey Fursov  wrote:
>
>> Please, disregard the part about failing tests, I had an old version
>> locally without this commit applied
>> https://github.com/django/django/commit/a57c783dd4e6dc73847081221827a1902eede88b
>> Anyway, I think it might be confusing (at least it is confusing to me)
>> that by default yaml serializer produces a different output than provided
>> in docs
>>
>> воскресенье, 26 декабря 2021 г. в 22:21:42 UTC+3, Sergey Fursov:
>>
>>> Hi there,
>>>
>>> Before opening a ticket I would like to discuss my problem here.
>>>
>>> Django docs say:
>>>
>>> ```
>>> YAML¶ <https://docs.djangoproject.com/en/4.0/topics/serialization/#yaml>
>>>
>>> YAML serialization looks quite similar to JSON. The object list is
>>> serialized as a sequence mappings with the keys “pk”, “model” and “fields”.
>>> Each field is again a mapping with the key being name of the field and the
>>> value the value:
>>> - fields: {expire_date: !!timestamp '2013-01-16 08:16:59.844560+00:00'}
>>> model: sessions.session pk: 4b678b301dfd8a4e0dad910de3ae245b
>>> ```
>>> https://docs.djangoproject.com/en/4.0/topics/serialization/#yaml
>>>
>>> But I cannot find a way to produce the same output. Both available to me
>>> environments (MacOS and Debian linux inside a Docker container) product a
>>> simpler output, like
>>>
>>> ```
>>> >> obj = core.ModelA.objects.get(id=1)
>>> >> print(serializers.serialize('yaml', [obj]))
>>> - model: core.modela
>>>   pk: 1
>>>   fields:
>>> created: 2017-02-05 16:11:44.310613+00:00
>>>  other fields ...
>>> ```
>>>
>>> I've found this after running full Django test suite when 6 test from
>>> django/tests/timezones/tests.py failed. Example failure:
>>>
>>> ```
>>> FAIL: test_naive_datetime_with_microsecond
>>> (timezones.tests.SerializationTests)
>>> --
>>> Traceback (most recent call last):
>>>   File
>>> "/Users/geyser/coding/opensource/django/tests/timezones/tests.py", line
>>> 671, in test_naive_datetime_with_microsecond
>>> self.assert_yaml_contains_datetime(data, "2011-09-01
>>> 13:20:30.405060")
>>>   File
>>> "/Users/geyser/coding/opensource/django/tests/timezones/tests.py", line
>>> 625, in assert_yaml_contains_datetime
>>> self.assertRegex(yaml, r"\n  fields: {dt: !(!timestamp)? '%s'}" %
>>> re.escape(dt))
>>> AssertionError: Regex didn't match: "\\n  fields: {dt: !(!timestamp)?
>>> '2011\\-09\\-01\\ 13:20:30\\.405060'}" not found in '- model:
>>> timezones.event\n  pk: null\n  fields:\ndt: 2011-09-01
>>> 13:20:30.405060\n'
>>> ```
>>>
>>> I've tried to install several different pyyaml versions (from 6.0 down
>>> to 5.1.2).
>>>
>>> Do I miss something or it is a valid issue for opening a ticket?
>>>
>>> Thanks,
>>> Sergey Fursov
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-developers+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-developers/6793d234-92f5-46fa-bc19-3996626dc456n%40googlegroups.com
>> <https://groups.google.com/d/msgid/django-developers/6793d234-92f5-46fa-bc19-3996626dc456n%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAMyDDM1JV95MzE0cOziaYQ17kMMDEBTCJBkTaPkphZQFqdaeiw%40mail.gmail.com
> <https://groups.google.com/d/msgid/django-developers/CAMyDDM1JV95MzE0cOziaYQ17kMMDEBTCJBkTaPkphZQFqdaeiw%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAG-zk-BGtwLNa2gA_NHwonAEoxie3T4uVN6GF%3DrKQANwJQqxhw%40mail.gmail.com.

Re: PostgreSQL Partial Indexes package

[Sergey Fursov]

Hi everyone,

In our company, we also use the ability to define our own indexes, but with
the current implementation of base Index class, we basically need to copy
several methods entirely from it.
What do we want is implement UNIQUE indexes with the UPPER function wrapper
around field names [1].
This provides two benefits to us:
1) case-insensitive uniqueness of the field or the fields combination
2) quick filtering using iexact lookup, which uses UPPER() LIKE
 statement in SQL (at least for the Postgres)

I think, adding more flexibility in current indexes build logic would help
to implement and further support custom indexes implementations (like
django-partial-index )
if we are not going to merge them in Django core or contrib.

[1] https://gist.github.com/GeyseR/363b4b9d2d1cd8c91d51ebfe9369a7ac


пт, 20 окт. 2017 г. в 14:37, Ashley Waite :

> I did a similar thing (but only tested in postgres) a while back, using
> Q's / filter clause to generate the where clause.
>
> Perhaps with our powers combined we're close to a generalised
> implementation?
>
>
> https://github.com/ashleywaite/django-more/blob/master/django_more/indexes.py
>
>
>
> On Saturday, October 7, 2017 at 7:56:00 PM UTC+11, Mattias Linnap wrote:
>>
>> Hi django-developers,
>>
>> I have written a package that implements PostgreSQL and SQLite partial
>> indexes on top of the new class-based indexes:
>> https://github.com/mattiaslinnap/django-partial-index
>> The most common use case is partial unique constraints, but I have a few
>> projects where non-unique partial indexes have turned out useful as well.
>>
>> I have a few questions on how to continue with this:
>>
>> 1. Right now the "where condition" expression is provided as a string,
>> and has to be different for PostgreSQL and SQLite in some common cases (for
>> example boolean literals). Is there a good abstraction for SQL expressions
>> somewhere in Django internals that I could use instead, something similar
>> to Q-expressions perhaps? In particular, to add validate_unique() support
>> to ModelForms, I would need to be able to extract all fields that are
>> mentioned in the where condition.
>> 2. I've seen mentions of a "check constraints" support being in
>> development (https://github.com/django/django/pull/7615). Will that
>> include partial unique constraints, or is just for the per-column checks?
>> 3. If separate, then it would be nice to one day get partial indexes
>> merged into the contrib.postgres package. Do you have any suggestions on
>> what needs to happen before that - more test coverage, more contributors,
>> more users, or similar?
>>
>> Best,
>>
>> Mattias
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/84f4431a-cdab-43c7-b96b-de4d2eb9805b%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAG-zk-CJnbuQASEmey6%2BWEewVrn8TPhxpUfv8TfzQEakAgwM%2Bw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Exceptions in model._meta._property_names with some 3rd-pty libraries

[Sergey Fursov]

Hi everyone,

I just want to mention here that in django-jsonfield repo proposed solution
to solve this problem on third party side:
https://github.com/dmkoch/django-jsonfield/issues/189. Maybe this is the
better way.

Thanks,
Sergey

2017-06-03 0:02 GMT+03:00 Zack Voase :

> Hi all,
>
> I'm encountering exceptions in a number of popular third-party Django
> libraries when upgrading from 1.8 to 1.11. The code path is typically
> `MyModel.objects.get_or_create(...)`, which causes
> `model._meta._property_names` to be checked (to make sure we're not
> querying/setting a property). The problem is, `_property_names` is
> implemented as follows:
>
> # in django/db/models/options.py:
> def _property_names(self):
> return frozenset({
>attr for attr in
>dir(self.model) if isinstance(getattr(self.model, attr), property)
> })
>
> The problem is when a custom field installs a field descriptor on the
> model class (during `contribute_to_class()`), with a `__get__()` method
> like this:
>
> class SomeFieldDescriptor(object):
> # ...
> def __get__(self, instance, type=None):
> if instance is None:
> raise AttributeError("Can only be accessed via an instance.")
> # ...
>
> Libraries which install such descriptors include [django-fsm](
> https://github.com/kmmbvnr/django-fsm/blob/2d2eaee/django_fsm/__init__.
> py#L225) and [django-jsonfield](https://github.com/dmkoch/django-
> jsonfield/blob/2.0.1/jsonfield/subclassing.py#L35). I think the problem
> is that we can't assume that all results of `dir(model_class)` can be
> accessed via `getattr(model_class, attr)` without exception; indeed, the
> Python docs state (c.f. https://docs.python.org/2/
> library/functions.html#dir):
>
> > Because dir() is supplied primarily as a convenience for use at an
> interactive prompt, it tries to supply an interesting set of names more
> than it tries to supply a rigorously or consistently defined set of names,
> and its detailed behavior may change across releases.
>
> A potential fix would be to adjust the definition of `_property_names`
> like so:
>
> def _property_names(self):
> attrs = []
> for attr in dir(self.model):
> try:
> if isinstance(getattr(self.model, attr), property):
> attrs.append(attr)
> except AttributeError:
> pass
> return frozenset(attrs)
>
> Does this seem like a good solution, or even a problem worth solving?
>
> Thanks!
> -- Zack
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/django-developers/ec0e2506-4c4b-441f-b005-
> 5623d6521ae3%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAG-zk-BDvJOQfhgNBFBpRCBuPkB7MTtJCYLp6D1jxgbppMdr%2BQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.