Sorry, I meant to write Scott, not Tim. I shouldn't write emails late at night.
- Taylor On Tuesday, April 20, 2021 at 10:29:46 PM UTC-7 Taylor wrote: > Hey Everyone, > > Sorry to open up an old thread. > > Tim - were you ever able to open source your Snowflake backend? We would > love to use it and even devote resources (developers or funding for > contractors) to improving it and getting the tests passing. At Cedar, we > were planning on creating our own Snowflake backend, but it would be great > to not duplicate work here. What are your thoughts? > > Best, > Taylor > > On Wednesday, January 27, 2021 at 1:08:13 AM UTC-8 f.apo...@gmail.com > wrote: > >> Hi Scott, >> >> Thank you for your response, this is very helpful. >> >> On Tuesday, January 26, 2021 at 11:38:18 PM UTC+1 foug...@apps.disney.com >> wrote: >> >>> Snowflake does not support lastrowid. So, we grab the last ID inserted >>> with a 'SELECT MAX(pk_name) FROM table_name'. This is obviously prone to >>> failure. Assigning an ID during the INSERT would provide similar results >>> on all backends. >>> >> >> Uffff, the 'SELECT MAX()' is not going to fly well, you are right. >> Assigning an ID during INSERT has it's own problems. In postgresql it would >> be possible because you could just select the next value from the created >> sequence (more or less), but with IDENTITY columns this might get harder. I >> do not think there is a sensible way to do this in MySQL at all. While >> lastrowid support is a nice to have, Django should work (mostly?) without >> it: >> https://github.com/django/django/blob/464a4c0c59277056b5d3c1132ac1b4c6085aee08/django/db/models/sql/compiler.py#L1372-L1387 >> >> -- the requirement here is that your database is at least able to return >> values after insert. From the looks of it, it does not I think? Or >> differently put: Which ways does snowflake offer to get an ID? Solely by >> providing it directly into insert? >> >> The feature flag `supports_transactions` really means >>> `supports_nested_transactions`. Snowflake supports a single level of >>> transaction, BEGIN + (ROLLBACK|COMMIT). Multiple BEGINS contribute to the >>> current (only) transaction. Since I have to set this flag to False, no >>> transactions are used, even ones that are supported and testing grinds to a >>> crawl with all of the table truncations and repopulation. Since Django >>> *normally* operates in auto-commit mode, this isn't a huge deal. >>> Snowflake also doesn't support save points, but the feature flag seems to >>> do what is expected when disabled. >>> >> >> Hu, which database support nested BEGIN? As far as I am aware Django does >> never nest BEGINs -- do you have an example for me? I'd very much like to >> fix this. From a quick glance at the code we only start a transaction if we >> are not already in one: >> https://github.com/django/django/blob/master/django/db/transaction.py#L196-L208 >> >> Snowflake does not support column references without a FROM or JOIN >>> clause. I've only encountered this used in the unit tests. >>> >> >> Ok, see below. >> >> I've been able to work around most of the function differences by adding >>> as_snowflake methods to the appropriate classes. There are a few cases >>> like JSON_OBJECT that don't translate well, which work, but not entirely as >>> expected. >>> >> >> Perfect, this sounds as if our extension system works as intended in this >> area. >> >> A search for 'connection.vendor == ' in the core code shows one example >>> of where core backends are given privileged access to core Django inner >>> workings that 3rd party backends don't. >>> >> >> We have been working to get rid of those: >> https://github.com/django/django/commit/275dd4ebbabbbe758c7219a3d666953d5a7b072f#diff-69f332030b6f25f8f4d95842548d847139ee2cc163aef18f1c8d83b90f3f9e5f >> >> -- This is solely in 3.2, but Tim can suggest a workaround for earlier >> versions (he was doing something similar in his cockroachdb tests before >> 3.2). >> >> Please take these comments as observations, not complaints. I understand >>> project evolution and the resistance to change something that doesn't seem >>> broken. Maybe keep some of these thoughts in mind when a change is made to >>> the core backends or the compiler and consider how other backends will need >>> to implement that new feature. >>> >> >> No offense taken at all. If somehow possible I'd like to encourage you to >> work with us on your pain points. I think at least some of those are >> addressed or at least addressable. I am happy to offer code and fixes, but >> I'll need a few more details especially wrt transaction handling. >> >> Cheers, >> Florian >> > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/9d2c1410-e873-4efc-95e2-373f7b42faa6n%40googlegroups.com.
