In order to release the code, I need to provide a sustainment plan per our open source committee. Which is why I was reaching out. I can share some of the challenges conforming Snowflake to the SQL model Django uses.
It sounds like we may have had similar experiences writing backends, but possibly we differ in completeness. I was able to write a working backend that met all of our application needs in about 3 weeks. Conforming to the unit tests is another story. We pass ~90% of the tests, but that last 10% will take a lot more effort to complete. For reference, we're using the snowflake-connector-python module, which implements PEP 249, v2.0. Snowflake does not support lastrowid. So, we grab the last ID inserted with a 'SELECT MAX(pk_name) FROM table_name'. This is obviously prone to failure. Assigning an ID during the INSERT would provide similar results on all backends. The feature flag `supports_transactions` really means `supports_nested_transactions`. Snowflake supports a single level of transaction, BEGIN + (ROLLBACK|COMMIT). Multiple BEGINS contribute to the current (only) transaction. Since I have to set this flag to False, no transactions are used, even ones that are supported and testing grinds to a crawl with all of the table truncations and repopulation. Since Django *normally* operates in auto-commit mode, this isn't a huge deal. Snowflake also doesn't support save points, but the feature flag seems to do what is expected when disabled. Snowflake does not support column references without a FROM or JOIN clause. I've only encountered this used in the unit tests. I've been able to work around most of the function differences by adding as_snowflake methods to the appropriate classes. There are a few cases like JSON_OBJECT that don't translate well, which work, but not entirely as expected. A search for 'connection.vendor == ' in the core code shows one example of where core backends are given privileged access to core Django inner workings that 3rd party backends don't. Please take these comments as observations, not complaints. I understand project evolution and the resistance to change something that doesn't seem broken. Maybe keep some of these thoughts in mind when a change is made to the core backends or the compiler and consider how other backends will need to implement that new feature. Scott On Saturday, January 23, 2021 at 10:36:38 AM UTC-5 f.apo...@gmail.com wrote: > On Saturday, January 23, 2021 at 12:41:35 AM UTC+1 t...@tomforb.es wrote: > >> > Honestly, I don't think we'd release this as a third party package. We >> have had to code around a lot of core db specific code to get Snow SQL to >> work >> >> This is good feedback that is valuable for us to hear. Even if you don’t >> release it as a fully maintained package just understanding the pain points >> you encountered while developing this is great feedback that can help us >> improve. >> >> Would you be willing to share specific code snippets here, or just >> general areas that you found challenging while developing this backend? >> > > +1, if it were possible to show the full code (even if you are not going > to maintain it in the longrun) would be very much helpful to improve. > > Like Tim I wrote one or two database backends in the past (informix and > the likes) and it was not to bad. That said, mine where relatively easy to > implement due to their closeness to existing database and (somehwat) > conformance with SQL standards. I'd love to see the code for snowflake and > maybe we can identify a few areas that would need improvement. > > Cheers, > Florian > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/768b428d-8d7d-462a-b925-484a6e681b2bn%40googlegroups.com.
