Adding a script positional argument to dbshell
In Adam's blog post https://adamj.eu/tech/2022/06/29/run-a-django-migration-by-hand/, he shows a pattern I've used in the past. Would it be useful to support passing an optional positional argument to dbshell that is then passed to the dbshell program appropriately? e.g., with @filename for Oracle sqlplus or directly to mysql? I suppose that might introduce new code to DatabaseOperations, but not too much. I think only Oracle's DatabaseOperations would be different. I've had to extend Django to run some old PL/SQL via management commands. This helped me with my 16 year old database to replace scripts that used combinations of bash, ksh, sh, expect, perl, sqlplus, and PL/SQL with just two three technologies - namely Python, Django, and PL/SQL. However, since there are 20k lines of PL/SQL and 25k lines of Python, making the PL/SQL go away is not yet a priority. I've done it basically by using code like this in a management command: https://gist.github.com/danizen/a3b5e3f8514be90b796f298dfb52f99e If you look at the "run_report" method of the basereport.py, you'll see that I am using os.pipe() and os.dup2() to enable passing the output of a template to dbshell. This is in case there is any sqlplus reliance in these old reports. In my case, it will be better to assure that all of this report code can run in management commands or directly via cursor execute - and so I haven't proposed this for my use case. The one that Adam proposes makes a much better use case for this additional positional argument to dbshell. Has this been discussed before? -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/aeb4ef0d-e8d6-46c6-9aaa-167abd5de3e5n%40googlegroups.com.
AWS RDS Proxy and session pinning
Hi Django developers, At the National Library of Medicine we are doing a lot of Django and AWS. For some of the applications with heavier traffic, we are using RDS Proxy within AWS. RDS Proxy is sort of like a managed version of pgbouncer or pg-pool-II - it must be rather AWS customized because of the way it manages authentication. Anyway, we found some problems for two reasons: - Our DBAs make the default timezone of the database UTF-8 is AWS CloudFormation to create the databases. - When "ensure_timezone" runs and sets the timezone, that database session is "pinned", so that it will not be shared. Since we already have our own PostgreSQL backend for a couple of other reasons, I just wrote a version of ensure_timezone which fails loudly if the timezone is not UTF-8 rather than set the timezone. I probably should have discussed the issue on this list, and I am remedying this. The failure to manage database connections with RDS Proxy is a pretty severe error, and I am wondering what the community thinks about a connection specific setting about how the timezone should be handled? For lower traffic applications, SET TIMEZONE is fine. For higher traffic applications, raising ImproperlyConfigured is better. Does the group think that a Postgreql specific setting/option should be implemented to prevent session pinning? -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/bf7ac5e1-05e1-4d5b-a2a5-8d1dda04b66bn%40googlegroups.com.
Re: AWS RDS Proxy and session pinning
Sorry - lest I miscommunicate - the DBAs make the default equivalent to EST5EDT, rather than UTC. Django team leads (me and David), want USE_TZ = True to be on for all the applications, and because of this the postgresql backend will issue SET TIMEZONE UTC in django/db/backends/postgreql/base.py:209 (ensure_timezone). At the same time, different applications and frameworks have different opinions and the DBAs try to satisfy us all, but can forget. This issue of session pinning with connection pooling servers (such as RDS Proxy) may be more general, and it may be good to not always set the timezone, but I want to discuss before filing an issue. On Monday, January 31, 2022 at 5:24:46 PM UTC-5 dans...@gmail.com wrote: > Hi Django developers, > > At the National Library of Medicine we are doing a lot of Django and AWS. > For some of the applications with heavier traffic, we are using RDS Proxy > within AWS. RDS Proxy is sort of like a managed version of pgbouncer or > pg-pool-II - it must be rather AWS customized because of the way it manages > authentication. > > Anyway, we found some problems for two reasons: > >- Our DBAs make the default timezone of the database UTF-8 is AWS >CloudFormation to create the databases. >- When "ensure_timezone" runs and sets the timezone, that database >session is "pinned", so that it will not be shared. > > Since we already have our own PostgreSQL backend for a couple of other > reasons, I just wrote a version of ensure_timezone which fails loudly if > the timezone is not UTF-8 rather than set the timezone. I probably should > have discussed the issue on this list, and I am remedying this. > > The failure to manage database connections with RDS Proxy is a pretty > severe error, and I am wondering what the community thinks about a > connection specific setting about how the timezone should be handled? For > lower traffic applications, SET TIMEZONE is fine. For higher traffic > applications, raising ImproperlyConfigured is better. > > Does the group think that a Postgreql specific setting/option should be > implemented to prevent session pinning? > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/ecc0c527-465a-45c7-b6ff-3c35c56824bfn%40googlegroups.com.
Re: AWS RDS Proxy and session pinning
We already do that - our backend has some other work to do since we use AWS Secrets Manager to store database passwords, and also the hostname, so that our settings.DATABASES does not contain the PASSWORD but must contain a SECRET_ID. The question is whether this is worth promoting since AWS RDS Proxy (and other similar proxy managers) are quite common. On Monday, January 31, 2022 at 6:35:00 PM UTC-5 Adam Johnson wrote: > Hi! > > Good to hear about the use of RDS Proxy. I have considered looking at it > to help scaling. > > You should be able to bypass the timezone check yourself with a little > subclassing. You can implement this yourself with a subclassed database > backend like so: > https://docs.djangoproject.com/en/4.0/ref/databases/#subclassing-the-built-in-database-backends > > . Use it to replace the ensure_timezone() method with appropriate logic. > > That may be all that's required. It would be good to know if that works > before considering changing Django. > > Thanks, > > Adam > > On Mon, Jan 31, 2022 at 10:30 PM dans...@gmail.com > wrote: > >> Sorry - lest I miscommunicate - the DBAs make the default equivalent to >> EST5EDT, rather than UTC. Django team leads (me and David), want USE_TZ = >> True to be on for all the applications, and because of this the postgresql >> backend will issue SET TIMEZONE UTC in >> django/db/backends/postgreql/base.py:209 (ensure_timezone). At the same >> time, different applications and frameworks have different opinions and the >> DBAs try to satisfy us all, but can forget. This issue of session pinning >> with connection pooling servers (such as RDS Proxy) may be more general, >> and it may be good to not always set the timezone, but I want to discuss >> before filing an issue. >> >> On Monday, January 31, 2022 at 5:24:46 PM UTC-5 dans...@gmail.com wrote: >> >>> Hi Django developers, >>> >>> At the National Library of Medicine we are doing a lot of Django and >>> AWS. For some of the applications with heavier traffic, we are using RDS >>> Proxy within AWS. RDS Proxy is sort of like a managed version of pgbouncer >>> or pg-pool-II - it must be rather AWS customized because of the way it >>> manages authentication. >>> >>> Anyway, we found some problems for two reasons: >>> >>>- Our DBAs make the default timezone of the database UTF-8 is AWS >>>CloudFormation to create the databases. >>>- When "ensure_timezone" runs and sets the timezone, that database >>>session is "pinned", so that it will not be shared. >>> >>> Since we already have our own PostgreSQL backend for a couple of other >>> reasons, I just wrote a version of ensure_timezone which fails loudly if >>> the timezone is not UTF-8 rather than set the timezone. I probably should >>> have discussed the issue on this list, and I am remedying this. >>> >>> The failure to manage database connections with RDS Proxy is a pretty >>> severe error, and I am wondering what the community thinks about a >>> connection specific setting about how the timezone should be handled? For >>> lower traffic applications, SET TIMEZONE is fine. For higher traffic >>> applications, raising ImproperlyConfigured is better. >>> >>> Does the group think that a Postgreql specific setting/option should be >>> implemented to prevent session pinning? >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "Django developers (Contributions to Django itself)" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to django-develop...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/django-developers/ecc0c527-465a-45c7-b6ff-3c35c56824bfn%40googlegroups.com >> >> <https://groups.google.com/d/msgid/django-developers/ecc0c527-465a-45c7-b6ff-3c35c56824bfn%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/654dc44a-98d3-422a-aa45-fc719eae0da6n%40googlegroups.com.
Newer constraint declarations and Foreign Keys
Hi guys, I remember that there is a new way to declare constraints in class Meta on a model, and that this is preferable for unique_together constraints. I've long wanted a way with Django to have database foreign key constraints cascade in the database rather than via Django. Is there now a way to do this? Thanks! -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/3e28c4b0-896d-49fa-b768-a30c3db96c23n%40googlegroups.com.
