Re: Proposal for new field type: CompositeField

[VernonCole]

Let me give a hearty YES to this proposal.  I have been told that a
composite field, like this, is a violation of good database design,
and that a 'true' relational database cannot have this feature.  That
is why the guy who designed the database I helped implement in 1982
did not call his design 'relational'. He wanted the capability and
added it.  I still use that ancient database language (RDM -- you've
never heard of it -- worked on PDP-11s) today and still love the
feature. It's just darn handy.
  To the 'relational database' purists, my reply is this:
You are already using SQL, which is NOT a relational database
language. Codd himself said so. Look it up.
--
Vernon Cole

On Dec 24, 4:53 pm, Jerome Leclanche  wrote:
> I like it, always wondered if there was a way to do this in the core. Hope
> the patch gets through.
>
> J. Leclanche / Adys
>
> On Fri, Dec 25, 2009 at 1:22 AM, Michael P. Jung  wrote:
>
> > On 2009-12-24 19:45, Stephen Crosby wrote:
> > > In your address example, I'm not sure what advantage this
> > > CompositeField has over just creating an address model/table and
> > > using a regular foreign key to reference it. You mentioned the need
> > > to create a field composed of other fields which to me sounds a lot
> > > like a regular one-to-many relationship between two normalized
> > > tables. Can you explain the reasons you decided to do it this way and
> > > the advantages you see?
>
> > Actually the whole idea of this is to store the data denormalized. I
> > have to agree that the given example makes little to no sense, as it is
> > lacking a sane context. In one of our applications the AddressField is
> > used in several models, but the stored addresses are never shared, thus
> > normalizing them brings no benefit. Contrary, a separate table would
> > cause more complex and slower queries.
>
> > I agree that a 1-to-N relationship, with N being some rather low
> > constant number, shouldn't be realized that way, but I find it quite
> > practical for true 1-to-1 relationships.
>
> > Better and less synthetic examples for composed fields would be:
>
> >    class Length(CompositeField):
> >        value = models.DecimalField(...)
> >        unit = models.CharField(choices=UNIT_CHOICES)
>
> >    class Location(CompositeField):
> >        longitude = models.FloatField()
> >        latitude = models.FloatField()
> >        altitude = models.FloatField()
>
> >    class FullNameField(CompositeField):
> >        prefix = models.CharField(max_lenght=20, blank=True)
> >        first = models.CharField(max_lenght=20)
> >        middle = models.CharField(max_lenght=20, blank=True)
> >        last = models.CharField(max_lenght=20)
> >        suffix = models.CharField(max_lenght=20, blank=True)
>
> >    class PhoneNumber(CompositeField):
> >        country = models.CharField(max_lenght=4, blank=True)
> >        area = models.CharField(max_lenght=8, blank=True)
> >        subscriber = models.CharField(max_lenght=16)
>
> > I hope all that makes a bit more sense now?
>
> > --mp
>
> > --
>
> > You received this message because you are subscribed to the Google Groups
> > "Django developers" group.
> > To post to this group, send email to django-develop...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > django-developers+unsubscr...@googlegroups.com
> > .
> > For more options, visit this group at
> >http://groups.google.com/group/django-developers?hl=en.

--

You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Porting Django to Python 3

[VernonCole]

Dave:
  Wonderful!  I am presently working on a project to get adodbapi
(http://sourceforge.net/projects/adodbapi) working in django.  That
may be important to you since it is one of few db interfaces which has
a working python 3 version for Windows. Keep in touch.
--
Vernon Cole

On Jan 8, 11:25 am, Dave  wrote:
> Hello everyone,
>
> My name is Dave Weber, and I'm a student at the University of Toronto,
> studying Computer Science. For one of our undergraduate courses led by
> Greg Wilson (http://www.cs.utoronto.ca/~gvwilson/), myself and a group
> of 10 other computer science students will be trying to port Django to
> Python 3.
>
> Until the end of January, we'll be studying the existing Django code
> in order to gain an understanding of how the program works. We'll be
> doing this primarily through architecture documentation and
> performance profiling. In early February we plan on beginning work on
> the port.
>
> A few of us have experience working with Django, and by the end of
> January we should have a much better understanding of it. I've been in
> touch with Jacob Kaplan-Moss, who pointed me to this group, and he
> also provided me with links about contributing to the Django project
> and Martin van Lowis' port.
>
> We don't really have any specific questions right now as we're pretty
> unfamiliar with most of the project at this point in time. However, we
> are very eager to learn as much as we can, so if you have any advice,
> warnings, or anything at all to say to us, please feel free! We'd like
> to hear from all of you as much as possible.
>
> Best regards,
>
> Dave Weber
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Porting Django to Python 3

[VernonCole]

Having survived the update of pywin32 to python 3, let me say that
both comments are correct:
1) you do NOT create a fork, you convert the existing code so that it
will run through 2to3
2) it takes a LOT of hand refactoring of older 2.x code to get ready
for 2to3.
and, may I add:
3) it's worth the work.  The refactoring tends to clean up rough edges
that have been hanging around the old code from long, long ago.
IMHO it is absolutely necessary for one or more core developers to be
intimately involved with the conversion. Such things as conversion to
new style classes and byte buffer creation objects will very likely
reach into a majority of the existing modules, so the volume of
patches will be very large.  If these patches are not integrated into
the tip branch(es) rapidly, it is likely that new work will get very
confusing.  I personally found it very helpful to incorporate
suggestions from the guys doing the 2-to-3 conversion directly into
the development branch I was working on -- so that within a day my
development branch became the 2-to-3 conversion branch as well.  By
the time I finished my next incremental update, it was 2to3 ready.

Cheering from the sidelines is not enough.

  By the way, the pywin32 modules work in all versions of Python from
2.3 to 3.1 (mine works in IronPython as well.) 2.6 is helpful for a
conversion effort, but not necessary.
--
Vernon Cole


On Jan 13, 7:22 am, Karen Tracey  wrote:
> On Wed, Jan 13, 2010 at 4:21 AM, Hanne Moa  wrote:
> > 2010/1/13 Tobias McNulty :
> > > I am by no means an expert on the matter, but I remember seeing a comment
> > > awhile back suggesting that it generally makes more sense to fix the 2to3
> > > script than to maintain two branches of the same library. Might that be
> > the
> > > case here as well?
>
> > Py3K does not support old-style classes. Django uses these quite a
> > lot, for instance the Meta-class of a model is old-style. I don't
> > think it is in any way possible to have an automatic script convert
> > these in a sensible way as django is deliberately utilizing the
> > difference between old and new style in no doubt a django-specific
> > way. If django on 2.x could be rewritten to no longer depend on
> > old-style classes, and was made to depend on python 2.6 or newer, then
> > 2to3 would have a chance to do its magic.
>
> I'm no expert either, but as I understanding it maintaining single source
> for 2.x (where x can be lower than 6) and 3.x and using 2to3 to generate the
> 3.x version during install may be a viable option.  This is the approach
> that was taken by Martin v. Löwis when he got an initial port working back
> in late 2008:
>
> http://wiki.python.org/moin/PortingDjangoTo3k
>
> He cites bugs in 2to3 as a barrier to getting the approach to work at that
> time, but doesn't note anything insurmountable he ran across in the Django
> source.  It is true the port only verified that getting through the tutorial
> worked, but that covers the basics of models certainly.
>
> Karen
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Process discussion: reboot

[VernonCole]

Not to start a flame war --- but PLEASE! don't use git.  I already
have to use the other two leading DVCS's and all three are one too
many.
I personally prefer bazaar, but python itself and pywin32 are both
committed to mercurial.  I suspect that hg would be a better choice
for most people.
--
Vernon Cole

On Apr 19, 10:05 am, Dennis Kaarsemaker 
wrote:
> On ma, 2010-04-19 at 15:47 +, Peter Landry wrote:
>
>
>
>
>
> > On 4/19/10 11:41 AM, "Jacob Kaplan-Moss"  wrote:
>
> > > On Mon, Apr 19, 2010 at 9:54 AM, Peter Landry  
> > > wrote:
> > >> One suggestion that jumped out at me (which I admittedly know very little
> > >> history about with regards to Django or other projects) was the "trunk
> > >> ready" branch(es) [1]. Perhaps an effort to outline what that process 
> > >> might
> > >> entail in detail, to determine if it would address any concerns?
>
> > > FTR, I think this is a fine idea, and I think most (all?) of the other
> > > Django core developers do, too. It's just waiting on someone to Just
> > > Do It.
>
> > > Anyone -- preferably a group -- who wants to start such a branch could
> > > go ahead and start one using the DVCS tool of their choice (Django has
> > > semi-official clones on Github, Bitbucket, and Launchpad). Tell me and
> > > I'll start watching it; show some continued motion and I'll spend some
> > > time getting a buildbot going against the branch; show high quality
> > > and I'll start pulling from it more and more frequently; show
> > > incredibly quality and I'll suggest that the maintainer(s) get commit.
>
> > >> For my part, I see that it could be helpful to let some patches/ideas 
> > >> get a
> > >> shot at integration without having to endure the (necessarily) more 
> > >> rigorous
> > >> core commit trails.
>
> > > Quality is important, and if this branch is created it needs to
> > > maintain that quality. If this hypothetical branch is low-quality it's
> > > just a different tool for a queue of un-reviewed patches, and I've
> > > already got one of those. I'm not willing to compromise on quality: if
> > > patches don't meet our standards, they don't go in.
>
> > > Jacob
>
> > I fully agree regarding quality, my point being that this branch itself may
> > not be "trunk ready" at any given time, but patches/pulls from it would be;
> > quality of patches would obviously need to meet existing standards. Perhaps
> > that distinction isn't helpful, necessary, or desirable.
>
> I've been thinking of starting a proper contribution in django in a
> similar way: a github repo with per-ticket branches that are trunk-ready
> and regularly updated (rebased) against trunk until they are applied.
>
> So far I've only done so for a pony of mine as a test, but as soon as
> the ash cloud clears, I will look at existing tickets. Would this
> approach be useful for core committers to pull patches from?
>
> --
> Dennis K.
>
> They've gone to plaid!
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To post to this group, send email to django-develop...@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group 
> athttp://groups.google.com/group/django-developers?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Process discussion: reboot

[VernonCole]

Thanks for the advice. Trying to make a contribution is why I'm here.
That's why I worry about version control systems.  Last time I tried
to contribute to an open source project via a "semi-official
mirror" (this one actually run by a core developer) it did not work
and I ended up having to resubmit the work to CVS. I now have commit
permission on that project (pywin32) so it's no longer a problem.
Jacob seemed to be suggesting a single dvcs ("THE branch") so I
thought I would weigh in on the choice. Sorry I missed the discussion
two years ago.

   Meanwhile, I suppose I must stick with svn if I expect to have my
work accepted, because my contribution will very likely be
controversial, and I prefer to only fight one battle at a time.  On
the other hand, getting some test time in a less critical environment
would be a really good thing, and would increase the chances of my
work having a good examination by other developers.
  Umm -- time to explain:

  I am the principle maintainer of adodbapi on sourceforge. I have
spent the last little while making adodbapi django compatible.  (The
new version allows the programmer to change 'paramstyle' at run time.)
The existing MS-SQL back end for django is a fork taken from the
adodbapi project *before* it was changed to be ready for Python 3 and/
or IronPython.  The alpha-test version now running will operate in all
three environments -- and should be a real aide to both the IronPython
and Python 3 django integration projects which are now under way.  It
should be a fairly minor change to the existing MS-SQL back end to use
the (new) trunk version of adodbapi rather than the fork.

  That's not controversial -- here comes the controversy:

  There is finally a solid release of IronPython 2.n available for
Linux. (Ubuntu Lucid).  The existing adodbapi will not work there, due
to lack of a COM interface.  I now have to put in genuine .NET calls
to reach the database.  I started working on that project yesterday.
When I am done the result will be that django on IronPython (or Python
3?) will be able to use *only*  MS-SQL or SQLite databases. Not a
pretty picture. In order to access other database engines, the engine
specific code for all of the other database engines will have to be
modified to fall back to ADO when their native adapters are not
present.  That's a lot of fixing of things that are not broken.  I
think the final result will be a version of django which is a lot more
flexible, but it's gonna be a hard sell.
--
Vernon

On Apr 19, 8:05 pm, "Sean O'Connor"  wrote:
> The DVCS conversation has been had many times over the last year or two on
> this list and in other places.  I mention this not to say that you should
> know already as it isn't clearly documented, but as a suggestion that you
> should make sure that you are bringing something new and concrete to the
> discussion before starting it again.
>
> The current view of the core team is that the combination of a central,
> authoritative, Subversion server with semi-official mirrors for Git,
> Mercurial, and Bazaar is sufficient for the foreseeable future.  All of the
> benefits of the distributed systems are available via the mirrors and
> there's no disruption to users who are used to the current
> workflow/infrastructure.
>
> If you would like to make a contribution to Django, you can work with
> whichever of the three most common DVCSs and there will be several core devs
> able to accept your changes without any problems.
>
> 
> Sean O'Connorhttp://seanoc.com
>
> P.S.  I am not a core dev so any of the core devs should feel free to
> correct me on this. That being said this view has been clearly expressed
> several times on this list and in other venues.
>
>
>
> On Mon, Apr 19, 2010 at 8:35 PM, Jerome Leclanche  wrote:
> > If you contribute to open source projects, at one point you'll be
> > faced with the forced choice to use git. It is extremely popular (I
> > believe it's the most popular after svn), and unlike svn it's popular
> > for a good reason.
> > However, hg is decent as well; whatever the django team chooses, as
> > long as it's not cvs it can't really be worse than svn.
>
> > (bzr fan, personally, but I'd prefer it if django moved over to git)
>
> > J. Leclanche / Adys
>
> > On Tue, Apr 20, 2010 at 2:58 AM, VernonCole  wrote:
> > > Not to start a flame war --- but PLEASE! don't use git.  I already
> > > have to use the other two leading DVCS's and all three are one too
> > > many.
> > > I personally prefer bazaar, but python itself and pywin32 are both
> > > committed to mercurial.  I suspect that hg would be a better choice
> > > f

adodbapi 2.3.0 (the django version) released

[VernonCole]

I have just released a version of adodbapi which is intended
specifically for django compatibility.  It should be useful for four
different projects:

1) accessing Microsoft SQL and/or Microsoft ACCESS (JET) databases.

2) running django on IronPython

3) converting django to Python 3

4) multi-db support

This is intended to replace the forked versions of adodbapi which have
been used for the above projects, so that all of them can return to
this, the official trunk version.

http://sourceforge.net/projects/adodbapi contains the source code. Use
either the .zip downloads or the Mercurial feed.

The present version is for Windows only.  A future version will be
converted to use .NET (or mono) and therefore run on Linux IronPython.

 Please try this out if you can.
--
Vernon Cole

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django-mssql with the dream of passing the test suite

[VernonCole]

Micheal:

   I note that your code is using the Adam Vandenberg fork of adodbapi.  I 
have rolled Adam's changes back into the main branch of adodbapi, so that 
django can have the advantages of numerous bug fixes, as well as the 
ability to run in Python3 and IronPython.  I have attempted to make all of 
the features that Adam added work exactly the same way -- except that you 
will have to add a line of code to inform adodbapi that it needs to switch 
paramstyle to 'format' when it connects.  As far as I know, this is the 
only db-api module which is capable of switching paramstyle at run time -- 
and I did it all for you.

  Please, please, please import my new django-compatible adodbapi from 
https://sourceforge.net/projects/adodbapi  
(or PyPi, same version in 
both places).

  Let me know if you need any help or changes.  I will modify the main fork 
to accommodate your needs.  I _really_ want this to work.
--
Vernon Cole

On Wednesday, May 16, 2012 2:27:00 PM UTC-6, Michael Manfre wrote:
>
> A few weeks ago, I started down the path of updating django-mssql so that 
> it supports Django 1.4. I moved the project over to bitbucket (
> http://bitbucket.org/Manfre/django-mssql/), docs are deployed to read the 
> docs (http://django-mssql.readthedocs.org), and have made it a policy of 
> deploying packages to PyPi. 
>
> Django 1.4 support is mostly there and so far is passing all testing for 
> the site responsible for the backend's existence (http://www.src.org). To 
> help avoid surprises and ensure life as a 3rd party database backend 
> maintainer is a lot less painful, I have set a goal of having django-mssql 
> pass the test suite. This is a non-trivial task that will require changes 
> to django and django-mssql. I've already submitted a few tickets that bring 
> me closer to my goal and it was advised that I send this message as a heads 
> up regarding my intentions to avoid tickets being closed with a "mssql 
> isn't officially supported by Django" comment.
>
> If you review one of these tickets, please keep my goal in the back of 
> your mind. I welcome feedback on better ways of approaching my proposed 
> changes.
>
> Regards,
> Michael Manfre
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/9Rq_YaZ8lKsJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django-mssql with the dream of passing the test suite

[VernonCole]

I should mention that if you use pywin32 build 215 or newer, an appropriate 
version of adodbapi will already be included.  You do not need to download 
it separately from sourceforge or PyPi unless you are running IronPython. 
Since pywin32 is already a dependency for django-mssql, you should be 
covered.  Also, it should not be too hard to add support for ACCESS and 3rd 
party databases.
--
Vernon

On Wednesday, May 16, 2012 2:27:00 PM UTC-6, Michael Manfre wrote:
>
> A few weeks ago, I started down the path of updating django-mssql so that 
> it supports Django 1.4. I moved the project over to bitbucket (
> http://bitbucket.org/Manfre/django-mssql/), docs are deployed to read the 
> docs (http://django-mssql.readthedocs.org), and have made it a policy of 
> deploying packages to PyPi. 
>
> Django 1.4 support is mostly there and so far is passing all testing for 
> the site responsible for the backend's existence (http://www.src.org). To 
> help avoid surprises and ensure life as a 3rd party database backend 
> maintainer is a lot less painful, I have set a goal of having django-mssql 
> pass the test suite. This is a non-trivial task that will require changes 
> to django and django-mssql. I've already submitted a few tickets that bring 
> me closer to my goal and it was advised that I send this message as a heads 
> up regarding my intentions to avoid tickets being closed with a "mssql 
> isn't officially supported by Django" comment.
>
> If you review one of these tickets, please keep my goal in the back of 
> your mind. I welcome feedback on better ways of approaching my proposed 
> changes.
>
> Regards,
> Michael Manfre
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/i4qYqPpLAnoJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3: should we apply unicode_literals everywhere?

[VernonCole]

That seems to me (in my dark status as a lurker here) to be a brilliant 
idea. 
It is already established practice to say something like: "version 1.n of 
django requires 2.m or later of Python".
The practice then would change to: "version 1.n of django requires 2.m of 
Python or 3.3 or later".
I see from reading the text of PEP414 that there is an import hook 
available to make this feature also work in Python 3.2.
Would there be any advantage to requiring support for older versions of 
Python 3?  I can't think of any.
Python 3.3 will be an established thing long before a django version using 
it gains production status. We developers can use hooks and beta versions.
--
Vernon Cole

On Tuesday, August 21, 2012 4:03:57 PM UTC-6, DrMeers wrote:
>
> It's a shame we couldn't skip straight to Python 3.3 and take 
> advantage of PEP414... 
>
> On 22 August 2012 07:32, Adrian Holovaty > 
> wrote: 
> > On Tue, Aug 21, 2012 at 5:46 AM, Aymeric Augustin 
> > > wrote: 
> >> In my opinion, option (2) is a logical move at this point. However I 
> >> believe it deserves a public discussion (or at least an explanation). 
> >> What do you think? 
> > 
> > I prefer option 2 as well, because it seems like the Right Thing To 
> > Do. Of course, there's no rush to do everything -- we can just nibble 
> > off bits here and there. 
> > 
> > I'll have some free time soon and would be happy to help out migrating 
> > code. (Relatively) mindless refactoring like this is one of my 
> > favorite things to do. :-) 
> > 
> > Adrian 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups "Django developers" group. 
> > To post to this group, send email to 
> > django-d...@googlegroups.com. 
>
> > To unsubscribe from this group, send email to 
> django-develop...@googlegroups.com . 
> > For more options, visit this group at 
> http://groups.google.com/group/django-developers?hl=en. 
> > 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/qDLg0m7y-PUJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Will be % string formatting be preferred in the future, or will string.format() be moving in?

[VernonCole]

Dear Gurus:
  I am a regular lurker on this list, and I have accepted the invitation to 
use 1.5rc1 to help test it.  My group is putting together a small (but very 
important, we think) application which will use some features of 1.5 and 
geodjango.  I have also been reading the corresponding 1.5 documentation, 
and have done some testing under Python 3.   [Job well done, everyone!]
  Here comes my question:  The documentation for i18n (
https://docs.djangoproject.com/en/dev/topics/i18n/translation/) makes 
extensive use of percent string formatting (where the format string is the 
left-argument of a "%" operator.)  I have been trying to convert myself and 
my code to the more-recently-defined string.format() function. 
  I know that the document has been recently edited, because I can see 
"print" used as a function, and "smart_text', etc. But not one word is 
mentioned about the new string formatting function.  Will it be / is it 
supported by django translations, or should I revert to using % formatting 
as my habit?
  What do the "insiders" say?
--
Vernon Cole
eHealth Africa .org


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Moving database backends out of the core

[VernonCole]

Huh??  I am running sqlite with GIS on my laptop right now.  It wasn't THAT 
hard to install.  Yes, the documentation could use some cleanup, but it got 
me through the tutorial okay and gives me a platform to learn GIS on.
  I really support the idea of sqlite and postgres in the core and moving 
everything else outside.  
(Not that my support counts for much.)

On Wednesday, March 6, 2013 10:47:12 AM UTC-7, Florian Apolloner wrote:
>
> Hi,
>
> On Wednesday, March 6, 2013 3:32:45 PM UTC+1, Michael Manfre wrote: 
>>
>> The lack of data validation is definitely a nogo for production sites, 
>> but imo sqlite in production is also a nogo.
>>
>
> Right, but shipping Django with a non production db might send interesting 
> signals to endusers ;)
>
> The reference implementation should imo also have strong support for GIS, 
>>> which is somewhat okay on sqlite but quite hard to install. So if we were 
>>> to do that I'd either vote for postgres or supporting postgres and sqlite 
>>> inside of core (the later solely for fast tests).
>>>
>>
>> . I've never tried to install GIS for sqlite, but is the difficulty due 
>> to lack of documentation or just sheer number of steps?
>>
>
> Well it's not to bad, we did document it after all, but it usually 
> requires recompiling sqlite and pysqlite (most importantly you can't pull 
> it from your distros repos).
>
> Regards,
> Florian 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Moving database backends out of the core

[VernonCole]

I would love to volunteer for the project of MSSQL support.  It happens 
that django-mssql uses a fork of my adodbapi v2.1 code. I pulled those 
changes back into the main branch of adodbapi (and did a lot of useful 
refactoring thanks to the good work done on the fork) at version 2.3.  
Apparently the announcement of that did not get to the right people, since 
the peek I just took shows the forked code still in use.  

The two major obstacles I can see to MSSQL support (either as part of the 
core or as a supported add-on) are:
1) lack of an MSSQL server and test bed which the maintainers can use on a 
regular basis.
2) django-mssql is not Python 3 ready.

The first of those obstacles has already been addressed. 

The second is fairly easy.  I suspect (not having examined the code) that 
most of the Python3 problems will be found in the old fork of adodbapi.  If 
we simply change django-mssql to call the current version of adodbapi all 
of that goes away.  The current version is already present on their Python 
library, I am sure, because it ships as part of pywin32, which is also a 
prerequisite for the V2.1 code. Yes, all of pywin32, including adodbapi, 
runs on Python 3 -- has done for years.

Would there be a "market" for a supported version of django which runs 
MSSQL?  Does the bear live in the woods??? Take, for example, my present 
employer. (I escaped from the "we are a strictly Microsoft shop" employer 
where I used to work.) We here at eHealth Africa use Ubuntu and postgres as 
much as possible.  But it's not always possible.  For example, part of the 
project I am working on now will use ArcGIS -- which only runs on Windows.  
Then one of our funding organizations (ever heard of the Gates Foundation?) 
required us to host a database on an MSSQL server. We bit the bullet, 
bought the licenses (it's their money, after all) and set up the SQL 
server.  Do you think we want to use django on it?  Huh?
 
If it will help kill polio, we will use anybody's software.

As to the question of moving MSSQL into the core, or moving the two Oracle 
dbs out of it (you do remember who owns MySQL?) -- I would suggest pulling 
all non-FOSS out.  Would you accept my patches to the core?  Of course not. 
and you should not: I don't know django well enough.  Would you accept my 
patches to an MSSQL back end?  I should hope so.

There has been talk. from time to time, about extending django to have more 
multi-database capability.  The changes that would be needed to run three 
different db engines from outside the core would easily extend to other 
engines.  People tend to think of ADO as only talking to Microsoft 
databases.  Nothing could be farther from the truth.  When maintaining 
adodbapi, I normally test against MS-SQL Server, Microsoft "Jet" (a.k.a. 
ACCESS), MySQL, and postgres. I have also personally used it to get data 
from IBM DB2, an Active Directory server, and a .csv file.  If some idiot 
has written an odbc driver for it, I will let you read it from Python.

Build the hooks.
--
Vernon Cole

On Thursday, March 7, 2013 8:24:00 PM UTC-7, Michael Manfre wrote:
>
>
>
> On Thu, Mar 7, 2013 at 8:13 PM, Russell Keith-Magee <
> rus...@keith-magee.com > wrote:
>
>> I completely agree with Jacob's analysis of the status quo, and I agree 
>> largely with his position on having MSSQL in the core. 
>>
>> I'd have no problem seeing MSSQL in the core - it's at least as high 
>> profile as Oracle, and although the demand for a MSSQL backend is 
>> restricted to a particular subset of the community, there is at least 
>> *some* demand for it to exist. Having MSSQL in core would allow us to hold 
>> our head high and say we support Microsoft platforms. Microsoft is even a 
>> member of the DSF at this point, so they're at least notionally interested 
>> in Django as a platform. 
>>
>
> /slightly off topic
> I didn't realize they were a DSF member. I have been wanting to set up 
> Jenkins to run the test suite on Windows with the ultimate goal of getting 
> its results added to ci.djangoproject.com. Knowing that Microsoft is a 
> DSF member will probably help gain access to a VM on Azure to make that a 
> reality. I've so far managed to get an introduction to a PM on one of the 
> Azure teams to help get a more direct answer to some Django, Windows, and 
> SQL problems that have been raised to my attention.
>
> /back on topic
>
> The maintenance burden is the problem. Historically, we've already had to 
>> *remove* a MSSQL backend because of bit rot (see [1]) - this isn't a 
>> pattern I want to repeat in the future. I don't want to add a backend to 
>> core unless I'm *certain* that it will see long term maintenance.
>>
>> [1] https://github.com/django/django/commit/c30a050e41
>>
>
> The demand for MSSQL would probably increase if it were included in core 
> because companies could be more confident about its ongoing support.
>
> There are three options for using MSSQL with Django; django-pyodbc, 
> django-mssql, or 

Re: Moving database backends out of the core

[VernonCole]

[My reply is below, after the question...  scan down.  VDC]

On Friday, March 8, 2013 7:33:27 AM UTC-7, Javier Guerra wrote:
>
> On Fri, Mar 8, 2013 at 4:25 AM, VernonCole > 
> wrote: 
> > People tend to think of ADO as only talking to Microsoft databases. 
>  Nothing 
> > could be farther from the truth.  When maintaining adodbapi, I normally 
> test 
> > against MS-SQL Server, Microsoft "Jet" (a.k.a. ACCESS), MySQL, and 
> postgres. 
> > I have also personally used it to get data from IBM DB2, an Active 
> Directory 
> > server, and a .csv file.  If some idiot has written an odbc driver for 
> it, I 
> > will let you read it from Python. 
>
>
> nice. 
>
> does the client run on non-win32 plaforms?  from a cursory read of the 
> website, adodbapi seems to require pywin32. 
>
> -- 
> Javier 
>

Sorry about terse nature of the website information.  I need to repent and 
write better documentation.

ADO is a strictly Microsoft animal. It seems that Microsoft invented ODBC, 
which was quickly recognized as such a great idea that everyone adopted 
it.  So they had to invent something "better", and that was supposed to be 
ADO.  As it happens, the most redeeming feature of ADO is that in runs in 
ODBC mode by default. That having been said, it  is very powerful, and does 
have the ability to reach out to almost any form of tabular data. But it 
does so using COM calls, which are very proprietary.

I have been worrying over the problem of how to read an ADO dataset from 
Linux for years.  I was hoping that IronPython and Mono would provide the 
answer, but it turns out that ADO.NET also uses COM, and looses some of the 
power (like timely update of .rowcount) in the process.  It does not even 
work well in a pure Windows environment. I have abandoned that code fork.  
The open source odbc products leave things to be desired, too, the largest 
being that they only work in ODBC mode and cannot switch to ADO mode when 
the connection string requires it. I have begun to think that eGenix is on 
to something with their "mxODBC connect" product: they use a connection 
server.

Here's how it might be done:  Somewhere, there would have to be a Windows 
box running an ADO connection server program.  Perhaps it could be on the 
actual database server, perhaps not.  A copy of adodbapi which notices that 
it is running where os.name != 'nt' would bring up a link to the connection 
server (using RPC?) which would do the actual data access and pass the 
results back.  A simple server would be trivial to write.  One which worked 
well under a heavy thread load might not be so easy, but my thinking is 
that a heavily used django server which made frequent use of an MSSQL 
database would already be running on a Windows box itself and would not 
need any connection server.

If there is interest, I will start coding a prototype.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Moving database backends out of the core

[VernonCole]

On Friday, March 8, 2013 4:43:20 PM UTC-7, Florian Apolloner wrote:
>
> On Friday, March 8, 2013 3:35:53 PM UTC+1, Michael Manfre wrote:
>>
>> If we have MSSQL in core I'd really like to be able to talk with it from 
>> a Linux machine too, it would also make testing easier since we'd just need 
>> a VBox with MSSQL ;) Supporting a commercially available product but 
>> requiring to run even more commercial software seems like counter intuitive 
>> to me.
>>
>> Django needs a VBox running Windows for testing, regardless if MSSQL is 
>> in the core.
>>
>
> That doesn't mean we actually have to support Microsoft in making MSSQL 
> only accessible from Windows… Also there is a large difference between 
> running a vm with a database server and running a vm (read: jenkins slaves) 
> to execute tests.
>
> If Microsoft wants to use/support Django I guess they could easily publish 
> a Python db-api 2.x driver. I'll happily code the Django part if the 
> prerequisites would exist (assuming MSSQL actually does work ;))…
>
> Regards,
> Florian 
>
 
Oh, MSSQL actually works (and works, and works, and works...) and finally 
comes up with your data. It may perform rather slowly, but it does 
perform.  And many of us are forced to use it by events beyond our 
control.  Most of us would probably prefer a Posix box for our django 
server, too, but we don't always have that choice, either.  Django servers 
are expected to run on Windows, and it would be a Really Good Idea (tm) if 
it were tested there during development, no matter which database it is 
using.

As for django servers reading MS SQL databases, I can see three use cases:
1) django server running Windows in heavy production.  This works fine, 
Michael has a pile of them.
2) django server running Posix in a test environment.  I think I can make 
this work using a connection server -- well enough to handle runsever at 
least.
3) django server running Posix in heavy production. There we run into 
problems with lightweight support for database drivers.  But will this 
situation ever exist in real life?  I don't think so.  I suspect that any 
heavy-use shops will also be using Windows to run their django servers.

As for Microsoft coding a db api 2.x driver -- why should they, when a 
perfectly good FOSS one already exists (If  I humbly say so myself)?
--
Vernon Cole

 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: MySQL, Django 1.6, Python3

[VernonCole]

This thread interacts with another current thread: the one about MS-SQL 
database support. As I mentioned there, adodbapi is capable of accessing 
any ODBC compliant data source. MySQL is one of those.  adodbapi is Python 
3 compliant, and I have been using MySQL databases from Python 3 for 
several years.

If django supported adodbapi as a connection method -- separated from its 
use as a connection method specifically for the Microsoft SQL engine -- 
then it could also be used to reach MySQL and any other database engine 
people needed.  I should be able to use adodbapi rather than psycopg2, for 
example, if the hooks were in place.  On Linux, I should be permitted to 
select a prominent commercial ODBC product to do the same job, if I wanted 
to.

At the present time, the specification of the database engine is implied by 
selecting the backend in settings.py.  There is no convenient way of saying 
"I expect to be using connection method "X" to hook with engine "Y".  The 
driver-to-engine coupling has always been one-to-one.  I suspect that there 
are places deep withing django where this simplification has been used to 
advantage.  However, the two things should really be uncoupled.  Is it 
important?  No.  Might it have some presently unforseen advantages? I 
suspect so.

If the two Oracle database interfaces were to be re-tasked as supported 
options, rather than part of the core, then the driver-to-engine coupling 
would become looser. In the end, it ought to be possible for the selection 
of the engine to be deferred until after the database is actually 
connected.  Neither ADO nor ODBC actually know which engine they are 
talking to until the connection is complete.  After connection, I can look 
at the connection.dbms_name attribute and I can see which engine I am 
connected to.  If  the engine turns out to be postgresql, should I not be 
able to hook to the correct SQL compiler and have everything just work?

Then, with this looser coupling, We could separate the testing (and 
support) of different engines on different operating systems on different 
Python versions.
--
Vernon Cole

On Sunday, March 10, 2013 5:32:02 PM UTC-6, Norberto Bensa wrote:
>
> Hello,
>
> I wanted to run Django with Python 3 but mysql was a showstopper until 
> today. Thanks to G+ I found a few links of interest:
>
>
> http://stackoverflow.com/questions/13320343/can-i-use-mysql-on-djangodev-1-6-x-with-python3-x
>
>
> https://groups.google.com/forum/?fromgroups=#!topic/django-developers/CtTpgMjN5J8
>
> https://github.com/clelland/MySQL-for-Python-3 (m4p3)
>
>
> So I git-cloned django and m4p3 but I got a problem.
>
>
> 'use_unicode' is an invalid keyword argument for this functionRequest 
>> Method:GETRequest URL:http://localhost:8080/propietario/Django Version:
>> 1.6.dev20130310222819Exception Type:TypeErrorException Value:
>>
>> 'use_unicode' is an invalid keyword argument for this function
>>
>> Exception 
>> Location:/home/ubuntu/.virtualenvs/test3/git/MySQL-for-Python-3/MySQLdb/connections.py
>>  
>> in __init__, line 175Python Executable:
>> /home/ubuntu/.virtualenvs/test3/bin/pythonPython Version:3.2.3
>>
>>
> I thought, well, let's disable use_unicode in my settings.py. I added 
> OPTIONS: { 'use_unicode': False } to DATABASES.default, but no way. Same 
> error. So I patched 
> django/db/backends/mysql/base.py:DatabaseWrapper:get_connection_params() 
> and commented 'use_unicode':True. And now I can use Django, Mysql, and 
> Python 3
>
>
> The question is: Is there any option I'm overlooking and this mix can be 
> used with patching Django or is Mysql+python3 just not supported (for now)?
>
>
> Thanks!
> Norberto
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Moving database backends out of the core

[VernonCole]

My organization just hit a use case where we need MS-SQL support. 

 I am jumping on board, so there are at least two of us who can do
maintenance. 

I must say that I would prefer quasi-supported status (akin to admin 
and geodjango) rather than actually being in the core.  I think it would 
be a better fit for most situations.  We will always be a small minority of
django users.  I would just like some assurance that pull requests needed
to provide good hook support for external database backends got prompt 
attention from the core developers.
--
Vernon Cole

On Thursday, March 7, 2013 6:46:18 PM UTC+1, Jacob Kaplan-Moss wrote:
>
> On Wed, Mar 6, 2013 at 3:22 AM, Marc Tamlyn > 
> wrote: 
> > I don't know why Oracle is included and MSSQL is not [...] 
>
> It's essentially because a group of people made a concerted effort to 
> get the Oracle backend up to snuff (around the 1.0 timeline, I think?) 
> and then committed to maintaining it. Lately the original people who'd 
> made that commitment have dropped off a bit, but there's always been 
> enough attention to keep it up to date. 
>
> If someone -- preferably a group -- stepped up and committed to 
> keeping a MSSQL backend up-to-date in core, I'd be +1 on merging it 
> and giving those people commit access to keep it up to date. 
>
> [This is me speaking personally, not as a BDFL. It'd have to be a 
> discussion, not just my fiat.] 
>
> Jacob 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Extending the DATABASES = {'xx': {'NAME': construct to carry additional information and / or virtual names

[VernonCole]

Dear Gurus:
 
At the bottom of this text there will be a question.  In order for you to 
provide a quality answer to the question, you need more information.  
Perhaps a lot more information. The next few paragraphs will attempt to 
provide that. They will also be an introduction to a couple of concepts, 
and a bit of a 'sales pitch."  Please accept my apologies for this and bear 
with me.  The question will be delimited by a 'v v v v v' construction...

1) My present situation (not important to the discussion here, just so you 
understand how I got into this):  I am working for a company called eHealth 
Africa. We provide software and logistics support to help stop endemic 
diseases (such as polio) and promote improved health care. Our principle 
focus is sub-Saharan Africa, and our principle office is located in the 
heart of the last remaining reservoir of wild polio, in Kano, Nigeria.  
Yes, the same Kano where nine girls from a polio vaccination team were shot 
by Islamic extremists a few weeks ago.  It is not really safe here, 
particularly for someone who looks like a cowboy from Wyoming (USA).  But I 
digress.  We use django and Ubuntu whenever we can. However, the present 
project consists of a lot of data collected by field workers using a tool ( 
http://wwwn.cdc.gov/epiinfo/ )  which writes ACCESS data files.  The same 
tool will also be used to read and analyze the data after it has been 
collected into an SQL Server database.  I have to get the data from the 
field, clean and verify it, and get it into the big database.

2) In order to collect, clean and warehouse the data, I need a tool which 
will read and write those databases.  Fortunately, I am the maintainer of 
just such a tool: adodbapi .  
Unfortunately, it would be best done on a Linux webserver ... and ADO is a 
Windows only tool.  My past week has been spent cleaning up the code of 
adodbapi and writing a remote proxy so that I can call it from a Linux box 
using Pyro.  Last night it executed a query and returned a rowcount and 
description.  Today I hope to return an actual data row, too.

3) An early fork of adodbapi was used as the data engine for django-mssql.  
I rolled the improvements from that fork back in to production adodbapi a 
couple of releases ago.  I am working with Michael Manfre to re-integrate 
the two, so that the next release of django-mssql will be using the next 
release of adodbapi, and will, therefore, be accessible from Linux. It is 
hoped that the resulting product will be included in a future release of 
mainstream django.

4) Adding a database proxy server into the chain complicates data naming in 
django.  The present model assumes three computers: The user's, who types a 
URL to find the django server, (which may manipulate that URL to make use 
of other web resources.)  Then the django server uses the DATABASES 
dictionary from settings.py to find its data storage using the 'NAME", 
"USER", "HOST" and "PORT" items.  The database server (or cluster of 
servers) is the third computer in the chain. The proxy server adds an extra 
bump between the second and third logical nodes.  I need a way to address 
it and pass to it the identity of the actual third node.

5) The construction of the actual text to put build an ADO data source is 
something between science, magic, and luck.  There is an entire web site 
dedicated to this arcane language ( http://connectionstrings.com ).  The 
place where one injects information (such as the USER name) into this 
jumble varies depending on which database engine will be used and which 
route is used to get there. A completely different vocabulary is used to 
open the same database depending on whether ADO or ODBC is used for 
routing. If ODBC is used, the "ODBC Data Source Administrator" tool can add 
its own information, selecting the HOST, USER, PORT, and database NAME for 
you -- all you pass is a DSN. Here is a little sample (slightly altered) 
from my test program:

> from adodbapi import is64bit

if doMySqlTest:
> c = {'host' :"25.116.170.194",
> 'database' : 'test',
> 'user' : 'adotest',
> 'password' : '12345678',
> 'provider' : '',
> 'driver' : "MySQL ODBC 5.2a Driver"}
> if is64bit.Python():
> c['provider'] = 'Provider=MSDASQL;'
> cs = '%(provider)sDriver={%(driver)s};Server=%(host)s;Port=3306;' + \
> 
> 'Database=%(database)s;user=%(user)s;password=%(password)s;Option=3;'
> print('...Testing MySql login...')
> connStrMySql = test_connect(cs % c, timeout=5)
>
Note that I had to write a module to detect whether I am running on 32 bit 
or 64 bit Python, because the connection string changes.  
I don't really want to put that in settings.py -- especially the part about 
which python the proxy machine is running.

6) ADO is database agnostic. Last week, just to prove the point, I wrote a 
pair of programs, one of them creates a database on an .

Re: Extending the DATABASES = {'xx': {'NAME': construct to carry additional information and / or virtual names

[VernonCole]

Thanks for the feedback.  I have incorporated your suggestions into my 
design. There is nobody around here I can use for a design review, so, 
begging your forgiveness, I will have to do it here.  Here is how I am 
setting up the prototype...

1) re-implement the argument signature of adodbapi so that it agrees with 
the recommendation for positional and keyword arguments in PEP 249, except 
that the first positional argument is called "connection_string" rather 
than "dsn".  ((Also handle the case where "dsn" is defined but 
"connection_string" is not by using "dsn" as a default.)) The user may 
supply other keyword arguments not documented in PEP 249 -- for example 
"timeout".

2) permit the first positional argument to be a dictionary, which will be 
merged with **kwargs. This will make it easy to pass both the DATABASES 
dictionary and other keyword arguments in.

3) Look in the resulting merged dictionary for key "OPTIONS".  If present, 
merge that dictionary into the existing keywords.

4) Django specifies that database connection keywords are UPPER CASE.  Pep 
249 specifies the same tokens in lower case. Therefore, go through the 
dictionary,
creating lower case copies of any upper case keyword items (if they are not 
already defined.)

4) Create an adodbapi.remote module with the same signatures as adodbapi.

5) In the remote, use the dictionary keywords "'pyro_connection",  
"remote_proxy" and "remote_port" to connect to the server, supplying 
appropriate defaults.

6) Create adodbapi.server.py which will be a PyRO server daemon for 
adodbapi connections.

7) The remote client passes the entire connection keyword dictionary to the 
server.  The remote or the server may add keywords and supply defaults.

8) The server will attempt an adodbapi.connect(*args, **kwargs) using the 
merged keyword dictionary.

9) Internally, adodbapi.connect will execute the equivalent of:  
connection_string = kwargs['connection_string'] % kwargs
to allow the programmer to build her connection string by using Python 
"String Formatting Operations".  She will place "%(keyword)s" constructs in 
her connection 
string where appropriate. 

10) Dictionary keywords will be used for ADO option settings:   
ADO.Connection_timout = kwargs.get('timeout', 30)
---
How does that sound?
--
Vernon

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

currency of DATABASES = {'xx' : {'OPTIONS' = {'AUTOCOMMIT' ... flag.

[VernonCole]

I am confused by the documentation.  I cannot seem to figure out whether 
the AUTOCOMMIT switch is a new feature on the way in, or an obsolete 
feature on the road to deprecation.

I can easily add the feature to adodbapi, but should I do so or not?
--
Vernon

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: currency of DATABASES = {'xx' : {'OPTIONS' = {'AUTOCOMMIT' ... flag.

[VernonCole]

On Monday, April 8, 2013 3:17:52 AM UTC-6, Aymeric Augustin wrote:
>
> On 8 avr. 2013, at 10:33, VernonCole > 
> wrote: 
>
> > I am confused by the documentation.  I cannot seem to figure out whether 
> the AUTOCOMMIT switch is a new feature on the way in, or an obsolete 
> feature on the road to deprecation. 
>
>
> Defining AUTOCOMMIT = True within OPTIONS, as shown in the subject of this 
> message, was a PostgreSQL-specific feature that will be removed (well, 
> ignored) in 1.6. 
>
> Defining AUTOCOMMIT = False next to ENGINE and OPTIONS is new in 1.6. It's 
> a per-database replacement for the even-more-confusing setting called 
> TRANSACTIONS_MANAGED. You should never use it anyway! 
>
> Django 1.5 uses the behavior mandated by PEP 249 ie. no autocommit (unless 
> you're using PostgreSQL and you set the AUTOCOMMIT option). 
>
> Django 1.6 will use autocommit on all databases. 
>
> You'll find more information in my posts to this list, one or two months 
> ago. 
>   m
> -- 
> Aymeric. 
>
> Thank you! That explains my confusion.  
ADO runs in autocommit mode  by choice.  In order to get it to obey PEP 249 
we have to send it a "turn off autocommit" message after every commit.  I 
only need to tell it to quit doing that. An easy extension to add.  I will 
add support for the flag in the api and default it as True in the backend 
(for 1.6), so that it will normally override the PEP which overrides the 
default.  Then if someone sets the value to False in settings.py, he will 
override the override of the override.  (It is a good thing computers don't 
get confused.)


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Extending the DATABASES = {'xx': {'NAME': construct to carry additional information and / or virtual names

[VernonCole]

On Monday, April 8, 2013 3:31:48 AM UTC-6, Aymeric Augustin wrote:
>
> On 8 avr. 2013, at 10:21, VernonCole > 
> wrote: 
>
> > 1) re-implement the argument signature of adodbapi so that it agrees 
> with the recommendation for positional and keyword arguments in PEP 249, 
> except that the first positional argument is called "connection_string" 
> rather than "dsn".  ((Also handle the case where "dsn" is defined but 
> "connection_string" is not by using "dsn" as a default.)) The user may 
> supply other keyword arguments not documented in PEP 249 -- for example 
> "timeout". 
> > 
> > 2) permit the first positional argument to be a dictionary, which will 
> be merged with **kwargs. This will make it easy to pass both the DATABASES 
> dictionary and other keyword arguments in. 
> > 
> > 3) Look in the resulting merged dictionary for key "OPTIONS".  If 
> present, merge that dictionary into the existing keywords. 
> > 
> > 4) Django specifies that database connection keywords are UPPER CASE. 
>  Pep 249 specifies the same tokens in lower case. Therefore, go through the 
> dictionary, 
> > creating lower case copies of any upper case keyword items (if they are 
> not already defined.) 
>
> In my opinion, adodbapi should have an API that makes sense by itself, 
> regardless of Django. 
>
> Then the `get_connection_params` method of the database backend should 
> take care of mangling arguments to match that API. 
>
> Of course, that's bikeshedding: you write the software, you get to chose 
> the API that suits your tastes. 
>
 
 Doh!!! Of course that should be in the backend!   Thank you, the "feature" 
is gone.
 
 

> > 4) Create an adodbapi.remote module with the same signatures as 
> adodbapi. 
> > 
> > 5) In the remote, use the dictionary keywords "'pyro_connection", 
>  "remote_proxy" and "remote_port" to connect to the server, supplying 
> appropriate defaults. 
> > 
> > 6) Create adodbapi.server.py which will be a PyRO server daemon for 
> adodbapi connections. 
> > 
> > 7) The remote client passes the entire connection keyword dictionary to 
> the server.  The remote or the server may add keywords and supply defaults. 
> > 
> > 8) The server will attempt an adodbapi.connect(*args, **kwargs) using 
> the merged keyword dictionary. 
> > 
> > 9) Internally, adodbapi.connect will execute the equivalent of: 
>  connection_string = kwargs['connection_string'] % kwargs 
> > to allow the programmer to build her connection string by using Python 
> "String Formatting Operations".  She will place "%(keyword)s" constructs in 
> her connection 
> > string where appropriate. 
> > 
> > 10) Dictionary keywords will be used for ADO option settings:   
> ADO.Connection_timout = kwargs.get('timeout', 30) 
>
> Yes, overall, that design sounds reasonable. 
>
> It's similar to running PostgreSQL behind pg_pool (a connection pooler). 
> As long as Django connects to something that behaves like a database, it's 
> all right. 
>
> From the point of view of Django, your PyRO server daemon will be the 
> database, even though it's actually a proxy. Whether the proxy gets its 
> parameters from its own configuration or from Django doesn't really matter. 
> Just go for the most practical solution for your use case. 
>
> -- 
> Aymeric. 
>
 
Okay, Thank you.  I shall plow on.
--
Vernon 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

survey question: should UUID database fields be returned as instances of class "uuid"? (re: ticket 19463)

[VernonCole]

While mucking with the refactoring of adodbapi, I included a patch fixing a 
bug when storing data into columns of type "adGUID".  I had not been really 
aware that SQL Server has a dedicated data type for UUID until that ticket 
surfaced.  Then I started thinking about the column in one of my CDC data 
tables which contains text which always contains "uuid:" followed by a long 
hexadecimal looking string. I am pretty sure it was once a primary key, but 
pending something like ticket 19463 I don't use it as such.  Nevertheless, 
it started me thinking...

When I read data columns written in native decimal or date-time formats, I 
make the db-api module emit good, proper Python decimal.decimal and 
datetime.datetime objects. It would not take much effort to perform a 
similar action when I see that the incoming column is of the Windows type 
"adUUID" -- I could emit an instance of the "uuid" class.

Is that a good idea, or a bad idea?
--
Vernon Cole
 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: survey question: should UUID database fields be returned as instances of class "uuid"? (re: ticket 19463)

[VernonCole]

Russ:

 I am not entirely sure what I am asking, either, but your answer has 
already helped.  Indeed, PEP 249 is silent about support of UUID feilds, 
because it was written too long ago.  It is also silent about support of 
decimal and datetime fields for the same reason.  You have just taught me 
why we (meaning myself and other db-api authors) need a new, updated PEP.

As for "whoever is responsible" -- that's me!  I am the guy who maintains 
the Python ADO module.  I am in the middle of an effort to upgrade that 
module specifically for django's benefit. My problem is that I have so 
little experience as a _user_ of django that I'm not a good judge of what 
we (meaning other django users) really need. I feel like a guy who doesn't 
know how to drive designing a new car.  That's why I am asking so many 
off-the-wall questions to this group.  I need someone to say: "No, we 
really need the steering wheel to stay where it is... Now if you can do 
something about needing one foot for the gas, and one foot for the brake, 
and one foot for the clutch in order to start on a hill -- we could really 
use THAT,"

So, if I were to add direct UUID support to adodbapi, it might eventually 
get included in a new PEP, and eventually picked up by the likes of 
psycopg.  No guarantees, but it would be a start.  

My present project is to make a way for Linux users to have a door into 
ADO. That ought to help to gain some traction for django-mssql, which is 
why I am here.  I have been seeing a use for this for ages -- but am now 
working for a company who needs to have the feature working -- about three 
weeks ago. If I mention that the UUID field I am working with happens to be 
in a record of collected data points which I also need to put on a map in 
geo-django, which does not work with django-mssql (yet, and I am still 
learning geo-django) you have an idea why I need wise advice whenever I can 
get it.  Thank you. You are helping eHealth Africa to put an end to polio.

So -- is it worth my time to muck with direct database api support of uuid 
fields?  

Where am I in relation to: "Now is better than never.
Although never is often better than *right* now."?
--
Vernon

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

design decisions for django-mssql: (switching on autocommit and remote proxy) advice wanted.

[VernonCole]

Dear knowledgeable persons:

I have completed a beta-test version of a Linux remote access server for 
adodbapi, and have started the process if integrating them into 
django-mssql.   (This is going to be an interesting experience for me -- I 
have already identified two or three new features that adodbapi needs to 
have.) I would like to get your collective wisdom about how to best 
accomplish two things.

1) The transaction logic will change drastically for django 1.6.  For the 
supported backends it makes good sense to simply change that in the 
repository. However, it is reasonable to expect significant code change 
with the upgrade of this backend, and I think that we should maintain a 
single code base for a while.  I was thinking of behavior similar to what I 
do inside of adodbapi:

...if sys.version_info >= (3,0):  # note that the "if" is executed at 
import time, not run time.
..maxint = sys.maxsize
...else:
..maxint = sys.maxint

Is there something more appropriate than django.VERSION to use for this, or 
is it, perhaps, just a bad idea?

2)  In my unit test programs, I detect whether to use the proxy module or 
the regular module depending on whether their is a certain key in the 
connection arguments.

.. import adodbapi
...import adodbapi.remote

...if "proxy_host' in connection_keywords:
..db = adodbapi.remote
...else:
..db = adodbapi
...db_connection = db.connect(connection_keywords)

Would there be any technical or political reasons NOT to do that in 
sqlserver_ado/base.py ?

--
Vernon Cole

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Test failures under Oracle

[VernonCole]

How about creating django.contrib.db.backends for databases with a small 
maintainer "bus size" number?  Oracle and django-mssql could live there. 
 They would have less than full support, but still some level of official 
recognition?
--
Vernon Cole


On Wednesday, May 1, 2013 4:31:58 AM UTC-6, Florian Apolloner wrote:
>
> Hi,
>
> now that the 1.6 release plan is out, I'd like to push this topic again. 
> If you are using Oracle, now is the time to fix those bugs. Personally I am 
> toying with the idea of dropping Oracle support from core if we can't 
> ensure it's quality.
>
> Regards,
> Florian
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Progress report: django-mssql for Linux

[VernonCole]

Hello all:
   I just want to let everyone know what has happened on this proposal in 
the last little while.

1) The combination of a Linux (or Windows) database client with a Windows 
proxy database server is up and running.  The remote supports all of the 
features of adodbapi (when run locally) except: custom error handlers, 
user-defined conversions, and manipulation of module 
values-that-really-ought-to-be-constants.  I ended up using Pyro4 as the 
connection method.  It has performed well. The default operation was 
remarkably good, and after a bit of work on timing and retry logic, it 
passes tests even across VLAN on a less-than-wonderful Nigerian Internet. I 
have tested briefly using Python 2.5 and 3.3, IPv6, and extensively using 
Python 2.7 on IPv4. My final test consisted of the two suites (dbapi20 and 
adodbapitest) running on Ubuntu 13.4, using two Windows proxy servers, one 
serving SQL Server (on Windows Server 2008), the other (Windows 7) serving 
Jet, MySQL, and PostgreSQL. The latter two database were in America on 
different Ubuntu servers, so this was not a trivial test. I will not 
release this version of adodbapi until I have had time to do more extensive 
testing with IronPython and Python 3.2 -- besides, I am still making minor 
adjustments as django integration continues.

2) The code changes to have django-mssql.base call adodbapi, rather than 
its built-in fork are complete.  The forked code has been removed from my 
copy, and django is starting to run its tests.  The creation and deletion 
of the test database are done in autocommit mode, so I had to use switched 
autocommit (newly supported in adodbapi). The backend now defaults 
"autocommit = django.VERSION > (1,6)" and allows settings.AUTOCOMMIT to 
override the default. [The Manfre/Thompson fork already had a form of 
switchable autocommit, so the adaptation was trivial. They had already done 
the hard work.] This is all running django on Windows using the 1.5 stable 
branch.  

3) I will try to support 1.5 and 1.6 from the same code base.  I have not 
found any sticky spots yet.

4)  Today, I plan to try the same tests with django running on Linux using 
the proxy server. The only change should be adding   
<>  to my configuration. 
The backend will switch to using the proxy adapter when it sees that 
option. I am toying with an "auto_proxy" option which turns on 
auto-magically when it sees that it is running on non-Windows.  Is that too 
much, or a good idea?  What if it is called "macro_auto_proxy"?
--
Vernon Cole

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

django and paramstyle: what's the actual story?

[VernonCole]

I noticed in a recent post that there is an outstanding patch for Oracle 
for support of 'named' paramstyle.

I also found, during my morning's research on the question, that there has 
been at least one patch so that django will play nice with 'pyformat' 
paramstyle.

I got involved with this entire question when I back-ported Adam 
Vandenberg's paramstyle conversion code into the development fork of 
adodbapi.  It appeared to me, and seems to be widely believed in the 
community (enough so that I have stated it as a fact) that "django blindly 
expects 'format' paramstyle."  Not only django-mssql, but also the sqlite 
adapter convert incoming queries from 'format' into 'qmark' before handing 
it off to the SQL engine.  My revision to adodbapi permits the programmer 
to SET paramstyle (to one of 'named', 'qmark', or 'format') and the adapter 
does the conversion.  The version now in development adds the capability of 
making that selection using a keyword argument to the 'connect' call.  The 
patched version of the django-mssql back end just sets that keyword and 
goes on its merry way.

Recent traffic on the Python database discussion list has floated the idea 
that it is time for an upgrade to PEP 249, making a new db-api version 3.0 
with a new PEP.  The consensus opinion is running toward paring the 
available paramstyle options down to two, and forcing adapter authors to 
support both.  The two winners were 'qmark' and 'named'.  My latest input 
to that effort was a plea that 'format' must be kept because of the 
widespread use of it in existing code -- and I specifically mentioned 
django.  But was I wrong?

I am not sure whether I am sounding a warning, or asking a question.  If a 
programmer is using the ORM, the whole question of paramstyle confusion is 
unimportant -- the ORM takes care of it.  To a programmer who makes raw SQL 
calls, it is all important.  Do I make a parameter list, or a parameter 
dictionary?  

What is the feeling of this group?  Should I maintain my activism that %s 
'format' paramstyle be maintained into the future?  Or should I be helping 
the django community to prepare for a new future where one must (or may?) 
choose between a parameter list with "?" or a parameter mapping with 
":paramname"?  Personally, I do not really like the '%s' style would be 
happy if it goes away.  What do you think?
--
Vernon Cole

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Cursor closing or reuse: am I missing something?

[VernonCole]

My testing of remote mssql is proceeding nicely, or was until a few days 
ago.  After much debugging it looks like my database proxy server may be 
running out of worker threads: it uses one for each cursor.  My traces show 
that, when running the test suite in django-mssql (using ./manage.py test) 
cursors are neither re-used, nor closed.  My verbose trace shows, for each 
new test class, a connection is created, a cursor is created, one SQL 
statement is run, another cursor is created, another SQL statement run, 
etc., until the test completes. Then the connection is closed, which 
(inside my adapter) triggers the closing of all of those cursors, sometimes 
a dozen or more.

I assumed that my adapter was at fault, and have gone looking for where, in 
other adapters, they check for re-use of the cursor, or close it.  I have 
not been able to find anything.  The idea of re-using connections in 1.6 
causes me to _really_ worry.  Is this behaviour expected but only happens 
in testing? Surely a production server does not do this.  Can some kind 
person point me to some back end example code where in says "close the 
cursor here" or "find the already open cursor"?  A module name and line 
number (or text search reference) would be wonderful.

I am starting to panic.
--
Vernon Cole

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: django and paramstyle: what's the actual story?

[VernonCole]

Shai:

  I think that you are showing how rotten this whole "paramstyle" mess is: 
the thing you are describing is, IIUC, "pyformat" paramstyle.  "named" uses 
a ":name" SQL statement syntax, and expects a mapping of parameters.  My 
understanding was that Oracle expected that one.  I am only an egg.

Here's my understanding:

'qmark' uses '?' inside SQL statements, and expects parameters in a 
sequence, to be applied in order.

'format' uses '%s' inside SQL statements, and expects parameters in a 
sequence, to be applied in order.

'numeric' uses ":n" (where 'n' is an integer), and expects parameters in a 
sequence, to be applied by index number 'n'.

'named' uses ":name" inside SQL statements, and expects parameters in a 
mapping, to be applied by name.

'pyformat' uses "%(name)s", and expects parameters in a mapping.

The intent of PEP-249 is that the ORM should look at the value of 
adapter.paramstyle and use that format, whatever it happens to be.

Clearly, that is not happening.  Should it be put on the To-do list for 
some future version of django,  at least to supporting the two expected 
future winners in the parmstyle competition ('named', 'qmark')?
--
Vernon Cole
 
On Thursday, May 16, 2013 7:49:47 PM UTC+1, Shai Berger wrote:
>
> On Thursday 16 May 2013, VernonCole wrote: 
> > I noticed in a recent post that there is an outstanding patch for Oracle 
> > for support of 'named' paramstyle. 
> > 
>
> As the author of that patch, I should probably clarify that it is a 
> "format" 
> style -- %(name)s -- not a "named" -- :name -- style. 
>
> AFAIK, that style is currenly supported by all core django backends (well, 
> except Oracle, of course); I know our code which uses it has been 
> successfully 
> run against SQLite and Postgres, and I suspect django-pyodbc also supports 
> it. 
> It has its problems, both with respect to Python string interpolation and 
> to 
> SQL's pattern operator, but AFAIK it is the most common in use. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: python 3.x

[VernonCole]

"Once we're at a Django 2.6 minimum supported version, using 2to3 to
maintain
parallel implementations becomes a lot easier."

As much as I admire Russ, and I do, I don't think that the above
statement is correct.

For a short time on the pywin32 team we tried to "maintain parallel
implementations" and found that it was a mistake we had to  undo.  The
correct approach is to maintain a SINGLE implementation -- in Python 2
format -- and use 2to3 as a tool when the code happens to be running
on Python 3+.  2to3 should be run by distutils when it detects that
setup.py is being run by Python3. It should NOT be run manually by a
human.

Then, some years in the future when the last Python 2.7 engine fades
away, you will run 2to3 once for the last time, and THEN maintain in
Python 3 format. You do NOT write your code with print() functions,
etc.. Simply roll any needed refactoring into the trunk at the
earliest opportunity, and make sure you don't break them during
maintenance.

That's my advice from my experience. The code I am supporting runs on
any version of Python from 2.3 thru 3.1, including IronPython.
--
Vernon Cole

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: python 3.x

[VernonCole]

May I humbly suggest using IronPython as a first baby step?

It has the same syntax as CPython 2.6/2.7, but ALL text strings are in
unicode, just like in Python 3.x. 8-bit byte arrays must be declared
as such. I suspect that about half of the problems with Python 3
conversion will be in that very area.

To handle str vs unicode differences, I use...



def str2bytes(sval):
   """ convert a string literal to a byte array """
if sys.version_info < (3,0) and isinstance(sval, str):
sval = sval.decode("latin1")
return sval.encode("latin1")

if sys.version[0] >= '3': #python 3.x
StringTypes = str
makeByteBuffer = bytes
memoryViewType = memoryview
_BaseException = Exception
else:   #python 2.x
from exceptions import StandardError as _BaseException
memoryViewType = type(buffer(''))
makeByteBuffer = buffer
try:#jdhardy -- handle bytes under IronPython
bytes
except NameError:
bytes = str
StringTypes = (str,unicode)  # will be messed up by 2to3 but never
used


Perhaps something in that same spirit should be adopted as a standard
"drop in" for modules which are being converted?
That would keep everyone doing such patches on the same page.
--
Vernon

On Sep 7, 12:01 am, Russell Keith-Magee 
wrote:
> On Sun, Sep 5, 2010 at 5:05 PM, VernonCole  wrote:
> > "Once we're at a Django 2.6 minimum supported version, using 2to3 to
> > maintain
> > parallel implementations becomes a lot easier."
>
> > As much as I admire Russ, and I do, I don't think that the above
> > statement is correct.
>
> My apologies -- in the haste of getting a response out, I was a little
> lax in my choice of words. What you've described - i.e.,
>
> * A single maintained 2.X source tree
> * An auto-generated 3.X source tree,
> * When 2.X support is dropped, the migration script is run one last
> time to migrate the source tree to be 3.X
>
> is pretty close to what I had in my head as the likely path. In
> practice, I'm sure there will be some complications, but we won't
> really know what they are until we get serious about 3.X support.
>
>
>
> > For a short time on the pywin32 team we tried to "maintain parallel
> > implementations" and found that it was a mistake we had to  undo.  The
> > correct approach is to maintain a SINGLE implementation -- in Python 2
> > format -- and use 2to3 as a tool when the code happens to be running
> > on Python 3+.  2to3 should be run by distutils when it detects that
> > setup.py is being run by Python3. It should NOT be run manually by a
> > human.
>
> > Then, some years in the future when the last Python 2.7 engine fades
> > away, you will run 2to3 once for the last time, and THEN maintain in
> > Python 3 format. You do NOT write your code with print() functions,
> > etc.. Simply roll any needed refactoring into the trunk at the
> > earliest opportunity, and make sure you don't break them during
> > maintenance.
>
> > That's my advice from my experience. The code I am supporting runs on
> > any version of Python from 2.3 thru 3.1, including IronPython.
>
> It's good to know that there are people in the community that have
> done this in anger on other projects. If anyone can provide patches
> for refactors that are necessary in order to simplify the 3.X
> migration process, I'm happy to apply those patches -- and as I
> indicated, I already have applied a couple of patches for exactly this
> reason; for example changeset 13509 was to change the way we use
> sorting functions to avoid a keyword argument that has been deprecated
> for 3.X.
>
> Yours,
> Russ Magee %-)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Dropping support for old Python versions in Django 1.4

[VernonCole]

Jython is also at release 2.5, so droppin
ig 2.5 would be harmful to that implementation.
CentOS will be the same as RHEL.


On Feb 4, 11:26 am, John Anderson  wrote:
> On Feb 4, 2011, at 12:23 PM, Łukasz Rekucki wrote:
>
> > With Django 1.3 almost out, I would like to ask, what's the current
> > deprecation plan of old Python versions (namely 2.4 and 2.5). The
> > major argument against dropping 2.4 was RHEL support. RHEL6 seems to
> > support Python 2.6 (dunno about CentOS), so is that still a blocker ?
>
> > As for dropping 2.5, the potential issue is that it's still quite
> > widely used by hosting services (would be nice to have some fresh data
> > about this;http://www.djangosites.org/stats/is useful, but probably
> > a bit outdated).
>
> > I wanted to start this discussion before 1.3 is released, because I
> > think that if we decide to drop 2.4, we should give an early warning
> > in 1.3 release notes, so that people can switch over more smoothly.
>
> I think dropping 2.4 would be a good idea but I think 2.5 needs to be
> kept around so that Django can continue to run on appengine. I don't
> believe appengine has any plans of moving to 2.6+ any time soon.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Timezone-aware storage of DateTime

[VernonCole]

On May 31, 7:23 am, Daniel Swarbrick 
wrote:
> I can almost hear the collective sigh as this topic once again rears
> up ;-)
>
Thanks for bringing it up.  I lurk both here and on db-sig, and intend
to use your well done research to once again attempt an update of the
db-api PEP, which also does not address time zone aware time.  I like
the idea of standardizing a cross-platform TZ aware data column.
[...]
> The way I see it, there are a few options for storage of timestamps in
> SQLite (whose docs clearly acknowledge that it does not officially
> support a timestamp column type).
>
> 1. Store timestamps as a Unix epoch value (integer). This is fast, and
> will ensure correct date sorting. On the other hand, it's not really
> human-friendly if manually browsing table data in SQLite.

Not only is the epoch limited in range, I would be wary of using any
storage based on the python time module (as opposed to datetime).  We
have discovered enough subtle bugs in the Windows and IronPython
implementations that we gave up trying to get an accurate test of the
python time option on adodbapi. Things that worked correctly for me in
North America would fail for Mark Hammond in Australia. It is a maze
of twisty little passages. We simply recommend that everyone use
datetime.

On the other hand, I found it necessary to convert date-time values to
ISO format strings in order to feed them to Microsoft ACCESS databases
in some cases, and that works well. But again, as with SQLite, the DB
has no concept of time zones. In the absence of actual time zone
support in the database, any action taken by django is going to
inconvenience somebody, and will likely not be compatible with non-
django use of the same database.  Perhaps it would work to store the
pickled tz-aware datetime on brain-damaged databases.  But, is it
reasonable to use an application-specific method to extend the
capability of a database engine?
--
Vernon Cole

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3 and you

[VernonCole]

> One assumption of the strategy I outlined was the fact that Django is
> as close to 3.X as possible. Django 1.4 will require Python 2.5 or
> higher, but I'm not sure how quick we can do the jump to 2.6, which
> is recommended by the Python porting docs [1].
>
Don't confuse "recommended" with "necessary."  I maintain a module
which installs and runs very nicely on any version of Python >= 2.3.
The automatic operation of 2to3 during the install is like magic.  It
really works very well.

> >   Finally, a philosophical question on approach: Should we really be
> > doing 2to3 (leaving the Django codebase in Python 2.X for a long time)

Yes, that's exactly what you do -- just like you code everything in
Python 2.4 until everyone is ready (or able) to jump to 2.5.  You
simply avoid using new language features until that point.  If
fortunate people have a later version, then they can use new language
features in their code, just us module authors cannot use them.  But
we plan for them.  For example, we add __enter__() and __exit__()
methods to objects where that makes sense.  If the programmer is using
an older compiler, then they will never get called, but they will be
present when someone with a new compiler uses a "with" statement.  We
cannot use "with" in module code, though, until all python 2.4 users
have disappeared.

Same thing when moving to Python 3.x -- you cannot use any new
features, but you have to refactor your old code in such a way that
2to3 can do the right thing when it installs.  Mostly this means
cleaning up old code which was probably due for a maintenance pass
anyway.  Get rid of string exceptions.  Make all class definitions
into new style classes.  Things like that.

> > or would it be better to port Django over to Python 3 & use 3to2 for
> > existing Python 2.X installs? I confess I don't know much about the
> > current state of 3to2 (nor how most other Python libraries are
> > handling the transition). But I do know Django will continue to grow
> > over time & I worry that, at some point in the future we'll be making
> > more even more work for someone else to do the 3-only work.

N!   There is no 3-only work until the last 2.7.n user upgrades.
When that glorious day happens, we will make an archive copy of the
old version, and then run 2to3 for the last time.  After that, code
would be maintained in the oldest supported 3.x version.
>
> I personally haven't ported a 2.X library completely to 3.X yet, so I
> can also only guess. But from what I've seen in the community I'm afraid
> of a "clean cut" port because it has a high risk of leaving many projects
> and apps behind. In that sense it seems more sensible to me to see the
> port to Python 3 just as another step of our Python version deprecation
> policy, which we at some point take with a complete conversion. Basically
> a "burn bridges as soon as everyone is safe" approach :)

Yes, that is exactly correct -- and I HAVE ported a 2.x library. I'm
guessing that the jump will be from 2.7 to  about 3.3.No sense
supporting obsolete 3.x versions..
>
> I don't dare to guess when that moment could be though, but it would probably
> happen after a potential Python 2.7 only release of Django -- whenever that 
> is.
>
Years  lots of years, before 2.7 disappears.  Hopefully, by then,
others will have been running 3.x versions for years, and will have
long since worked out the kinks.
--
Vernon

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Passing parameters to raw()

[VernonCole]

Officially (according to PEP-249) you are not supposed to be able to do 
that (pass a dictionary  of parameters), but PostgreSQL does it as an 
extension to the spec.  SQLite and ms-sql (both of which use format 
converters to send your queries out with '?' where you put the '%s') do NOT 
support it.  Shai is supplying a patch for Oracle (which uses a different 
format conversion) to support it.
 
There is a very noisy discussion taking place right now on the DB-SIG about 
the design of a future db-api version 3.  I am suggesting that we support 
automatic switching (similar to what you want to use here) but at the 
moment, I am loosing, and I do not expect that it will happen.  So you can 
do that now, on some databases, but do not expect it to be either universal 
or future proof.  I would advise sticking with a list of parameters for 
your present development efforts.

On Wednesday, May 22, 2013 3:37:55 AM UTC-6, roman wrote:
>
> Hello, 
>
>
> the docs say: 
>
> """ 
> Passing parameters into raw() 
>
> If you need to perform parameterized queries, you can use the params 
> argument to raw(): 
>
> >>> lname = 'Doe' 
> >>> Person.objects.raw('SELECT * FROM myapp_person WHERE last_name = %s', 
> [lname]) 
>
> params is a list of parameters. You’ll use %s placeholders in the 
> query string (regardless of your database engine); they’ll be replaced 
> with parameters from the params list. 
> """ 
>
> howerver this woks just fine and I see no reason why this should not be 
> done: 
>
> >>> param = dict(lname = 'Doe') 
> >>> qs = Person.objects.raw('SELECT * FROM myapp_person WHERE last_name = 
> %(lname)s', param) 
>
> however still this fails: 
>
> >>> repr(qs) 
> /home/user/vpy/dev/lib/python2.7/site-packages/django/db/models/query.pyc 
> in __repr__(self) 
>1530 
>1531 def __repr__(self): 
> -> 1532 return "" % (self.raw_query % 
> tuple(self.params)) 
>1533 
>1534 def __getitem__(self, k): 
>
> TypeError: format requires a mapping 
>
> If no one objects, I could write a patch to the code and the docs that 
> implements the functionality. 
>
> Regards Roman 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Passing parameters to raw()

[VernonCole]

Understand that this will not affect django for the next several releases.  
Probably django 2.0.  The transition from db-api 2 to db-api 3 will be a 
lot like the transition from Python 2 to Python 3.  You will have to import 
a different module to get the different behavior, and today's api-2 
interface will linger around for innumerable years to come.

The reason that 'format' and 'pyformat' have fallen into disfavor is that 
they *look* like the %s and %(identifier)s used by the string "%" operator, 
but they don't quite *work* like them.  That causes no small amount of 
confusion and trouble -- such as having to use %%s sometimes but not 
others.  I need not explain to this group, you have all been bitten by it, 
or will be.  Someday, some poor human will, I suspect, have to look at 
every occurrence of %s in the entire django code base and decide whether to 
replace it with '?' because it is going to an SQL query, or with "{}" 
because it will be a string format operation. The only examples of %s still 
left will be in templates, and in ADO connection strings.  [Having heard 
myself say that, I may decide to write a .format() method for Python 2.5 so 
that I can keep them out of ADO connection strings -- so that that problem 
would never appear.  Must give us pause.]
 
Expect % as a format specifier in an SQL query to go away ... someday ... 
but not today, and not tomorrow.
--
Vernon Cole

On Wednesday, May 22, 2013 3:37:55 AM UTC-6, roman wrote:
>
> Hello, 
>
>
> the docs say: 
>
> """ 
> Passing parameters into raw() 
>
> If you need to perform parameterized queries, you can use the params 
> argument to raw(): 
>
> >>> lname = 'Doe' 
> >>> Person.objects.raw('SELECT * FROM myapp_person WHERE last_name = %s', 
> [lname]) 
>
> params is a list of parameters. You’ll use %s placeholders in the 
> query string (regardless of your database engine); they’ll be replaced 
> with parameters from the params list. 
> """ 
>
> howerver this woks just fine and I see no reason why this should not be 
> done: 
>
> >>> param = dict(lname = 'Doe') 
> >>> qs = Person.objects.raw('SELECT * FROM myapp_person WHERE last_name = 
> %(lname)s', param) 
>
> however still this fails: 
>
> >>> repr(qs) 
> /home/user/vpy/dev/lib/python2.7/site-packages/django/db/models/query.pyc 
> in __repr__(self) 
>1530 
>1531 def __repr__(self): 
> -> 1532 return "" % (self.raw_query % 
> tuple(self.params)) 
>1533 
>1534 def __getitem__(self, k): 
>
> TypeError: format requires a mapping 
>
> If no one objects, I could write a patch to the code and the docs that 
> implements the functionality. 
>
> Regards Roman 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Testing generated SQL under different backends

[VernonCole]

I have just run afoul of one of those "features that make no sense except 
when combined with the test suite."In BaseDatabaseFeatures it looks 
like this:

   # Does the backend allow very long model names without error?
supports_long_model_names = True

What this flag actually does is (when False) eliminate generation of one 
specific test table when setting up the test suite. The test creation logic 
only considers the setting of the flag on the 'default' database, so if, as 
in the case I was attempting, 'default' is PostgreSQL and 'other' is 
ms-sql, the construction of the test configuration fails.  I wanted to test 
that configuration specifically, since I think it is one of the wiser ways 
to run django with SQL Server involved. But I digress...

My point is that an example exists in the core code, and that, indeed, it 
is not very maintainer friendly.  In addition to not addressing the 
question of "how long is 'long'?", it requires a full text search of the 
source code to figure out the relationship.  If, instead, the failing setup 
module were coded with actual vendor names, and a comment like "# vendor_x 
table names are limited to 132 characters", it would be much easier to 
maintain the tests correctly, IMHO.

I maintain a test suite which has four target database engines, and the 
test code has plenty of examples if skipping (or modifying) a test based on 
code like "if db.engine is in ['mssql', 'postgress']:". I find that such an 
arrangement works very well.


On Tuesday, May 28, 2013 12:26:44 AM UTC-6, Anssi Kääriäinen wrote:
>
> On 27 touko, 20:15, Shai Berger  wrote: 
> > Hi Carl, 
> > 
> > On Monday 27 May 2013 19:37:55 Carl Meyer wrote: 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > > Hi Shai, 
> > 
> > > On 05/27/2013 09:26 AM, Shai Berger wrote: 
> > > > I'm working on fixing some failing tests under Oracle, and I ran 
> into 
> > 
> > > >   commands_sql.tests.SQLCommandsTestCase.test_sql_all() 
> > 
> > > > [...] 
> > 
> > > > For now, I will only special-case Oracle -- that should solve a 
> standing, 
> > > > release-blocker bug, and not change the semantics of the test 
> otherwise; 
> > > > but I'd like to achieve something better, more general and 
> > > > 3rd-party-backend- friendly, for the future. 
> > 
> > > It seems to me that ideally a test for backend-specific behavior 
> should 
> > > become a test for that backend (and thus skipped on other backends). 
> > > This also solves the third-party-backend problem; said backend should 
> > > have its own tests as needed, and Django's tests for backend-specific 
> > > behavior should be skipped under any unrecognized backend. 
> > 
> > I agree in general, but this can lead to DRY violations if we're not 
> careful. 
> > A better example of the problem is a test next to the one I needed to 
> fix: 
> > 
> > def test_sql_delete(self): 
> > app = models.get_app('commands_sql') 
> > output = sql_delete(app, no_style(), 
> connections[DEFAULT_DB_ALIAS]) 
> > # Oracle produces DROP SEQUENCE and DROP TABLE for this command. 
> > if connections[DEFAULT_DB_ALIAS].vendor == 'oracle': 
> > sql = output[1].lower() 
> > else: 
> > sql = output[0].lower() 
> > six.assertRegex(self, sql, r'^drop table .commands_sql_book.*') 
> > 
> > Does any backend beside Oracle produce extra SQL for this command? Does 
> the 
> > different command index for Oracle justify separating this test into two 
> > separate methods? And if you separate, would you mark the non-oracle 
> case with 
> > skipIf(Oracle) or with skipUnless(sqlite or postgres or mysql) ? 
> > 
> > I think a better solution for this is to keep the original method, and 
> mark it 
> > with skipUnless(is_core_db) -- we'd need to define is_core_db for that, 
> of 
> > course; this could also serve as an easy-to-grep marker for "general 
> > functionality test, with backend variations" -- which I think would be 
> quite 
> > useful for 3rd-party backend writers. 
> > 
> > Shai. 
>
> Traditionally a backend features has been used for this. So, add some 
> feature like "produces_extra_sql_for_drop_table", if so we skip based 
> on that. This unfortunately leads to features that make no sense 
> except when combined with the test suite. 
>
> One idea is to somehow include a subclass of problematic test classes 
> in each backend. The backend can alter or skip those tests that are 
> backend specific. There are a couple of test apps that should work in 
> this way (introspection, backends, inspectdb, maybe more). 
>
> I don't know how to make this actually work so that Django's test 
> runner finds the correct test class from the backend's test module 
> instead of the generic backends test class. Optimally a backend could 
> subclass any test class to specialize or skip tests. 
>
> For 3rd party backend support in general: My opinion is that we should 
> include them all in core. The rules would be: 
>   1

Re: Supported Python versions for Django 1.7

[VernonCole]

Dropping support of 3.2 would potentially aid projects which have not yet 
converted to Python 3, since Python 3.3 supports u"Unicode Literals" but 
3.2 does not. Skipping over the early 3.x versions vastly eases transition 
from 2.6 to 3.3. 

Besides, what are the chances that someone running Python 3.n is _not_ 
running 3.3?
--
Vernon Cole

On Friday, June 28, 2013 8:17:22 AM UTC-6, Aymeric Augustin wrote:
>
> Hello,
>
> We just forked the stable/1.6.x branch. The development of Django 1.7 
> starts now!
>
> As far as I can tell, there's a consensus on dropping support for Python 
> 2.6. That will allow us to remove the vendored copy of unittest2 and to 
> take advantage of datastructures introduced in Python 2.7 like OrderedDict.
>
> I think we can continue supporting Python 3.2 in addition to Python 3.3 
> and 3.4. But if you see good reasons to drop it, I'd like to hear them!
>
> Thank you,
>
> -- 
> Aymeric. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Deprecate FCGI support in Django 1.7

[VernonCole]

Let me second the need for a good "How To" document on deployment.  As a 
neophyte django user I suddenly felt very lost last week. After several 
months of development, the ADO interface to the Microsoft SQL server from 
Linux is working (it still needs some tuning, but the data does go in and 
out using runserver) and it is now time to deploy.  The boss requested an 
Nginx server, so I started searching the web for how to do it.  I had come 
the the conclusion that FastCGI was the only way that I could deploy and 
was planning on starting that this morning, until I read this thread.  Now 
I know that I was about to start down an obsolescent path. A "How To" in 
the official documentation would have prevented this false start.

Does anyone know of an existing cookbook that I can refer to as a starting 
point? Or should I be making accurate records of my work with a view to 
contributing a new document?
--
Vernon Cole
 

On Saturday, July 20, 2013 5:32:44 AM UTC-6, Christian Schmitt wrote:
>
> I would recommand to remove FastCGI. 
> That's the thing i love with the Python Community. Remove depracted or 
> obsolete things.
>
> The thing is, there are way more important things to do, than supporting a 
> 'depracted' way to deploy. And it's really not that hard to get mod_wsgi 
> with a httpd running.
> You could strip that down so easily and compile it, even on CentOS.
>
> Maybe it's better to improve the Docs about Apache2 / nginx / HAProxy 
> deployment with mod_wsgi than supporting FCGI.
> I mean some things like that:
>
> > 
> http://ontwik.com/python/django-deployment-workshop-by-jacob-kaplan-moss/
>
> I think that these things are really important. Howto get from development 
> to Production, with security.
> (btw. the talk is from 2011 and some things aren't up 2 date, like the 
> pgpool single point of failure thing, etc. today you could really well 
> deploy a good enviroment or 2 or more servers where you don't have any 
> single point of failure where you don't need to add any extra line of code. 
> okai maybe you need to find a good way to hold cookies in a good memory 
> based database, that you could share them between machines, but that is the 
> only thing you should care about, since that is the only you shouldn't hold 
> in your legacy database.)
>
> Am Sonntag, 14. Juli 2013 21:17:00 UTC+2 schrieb Florian Apolloner:
>>
>> Hi,
>>
>> I'd like to get rid of everything FCGI-specific in Django sooner or later 
>> (rather sooner). Flup isn't maintained since a long time and there is no 
>> ticket tracker to report stuff. Graham pointed out that if someone wants to 
>> use FCGI they can use 
>> http://uwsgi-docs.readthedocs.org/en/latest/Options.html#fastcgi-socketwhich 
>> doesn't even require flup, which sounds like a good compromise to me. 
>> I'd need some help for the docs from some uWSGI users, since I have no idea 
>> about it ;)
>>
>> Thoughts, objections?
>>
>> Cheers,
>> Florian
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: tests failing/throwing errors - not sure where to start

[VernonCole]

Kris:
  9 failures and 17 errors are not an unusual number (out of 556 tests) 
given the complexity of django and the number of platforms it works on. 
 The number and selection of failed tests will vary depending on the exact 
implementation you are running. However, I notice that _none_ of the errors 
your are seeing are in core django -- all are in either django.contrib or 
another package.  Also -- you are not running a development (or indeed, 
even a current) version of django, so the errors you have seen have very 
possibly been fixed in the latest version(s).
  Given that "The project I'm working on seems to work fine" I would say 
that there is no cause for concern.  If you wish to help test and/or debug 
django, you should do so using the latest version.
--
Vernon

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Order of INSTALLED_APPS

[VernonCole]

It also affects _how_ South works, not just _whether_ it works.  Only this 
week I had to install a patch for the formhub package which consisted of 
re-ordering INSTALLED_APPS so that tables were built in the correct order 
on PostgreSQL installations.

On Sunday, September 1, 2013 11:04:03 PM UTC+1, Marc Tamlyn wrote:
>
> I don't think it would break South installations as custom app commands 
> will always override builtins. Could you open a ticket for that? The 
> inconsistency is problematic. Obviously it would be backwards incompatible, 
> but we need to gauge the size of the impact.
>
> Marc
> On 1 Sep 2013 21:31, "Kevin Christopher Henry" 
> > 
> wrote:
>
>> Sorry to be late to this thread, I just came across it.
>>
>> There's another place where the order of INSTALLED_APPS matters: 
>> management commands. Management commands associated with apps that come 
>> later in INSTALLED_APPS will replace those with the same name that are 
>> listed earlier. I can't find this documented anywhere, but a look at the 
>> code confirms it. South, for example, takes advantage of this to override 
>> the syncdb command (and hints at the ordering issue when the documentation 
>> says: "add 'south' to the end of INSTALLED_APPS").
>>
>> So a documentation update should probably note this specifically in the 
>> management documentation, as well as mentioning it in the box for 
>> INSTALLED_APPS. (I also think static files should also be mentioned in the 
>> box, since as Aymeric points out that is another area dependent on the 
>> order of apps.)
>>
>> One thing that's unfortunate is that the semantics of ordering for 
>> management commands is opposite that of the other cases. For templates, 
>> static files, and translations, listing an app *first *gives it 
>> precedence, whereas with management commands it's listing it *last.*Ideally 
>> this would be changed so that management commands were consistent 
>> with the other cases, but that would create serious backwards compatibility 
>> issues. Like, breaking everyone's South installation. :-O
>>
>> Cheers,
>> Kevin
>>
>>
>> On Wednesday, August 14, 2013 5:26:47 AM UTC-4, Stefano Crosta wrote:
>>>
>>> Done! 
>>> https://code.**djangoproject.com/ticket/**20914#ticket
>>>
>>> thanks!
>>>
>>> On Tuesday, August 13, 2013 12:20:48 PM UTC+2, Aymeric Augustin wrote:

 2013/8/13 Stefano Crosta 

> My proposal would then be to simply add another box to the 
> https://docs.**djangoproject.com/en/dev/ref/**settings/#installed-apps
>  to 
> say "order matters" once more and link the other two pages for 
> translations 
> and templates.
> *if you think this would* help I could do it as well as a ticket. To 
> save everybody's time no answer will mean it's not worth it!
>

 Yes, please file one, and include a link to this discussion.

 -- 
 Aymeric. 

>>>  -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com .
>> To post to this group, send email to 
>> django-d...@googlegroups.com
>> .
>> Visit this group at http://groups.google.com/group/django-developers.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Add strutctured settings module to django 1.7?

[VernonCole]

I find myself using up lots of time and keystrokes explaining about the 
benefits and methods of a structured settings module in django.  
For example: 
Using-a-Structured-Settings-environment

It occurs to me that life would be easier if django shipped already set up 
for structured settings, rather than having to retro-fit it.  The pull 
request I sent to formhub could be a starting point for the conversion. (
formhub/pull/1240)   The 
changes to core are simple enough that I could do it.

Your  opinions please, Would such a change be good for django 1.7?
--
Vernon Cole
eHealth Africa

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Add strutctured settings module to django 1.7?

[VernonCole]

Okay, I read it.  I think that it argues for structured settings, not 
against them.

What I read was a set of guidelines, rather like the Zen of Python. They 
look like good, reasonable rules-of-thumb to me, and I have been around 
long enough to recognize such.  One of the reasons I adopted Python was 
that I found my self in such complete agreement with the Zen the first time 
I read it. I particularly liked its built-in admission that there will be 
exceptions to its principles.  

Let's talk about three of these twelve.  I have cut-and-paste-ed them from 
the web page.

I. Codebase <http://12factor.net/codebase> One codebase tracked in revision 
> control, many deploysIII. Config <http://12factor.net/config> Store 
> config in the environmentX. Dev/prod 
> parity<http://12factor.net/dev-prod-parity> Keep 
> development, staging, and production as similar as possible


Note that number three says "in the environment" not "in a large collection 
of operating system environment variables". Those are different things. 
More about that later.

First, note that rule number ten works against rule number one.  The reason 
that the codebase we call "django" gets deployed so often is because it is 
so flexible.  The same codebase runs on a Windows laptop and a Java server 
in a dark server room.  "As similar as possible" gets stretched a very long 
way.  Yet the fact that I am actually running the same codebase in both 
situations means that there are a _lot_ of similarities - more than there 
are differences -- because the codebase is programmed to minimize the 
differences.   But there must be a way to tell that large, flexible 
codebase what we need it to do today -- and today in django, that is done 
in settings.py.

Other products have other methods of specifying their configurations.  The 
database system I helped write in 1982 had a database-of-databases: you 
switched environments by specifying (using a command line switch or an 
operating system environment variable) which root database you were using 
at that moment.  It eventually worked on eight different operating systems 
from RT-11 to Windows-32.  Microsoft dotNET uses xml files stored alongside 
the executable files for their configurations.  If you have ever had to 
modify one of those monsters, you will appreciate the simplicity of 
settings.py.  At times, I have written Python scripts to modify the dotNET 
config files in place. Ugly! Then there is the Windows registry, lets say 
no more about that.  The point is: configurations have to be stored 
somewhere.

So how do we store them "in the environment"?  

1) As multiple operating systems environment variables.  Each operating 
system will need to be separately documented as to how to set environment 
variables.  (Have you ever tried to explain that to a beginning Windows 
user?)  Each *nix shell will also require separate documentation, and a 
separate shell script to set them.  Here there be dragons! Or, perhaps, 
someone will write a Python script to set them all.  :-o

2) In a pre-defined database-database, perhaps overridden by an environment 
variable or command line switch.

3) In a generic file with a known name, perhaps overridden by an 
environment variable or command line switch, in a format as yet to be 
defined and fought over.

4) In a Python file with a known name, perhaps overridden by an environment 
variable or command line switch, which can import more generic 
configuration files, which can import others, until a base standard 
configuration is reached.

5) In a Python file with a known name, perhaps overridden by an environment 
variable or command line switch, which may import another Python file with 
a known name, which cannot be stored in version control.

Numbers three or four could be included in version control, and selected at 
run time as needed.  I think that is a much more secure "environment" than 
a motley set of instructions of how to set environment variables in shell 
xyz, or how to encode some kind of URL giving the same information.

Numbers four and five use a format already familiar to django users, and 
which is extensible so that more advanced methods of configuration setting 
(such as the two mentioned elsewhere in this thread) can easily be 
implemented, if desired.

Number five is what we have today.

I am suggesting number four.

Did that sell anyone?
--
Vernon


On Thursday, September 26, 2013 12:45:20 AM UTC+1, Russell Keith-Magee 
wrote:
>
>
> On Thu, Sep 26, 2013 at 4:21 AM, VernonCole 
> > wrote:
>
>> I find myself using up lots of time and keystrokes explaining about the 
>> benefits and methods of a structured settings module in django.  
>> For example: 
>> Using-a-Structured-Settings-environment<https://github.com/modilabs/formhub/wiki/Using-a-Structured-Settings-environment>
>>
>

Re: Add strutctured settings module to django 1.7?

[VernonCole]

(Memo to self: always think about it for a day before replying.)

Okay, I see.  And I certainly agree that a few strategically placed 
instances of (for example) 
>>> psw = os.environ.get('DBPASSWORD', 'DumbDefault')
would help security in most cases.

I will get back to the drawing board, then, and see if I can still get the 
optimization I am looking for without making the system more difficult for 
those who don't want it.

My use case, is, I'm afraid, even more wrong than you suggest, Marc.  I 
develop on Linux, but have to plan for instances where unfortunate 
individuals must deploy on Windows.  Administrators of those systems are 
often neophytes at systems administration in general and Python in 
particular.  Installation of something like envdir would be far beyond 
hope.   More frequent will be deployments by people who may know Windows 
systems administration, but this will be their first experience with 
Linux.  Simplicity is everything.

Thank you, everyone, for the quality input.
--
Vernon

On Thursday, September 26, 2013 11:26:01 AM UTC+1, Marc Tamlyn wrote:
>
> No, when 12 factor says in the environment, it means exactly that. *NOT* 
> checked into version control, and in the environment of the shell. 
> 12-factor is talking *only* about number 1 for differences in your 
> environments.
>
> Yes windows is not very good at environment variables in general, but I 
> would argue if you're developing on windows and deploying to *nix you're 
> already doing it wrong. I refer you to point 10. You may however be 
> interested in Jannis' python port of envdir - 
> https://github.com/jezdez/envdir - which works on Windows.
>
> Personally, I find the multiple settings files (local_settings.py, 
> structured settings) to be a pattern I'm moving away from, and I'm unlikely 
> to support changes to Django encouraging it more. Jannis' configurations is 
> a bit different, I find it's a nicer way of grouping your settings 
> together, making them overrideable from external apps etc. The end game is 
> still one settings file, with environment switches.
>
> The advantage of the current settings.py is that it's very simple and 
> flexible, which allows these patterns to all work together. At some point 
> we may change to another system, but I don't feel we should be actively 
> encouraging any particular path at present. There isn't a clear consensus 
> as to the best way forwards.
>
>
> On 26 September 2013 11:11, VernonCole  >wrote:
>
>>
>> Okay, I read it.  I think that it argues for structured settings, not 
>> against them.
>>
>> What I read was a set of guidelines, rather like the Zen of Python. They 
>> look like good, reasonable rules-of-thumb to me, and I have been around 
>> long enough to recognize such.  One of the reasons I adopted Python was 
>> that I found my self in such complete agreement with the Zen the first time 
>> I read it. I particularly liked its built-in admission that there will be 
>> exceptions to its principles.  
>>
>> Let's talk about three of these twelve.  I have cut-and-paste-ed them 
>> from the web page.
>>
>> I. Codebase <http://12factor.net/codebase> One codebase tracked in 
>>> revision control, many deploysIII. Config <http://12factor.net/config> 
>>> Store 
>>> config in the environmentX. Dev/prod 
>>> parity<http://12factor.net/dev-prod-parity> Keep 
>>> development, staging, and production as similar as possible
>>
>>
>> Note that number three says "in the environment" not "in a large 
>> collection of operating system environment variables". Those are different 
>> things. More about that later.
>>
>> First, note that rule number ten works against rule number one.  The 
>> reason that the codebase we call "django" gets deployed so often is because 
>> it is so flexible.  The same codebase runs on a Windows laptop and a Java 
>> server in a dark server room.  "As similar as possible" gets stretched a 
>> very long way.  Yet the fact that I am actually running the same codebase 
>> in both situations means that there are a _lot_ of similarities - more than 
>> there are differences -- because the codebase is programmed to minimize the 
>> differences.   But there must be a way to tell that large, flexible 
>> codebase what we need it to do today -- and today in django, that is done 
>> in settings.py.
>>
>> Other products have other methods of specifying their configurations.  
>> The database system I helped write in 1982 had a database-of-databases: you 
>> switched en

Re: Add strutctured settings module to django 1.7?

[VernonCole]

Not off topic at all, Carl, this back-and-forth is helping me, at least, 
consider some new things.

Here's what I have learned from this discussion so far:

1. Clearly, any change needs to avoid use of the word "structured", since 
that is now yesterday-ish.

2. Any change needs to lead toward and encourage and/or ease use of 12 
factor methodology.

3. The settings.py file needs to stay where it is and do what it does 
without much change.

4. There is a quiet wish for something better than the status quo, but it 
must  be flexible enough to allow new and different advanced solutions.

5. It needs to be simple to use for development, demo, test, tutorial, and 
semi-skilled implementations.

6. A URL-like scheme might be a good default option.

The original idea with which I started this thread miss-fired on points one 
and three, and I was completely unaware of point two. Point six was 
something I had rattling in the back of my mind for years, but could not 
envision a way to do it -- especially with something as simple and elegant 
as Kenneth Reitz's work. 

So I started mucking around and I like the emerging design much better.  I 
have received a bit of positive feedback on it already, and I think I may 
have found a decent compromise where practicality and purity can co-exist 
as multiple example files, and the implementer can pick from them his own 
best options.  I have several example files there now, which effectively 
replace the not-verbose-enough commentary in settings.py which tries to 
tell how to set up the DATABASES dictionary. They include working examples 
which I use for actual testing against multiple db engines. (My intranet is 
like my house: just enough security that breaking in is not trivial, and 
nothing inside worth stealing.)   Since each of the small, individual 
examples is simple, I have started to add 12-factor features (like getting 
a password from the environment) to some of them.  I will add samples of at 
least one "pure" 12-factor method.  I may decide to use a version of 
dj-database-url as the default.

I moved settings.py back where it belongs.  Since the folder with the 
examples cannot also be named "settings", I renamed it "preset". So, 
presently, my modified manage.py sets the default value of 
DJANGO_SETTINGS_MODULE to "formhub.preset.default_settings".   

As always: any suggestion is welcome.
--
Vernon

On Friday, September 27, 2013 11:07:28 AM UTC-6, Carl Meyer wrote:
>
> On 09/27/2013 10:52 AM, andres@gmail.com  wrote: 
> > What is the difference between privately tracking a shell file and a 
> Python config file? 
>
> Not a lot of difference, but the env vars give you more flexibility. In 
> a single settings file using env vars, I can easily have things like a 
> "mode switch" in a single env var that affects the defaults for a number 
> of related settings, without having to repeat myself on redefining all 
> those settings. That kind of thing is harder to do cleanly when you're 
> "inheriting" one settings file from another using "from settings.base 
> import *", or using local_settings.py. 
>
> (Since I don't think Django core is going to take any steps to endorse 
> any particular approach to handling settings anytime soon, I think this 
> discussion is really off-topic and should move somewhere else. I should 
> have considered that before replying to Tom.) 
>
> Carl 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Using setuptools to make django-admin.py runnable on Windows (#21340)

[VernonCole]

Waylan is correct, but does not go far enough.

When one installs Python 3.3, the Python Launcher for Windows is also 
installed. I install Python 3 even on systems where I do not (at present) 
plan to use it, just to get the launcher. The launcher is the program which 
should be associated with .py files on Windows. It is responsible for 
reading the #! line at the top of your Python file and running the correct 
version of Python. The launcher works so well at handling different 
versions that I keep wanting to install it on my Linux boxes as well.

So, I would agree that the Windows installation instructions might indeed 
be altered -- to include installing Python 3.3 (or later), and associating 
the c:\windows\py.exe with .py files.

If you plan to use only a single version of Python, putting that directory 
(and its "scripts" directory) in your PATH is a good idea. But, I feel that 
virtualenvwrapper-win is 
the best way to go.  It works almost exactly like virtualenvwrapper for 
Linux, and if used with PyCharm 3.0 one can almost forget which OS one is 
using.

I am going to investigate using the launcher to pick the correct 
virtualenv.  Meanwhile -1 on the patch.

a
On Sunday, November , 2013 4:58:49 PM UTC-7, waylan wrote:
>
>
>
> On Sunday, November 24, 2013, Rémi Rampin wrote:
>
>> Hi developers,
>>
>> On Windows, running the django-admin.py tool is painful[1], because
>> .py scripts are not "executable". You might be able to run it using
>> the full path (if Python is the default handler for .py files, which
>> it really shouldn't be). Most probably you'll need to copy it to your
>> project directory and prefix it with "python " each time.
>
>
> There are a few things I do whenever I install python on windows.
>
> 1) first, make sure the .py extension is associated with python.exe.
> 2) add the python directory to my path.
> 3) add the scripts directory to my path.
>
> Then, when any packages install scripts (like django-admin.py), they ‘just 
> work’.
>
> If you have multiple versions on python installed, then you can copy 
> python.exe to pythonX.X.exe and make sure the defult version (the version 
> associated with .py) is listed first on your path. The order of dirs on 
> your path in key here, but you can list all the versions on your path. Then 
> you can just type `python3.3` for example, and you'll get that version just 
> like on unix systems.
>
> Of course, on unix based systems we all use virtualenv. There are a few 
> windows virtualenv clones implemented for windows either as batch scripts 
> or powershell scripts. You might want to look into those. Although, I 
> haven't used them myself, so I can't speak to their completeness, 
> reliability or usefulness. I always have git installed which comes with 
> mssys and gives me a bash shell on windows - the best solution IMO 
> (although virtualenv can be a little janky).
>
> Of course, this has entered django-users territory. And additional 
> discussion about how to work around windows limitations should probably be 
> discussed there. My point is that with a correctly configured system, this 
> is not a problem on windows at all.
>
> If there are any changes to make to Django, perhaps the install docs could 
> suggest the additions to the path I mention about as specific to installing 
> on windows.
>
>
> -- 
> Waylan Limberg
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/c1df7c13-684e-4c8c-9fe6-3c169b089645%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

An improved settings.py configuration (revisited)

[VernonCole]

Several months ago, I floated an idea on this forum for re-structuring the 
settings.py environment.

It generated a fair amount of discussion.  The consensus was that something 
really ought to be done, but no one was sure just what.
One strong opinion was that any new changes must involve a major move 
toward 12-factor configuration.
Another opinion was that 12-factor was an okay idea, but is not the best 
answer to many problems, and simply cannot replace settings.py in any 
practical way.

So, I have been fiddling with the problem on and off, and I think I have a 
solution which would be acceptable to pretty much everybody.  Here is a 
not-so-short description:

1) Some people (like me, for example) really like the idea of structured 
settings and using DJANGO_SETTINGS_MODULE to switch their configurations 
around easily.
  So I created a new module directory, and named it "preset" (because the 
scripts in it are imported _before_ settings.py).  We can't call it 
"settings" because that would conflict with the name of "settings.py", 
which Python does not allow. (I was thinking about the six preset buttons 
on the radio of my Dad's 1957 Chevy.) These "preset" scripts import each 
other as in a classic J.K-M. structured settings design, until finally they 
import the common base settings. 
  For convenience sake, I added preset/test_* to my .gitignore file so that 
I can create test scripts willy-nilly as needed.  I can happly switch 
between them using a simple
"./manage.py somecommand --settings=myproject.preset.test_whatever"
  Also, the preset directory is pre-populated with several example 
scripts.  Rather than trying to explain in settings.py how to create a 
DATABASES dictionary of dictionaries, there are several working examples 
for new guys to copy.  One even opens two databases using a router. This 
would have helped me a lot when I was getting started. There are also 
example intermediate-level presets to define local-ish things like time 
zone and email addresses and similar whatnot, with some of the verbose 
explanations from settings.py moved into them.

2) Some people want a pure 12-factor configuration with no messing around 
with configuration files.
  They are in luck.  The default preset is 
".preset.default_settings".  So if you don't select any 
(other) preset you get this one.  It is special, because it is a reworked 
version of Kenneth Reitz's fine dj_database_url module.  His idea was that 
you need about the same information to open a distant database as to access 
a distant file, so rather than burn up four or five environment variables, 
just lump all that information into a URL.  So you can say:
export 
DATABASE_URL="mysql://vernon:letm...@my.server.info:3606/vernons_big_db?production=true;secret_key=gobbldygook"
  The query keys are passed back as a dictionary, so your settings modules 
can do whatever you want with them. You have two local-programmed 
presets,either staging.py or production.py, depending on the value of the 
"?production=" query.  They can be as different or as similar as you wish, 
or one import the other. Do you really want to set the time zone and the 
administrator email-address using environment variables?  Go right ahead.

3)  Some people like things fine just the way they are.  If it ain't broke, 
don't fix it.
   Relax, settings.py is right where you expect it, and still does 
everything it did before.  If you really want to keep editing this one 
file, rather than moving your site specific information down into the 
presets, we will not stop you.  All  presets must eventually import 
settings.py. Therefore, settings.py will always maintain its most important 
function: it defines which applications are part of this project. If proper 
presets are maintained, all implementations of a project will use identical 
settings.py files.

  One potential problem which I have been made aware of is: if settings.py 
imports local_settings (and expects those overrides to be final) there 
could be an unpleasant surprise when a preset overrides them. I added code 
to preset/defalt_settings.py to issue a warning if it sees that module 
local_settings has been imported -- but I don't know whether I like that 
solution.

You can see the present version of the code at 
https://github.com/vernondcole/formhub in branch *make_presets_neater 
* , if you 
really want to see how it would look in a production system.  There are a 
few rough things in the code which would be better with help from the core 
-- like the need to trap and modify ImportError exceptions in order to get 
proper stack traces for errors in settings modules. Also, I wonder if 
DATABASE_URL should be more like DJANGO_DATABASE_URL?

So -- is this worth building a Patch so that people can _really_ tear it 
apart?
--
Vernon Cole


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To u

Re: An improved settings.py configuration (revisited)

[VernonCole]

Russ & Carl:

Thanks for the course correction. I guess one tends to read what one wants 
to see, rather than what the writer wrote.

Yes, this would make a good package, and I can do some different things 
with it that way.  For example, I really want to type:
>dj -s test_vernon 
rather than:
>python manage.py  --settings=myproject.preset.test_vernon

I may call it something like django-quick-settings.

Again, thank you for your always kind and valuable input.
--
Vernon Cole


On Saturday, March 8, 2014 5:01:30 PM UTC-7, Russell Keith-Magee wrote:
>
>
> On Sun, Mar 9, 2014 at 3:14 AM, Carl Meyer 
> > wrote:
>
>> Hi Vernon,
>>
>> On 03/07/2014 07:44 PM, VernonCole wrote:
>> > Several months ago, I floated an idea on this forum for re-structuring
>> > the settings.py environment.
>> >
>> > It generated a fair amount of discussion.  The consensus was that
>> > something really ought to be done, but no one was sure just what.
>>
>> There was not consensus that "something really ought to be done." I was
>> part of that discussion, and my opinion was (and still is) that nothing
>> needs to be done (in Django core).
>>
>> (For reference, a link to that discussion:
>> https://groups.google.com/d/topic/django-developers/O89Qb_9pwmo/discussion)
>>
>> Your proposed technique covers a lot of options - that's great. It works
>> fine without any changes to Django, and you can package it up as a
>> startproject template, use it, and promote it.
>>
>> I think the default startproject template should remain as simple as
>> possible (that is, a single-file settings.py) and not try to provide all
>> possible ways of handling multiple-deployment settings.
>>
>
> I agree with Carl. It takes a very selective reading of that discussion to 
> say that there was consensus on anything. 
>
> I also agree that the default start project template should be kept 
> simple, and any changes in project structure such as you have proposed 
> should be maintained on a per-project basis, depending on the needs of 
> individual projects.
>
> Yours,
> Russ Magee %-)
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/e97b2c6e-566a-4fd5-924e-6f8b4f85e569%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Proposal for prepared statements API

[VernonCole]

There was a recent query about prepared statements on the db-sig mailing 
list, too. Apparently thought is being given to adding such functionality 
to Psycopg. If such functionality is added, it could be useful to support, 
I suppose. Some SQL engines apparently benefit from the techinque. MS SQL 
Server is not one of them, so I have not bothered to add support for them 
to adodbapi.  If I do so, it will use the same api as mxodbc uses now (a 
copy of the SQL statement is kept with the cursor). [note: my reading of 
Microsoft's recommendation is not "don't do that", it is "why bother?".]

Pep-0249 is silent on the subject of how to support prepared statements, so 
any existing systems are likely to do so differently.  In particular, there 
can be no expectation that there is any support whatsoever for the concept, 
so it will have to be emulated where not present (-- i.e. almost 
everywhere).

On Tuesday, March 25, 2014 9:17:13 AM UTC-6, Michael Manfre wrote:
>
> Is falling back to a direct queries being considered? Not all backends 
> support prepared statements or recommend using them. The native mssql 
> drivers support prepared statements, but the other drivers django-mssql 
> supports do not. Also, "In SQL Server, the prepare/execute model has no 
> significant performance advantage over direct execution, because of the way 
> SQL Server reuses execution plans." [1]
>
> Given Microsoft's recommendation against using prepared statements, my 
> motivation is geared toward having this be an truly optional feature that 
> will not prevent using 3rd party apps with django-mssql. I do like the 
> proposal with an explicit .execute() and think documenting corner cases is 
> the sane way to proceed. 
>
> Regards,
> Michael Manfre
>
> [1] http://technet.microsoft.com/en-us/library/ms175528(v=sql.105).aspx
>
>
>
> On Tue, Mar 25, 2014 at 10:22 AM, Anssi Kääriäinen 
> 
> > wrote:
>
>> On Tuesday, March 25, 2014 2:53:42 PM UTC+2, Curtis Maloney wrote:
>>>
>>> ps = MyModel.objects.filter(foo__lt=Param('a').prepare()
>>>
>>> The result is now a callable that accepts one parameter - "a".  To 
>>> invoke the query:
>>>
>>> results = ps(a=1000)
>>>
>>>
>>> Clearly it's early days yet - I've written no code.  And akaariai has 
>>> pointed out already there's some corners cases which won't work well with 
>>> existing behaviours (e.g. foo=None being silently translated to 
>>> foo__isnull=True), but it's best to get this idea under wider public 
>>> scrutiny earlier, rather than later.
>>>
>>
>> I like this style of prepared statements. It is explicit, and 
>> implementation should be achievable without too much added code complexity. 
>> I prefer ps.execute(a=1000) personally, but the exact syntax isn't that 
>> important at this stage.
>>
>> There will be a couple of corner cases that will be hard to solve. The 
>> problems are around value preparation during .filter() calls and how 
>> certain special values are dealt with. Quickly thinking the value 
>> preparation (basically get_prep_lookup() call) shouldn't be that much of a 
>> problem - it is currently done during .filter() calls, but it should be 
>> possible to defer it to execution time.
>>
>> The foo=None case is something that likely can't be solved. The problem 
>> here is that foo=None translates to WHERE foo IS NULL, while foo=1 
>> translates to WHERE foo = 1. These are syntactically different queries, and 
>> thus single prepared statement can't handle both of these. There are also 
>> cases where isnull=True/False require different join promotion depending if 
>> True or False is supplied. These again can't be handled.
>>
>> I am OK for just documenting these corner cases. They aren't something 
>> that should happen too often. The implementation for prepared statements is 
>> relatively straightforward (generate SQL, prepare it once, execute using 
>> given values), but if the corner case needs to be handled the 
>> implementation will be more complex, likely so much more complex that 
>> nobody will implement this feature.
>>
>> In short: +1 for implementing this with documentation of the corner cases.
>>
>>  - Anssi
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com .
>> To post to this group, send email to 
>> django-d...@googlegroups.com
>> .
>> Visit this group at http://groups.google.com/group/django-developers.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/3123fda6-d7b3-46d3-82ec-28ed3003e837%40googlegroups.com
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google G