Re: 1.2 Proposal: Database savepoint refactoring
On Tue, 2009-08-11 at 10:21 -0700, Richard Davies wrote: > > I don't agree the current savepoint use within Django is inconsistent. As > > far as I can tell, savepoints are used internally in the one case where > > Django itself catches and suppresses an IntegrityError. > ... > > Right now it is pretty simple: if your app code catches a database error it > > must > > do something to restore the DB connection to a usable state. With the patch > > for #9205 would we be able to switch that to state that the app code will > > never have to worry about clearing the error on the DB connection when it > > catches a database error? If not then we would have less consistency than > > before. > > Thanks Karen - that's an interesting take on the #9205 situation. I > agree that our current situation is that "if your app code catches a > database error it must do something to restore the DB connection to a > usable state" and that is a consistent position. This isn't a new interpretation, by the way. It's the same position I've explained to you, Richard, every time you've opened a ticket to automatically add savepoint() handling to Model.save() or tried to introduce savepoint() elsewhere in the code. Karen's understanding and expectation completely matches mine: get_or_create() needs to do transaction handling the way it does to handle some real-world race conditions that must be handled inside the model. > However, this need to "do something" is specific to PostgreSQL - on > all other databases which I'm aware of the problem does not arise and > the DB connection always returns in a usable state (someone please > correct me if I'm wrong here...). Just because it's the only database in Django's core that uses a single connection status, doesn't mean it's the only database in the world that does so. PostgreSQL's particular connection handling has advantages and disadvantages (one of the disadvantages being the connection handling), but the ability for all cursors to operate in the same state is handy. > > That doesn't strike me as nice - effectively we end up with PostgreSQL- > specific code in the app code. Don't think of it as PostgreSQL-specific. Think of it as handling connections that use this particular state model. The calls are no-ops on other databases anyway. Savepoints definitely aren't a perfect solution, since we've never officially said we no longer support PostgreSQL 7.x (although at some point I'm going to propose we say that for the Django 1.2/1.3 timeframe) and savepoints don't exist in that version. > You can see many examples of this in > the current test suite where various calls to create(), save(), etc. > are wrapped in savepoints for the benefit of PostgreSQL only. This > also makes it easy to write a Django app (or Django core code!) which > tests and runs fine on other databases, but is buggy on PostgreSQL > since these savepoint wrappers were omitted. > > > I'd much prefer to move towards a uniform position of "the Django > framework will always return a usable DB connection". "...and the user has no choice how to handle the problem." That's why we don't do this currently. Good that there's a discussion going on around the trade-offs, but that alternate side needs acknowldgement and addressing as well. > We're already > there on all non-PostgreSQL databases. We're already there on > PostgreSQL for some calls (like get_or_create). I think that we can > easily get there in the remainder (like create, save). > > Does this fix ALL cases where an IntegrityError results in an unusable > PostgreSQL db connection? I'd need more work here to really believe > this. That would be an absolute minimum for a proposal like this. It's all or nothing. Right now we have completely consistent behaviour. Changing it to be inconsistent would be a regression. Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: FileFields and file ownership
On Tue, 2009-08-11 at 04:39 -0700, Ole Laursen wrote: > Hi! > > There are a couple of bugs open/closed about what happens when you > upload a new file to a file field that already has a file: > > http://code.djangoproject.com/ticket/11663 > http://code.djangoproject.com/ticket/2983 > http://code.djangoproject.com/ticket/4339 > > Progress is currently halted because a design decision is needed, > maybe the problem is conflicting visions of what FileField is. > > First: what happens right now is that the old file is left behind. If > the new file has the same name as the old, it is mangled so both can > stay. > > As has been pointed out in 2983, this is all else set aside a security > problem because the old file is essentially garbage This is an assumption that isn't universally correct and it's implications appear to be vastly under-appreciated. One of the reasons we *don't* delete files automatically is that you cannot you know for certain, given no other information, that no other process on the system or elsewhere is still needing that file. You are implicitly assuming that the file upload through Django means that Django instance is the only thing using the file. Other Django installs could be using. Other processes could be. It's impossible to tell. So knowing when it is safe to delete files requires domain-specific knowledge. We need good ideas on addressing the denial of service via disk exhaustion attack. But, for example, there are patches floating around (SmileyChris created a new version recently) to allow a FileField to optionally delete files after saving a replacement. I'm not sure if Chris's patch saves and then deletes to handle errors correctly right now, because I haven't reviewed it in that level of depth yet. That would be something you might enable if you were allowing public file uploads. Or you might have a cronjob that periodically looks for and cleans things up (an easy enough approach today) and the frequency with which it runs -- or how it is triggered externally -- depends upon your available resources. Realise that uploading with the same filename isn't the only way to use up disk space. The filename is a pretty random string to use as a save thing and it takes about 35 seconds to write a script to generate hundreds of thousands of filenames for the same file if different filenames are the only impediment to DOS'ing a system. Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Change list default sort modification proposal
For those, like me, wondering what this proposal was about, it's concerning changing sorting in the admin interface to initially use the full set of fields specified in Meta.ordering on the model. What I can't work out yet, due to difficulty in reviewing the patch, mentioned below, is whether it allow sorting by more than one column via the column-clicking method. That would almost certainly seem to be a request that is going to arrive 10 seconds after a feature like this is committed, although the difficulty has always been with how the UI works for that situation. On Tue, 2009-08-11 at 20:21 -0100, Joshua Russo wrote: > On Tue, Aug 11, 2009 at 7:46 PM, Rock wrote: > > Yes. I have a project that could use this immediately. Looking > forward > to seeing the ticket and a patch I can try out. > > > Ok, let me know what you think. > http://code.djangoproject.com/ticket/11695 Please read contributing.txt about how to submit patches. I looked at the latest patch on that ticket and it wasn't complete. Then I realised you have filed three patches that all need to be read together. This makes reviewing extremely difficult. It also means that if somebody fixes a problem in one of the patches, it becomes incredibly unclear which are the relevant patches. Creating svn diffs isn't difficult. You change to the top of the source tree and run "svn diff". All your changes in one patch with full paths. Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: App Engine support
On Wed, 2009-08-12 at 02:32 -0700, Waldemar Kornewald wrote: > On Aug 12, 2:09 am, Russell Keith-Magee > wrote: > > > Is the consensus that further refactoring or rethinking of things like > > > QuerySet and Query are required to make this happen? > > > > Not really a consensus - more a general feeling that there are some > > SQL-specifics that still need to be purged. The Query/QuerySet > > interface was designed with support for non-SQL backends as a design > > goal, but without an actual non-SQL backend implementation to prove > > the design. Finding and purging these SQL-specific components is the > > work in progress. > > Is there a way to override sql.subqueries? Not yet. As mentioned earlier in the thread, a large chunk of the process of making non-SQL support is to allow wholesale overriding of the django.db.models.sql namespace. Everything in there is SQL specific and I would hope that that is the layer that is entirely replaced by alternate storage systems. So overriding subqueries feels like too low level, because it has plenty of builtin SQL assumptions. So anything that imports and uses things from django.db.models.sql needs to instead use a common API to fetch the appropriate module and we need to thin down the interface. Nothing outside of django.db.models.query should be importing from that namespace, ideally (I believe that should be true now, unless we've introduced bugs) and then we need to introduce something like a (*shudder*) set of factory methods or otherwise split up the features. That's certainly where the design changes are initially going to happen in this, we already know that. Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: App Engine support
On Wed, 2009-08-12 at 03:04 -0700, Waldemar Kornewald wrote: > On Aug 12, 11:40 am, Malcolm Tredinnick > wrote: > > On Wed, 2009-08-12 at 02:32 -0700, Waldemar Kornewald wrote: > > > Is there a way to override sql.subqueries? > > > > Not yet. As mentioned earlier in the thread, a large chunk of the > > process of making non-SQL support is to allow wholesale overriding of > > the django.db.models.sql namespace. Everything in there is SQL specific > > and I would hope that that is the layer that is entirely replaced by > > alternate storage systems. So overriding subqueries feels like too low > > level, because it has plenty of builtin SQL assumptions. > > OK, I think I misunderstood how far you want to go with the > refactoring. Non-SQL support could be achieved with a few smaller > changes (my impression was that the preferred solution is to override > sql.Query), but it looks like you want to do a much bigger > refactoring. You seem to be oscillating between extremes. It's not a large refactoring. The large refactoring was queryset-refactor so that these changes will be small and fairly self-contained. But, really, we could debate "large", "small", "medium" until the cows come home. They are relative. I expect it will be smaller than the queryset-refactor and larger than a couple of evenings' work, but since the work hasn't been done yet, that's only an intuitive guess. My understanding of the problem -- whilst I don't have your AppEngine knowledge, I have fairly good understanding of non-SQL storage systems in general and excellent understanding of the ORM side of Django -- is that trying to patch non-SQL support into django.db.models.sql won't be flexible enough. There will be too many alternate paths because there are a lot of relational-tree data structures and column-based storage and join-based single-query features in that code. And it's already on the performance critical path. So it's a candidate for replacement with more appropriate object models for other storage backends. > > > So anything that imports and uses things from django.db.models.sql needs > > to instead use a common API to fetch the appropriate module and we need > > to thin down the interface. Nothing outside of django.db.models.query > > should be importing from that namespace, ideally (I believe that should > > be true now, unless we've introduced bugs) and then we need to introduce > > something like a (*shudder*) set of factory methods or otherwise split > > up the features. That's certainly where the design changes are initially > > going to happen in this, we already know that. > > When do you plan to start with the changes? It's already started. This thread is part of the design work and validation of the ideas around the area. And it's not "you", it's "us". Anybody can start trying out things and publishing a repository or putting up some patches to review for ideas or prototype implementations. I'm hoping I personally will have some time to devote serious development work in this area over the next few months, as it's something I've been working incrementally on in Django for over three years now. > I'm trying to get some more App Engine developers involved, so this we > can get this done faster. Mitch, do you have anyone else on the > SimpleDB side who could help, too? Whilst enthusiasm is good, please realise that the goal isn't "AppEngine support as fast as possible". It can't be. That's too short-sighted for a feature change like this in Django. If we're going to do the work, we have to really work out the changes to support as broad a range as possible. We've made some pretty good progress in this thread already and you're collecting the broader problems on the new wiki page is going to be useful. I'd like to hear any other people who've spent serious time (I think you'll find you've spent more time looking at this than anybody except a couple core developers, Waldemar, and you know it's a large-ish area of code to think through) with other storage engines. I've been thinking about this on and off over the last couple of days as I have time and I've put some serious thought into this aspect a few times in the past -- which is why I can comment on some of the things fairly quickly. I know Jacob and Russell have also spent time here. So we collect the ideas and start thinking about what works. Maybe try out a few things and see. Start writing some prototype code and see what happens. I'm certainly one of the people happy to review and critique. Jacob, Russell and I have given some indications in this thread of the direction we'd like to g
Re: Shouldn't custom fields in ModelForms pick up model Field options by default?
On Wed, 2009-08-12 at 20:05 -0700, Kevin Henry wrote:
[...]
> But if you decide to customize the widget, for example:
>
> class AuthorForm(forms.ModelForm):
> name = forms.CharField(widget=forms.TextInput(attrs={'size':80}))
>
> class Meta:
> model = Author
>
> the form field no longer reflects these model field options (that is,
> there is no help text, no default value, and you can't save it blank).
Correct. You told Django that you wanted to specify the field yourself
and it believed you. Django cannot know which attributes you intended to
leave out and which you wanted inherited, so it does the only possible
thing and leaves things entirely in your control.
>
> This seems like undesirable behavior. At the very least it's
> surprising,
Well, surprise is in the eye of the participant. As noted above, it
would be surprising to a lot of us if it did pick up the attributes
because it removes control from the user. The current behaviour isn't
illogical.
> and should probably be documented here:
> http://docs.djangoproject.com/en/dev/topics/forms/modelforms/#overriding-the-default-field-types.
Definitely. Please open a ticket (if one doesn't already exist in the
documentation component for this) and feel free to attach a patch with
some suggested wording.
>
> I'm fairly new to Django, so unfortunately I'm not sure of the right
> way to do this.
If all you want to do is change the widget, you can do that in the
form's __init__ method by updating self.fields["name"].widget, for
example.
Regards,
Malcolm
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Re: GSoC10 Idea: Additional queryset methods
On Fri, 2010-03-19 at 05:39 -0700, Sh4wn wrote:
[..]
> I want to refactor the QuerySet and other databases related API's,
> which adds an additional method of selecting records, and provides
> more control of what data you'll select, but probably is a bit more
> complex than the current API. It's a bit inspired by Doctrine, a PHP
> ORM, and yes, one of the few decent PHP projects.
>
> Example 1.1:
> qs = QuerySet().select(News.all, User.username,
> Count('c__id'), comment_average=Avg(OtherModel.field)
> ).from(News) \
> .leftJoin('comments c')
> .filter(news__title="value") \
> .order_by('comment_average')
> (This is just a not-working dummy example)
>
> As you can see you can explicitly set the fields and tables you want
> to select. Beside that the existing queryset methods will remain to
> work. There'll be a slight difference though: instead of the related
> name in filter/order_by/other methods you don't use the related name,
> but the table name specified in the from method.
In other words, you want to write raw SQL, but in Python. Django
intentionally doesn't do this, focusing on functionality, rather than
describing the SQL that is created. This, for example, makes things
easier for using non-SQL storage and keeps the ORM "Pythonic", rather
than SQL-based. Any time you get to the level of having to think about
"left outer join" or database table names, you might as well just use
raw SQL.
Your motivational examples aren't particularly convincing as written,
since you don't explain *why* those tags are making multiple queries. I
will grant you that there are many Django applications around in public
and prviate use that could quite possibly make more efficient use of the
database. Sometimes this is because the maintainers have missed an
alternate approach, sometimes because the particular tag or method isn't
used enough to warrant extra optimisations just yet. However, I don't
see many of these cases as being symptomatic of a failure in the ORM
that requires a more complex API.
(By the way, adding extra methods too QuerySets are easy, since you can
subclass the Queryset class and use that instead. For anything that
would end up trying to justify the sort of step backwards -- which is
what you're proposing here, as you want to leak more about the raw
database layer into the Python API -- you would want to start by showing
how these are unavoidably useful via a set of subclasses that are being
used in various applications.)
Regards,
Malcolm
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: GSoC: Data importation class
Hi, I see a few problems here. The gist of what follows is that it seems a bit abstract and as one tries to nail down the specifics it either devolves to a more-or-less already solved problem that doesn't require Django core changes, or a problem that is so unconstrained as to not be solvable by a framework (requiring, instead the full power of Python, which is already in the developer's hands). On Thu, 2010-03-25 at 09:07 -0700, subs...@gmail.com wrote: > If we could visualize the entirety of data within django-projects, we > would probably see that this 'data economy' is growing exponentially > year-over-year. However, I know of no guided way to actually get this > data into a project that's been converted to Django. There are two > methods I generally hear about when asking people how to move between > schemas: purely SQL solutions and one-shot scripted solutions (or a > mix). With talk of model-level validation, the first approach is > becoming increasingly invalid, That's not a correct statement, since Django models can often be used to proscribe conditions on new data that is created via the web app, yet those conditions might not be required for the universal set of data that already exists. For example, webapp-generated data might always require a particular field, such as the creating user, to be filled in, whilst machine-generated data would not require that. Don't equate validation conditions at the model level with constraints on the data at the storage level. > but I wonder if we could include some > batteries for the second approach? > > My proposal is a new django class which provides a mapping for how > this data should move from its legacy schema to a django project. > I've > got a sort of proof-of-concept already working but it lacks the polish > of a refined community contribution. Moreover, with multi-database > support coming, I see this concept getting a shot in the arm, > especially in cases where the legacy db is a currently supported one. > > I imagine the usage going something like: > > 1) User creates django project > > 2) User runs a 'startconversion' app which creates a stage folder for > holding an inspectdb of the legacy data, a default router for the > legacy data, and some other empty files. The last bit sounds a bit nebulous. You could optimise it by not including any empty files, or be a bit more specific about what the empty files are meant to represent. :) > > 3) User defines the classes which defines the map between the legacy > and new schema, and defines clean functions according to their needs, > 'foreignkeys' to other conversion classes, etc. It seems that you are talking about the cases where, by default, a different schema is required. The first approach is to make the models match the existing schema, on the grounds that the existing schema is a reasonable representation of the data. In the case where that isn't true, a migration is required, but the possibilities for such migrations are endless unless the original data can already be put into natural Django models. If inspectdb can already be run on the existing data, why not use that as the starting point and then the dev can use something like South to migrate to their schema of choice? It seems that we already have all the tools in that case. If inspectdb cannot generate a useful schema that can be modelled by Django, the user is going to have write a generic Python script in any case and the possibilities there are boundless and best left to the best tool available for the job at hand: Python itself. > 4) User runs a command at the top branch of their schema (some distant > relation) and the command inspects these classes and runs them from > the ground up. As it does this measures are taken (such as use of > pagination) to avoid server CPU/memory thrashing, as well as model- > level measures such as OneToOne's being respect, etc. Adding system administration functionality to Django, which is what this monitoring is, feels like the wrong approach. It's not intended to replace everything else in your computing life. What is appropriate load usage for one case will be highly inappropriate elsewhere. How will you detect what you are labelling as "thrashing"? Regards, Malcolm -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: magic-removal cheat sheet
On Fri, 2006-05-12 at 00:19 -0500, Eugene Lazutkin wrote: > I added to the wiki a new document "Removing the magic: the cheat > sheet": http://code.djangoproject.com/wiki/MagicRemovalCheatSheet. > Essentially it is an index to "Removing the magic" organized by > functional areas with small hints. It reflects my experience with > converting web sites to the "magic-removal". I was watching you build this piece by piece today and I think it's brilliant. Really good idea, Eugene! The layout is great. Nice work. :-) Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: CONTENT-LENGTH not being set
On Sat, 2006-05-13 at 09:11 -0700, Martin Ostrovsky wrote: > Hello, > > Is there a reason why CONTENT-LENGTH is always blank in the environ > dictionary that gets passed to WSGIRequest ? > (django/core/handlers/wsgi.py) What web server front-end are you using to handle the request? Development server? Twisted? Other? > > Why isn't it just automatically computed based on the length of the > request's content? I'm not sure in this specific case, but in general this is not going to always be possible. For example, if the incoming content is passed to the handler before it is fully received (streaming input). Django does not have good streaming input handling at the moment, so that will not be the case here, but I just want to point out that the solution is not necessarily as simple as this. > > The reason I ask is because I'm getting an exception raised on line 119 > in wsgi.py when the WSGI object tries to return its raw post data: > > self._raw_post_data = > self.environ['wsgi.input'].read(int(self.environ["CONTENT_LENGTH"])) > ValueError: invalid literal for int(): > > , which fails because of the empty string value. On all requests, or only on a particular type of request? Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Exceptions in Templates
Hey Michael, On Mon, 2006-05-15 at 05:30 +0200, Michael Radziej wrote: > Hi, > > Django silently drops certain types of exceptions during exception > processing. > > Now, while this is in general useful, it really obscures your bugs. > > I'd propose to optionally log those exceptions to stdout when, say > settings.log_template_exceptions, is set. > > What do you thing? At some point we start to drown in config settings. It's probably better to just "Do The Right Thing" here (first, decide what the right thing should be). Having a way to always show the exception (if possible) seems like the right thing to me. The patch in ticket #1852 looks like a good start. I was looking at it this morning and trying to trace through whether anything else needed similar modification. Not sure the output it produces is perfect, but it looks like a good start. Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Targetting SQLite versions
Michael Radjiez quite reasonably raised a problem with a recent change I made to fix a bug with QuerySet.count() (it was previously ignoring any "distinct" modifiers). The problem is that if you are using a version of SQLite prior to version 3.2.6 (released in September 2005), QuerySet.distinct().count() generates a query that is not understood by your version of SQLite. :-( So what is our policy on working around arguably broken databases? This is a little trickier than "some MySQL installs don't do transactions", since it isn't just ignored -- it crashes Django. A couple of solutions (from my favourite to absolutely least favourite): (1) If you are using a sufficiently old version of SQLite, the SQL produced by distinct().count() does not respect the "distinct" modifier. GOOD: - Code runs to completion. You always get an answer. It is, in fact, the same as you got with Django last week. If you really need distinct counting, upgrade SQLite. - Analogous to the way we treat versions of MySQL without transactions (and probably a reasonable approach). BAD: - You get different answers to other people in some queries. And your answers are wrong (see the final point for why this might be a problem). - The tests are going to be a real pain to make work conditionally like this (again, maybe the MYSql-transaction approach might be reasonable: the test will always fail and you will know it's because of your SQLite?!) (2) Inside the SQLite backend, try to work out if the version is sufficiently old and do the "count(distinct(..))" bit by hand by doing a "SELECT DISTINCT..." and then counting the number of results (sadly, in pysqlite2, cursor.rowcount is always -1 after a query, so can't cheat). GOOD: - you get the same (right) answer under all versions and so your applications work the same, no matter which database is installed. BAD: - it's going to be a little inefficient and very memory hungry if you have a "bad" version of SQLite. Counting the results is going to require calling len() on the queryset (or equivalent). Very large result sets on a shared server will run out of memory. (3) We don't support QuerySet.distinct().count() on older versions of SQLite. We can make the tests conditional, so that if you get the "unsupported" error, it just informs you (raises an exception?). We might even be able to make the tests always pass. GOOD: - no more accidental huge memory consumption. You are explicitly told when you're in trouble. - upgrading SQLite if you want things to work properly is really not that big a deal (as compared to, say, PostgreSQL or MySQL), since it's such a tiny, binary (36K when stripped on my machine) linked to some very common libraries. Even a static version is pretty small. It was originally designed to be built into distributed packaged. So we are still not making it that hard for people to use SQLite + Django. BAD: - if you have an older version of SQLite and want to run an app that uses distinct().count(), you have to do some sysadmin work (that is why the first version may be better -- at least the code runs, if not necessarily very efficiently). (4) We back out this change. GOOD: - Works the same for everybody again. BAD: - QuerySets now give wrong results and behave counter-intuitively. - Some not uncommon situations, such as pagination, become problematic without manually coding SQL. A typical example is as in the test suite: articles can be assigned to multiple publications. You want to display all articles from a set of publications. So you need distinct(). Trying to work out the pagination, you obviously need distinct().count(), since count() over-counts the number of articles (every article that is in multiple magazines is counted multiple times) and would result in page breaks coming too early. Cheers, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Targetting SQLite versions
On Mon, 2006-05-15 at 20:43 +1000, Malcolm Tredinnick wrote: > Michael Radjiez quite reasonably raised a problem with a recent change I > made to fix a bug with QuerySet.count() (it was previously ignoring any > "distinct" modifiers). The problem is that if you are using a version of > SQLite prior to version 3.2.6 (released in September 2005), > QuerySet.distinct().count() generates a query that is not understood by > your version of SQLite. :-( I may have omitted to be clear about the details here: in versions prior to 3.2.6, SELECT DISTINCT ... and SELECT COUNT(...) ... both work. But SELECT COUNT(DISTINCT(...)) ... is considered illegal SQL. COUNT() was enhanced in 3.2.6 to support the DISTINCT function. Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Standalone template -- updated patch
A friend hit the standard problems with trying to use Django's templates in another application at his work last week. So we sat down on Saturday and polished Luke Plant's existing patch a little. I have put the new patch into ticket #1321. I would appreciate some review of this, since I believe it is pretty close to something that is complete. There are documentation updates in the patch, so I won't go into too much detail: will be a good test of whether the docs make sense. It is all still heavily based off the earlier work of Luke and Fredrik Lundh. We are still working on making this stuff get tested in the automated testing, but for the moment you can run tests/othertests/templates.py on its own and it will use the standalone configuration (prove this by unsetting DJANGO_SETTINGS_MODULE from you environment before running, or setting it to some nonsensical value, even better). A few design decisions we made: - at any point prior to accessing django.conf.settings for the first time (you can import it, but not read or write), you can decide to configure it manually (by calling settings.configure()). Otherwise DJANGO_SETTINGS_MODULE is read and life works as before. So existing Django apps require no changes. This was all discussed previously; nothing new here. - By default, the fallback value for each config settings is django.conf.global_settings, but you can change that as well by passing in any module or class containing defaults that you like. - We have not put any effort into making it possible to just distribute the templating module on its own yet. It is not precluded by this patch, but it is not necessarily an immediate win: a bzipped Django tarball at the moment is 1.1 MB, so if downloading is a problem, this is still sufficiently small. Templating imports code from a few places outside django.templates, with good reason (mostly django.conf, obviously, and django.utils). It is not too hard to write a script to track all these imports if somebody did want to package separately, but the problem then is that if somebody has django-templates installed and (say) django-views and then installs a full Django setup, it is going to get confusing working out which modules are being imported from where. And making Django have three packages by default seems overkill for something so small. - These changes make it possible to also use the view infrastructure in another application (i.e. does not require DJANGO_SETTINGS_MODULE to be set, etc). Models as well. The only tricky bit is knowing which config variables needs setting for each component and we can document them over time (probably in an advanced settings document). I'd like to keep the momentum going on this, since it's coming up again and again on the users list and in conversations I'm having with other people. Cheers, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Suggestion: Better slugifying of scandinavic characters
On Tue, 2006-05-16 at 09:00 +0200, Gábor Farkas wrote: > Jeroen Ruigrok van der Werven wrote: > > On 5/16/06, Ville Säävuori <[EMAIL PROTECTED]> wrote: > >> I think that this problem applies in most european languages, too. > >> Like, say, Swedish, German and French. > > > > The same appliesa for Dutch where we use trema's (sort of umlauts) to > > denote any possible ambiguity in reading. So having the accent > > stripped would be way better than having the entire letter stripped. > > The same applies of course to say Spanish with the tilde-n, or even > > some slavic languages or Romanian. > > > > also in Hungarian and Slovak the preferred way is to just strip the accents. > > maybe the best way would be to make this locale-dependent... At the risk of offending everybody who uses a language requiring accents, but this one of those "it's harder than it looks" problems in Unicode. You need to have a mapping from every accented character (or a reasonable set of them) to their unadorned equivalents. Many characters are a single unicode character, not a unicode composition of two characters, so it's not just a matter of "stripping the accent". So either we're going to end up carrying around a fairly large mapping table in the Javascript or we need a better solution. To put the problem into context: it's only a small generalisation to attempt to do the same thing for mapping Japanese characters to ASCII-based URLs. The accent version just sucks you in by making you think the characters are close because they look the same. Your computer has a different "vision". Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Suggestion: Better slugifying of scandinavic characters
On Tue, 2006-05-16 at 09:36 +0200, Gábor Farkas wrote: > Malcolm Tredinnick wrote: > > On Tue, 2006-05-16 at 09:00 +0200, Gábor Farkas wrote: > >> Jeroen Ruigrok van der Werven wrote: > >>> On 5/16/06, Ville Säävuori <[EMAIL PROTECTED]> wrote: > >>>> I think that this problem applies in most european languages, too. > >>>> Like, say, Swedish, German and French. > >>> The same appliesa for Dutch where we use trema's (sort of umlauts) to > >>> denote any possible ambiguity in reading. So having the accent > >>> stripped would be way better than having the entire letter stripped. > >>> The same applies of course to say Spanish with the tilde-n, or even > >>> some slavic languages or Romanian. > >>> > >> also in Hungarian and Slovak the preferred way is to just strip the > >> accents. > >> > >> maybe the best way would be to make this locale-dependent... > > > > At the risk of offending everybody who uses a language requiring > > accents, but this one of those "it's harder than it looks" problems in > > Unicode. You need to have a mapping from every accented character (or a > > reasonable set of them) to their unadorned equivalents. Many characters > > are a single unicode character, not a unicode composition of two > > characters, so it's not just a matter of "stripping the accent". So > > either we're going to end up carrying around a fairly large mapping > > table in the Javascript or we need a better solution. > > i agree that this problem is quite hard to solve "for everyone". > > but this actual problem (stripping accents) is not that hard. > > in unicode, there are several "normal forms" defined. > > for example, take the character [a] with an accent ['] : [á]. > in unicode it can be represented either as one character, or as two > characters: the [a] symbol and the accent['] symbol. > > but there is a normal form, where every character is in it's decomposed > (two character) form. Yes, but think about how you move to the normal forms. It requires lookup tables and they are pretty long. In Unicode 3, the normalisation table was about 2500 lines long (that is the only ones I have sitting around at the moment). Sure, you can compress it a bit, but it's still a lot of data and computation. Now, we could try working with a small subset, which may be the most practical solution. But we will continually be receiving mail from people with ø or Ô or Å or some other not-so-common-outside-of-country-X character in their title saying that we missed their favourite character. It's possible, just fiddly. I'm not arguing for punting the problem: I have some Norwegian and German friends I'd like to still have talking to me. Just pointing out that a bunch of "that sounds good" mail isn't solving it. Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Standalone template -- updated patch
On Tue, 2006-05-16 at 16:26 -0500, Adrian Holovaty wrote:
> On 5/15/06, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote:
> > A friend hit the standard problems with trying to use Django's templates
> > in another application at his work last week. So we sat down on Saturday
> > and polished Luke Plant's existing patch a little. I have put the new
> > patch into ticket #1321.
> >
> > I would appreciate some review of this, since I believe it is pretty
> > close to something that is complete. There are documentation updates in
> > the patch, so I won't go into too much detail: will be a good test of
> > whether the docs make sense. It is all still heavily based off the
> > earlier work of Luke and Fredrik Lundh.
>
> This is excellent stuff! I've committed it to trunk. Thanks very much
> to all who contributed to the patch.
>
> One tiny question:
>
> Why is LazySettings.__setattr__() using this:
>
> self.__dict__['_target'] = value
>
> ...instead of this?
>
> self._target = value
As Luke mentioned in the ticket report (just posting here for the
archives), it's because in Python
self._target = value
is implemented as
self.__setattr__('_target', value)
for attributes that are not specifically attached to the class at
construction time. See also: "infinite loop". I started off with the
same thinking you did, stripped all that stuff out and then spent half
an hour rediscovering that Luke is pretty smart and had no doubt been
through the same process.
Malcolm
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---
Re: ImageField and its use in templates
On Wed, 2006-05-17 at 07:20 +, wegen wrote:
> hi, all.
>
> i'm new to django and i have a couple of questions regarding images
> upload in django.
>
> i followed the step-by-step instruction given in the FAQ of the django
> homepage
> (http://www.djangoproject.com/documentation/faq/#how-do-i-use-image-and-file-fields),
> however, it does not seem to work and i just can't figure out why.
>
> i changed the MEDIA_ROOT and MEDIA_URL in the settings and set
> ImageField in my model, and when i tried to view the image via one of
> my templates by {{ object.get_xxx_url }} all i get is an empty page.
>
> could anyone please give me an example of doing this?
>
> my setting is as the following;
> 1. i store all car images in /blahblah/media/images/cars/.
> 2. in settings.py;
> MEDIA_ROOT = '/blahblah/media/'
> MEDIA_URL = 'http://localhost:8080/media/'
> 3. in models.py;
> carimage = models.ImageField(upload_to = 'images/cars/%s' % carID, null
> = True, blank = True)
> 4. in cardetails.html (i.e. template)
>
>
> thanks very much for your time.
A first step at debugging this would be to see what the value returned
from object.get_carimage_url is. Either by viewing the source or by just
dumping it into the template temporarily.
Regards,
Malcolm
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---
Re: ImageField and its use in templates
On Thu, 2006-05-18 at 06:16 +, wegen wrote:
> thanks very much for your help, arthur.
>
> now i can get the value returned from {{ car.get_carimage_url }} as
> something like
> http://localhost:8080/media/images/cars/car1.png.
>
> however, if i try to view the image in my template by using the
> following tag,
> , all i get is nothing.
>
> now, i have a correct link to the image and i don't see anything wrong
> with the img tag i'm using as above.
>
> does anyone have a clue? feel free to correct me if i'm wrong.
Assuming you are using the development server, I've been bitten by this
before. The development server (./manage.py runserver) cannot handle
having your MEDIA_URL being a suburl of the ADMIN_MEDIA_PREFIX.
Normally, with something like mod_python + Apache, this would not be a
problem, but the way django.views.static.serve handles things, it gets
confused.
The solution is to change ADMIN_MEDIA_PREFIX in your settings.py to
something like /static/, rather than /media/, You don't need to change
anything else, just that one setting, then restart (or change MEDIA_URL
to be something like localhost:8080/something_else/...
Of course, if you are not using the dev server, we are still on square
one.
Regards,
Malcolm
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---
Re: Document translations
On Thu, 2006-05-18 at 18:41 -0700, ymasuda wrote: > I have finished Japanese (ja-jp) translations for revision 2933, and > I wish to contribute them (after proofread patches for tutorials). > > Although there are no explicit rules for translated doc, I think there > should be some guidelines: for example, reStructuredText, with ".txt" > extension, UTF-8 for non-ASCII charsets, etc. If there are any > additional > requirements, please point them out. > > # You may find latest version of Japanese translations at: > # http://ymasuda.jp/python/django/docs/ This is an enormous amount of work. Well done. Django looks impressive in Japanese. :-) Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: ImageField and its use in templates
On Fri, 2006-05-19 at 03:00 +, wegen wrote: > thanks for letting me know about the issue you had before, malcolm. > > i'm using the development server as you suspected, however, > changing the value of the ADMIN_MEDIA_PREFIX does not seem to fix the > problem. > > rather than asking you to what to do in order to solve this specific > problem, > i'd like you to clarify the some ImageField-related things that keep > confusing me, if possible. This thread is starting to get more and more off-track for django-developers. It would probably be a good idea to move any follow-ups to the django-users list, since there will be more people there who may have hit the same problems you have. > 1. do we need to make any changes to the URL file (i.e. urls.py)? Yes. I assumed you had already done this. Sorry 'bout that. Have a read of this page for the information you need: http://www.djangoproject.com/documentation/static_files/ This is almost certainly what is stopping you if you have omitted this step. > 2. if your page is supposed to have images in it, then do we need to > manually set the content-type of the page so that images will be > displayed properly? No. The page will still be text/html or whatever. The images themselves will have the right content type set in their headers when the webserver sends them. Your web page does not actually include any images, remember, it just includes links to the images. The client's web browser is responsible for actually fetching the data at the other end of those links and displaying it. Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: [Changeset] r2931 - django/trunk/docs
Hey Luke, On Fri, 2006-05-19 at 13:15 +0100, Luke Plant wrote: > Malcolm, > > I noticed this addition to the docs in django-updates: > > + Note that QuerySets are not callable, so > + if you want to pass in a QuerySet in ``extra_context`` and have it > + evaluated at render time, you need to wrap it in a function (that > + returns the QuerySet). > > I think this confuses the issue slightly - the fundamental problem is > that QuerySets cache the data they get back from the db when you > evaluate them. Could I suggest something like this instead: > > Note that QuerySets retrieve and cache their data when they are first > evaluated, so if you want to pass in a QuerySet in ``extra_context`` > that is always fresh you need to wrap it in a function or lambda that > returns the QuerySet. Thanks heaps for the feedback. Agree fully that my text is confusing and I would like to have something much shorter there. Your explanation is correct, but also sliding past the issue a little, isn't it? The reason I attempted to clairfy this is because a couple of people were confused about the "liveness" or otherwise of extra_context members. Maybe singling out QuerySets for special attention actually confuses things more than helps. Should we just omit that "Note..." sentence altogether? So it becomes: * ``extra_context``: A dictionary of values to add to the template context. By default, this is an empty dictionary. If a value in the dictionary is callable, the generic view will call it just before rendering the template. If it is not callable, it will be evaluated once, at import time. (I'm not really thrilled about "at import time" there, either, now that I reread it. What a lame way to try and nail down the evaluation time. I'll work on that.) Is it clearer if we leave it out altogether? Trying to put in a repetitive long explanation about extra_context for every function looks poor, I think. Maybe we should just suck out an explanation of extra_context to the top of the document and explain QuerySet evaluation up there. If you don't have any really strong feels that either of these are an improvement, I think we should just commit your change, since it at least addresses the main area people are going to mess this up. But I wouldn't mind if we could do better still. I'll think about it overnight, but I'm not going to mind if you just dive in and edit it. I'm just tweaking the docs as things come up: I'm pretty thick-skinned about being edited. Best wishes, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: bug? missing foreign key constraints in SQL
Hi Shaun, On Sat, 2006-05-20 at 10:44 -0700, shaunc wrote: > Is this the right place to ask about what looks to be a bug? If not, > sorry > > If so, the following code will cause generation of SQL that is missing > constraints: [... informative example snipped ...] > But the bottom line is that only one set of ALTER TABLESs can be > generated in this situation, where we might need more than one. Nice debugging (and thanks for the solution in subsequent emails as well). I think this is the same problem as was reported in ticket #1968 (http://code.djangoproject.com/ticket/1928 ), so I've added a pointer to this thread to the bottom of that ticket -- since you have gone deeper than the original report. When somebody gets a chance to review this, your work should help. Regards, Malcolm X-Google-Language: ENGLISH,ASCII-7-bit Received: by 10.54.68.11 with SMTP id q11mr158624wra; Sat, 20 May 2006 18:08:22 -0700 (PDT) Return-Path: <[EMAIL PROTECTED]> Received: from bigben2.bytemark.co.uk (bigben2.bytemark.co.uk [80.68.81.132]) by mx.googlegroups.com with ESMTP id v11si524940cwb.2006.05.20.18.08.22; Sat, 20 May 2006 18:08:22 -0700 (PDT) Received-SPF: neutral (googlegroups.com: 80.68.81.132 is neither permitted nor denied by best guess record for domain of [EMAIL PROTECTED]) Received: from sharp.pointy-stick.com ([80.68.90.23]) by bigben2.bytemark.co.uk with esmtp (Exim 4.52) id 1FhcQf-0008Hg-Sm for django-developers@googlegroups.com; Sun, 21 May 2006 01:08:21 + Received: from counterweight.tredinnick.org (cust7071.nsw01.dataco.com.au [203.171.88.159]) by sharp.pointy-stick.com (Postfix) with ESMTP id DE6EAC38C for ; Sun, 21 May 2006 11:08:18 +1000 (EST) Subject: Re: bug? missing foreign key constraints in SQL From: Malcolm Tredinnick <[EMAIL PROTECTED]> To: django-developers@googlegroups.com In-Reply-To: <[EMAIL PROTECTED]> References: <[EMAIL PROTECTED]> Mime-Version: 1.0 Content-Type: text/plain Date: Sun, 21 May 2006 11:08:13 +1000 Message-Id: <[EMAIL PROTECTED]> X-Mailer: Evolution 2.6.1 (2.6.1-1.fc5.2) Hi Shaun, On Sat, 2006-05-20 at 10:44 -0700, shaunc wrote: > Is this the right place to ask about what looks to be a bug? If not, > sorry > > If so, the following code will cause generation of SQL that is missing > constraints: [... informative example snipped ...] > But the bottom line is that only one set of ALTER TABLESs can be > generated in this situation, where we might need more than one. Nice debugging (and thanks for the solution in subsequent emails as well). I think this is the same problem as was reported in ticket #1968 (http://code.djangoproject.com/ticket/1928 ), so I've added a pointer to this thread to the bottom of that ticket -- since you have gone deeper than the original report. When somebody gets a chance to review this, your work should help. Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Some thoughts about the new settings.configure() and decoupling
Hi David, On Mon, 2006-05-22 at 15:06 -0700, David Elias wrote: > I think one way to go is with factories. > The packages don't know nothing where settings coming from. These > settings are passed within a constructor or something and django would > use factories that know to look from django.conf.settings and passed > them to the new object. That is precisely how it is currently implemented, in effect. Although it's not a completely plain "factory function", when you first access django.conf.settings or call configure(), it sets up the right settings-producing class that is used on all subsequent uses of django.conf.settings. It's not what a Computer Science student might immediately identify as a factory function from the shape of the code, but it is doing the same thing. And if, for some reason, you wanted to override the settings "producer", dropping a new implementation of UserSettingsHolder into django.conf is not too hard either (not documented, since there should be almost no reason for it and it's not that hard to read the code if you need to). Something like from django import conf conf.UserSettingsHolder = MyUserSettingsHolder conf.settings.configure(...) will work if you do it before anything else accesses the settings. > Although i don't get it yet, i think dependency injection > (http://www.martinfowler.com/articles/injection.html) could be a > possible solution :-) I'm not entirely sure what you are suggesting here. How does this help with the usage of this feature (which I assume is what you're talking about)? At some point, you still have to tell Django what the values for your particular set of configuration variables are. Or use the defaults. The configure() function provides that functionality at the moment. I'm really not sure what pieces you are trying to "decouple" here. Things like the templating system and ORM need configuration settings. Some some infrastructure is required. Omitting anything from under django.conf entirely would not really save lines of code overall. And the usage API seems about as minimal as you can get (set what you need, use default for the rest). So can you explain you thinking further? Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Buglet in tutorial 2
On Mon, 2006-05-22 at 20:27 -0500, Jeremy Dunck wrote: > At: > http://www.djangoproject.com/documentation/tutorial2/ > > This: > "By default, Django displays the repr() of each object." > > It actually uses str() now, right? Indeed. Fixed now. Thanks. :-) Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Patch review procedure?
I'm arriving late to this thread; been travelling on business last week and only catching up on email in some downtime this weekend. The topic here is something I've been tossing around in my head a bit, too, since I'm not sure how best to help without making life harder for Adrian and Jacob. Anyway, some thoughts/additions to Luke's excellent post... [This might get a bit long; apologies in advance.] On Thu, 2006-06-01 at 19:24 +0100, Luke Plant wrote: [...] > As a moderately experienced django hacker, I'm happy to help out with > this stuff (I have already been doing a bit, if in a rather haphazard > way), and I've had a think about it and detailed some ideas below. [...] > Principles > == > The solutions to this problem have to be: > - low maintenance > - opt in, to allow experienced django developers to > contribute as much or as little as they want, as their > experience and time allows. > - rigorous -- patches shouldn't wait months with no feedback > while others get applied immediately, there should be some > system to ensure everything gets reviewed. > - actually remove work-load from Adrian and Jacob, including the > time spent looking through bug lists etc. > - integrate with existing stuff -- no extra mailing lists > or websites please. Agree with all this 100%. In recent years (not so much lately, but particularly 2000 - 2004), I spent a lot of time working in the GNOME project which is a couple of orders of magnitude larger than Django in terms of lines of code, modules, bugs, even mailing lists (although not quite as many as 100x more than Dango there, thankfully!) At various points, the bug lists and patches got out of control and only a concentrated effort got things manageable again. However, when things were mostly under control and people understood the process, a little (or even long) delay was understood. Not killing the volunteers working on the project is a pretty key idea. I hit the wall a few times in busy GNOME periods, as did other developers working much harder than me. In times like that, being able to hand things along without too much trouble or notice is necessary. > > Suggestions > === > > I think it would be good to aim to add an initial comment to each > patch within N days (where Adrian and Jacob should decide the value of > N!). I would suggest that some of the experienced Django hackers should > be allowed to do this initial review (especially for obvious > WONTFIXes). I'll call these developers 'exp devs', Adrian and Jacob > and probably other commiters 'core devs', and both groups together 'all > devs'. > > A number of stock answers could be helpful. They would be > different for new feature patches and bug patches, and might be > something like below: > > For bugs: > - [core devs] patch looks OK, I aim to apply within N days/week > - [all devs] please add a test that supports the bug [obviously not > always feasible] > - [all devs] needs more review, I aim to do this in N days/weeks > - [exp devs] patch tests OK, I suggest to the core devs that this patch > (or my tidied up version) is applied immediately. This last one is where some clarification about expectations from the Adrian & Jacob show might be useful: in theory, most people committing to svn lately have pretty good sense and coding style (I'll ignore my own bits and let others evaluate those). So for pure bug fixes, it would not be unreasonable for your "exp devs" group to commit the fixes. I know both you and I, Luke, have dropped in small fixes of late that are "obviously correct", but I've always felt a bit nervous about doing this just due to lack of feedback. Does Adrian spit out his cereal everytime a change like this flies past on the change list? Or are we making things easier? It's the more complicated things that are correct on their own, but require a call as to whether they are in scope that I tend to punt upstairs. > > For new features: > - [core devs] this patch is almost certain to be accepted, I aim to > apply it within N days/weeks. > - [all devs] this patch is likely to be accepted at some point, I aim > to review again in N days/weeks. > - [all devs] this is interesting, but will have to wait for Django 1.1 > - [all devs] this patch is unlikely to be accepted, as I don't see > that the need is common enough, or it can be implemented in other ways > (featuritis), unless you can convince me otherwise. This one is hard and I've seen a few that are borderline here lately. Even ignoring the more extreme "Django is useless without this" claims on some items, evaluating the potential applicability of a change is sometimes tricky. We might need to maintain a list of "useful, but not yet core" patches and then ensure we come back and re-evaluate in a few months. Some more thoughts about how this could work below. > - [all devs] clean up your patch, add some documentati
Re: Solving the POST-data-lost-after-redirect problem
On Tue, 2006-06-06 at 22:11 -0500, Adrian Holovaty wrote: > http://code.djangoproject.com/wiki/NewbieMistakes#POSTtoviewslosesPOSTdata > > It's about time to solve this one. Who has a creative, elegant solution? :-) > > Some ideas, just to start discussion -- > > * THROW ERROR ON POST: The CommonMiddleware, when adding a slash and > redirecting, can check for POST data. If it finds any, it somehow logs > the error or otherwise alerts the developer. Maybe if DEBUG=True it > can actually display an error, with a custom-built message saying "You > need to change so-and-so form action to so-and-so URL"? I can't think > of any case in which a POST to a page that needs to be redirected > would be intentional, and if we only do it for DEBUG=True I think this > may be acceptable. I like this one. I would prefer to see the error is thrown whenever we are going to add a slash and the method is POST. Checking for POST data is probably over-specialising. The problem we are trying to solve is User Agents rewriting a POST as a GET, so if the original submission intended POST then -- whether or not there is data -- we should try to avoid the User Agents' bug in all circumstances. This really feels like a good solution. We are alerting the site developer to the problem as soon as we know about it and in a way that does not add a lot of extra processing time on the Django side. Also, what is the behaviour for other non-GET methods (DELETE, PUT)? Arguably, they are less vulnerable, since they will be sent by (more likely) non-browser clients and the coder should be able to read the spec, but should we be proactive and trigger the same error whenever the method is not GET? I would vote "yes" on this, but from a position of ignorance, I'll admit. > * THROW ERROR ON FORM DISPLAY: If the APPEND_SLASH setting is set to > True and the CommonMiddleware is activated, Django could check the > output of every page for a whose method is POST and has an > "action" whose URL doesn't end in a slash. Then it could somehow alter > the page to display an error on it. This solution is horrific, but I'm > mentioning it here for diversity. Err ... no. :-( Could we not mention this one ever again? Please? Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: runtests: 'invalid_models' module: Validator found 47 validation errors, 47 expected
Michael, On Wed, 2006-06-07 at 11:11 +0200, Michael Radziej wrote: > Hi, > > is this again just me :-), or has some checkin changed an error message > recently? > If I run the test suite, I get: > > 'invalid_models' module: Validator found 47 validation errors, 47 expected > == > Missing errors: > invalid_models.fielderrors: "choices": "choices" should be either a tuple or > list. > > Unexpected errors: > invalid_models.fielderrors: "choices": "choices" should be iterable (e.g., a > tuple or list). It's not just you. Checking 3094 (http://code.djangoproject.com/changeset/3094) changed the possibilities for choices. Somebody should talk to that Holovaty character about the importance of running the tests. :-) Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Solving the POST-data-lost-after-redirect problem
On Wed, 2006-06-07 at 02:12 -0700, Luke Plant wrote: > > Simon Willison wrote: > > > It's certainly a nasty solution here, but we shouldn't rule > > inspecting generated pages for errors out completely - just as long > > as anything like that is hived off in to a specialist tool. > > You could do this pretty easily as a 'debug' middleware that is enabled > by default. Ideally people would remove it from their list of > middleware for their deployed app, but as a precaution it would simply > do nothing if DEBUG=False. I already wrote a validator app that checks > outgoing HTML for validity, using a middleware to intercept the > response. It also stores all details of failed pages so you can go > back to them, which would be overkill in this case. Not entirely on-topic, but I must say that I think your HTML validation middleware deserves to be much better known. I was using it last week to check out some pages prior to release. It's very handy in development. Thanks. :-) Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Multiple database support (#1142) roadblock
On Wed, 2006-06-07 at 22:25 +, [EMAIL PROTECTED] wrote: [...] > However, I've run into a problem that can't be fixed with little > patches, in django.core.management. Specifically, all of the many > get_sql_* functions in there pull together sql from multiple models and > execute it all with the default connection. That's not going to work if > model A and model B want different connections. > > I'd argue that the right solution here would be to push the brains > farther out to the edge. Have management functions call class methods > on models to execute table creation, initial data loading, etc, rather > than having them poll the models for information and construct and > execute the sql themselves. Something like: > > def install(app): > # ... validation, start a transaction, etc > for model in models.get_models(app): > model.install() > > Rather than the current: > > def install(app): > # ... > sql_list = get_sql_all(app) > > try: > cursor = connection.cursor() > for sql in sql_list: > cursor.execute(sql) > > That would be a pretty substantial change, but I think it would open up > a lot of interesting possiblities for models that behave differently > from the default -- and it certainly would make supporting multiple > databases a whole lot easier. I don't think you can completely remove the controller portion here, although you can push a lot of the mechanics down into the model managers. The difficulty is that models do not exist completely independently of other models. Think about relations between models. ForeignKey and friends now have to be implemented differently or can only apply to models using the same database. And, in any case, they need to know the table name (and quite possibly database name and connection proxy) for the related tables. So you are going to have do a pass through all the models and build up the graph of dependencies and make that available to each model at construction time as well, aren't you? [As an aside: I think we are also going to discover that ForeignKey is unfortunately named, because one-to-many relations to tables in another database is not complete nonsense; I've worked on systems in the past that separated frequently read tables from frequently updated, less frequently read tables for performance reasons.] > > What do you all think? Nice work. :-) Best wishes, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Testing required for r3116
Can a couple of people with access to SQL Server or Oracle setups running Django please run the tests after updating to r3116. I just checked in a fix for ticket #2108 that I'm not 100% certain is correct for those two databases, although I suspect it's right. The problem is how to specify a default value in an insert statement. The SQL standard is DEFAULT and that works for PostgreSQL and MySQL (and I think I've used it on Oracle in the past). SQLite does not implement that, so I needed a workaround. The test I am particularly interested in is modeltests/empty, so "runtests.py empty" is the interesting one. The guts of the change are in r3115, but I forgot to commit the test at the same time. If it doesn't work, no existing code breaks, fortunately. It is only problematic for models with values set to the default. Thanks, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Should filters fail silently?
What's our policy on template filters failing? If applying a filter raises an error, should we just trap it and return the empty string, or raise an error? I can't find this written down anywhere, but maybe I've missed it. Should the answer depend upon the DEBUG setting? This is in particular reference to ticket #2127, but the answer will be useful elsewhere. Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Should filters fail silently?
On Sat, 2006-06-10 at 14:57 +, Gary Wilson wrote:
> Simon Willison wrote:
> > On 10 Jun 2006, at 07:39, Malcolm Tredinnick wrote:
> >
> > > What's our policy on template filters failing? If applying a filter
> > > raises an error, should we just trap it and return the empty
> > > string, or
> > > raise an error? I can't find this written down anywhere, but maybe
> > > I've
> > > missed it.
> >
> > The original policy has always been that errors in template filters
> > should fail silently - the idea being that it shouldn't be possible
> > to "break" a site by editing a template.
>
> Just to be clear, this following is quoted from
> http://www.djangoproject.com/documentation/templates_python/#writing-custom-template-filters
>
> "Filter functions should always return something. They shouldn't raise
> exceptions. They should fail silently. In case of error, they should
> return either the original input or an empty string -- whichever makes
> more sense."
>
> > With hindsight, I think that this was a mistake. An error in a
> > template is still an error, and needs to be caught. Failing silently
> > just means pages stay broken without the website maintenance team
> > hearing about it.
Ah, I checked the wrong document: I looked in the "template users"
document, not the "template developers" one.
> What are the altervatives though? If you raised an error for a
> template variable that wasn't defined, you wouldn't be able to do
> {{ possibly_undefined }}
> or
> {% if possibly_undefined %}{{ possibly_undefined }}{% endif %}
>
> Letting undefined variables return empty strings is both flexible and
> convenient.
This was actually the motivation for asking the original question. Your
above example uses only template *tags*, not *filters*. I was thinking
that if somebody had made the effort to include the data, then it's not
unreasonable to expect them to include it in the right form. And if the
data is going to possibly undefined, then something like the above test
should be used first.
But since we have a documented policy and the problem here is just my
inability to read, this issue is moot.
Thanks,
Malcolm
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---
Re: Bug in django.db.models.base.Model.save() for emptry models?
Hi Michael, On Mon, 2006-06-12 at 14:33 +0200, Michael Radziej wrote: > Hi, > > I wonder if there's a bug in the case that save() creates a new record for a > model without data. [...] > The pk value isn't retrieved. There should be a: > > setattr(self, self._meta.pk.attname, > backend.get_last_insert_id(cursor, self._meta.db_table, self._meta.pk.column)) > > Or, probaby better, move the settattr from the other case around: Well spotted! You are right, it's a bug and the tests I checked in when I made this change didn't test for this. It is fixed (with a test addition) in revision 3118. Thanks for checking this out. Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: RFC: Django history tracking
On Sat, 2006-06-17 at 07:27 +, william wrote:
>
[...]
> Sounds nice, this is a feature I'm currently looking for... but I've
> already started my own implementation.
>
> I would just share it with you.
>
> I've build a single table History with :
> - "change"; a text field which will contain a python pickled
> dictionary: { field: old_value} in case you update a record.
A drawback of this is that you pay the price when searching for "all
changes to field X since date D" or "show me a change history for field
X". You have to read and unpickle every single row before you can know
whether to discard it or not.
Another thing that occurs to me -- may not be relevant in your
particular situation, but does have general application -- unless you
are setting your timestamps manually (and it has sufficiently fine
granularity), you don't get a concept of a "changeset" of changes that
happen all at once, which does fall out of Uros's implementation.
> - type: type of modification (update, delete, insert).
> - "obj": the table object. This can come from ContentType
> - "obj_id": the id of the impacted object.
> - create_date: a timestamp automatically set.
Regards,
Malcolm
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---
Re: Proposal for documentation reform
[OK, this is longer than I intended. Sorry 'bout that. :-( Short version: yes, I agree. Focus on the organisation, too; we already have a lot of content; people apparently don't read the docs anyway; how we can kickstart this a bit; and some clues from other projects I've help on and hung around.] On Sat, 2006-06-17 at 23:24 -0500, James Bennett wrote: > I've had some thoughts bouncing around in my head for a while about > Django's documentation, and I think it's time to finally commit them > to tangible form, so here goes: [...] > So I got to thinking about reforming Django's documentation, and > basically came up with three types of documents that I think we need: Prologue: I'm not sure why you titled this "reform", since I think it's pretty much a natural extension of where we are travelling already. Just attempting to lay out and possibly coordinate the road-map a bit more carefully. One problem is that even a large table of contents makes it hard to find what you are looking for. So how do we manage the fire hose of information? (This is fundamentally hard, I think -- the slides at [1] are from a talk I gave about this same problem in the "desktop" space last year. Most of the points I make there -- particularly the "bad" ones -- carry over to something like Django.) [1] http://www.pointy-stick.com/desktopcon-2005/ In what follows, take it as read that I agree with your ideas and most of your hypothesis -- the current documentation is pretty good and comprehsnive. Witness how many times a correct answer to a mailing list query is a URL pointing to the existing documentation (but don't think about how many times it's the same URL you posted three days ago; that just gets depressing). I'm going to approach this from the direction of what do we have now and how can we extend that. > > 1. High-level overviews of Django, both in the form of tutorials and > in documents that look at Django as a whole to show how the different > parts fit together. This would be a great starting point for new > Django users. Agree. The current tutorial clearly works for some people. But everybody learns in different ways and repetition doesn't hurt, either. So agree that we could use some more things here in the tutorial department. I don't see a problem with having these scattered around various places in the web initially. How things fit together is something we don't entirely have at the moment. Your detailed "lifetime of a request" post fills in a large hole here. There are a few other summaries like this I can think of, too, mostly shorter things that answer some of the common queries we are seeing like - where can I put my views? (anywhere, because they just have to be importable and callable) - where do templates go? (it's in the templates-python docs, but easily overlooked) - and so on... Challenge for document writers: what do we want to do? How to avoid repetition? How to track what we've done? Where to put it? I guess these are the kind of things you had in your point #3. > 2. Lower-level documentation of the individual pieces, featuring as > obsessive a level of detail as possible. Experienced Django users, and > experienced web developers in general, will get a lot of mileage out > of this. I would have thought that many of the existing docs tend to fall into attempting to fill this category somewhat. And that can get confusing. Because it's maybe not enough detailed for the very experienced and "Too Much Information. Make It Stop. Aargh!"(tm) for people starting out. Although, if I step back and try to be objective, I think we probably get things pretty right for the people getting started who want to do more than just generic views and admin interface. I'm not entirely sure what you (James) have in mind here, but I am thinking something like API reference docs, but not auto-generated, since they are less than useless without some summary wrapping and guide for tourists. There's just too much machinery that cannot be differentiated from real public methods inside Django for auto-generated to not be confusing. Something else that falls into this category is documentation for developers. Things like - where are things in the source code - Your "lifetime of a request" write up is in this area. - What happens when a model is constructed - How do field classes work? How do we write new ones? - What were the developers smoking when.. (no, wait... better leave that one out). These don't have to be too detailed, since it's fair to assume that if you are wanting to hack on Django you are capable of opening up a source file and reading. And it is Python, after all, so the bar is set pretty low. > 3. More documents in the line of the "solving specific problems" items > we already have. This can cover a lot of things, from specific types > of server setups to implementing particualr
Regressions tests: a suggestion
There are a couple of problem I am looking at where it will be nice to put in some regression tests just to ensure they don't reoccur. However, putting them in tests/modeltests/* doesn't seem like the right place, because we are simultaneously using that directory as a source for examples. This traditionally doesn't work (mixing "good practice" and "evil, corner-case regression tests"). What I would like to propose is that we create a tests/regression/ subdirectory for these slightly more mind-numbing but important tests. Same sub-directory structure, etc, as the the modeltests/ directory, but not in any way intended to be examples of good model technique. Thoughts? Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Regressions tests: a suggestion
On Mon, 2006-06-19 at 12:28 -0500, Adrian Holovaty wrote: > On 6/18/06, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote: > > What I would like to propose is that we create a tests/regression/ > > subdirectory for these slightly more mind-numbing but important tests. > > Same sub-directory structure, etc, as the the modeltests/ directory, but > > not in any way intended to be examples of good model technique. > > Sounds like a good idea. Go for it! I've done this in r3176. It seems to actually require some tests in each directory for the framework to not explode, so I couldn't commit just the runtests.py changes and then the first bunch of tests. :-( Because of the way tests are imported in runtests, it is currently not possible to have a subdirectory of modeltests/ and a subdirectory of regressiontests/ with the same name (so, for example, I created regressiontests/one_to_one_regree/ ). If somebody wants to play around with that and fix it, be my guest; I intend to just leave it as a slight wart for now. Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: MultiValueDict and manipulators
On Tue, 2006-06-20 at 20:40 +, [EMAIL PROTECTED] wrote: > i'm having the same issue that is described in ticket #2079 (except the > fields are just CharFields). the ticket says that the issue was fixed. > so i updated to the latest build, but i'm still having the problem. > anyone else seen this issue with manipulators? Ticket #2079 is about a syntax error when working with form submissions. I'm not seeing the link between that ticket and CharFields, so can you provide some more details, please? If your code is raising an exception as in that ticket, maybe reopen the ticket and supply the details of the traceback. Thanks, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: trac updates/changesets not being sent?
On Wed, 2006-06-21 at 05:35 -0700, Luke Plant wrote: > > Jacob Kaplan-Moss wrote: > > > Yeah, the media server failed early this morning. > > > > It should be fixed now, but I think the updates still aren't being > > sent and I'll look into it. > > Cheers, updates seem to be working now. The community aggregator still > doesn't though. I noticed earlier this evening that automatic documentation rebuilds aren't happening, either (models-api.txt and templates.txt have both been updated today). Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Empty models don't get created (#1972)
On Wed, 2006-06-21 at 13:35 -0700, Tyson Tate wrote: > I posted a ticket (#1972) a while ago and there's been a few other > tickets reporting the same issue, so it sound like an issue that > should be addressed. > > Basically: A model with no explicit fields doesn't get created by > syncdb, but syncdb doesn't throw any warnings or errors. Consider the > following: > > --- > > class SignupList(models.Model): > # Has many SignupSlot objects > pass > > class Event(models.Model): > list = models.OneToOneField(SignupList) > > class SignupSlot(models.Model): > parent = models.ForeignKey(SignupList) > > --- > > (i.e. Every event has a signup list, which has a one-to-many > relationship with signup slots.) > > I have two ideas on how this could be addressed: > > 1. Throw a warning or error in Syncdb, letting the user know that > every model must have at least one explicit field. > > 2. Build the field as the programmer defined them. What are the use cases for this sort of construction? (1) Somebody wants to do this in production code, because they don't want their application to run very efficiently and can't be bothered to do the small rewrite necessary to eliminate the empty class. (2) During rapid development, a developer throws down a few placeholders, expecting to go back and fill in the necessary details as they implement them. Hopefully, case (2) is going to be more common (eventually all the people in case (1) will become unemployed programmers). For this reason, I would be in favour of just creating the class in the database and not issuing an error. On the grounds, that it gets the framework out of the way of the developer and lets them get on with their preferred method of working. Will be interesting to hear what others think, since I know James already wants to go the other way. Either solution is survivable in the end. In the interim, follow the advice at the end of the old joke: "Doctor, it hurts when I do this...". :-) Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: trac updates/changesets not being sent?
On Wed, 2006-06-21 at 19:42 -0500, Jacob Kaplan-Moss wrote: > On Jun 21, 2006, at 7:28 PM, Jeremy Dunck wrote: > > Was it Unicode? How did you find the offending bits? Is it something > > we could have helped with? (Can we help next time?) > > Nah, just a small change in the layout of the test directory that > broke the update script (see http://code.djangoproject.com/changeset/ > 3190) Sorry about that. I had not properly remembered that portions of runtests.py were used to generate the docs. :-( Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: ManyToManyField looks broken
On Sun, 2006-06-25 at 14:04 +0400, Ivan Sagalaev wrote: > I'd like to draw some attention to ticket > http://code.djangoproject.com/ticket/2232 (mainly russelm's attention as > it appeared after his checkin 3195). > > Basically ManyToMany relations don't work when you try to get related > queryset from a model where ManyToManyField is defined. Or is it just me? Pretty much "just you", but it's obviously a little trickier than that. This is basically ticket #1796 occurring again. You'll find it will only fail in some very specific cases and it's dependent upon both your machine and your Python build. The patch labelled ManyToMany.patch in ticket #1796 might well fix your problem in the short-term. I tend to agree with Luke (see his last comment on that ticket) that it's probably the completely correct fix, rather a correction after the fact of the problem. But we might have to check it in for now until we work out the deeper mystery there (which could well just be that things imported from different paths cannot be identified with each other reliably and so we can't fix this). Are the tests failing for you, too, Ivan? I just ran them (and my code is up to date) and also tested out a few apps I have here using ManyToMany and I saw no problem. I'm looking at #1796 now. Will try to check in something reasonable if I can in the next couple of hours (although I can't actually replicate the problem, so I'll probably just have to Use The Force or something). Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: ManyToManyField looks broken
On Sun, 2006-06-25 at 20:20 +1000, Malcolm Tredinnick wrote: > On Sun, 2006-06-25 at 14:04 +0400, Ivan Sagalaev wrote: > > I'd like to draw some attention to ticket > > http://code.djangoproject.com/ticket/2232 (mainly russelm's attention as > > it appeared after his checkin 3195). > > > > Basically ManyToMany relations don't work when you try to get related > > queryset from a model where ManyToManyField is defined. Or is it just me? > > Pretty much "just you", but it's obviously a little trickier than that. > This is basically ticket #1796 occurring again. You'll find it will only > fail in some very specific cases and it's dependent upon both your > machine and your Python build. > > The patch labelled ManyToMany.patch in ticket #1796 might well fix your > problem in the short-term. I tend to agree with Luke (see his last > comment on that ticket) that it's probably the completely correct fix, ".. probably NOT the completely correct fix." Oh dear, an unfortunate typo. > rather a correction after the fact of the problem. But we might have to > check it in for now until we work out the deeper mystery there (which > could well just be that things imported from different paths cannot be > identified with each other reliably and so we can't fix this). > > Are the tests failing for you, too, Ivan? I just ran them (and my code > is up to date) and also tested out a few apps I have here using > ManyToMany and I saw no problem. > > I'm looking at #1796 now. Will try to check in something reasonable if I > can in the next couple of hours (although I can't actually replicate the > problem, so I'll probably just have to Use The Force or something). Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: ManyToManyField looks broken
On Sun, 2006-06-25 at 22:39 +0800, Russell Keith-Magee wrote: > On 6/25/06, Ivan Sagalaev <[EMAIL PROTECTED]> wrote: > > > > I'd like to draw some attention to ticket > > http://code.djangoproject.com/ticket/2232 (mainly russelm's attention as > > it appeared after his checkin 3195). > > Paying attention now :-) > > > Basically ManyToMany relations don't work when you try to get related > > queryset from a model where ManyToManyField is defined. Or is it just me? > > For me, all the regression tests passed at time of checkin. Following > the rest of the messages on this thread, it looks like the problems is > with model importing. 2 quick observations which might help find the > problem: > > 1) r3195 did make some changes to the model importing process, so this > might have changed the conditions under when any previous __import__ > related bug revealed itself > > 2) I have seen the behaviour from r1796 before - for me, the problem > ultimately turned out to be caused by a PYTHONPATH that included the > application directory (so "from myapp.models import Foo" and "from > myproject.myapp.models import Foo" were both legal). At the time, I > chalked it up to the pythonpath, and moved on (sorry - can't give any > more specific details as to SVN revision, exact code structure, etc). > > I don't know if this helps anyone - however, I thought it might yield > a reliably broken test case. I've just checked in r3202 which I believe fixes the root cause of #1796 in a reasonably correct way. I think it also fixes #2232 for the same reason. It all seems to come down to the import paths being used as dictionary keys and models being registered twice. See the change for the way I am working around this. Russ: can you see if I've overlooked any problem there, please. It took a little while to track everything down and my brain may have gotten twisted in the process. Also, I left it alone for now and I realise you were just cutting-and-pasting, but __import__ takes dictionaries as the second and third arguments (globals() and locals(), basically, although the third arg is essentially ignored); why are we passing in strings??? I can no longer cause #1796 or anything like it to fail -- and I had a couple of failing examples there. I believe relative imports (appname.models, etc) might now work reliably too (experiments suggest they do), which means you no longer are reliant on your project name and can create portable apps. Feedback welcome/appreciated. I may have screwed up in some subtle way. Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: ManyToManyField looks broken
On Mon, 2006-06-26 at 00:58 +1000, Malcolm Tredinnick wrote: > I've just checked in r3202 which I believe fixes the root cause of #1796 > in a reasonably correct way. I think it also fixes #2232 for the same > reason. It all seems to come down to the import paths being used as > dictionary keys and models being registered twice. See the change for > the way I am working around this. > > Russ: can you see if I've overlooked any problem there, please. It took > a little while to track everything down and my brain may have gotten > twisted in the process. Also, I left it alone for now and I realise you > were just cutting-and-pasting, but __import__ takes dictionaries as the > second and third arguments (globals() and locals(), basically, although > the third arg is essentially ignored); why are we passing in strings??? > > I can no longer cause #1796 or anything like it to fail -- and I had a > couple of failing examples there. I believe relative imports > (appname.models, etc) might now work reliably too (experiments suggest > they do), which means you no longer are reliant on your project name and > can create portable apps. By the way, I'm pretty sure there are (or should be) some "reverse relation" bugs still present, for the same reasons as these two bugs. They were there before and are no worse than previously. I just need to trace through the reverse lookup stuff a bit more carefully. I'll fix them in the morning, but my brain is fried right now and it's nap time. Best wishes, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: ManyToManyField looks broken
On Sun, 2006-06-25 at 18:36 +0200, gabor wrote: > Malcolm Tredinnick wrote: > > > > I can no longer cause #1796 or anything like it to fail -- and I had a > > couple of failing examples there. I believe relative imports > > (appname.models, etc) might now work reliably too (experiments suggest > > they do), which means you no longer are reliant on your project name and > > can create portable apps. > > could you please try what i described here: > http://tinyurl.com/kkpgw > > for me it still fails :-( I just sent of another post before this one arrived: I'm aware that reverse relations still have problems. It's the same root cause, but not the identical problem as #1796. I'll fix it when I can. Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: URL confs should take callables as well as strings
On Mon, 2006-06-26 at 10:30 +0100, Simon Willison wrote:
> I've been playing around with making the URL resolver accept a
> callable in addition to accepting 'module.blah' strings, and I think
> it's a big improvement.
>
> The change itself is very simple - when a URL is resolved, callable
> (callback) is used to decide if the callback is already a callable
> object; if it is, it is called directly rather than being resolved in
> to a module/method first. It ends up being a two line patch.
>
> This then makes urls.py a whole lot more flexible. Here's an example:
>
> def redirect(url):
> def fn(request):
> return HttpResponseRedirect(url)
> return fn
>
> urlpatterns = patterns('',
> (r'^$', redirect('/HomePage/')),
> (r'^(\w+)/$', 'djwiki.views.wikipage'),
> ...
> )
>
> That redirect function can either exist in urls.py itself, or can be
> dropped straight in to djange.conf.urls.defaults (since all urls.py
> files start with 'from django.conf.urls.defaults import *').
>
> Allowing callables in urlconfs opens up a whole bunch of other
> opportunities as well. Setting up generic views could be made much
> less fiddly:
>
> from django.views.generic import ListDetail
> from models import Things
>
> thingviews = ListDetail(Things.objects.all())
>
> urlpatterns = patterns('',
> (r'^$', thingviews.object_list),
> (r'^(\d+)/$', thingviews.object_detail)
> ...
> )
>
> In fact, a whole bunch of stuff that currently has to be done with
> global settings or duplicated arguments could be handled in the URL
> configuration Python file instead.
I'm a little wary of encouraging people to put too much logic in the
URLConf files, but at the same time, I can see that it might make some
things easier and arbitrary restrictions are probably bad.
I guess I'm +0 on this at the moment, but I suspect that is because it
doesn't grab me in the "wow! nice!" way that some changes do. So pretend
I have no opinion. My real contribution to the thread is the next
paragraph.
>
> Another advantage that this offers is a more elegant alternative to
> the common prefix problem. Right now, if you are using a bunch of
> views from a certain module the recommendation is to do the following:
>
> urlpatterns = patterns('path.to.some.moduleviews',
> (r'^$', 'index'),
> (r'^(\d+)/$', 'detail')
> ...
> )
>
> This only works if EVERY view comes from the same module - if you
> want to mix and match views from different modules you're back to
> specifying the full path for each one (or futzing around with include
> ()).
Another thing we need to document better here, because this claim is not
quite correct (didn't you help write this?). The patterns() function
returns a normal Python list. So you can just append successive calls to
patterns() to each other:
urlpatterns = patterns('website.weblog.views',
# Latest entries.
(r'^$', 'display.section'),
...
)
urlpatterns += patterns('',
(r'^feeds/(?P.*?)$',
'django.contrib.syndication.views.feed',
{'feed_dict': feeds.feed_mapping}),
)
So (Jedi-like wave of hand), these are not the... er.. there is no
prefix problem.
Best wishes,
Malcolm
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---
Re: problem with revers lookup
On Mon, 2006-06-26 at 06:33 -0700, [EMAIL PROTECTED] wrote: > Hello > > I run into trouble when I tried to do backward reverse lookup. > Exampe in doc: Blog.objects.filter(entry__headline__contains='Lennon') > > It doesn't work and raises TypeError: Cannot resolve keyword 'blabla' > into field This is an ongoing. It has come up within the last 24 hours on the list and in a few Trac tickets. Ticket #1796 is kind of a reasonable summary of the difficulties (and Trac is usually the better place to report bugs; they may get lost on the mailing list). I am currently working on fixing it, but the problem is a bit subtle and because it's basically in the meta-programming portions of the code, mistakes have far-reaching consequences. So it will take another day or two before this is fully fixed, I suspect. It's not that we actually suck at fixing this. It's just that realiable failures are hard to come by. Once we fix one and think we understand the fix, it takes a good three or four hours for Ivan to break it again. > > I found that the problem is in get_all_related_objects method of class > Options. > > options.py:124 if f.rel and self == f.rel.to._meta: > > this comparison does not always yield True for the same models as > expected. > I have changed this line to: > > if f.rel and str(self) == str(f.rel.to._meta): > > and it works for me. This won't work in a large number of cases (try having multiple relations of the same kind between classes, for example; I suspect that will lead to ambiguities sometimes). Your diagnosis of the ultimate failure is correct, but the problem is that we should only have one of these Option instances floating around for each class and we end up with two. Anyway, when it's fixed, I'll put a page in the Wiki about how model construction happens internally so that people don't have to work it out again when debugging this sort of thing. Thanks for the report, anyway, Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Time field issues
Hi Tom, On Wed, 2006-06-28 at 03:09 -0700, tomass wrote: > Hi Folks, > > I've just tried to add a time field to one of my models, and then I > ran: > > ALTER TABLE blah ADD COLUMN blah_blah TIME; > UPDATE blah SET blah_blah='09:00:00' > > I restarted apache, and I can see my current entries are showing 9am > correctly in the new "blah_blah" field, but when I try to add a new > entry into the "blah" table I get the following error: > > AttributeError at /admin/runsheets/item/add/ > 'str' object has no attribute 'strftime' [...] > Any ideas what's happening here? Take your pick of ticket #1754, #2211, and #2192 (they are all the same problem). I'm going to look at this tonight (which is now, my time) and check in a fix. Too many people are tripping over it to delay much longer. Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Time field issues
Tom, On Wed, 2006-06-28 at 03:09 -0700, tomass wrote: > Hi Folks, > > I've just tried to add a time field to one of my models, and then I > ran: > > ALTER TABLE blah ADD COLUMN blah_blah TIME; > UPDATE blah SET blah_blah='09:00:00' > > I restarted apache, and I can see my current entries are showing 9am > correctly in the new "blah_blah" field, but when I try to add a new > entry into the "blah" table I get the following error: > > AttributeError at /admin/runsheets/item/add/ > 'str' object has no attribute 'strftime' > Request Method: GET > Request URL: http://example.com/admin/runsheets/item/add/ > Exception Type: AttributeError > Exception Value: 'str' object has no attribute 'strftime' > Exception Location: > > /usr/lib/python2.4/site-packages/Django-0.91-py2.4.egg/django/db/models/fields/__init__.py > in flatten_data, line 750 > > Any ideas what's happening here? I'm so sorry I misread where this was occurring in my earlier response (and that you were using a TimeField and not a DateField). It is not related at all to the tickets I pointed out. And I'm a bit unsure what is happening here. Could you post the full traceback that appears when you have the problem (there is a little link lower down the screen that says "cut and paste version"... click on that to get something that is suitable for email). Also, what doe the model field say? In particular, I am interested in what attributes you may have set in the TimeField(...) call. Again, sorry for not paying attention earlier. I was on autopilot a bit too much. If you can give a bit more information, we should be able to work this out. Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Request for history: incoming data processing
I've been writing an application this evening to add the Atom Publishing Protocol to my blog and some other projects I have in mind and hit a problem that I now realise Hugo, and no doubt others, have stumbled across in the past, too: Django wants to treat all incoming HTTP as though it was a form submission. We detect multipart submissions to get at file uploads. But everything else is run through cgi.parse_qsl(). It seems like correct handling is that that should only be done for application/x-www-form-urlencoded MIME types (maybe I haven't got the full set right; too late at night to go read all the specs now). For things like applications/atomserv+xml and related ones (basically, everything else), data should be left unprocessed by the Django layers, I would have thought. For example, an Atom publishing request contains XML data that can't be treated like a form. And the application needs access to the raw data to process it. The fix I have in mind is to add a "data" attribute to the request object that contains the unprocessed data in cases like this and NOT put the data into the POST dictionary unless it really is form-encoded (since we are treating that as a FORM dictionary, rather than a POST dictionary). But I wanted to find out if there was a reason for this behaviour in the original design (Adrian? Jacob? Simon?). Did you have something in mind to work around it? Or was it a case of "never came up before"? Coincidentally, with Jacob working on some serializer stuff, I suspect he may be encountering the same problems at the moment, so this could be fortuitous timing. Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Request for history: incoming data processing
On Wed, 2006-06-28 at 14:15 -0500, Jacob Kaplan-Moss wrote: > On Jun 28, 2006, at 11:45 AM, Malcolm Tredinnick wrote: > > For things like applications/atomserv+xml and related ones (basically, > > everything else), data should be left unprocessed by the Django > > layers, > > I would have thought. For example, an Atom publishing request contains > > XML data that can't be treated like a form. And the application needs > > access to the raw data to process it. > > Am I missing something or isn't ``request.raw_post_data`` what you > need? AFAIK, ``request.POST`` and friends are lazy-loading, and you > can just access ``request.raw_post_data`` to get the, um, raw post data. > > (See http://www.djangoproject.com/documentation/request_response/ > #attributes). > > Is that what you need, or am I being dense? One of us is being dense, but it isn't you. :-( All this looks much easier after a bit of sleep. Not sure what I was doing last night. Thanks for pointing out the obvious: raw_data will do what I want and all the HTTP headers are supplied correctly. So there's nothing much missing. Just my brains. Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Multiple db branch: tests
On Thu, 2006-06-29 at 14:40 +, [EMAIL PROTECTED] wrote: > Adrian Holovaty wrote: > > On 6/28/06, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > > > Is it appropriate to have a test that requires sqlite, or should tests > > > always use the database engine(s) specified in settings? > > > > Tests should always use the database engines in settings. > > Ok. > > > > If they should always use the engines in settings, when needing > > > multiple databases, is it appropriate to kill the test with an error > > > specifying how the settings must be set up for the test to run when the > > > needed databases are not defined? Or should the test create the > > > databases it needs using the default engine? > > > > It should work the way it works currently: It should attempt to create > > a test database (or databases). If any database of that name already > > exists, it should display a confirmation message. (Does this answer > > your question?) > > Almost. I'll try to ask more clearly. The question is: say I have a > test that requires connections in settings.DATABASES named 'a' and 'b'. > If there are no such DATABASES keys, should the test just stop with an > error saying: "This test requires connections in settings.DATABASES > named 'a' and 'b'", or should it create test databases for the keys it > needs, using the default settings as the template? Or is there a better > way to handle this that I haven't thought of? (If the keys do exist, I > assume that creating test databases and attaching to those names is the > right thing to do, and that will be done in runtests.py.) A comparison with the current tests might be useful here: although I believe you have to specify a DATABASE_NAME setting in the settings file you pass to the tests, it isn't used at the moment. Instead, the django_test database name is always used (except for SQLite's in-memory case). And we assume the user running the tests has "create database" permissions, since we create the database when the tests start and delete it at the end. So wouldn't the analogous situation in your case be to also reserve django_test_a and django_test_b names as databases we will create in the test framework and then use those? > > Also, for sqlite, as far as I know there's no way to create multiple > in-memory databases at the same time; is it ok to use temp files for > this, or should the tests just not run with sqlite? Use temp files. We cannot avoid testing one whole database arm (unless it's because the database in question is missing a feature, such as transactions with MySQL). > > > > Are there any objections to my adding setup/teardown support for model > > > tests? (Basically: run module.setup() if defined before running tests; > > > run module.teardown() if defined after running tests.) > > > > Yeah, I'm a bit hesitant to do this -- only because the model tests > > double as our "model examples" documentation, and for that use it's > > essential that the examples be clear. > > I should have looked at the doc generator before asking that, sorry -- > I didn't understand how it worked. How about, instead, updating > runtests so that it resets settings between tests? The reason this is > needed is that the multiple db test needs to test and document how to > create DATABASES settings, and I don't want to leave side-effects > sitting around after the test. Now much variation is there going to be in your test frameworks? From your earlier descrption of how the configuration works, it sounded like the settings were pretty fixed: a mapping of strings to database params. So can't you just use the same settings throughout, or are there various setups that induce different behaviour? Our current tests do not play around with configuration at all, so I'm wondering if you need to do this or if we can just assume a fixed settings setup. The documentation of this should go in the docs, not the tests. Unless there is some subtlety that means users of the code need to have a deep awareness of the settings at the time they write the code (which would seem to be problematic). I'm sure you are not just throwing these out to see how they sound, so I'm probably missing something. Is changing the settings in the midst of a test run necessary? Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Multiple db branch: tests
On Fri, 2006-06-30 at 02:59 +, [EMAIL PROTECTED] wrote:
> Malcolm,
>
> Thank you. You questions have actually answered a few of mine. For
> those that are left, I'll try to clarify...
>
> > So wouldn't the analogous situation in your case be to also reserve
> > django_test_a and django_test_b names as databases we will create in the
> > test framework and then use those?
>
> The runtests.py that I checked in today does something similar --
> increments a counter for each test database it creates. But that
> doesn't solve the problem, which I can see I'm not explaining very
> well. An example might help. In the implementation I'm working on, a
> DATABASES property is added to settings, which is a dict, like this:
>
> DATABASES = {
> 'a': { 'DATABASE_ENGINE': 'postgres',
>'DATABASE_NAME': 'whatever',
># ...
>},
> 'b': { 'DATABASE_ENGINE': 'postgres',
>'DATABASE_NAME': 'something',
># ...
>},
> 'tmp': { 'DATABASE_ENGINE': 'sqlite3',
> 'DATABASE_NAME': ':memory:'
> }
> }
>
> What I need for testing are predictable keys in that dict, since those
> are the names of the connections. The name is what a model specifies to
> indicate that it uses a non-default connection, like:
>
> class Zoo(models.Model):
> # ...
> class Meta:
> db_connection = 'a'
>
> (Likewise for transactions: transaction.commit(['a','b'])... ) So if my
> test models need connection 'a', I can either have the tests fail if
> there's no 'a' in DATABASES, or (if settings are reset between tests) I
> can set the settings I need inside of the tests themselves. The spirit
> of earlier answers seems to be "don't touch settings," so I'll go with
> the error option unless it just won't work at all, or someone has a
> better idea.
I understand what you are wanting to do. I'm wondering why you can't
create the dictionary inside runtests.py and then poke it inside
settings.DATABASE? Just like we do with settings.DATABASE_NAME (see line
160 of the current trunk's runtest.py). The settings file passed in by
the user then just specifies what database driver(s) to use and they
don't need to set things up according to what the tests want: you can do
that yourself inside runtests.py. That way your keys are completely
predictable and the burden on the user for setup is kept small.
Sure, writing to the settings class is officially "no-no", but breaking
the rules a little in the internal test framework supporting machinery
is not that horrible (and you won't be breaking the rules, so much as
following the pattern). It's tightly tied to the core anyway, so it's
not like we're going to not update things if conf.settings changes
internally. We already write to settings.DEBUG and settings.DATABASE
inside the test framework.
I guess one change you might want to make/require in the passed in
settings files is a list of available database drivers. So that you if
you want to generate a DATABASES dictionary containing multiple
connections to different databases, you know what you can choose from.
>
> > Now much variation is there going to be in your test frameworks? From
> > your earlier descrption of how the configuration works, it sounded like
> > the settings were pretty fixed: a mapping of strings to database params.
> > So can't you just use the same settings throughout, or are there various
> > setups that induce different behaviour? Our current tests do not play
> > around with configuration at all, so I'm wondering if you need to do
> > this or if we can just assume a fixed settings setup.
>
> There shouldn't be any need to change settings, beyond what I've
> described above. If the right thing to do is to just punt on the tests
> if they can't find the settings they need, then there's no need to
> change any settings at all.
>
> > The documentation of this should go in the docs, not the tests.
>
> This is where I really misunderstood things, and I think that's why
> some of my earlier questions were sort of obtuse. I thought that all of
> the docs on the site were built from the doctests. I hadn't looked at
> the *.txt files in docs to see that they aren't meant to be executable.
> So that really answers the biggest question, which was "how can I write
> tests for these settings values and have them get into the docs" -- the
> answer is, I don't, the docs and tests are separate. On the one hand
> that makes things a lot simpler, but on the other hand it makes me
> concerned about docs being wrong or out of sync with the tests -- how
> do you generally handle that?
The way I tend to work (can't speak for anybody else) when I'm checking
in patches is always to be asking myself whether there is a test and/or
a doc change required. This does require a good familiarity with the
docs, because some things (e.g. relations between models) are documented
in various places in more than one document. And sometimes something
get
Re: Multiple db branch: tests
On Fri, 2006-06-30 at 13:59 +, [EMAIL PROTECTED] wrote: > > I understand what you are wanting to do. I'm wondering why you can't > > create the dictionary inside runtests.py and then poke it inside > > settings.DATABASE? > > That's a good idea, I hadn't thought of that. Are there any objections > to reserving the keys 'django_test_db_a' and 'django_test_db_b' in > settings.DATABASE for use in multiple db tests? If that's ok, where > would I document that they are reserved? There's a constant at the top of runtests.py called TEST_DATABASE_NAME at the moment. If you put your constant(s) (tuple?) around there with a similar name, it will be easy enough to find. The current testing framework detects if a database with TEST_DATABASE_NAME already exists and asks for permission to stomp on it; you should do the same. Try running the tests with a django_test database already created and you'll see what I mean. So it's not vital that everybody knows about these database names, since we do not destroy data without permission. But if you put them in constants at the top of the file, finding them all will be easy. Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Cutting applications vertically, like multiple weblogs software
Hi Ahmad, On Fri, 2006-06-30 at 09:49 -0700, Ahmad Alhashemi wrote: > Hello, > > Immagine that I created a weblog application. Then I decide to make it > accomodate multiple weblogs, by adding a weblog model and weblog_id to > all records in the application. > > Is there an easy way to achieve this without having to change all the > model code in the application? I mean so that I don't have to go to the > "post new entry" view for example and add some code to make sure the > post is added to the correct weblog, then change the "view entries" > view to filter comments to only display the ones made in that weblog, > ..etc. > > What I was thinking is someway of partitioning certain models then > deciding the partition to be used somewhere outside the scope of any > certain view. > > I hope that my point is clear. This thread is more appropriate for the django-users list than django-developers. Please send any follow-ups there. To attempt to answer your question, though: I can't think of any automatic way to do this. I am not entirely clear about how you have set up your models (maybe post an example?), but if you need to ensure that the weblog_id field is set correctly then you are going to have to write some code to do that. Django can't read your mind; It does not know that you want to filter all your views and additions of new instances (via views) based on username and weblog_id or something like that. Filtering based on username (or some other way to tie the request to the weblog_id). And this is a feature of views, not of the model itself (since models aren't tied to web requests). So you are going to have to edit your view logic to convert from the current environment (logged-in username or whatever) to the right filter condition. Best wishes, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: i18n: Removing dependency on django.utils.translation from settings files
Hi Adrian,
On Sat, 2006-07-01 at 12:52 -0500, Adrian Holovaty wrote:
[...]
> The problem with this arrangement is that it causes any import of
> django.utils.translation to load django.conf.settings -- and the
> settings file in itself imports the translation hooks, because of the
> LANGUAGES setting in global_settings.py:
>
> from django.utils.translation import gettext_lazy as _
> LANGUAGES = (
> ('bn', _('Bengali')),
> ('cs', _('Czech')),
> ('cy', _('Welsh')),
> ('da', _('Danish')),
> ('de', _('German')),
> # ...
> )
>
> So, in order to make this work, I'm proposing that we *disallow*
> translation strings in settings files -- so that the
> django.utils.translation file can import from settings and we avoid
> circular import references.
>
> This leaves the problem: Where do we put the translation strings for
> the available languages? For that, we could put the translation
> strings in in django/utils/translation/trans_null.py and change any
> instance of LANGUAGES to apply the gettext call at runtime rather than
> compile time. Or is there a better way to solve the problem?
The way the translation extraction code (xgettext) works is to search
for particular keywords that are the names (or aliases) of functions
marking translatable strings. One of these keywords is gettext_noop,
which is the "mark for translation, but don't translate yet" function.
So you could leave the strings in settings.py, providing you define a
function called gettext_noop. All this function does is return the
string passed to it (that is all it does for real, anyway). Then the
strings are marked with as gettext_noop('Bengali'), etc, and it should
all Just Work(tm).
There are no namespacing issues here because xgettext just looks for
strings matching the keyword. So having a little
def gettext_noop(s): return s
in global_settings.py shouldn't hurt anything. You still need to then
wrap all uses of settings.LANGUAGES in a call to gettext() or _().
So does this help or hurt? For translators, it doesn't really matter
where these strings are, just as long as they are picked up by the
xgettext runs to generate the .po files. For i18n maintainers (Hugo),
they need to be able to find this list reasonably quickly for adding to
it. Splitting up the list so that two places have to be updated (the
list of codes in global_settings -- which I don't think you can remove
without making things awkward -- plus the mapping of code to translated
string) is not ideal (leads to mistakes).
Translatable strings in the project's settings.py are a small problem.
They would have to define their own gettext_noop() as above. Not a real
burden in those cases, I guess.
If the extra gettext_noop function in global_Settings.py doesn't offend
your sense of style too much, I would prefer that approach, just to keep
everything in one place.
Best wishes,
Malcolm
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---
Re: Cutting applications vertically, like multiple weblogs software
On Sun, 2006-07-02 at 00:54 -0700, Ahmad Alhashemi wrote: > Hi Luke, > > I agree that explicitness is better. But I think implicit filtering > here is not that bad for two reasons. The first is, as I said, it is > the rule not the exception. The second is that it makes it extremely > easy to start your application as a single site application then turn > it into a multi-site application. > > In all the other solutions, a lot of view code has to be changed when > you decide to turn a single site application into a multi-site one. > > Besides, I think of this feature more as DRY than as implicit parameter > setting. It is in a way similar to authentication code. You don't have > to repeat authentication code in every view becasue authentication is > the rule not the exception. > > The same way, you don't have to add code to tell the view that the > table contains records for many weblogs and we are only interested in > one weblog at a time. You will almost never run a query without > specifying the weblog_id because the weblogs are completely seperate, > and as I said before, you can almost put each weblog's data in its own > database, but you don't want to do that. To avoid going round and round in circles here with a lack of specifics, maybe you could suggest how you see this kind of API working. What would be added where to configure this type of filtering? And how will a request know what the right value to filter on would be? I am struggling to see how you can do this without tying views to models. I think you are going to have to pass in some kind of identifying information from your view to indicate the filtering parameter's value. And that level of parameterisation can already be done with customer manager methods. You are always going to have to specify the weblog_id (or whatever condition you are filtering on) somewhere and you are going to have to put in some logic somewhere to convert a request into the right weblog_id. The latter can be done by request context processors, so there's no repetition there, either. Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: i18n: Removing dependency on django.utils.translation from settings files
On Sun, 2006-07-02 at 21:16 -0500, Adrian Holovaty wrote:
> On 7/1/06, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote:
> > So you could leave the strings in settings.py, providing you define a
> > function called gettext_noop. All this function does is return the
> > string passed to it (that is all it does for real, anyway). Then the
> > strings are marked with as gettext_noop('Bengali'), etc, and it should
> > all Just Work(tm).
>
> I've made the changes you've suggested (in a local sandbox), but
> there's one problem left: The bottom of django/conf/__init__.py does
> the following:
>
> from django.utils import translation
> translation.install()
>
> This registers the _() function in the __builtins__ namespace.
>
> That code can no longer live there, because the new
> django.utils.translation depends on the USE_I18N setting. But if it
> doesn't live in django/conf/__init__.py, where should it live? What's
> a place where we can put it where we can be sure it'll always be run?
> I'm a bit stumped...
I'm not sure I like what I'm about to suggest, but let's throw it out
and see how it sounds...
How would you feel about making a requirement that settings must be
configured prior to attempting to use translations. By this, I mean,
Django must know that it is either using DJANGO_SETTINGS_MODULE or
manual configuration prior to trying to use _(). In that case, you could
put the import into __builtins__ into LazySettings._import_settings()
and LazySettings.configure(), because you will know the value of
USE_I18N at the end of both of those methods.
The reason I don't really like this (and I think it's a loser if I go
with "trust my gut", but I'm not 100% sure why) is the requirement to
"clarify" which settings are being used first. It might be a bit too
easy to forget to do that and it will be an absolute bear to debug.
The other thing I can think of is to install a function initially that
installs the "real" function over the top of itself the first time it is
called (see attached lazy-install.py for a proof of concept). The first
time the lazy _() is called, it can check settings.USE_I18N, which will
always be accessible at that point (the first access to "settings"
configures it if it has not already been done).
This is all starting to get a bit twisted, but might be unavoidable
unless you go back to the original plan of shipping the strings out of
global_settings and into another file.
I'd probably go for option 2 and a bunch of comments for future
explorers.
Regards,
Malcolm
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---
#!/usr/bin/env python
import sys
def new_f(*args):
print "Now calling new_f() with", args
def f(*args):
print "Calling f() the first time."
setattr(sys.modules['__builtin__'], '_', new_f)
return new_f(*args)
sys.modules['__builtin__']._ = f
if __name__ == '__main__':
_('fred')
_('barney')
_('wilma')
# vim: se ts=4 sw=4 et:
Re: #2237 - TEMPLATE_STRING_IF_INVALID and template IF statements
On Mon, 2006-07-03 at 22:49 +0800, Russell Keith-Magee wrote:
>
> Hi all,
>
> I've been tracking bug #2237, which has revealed a few weird cases in
> the way the TEMPLATE_STRING_IF_INVALID setting gets used. These cases
> are easy to find - put TEMPLATE_STRING_IF_INVALID='invalid' in your
> settings file, load the admin page, and watch the breakage that
> ensues.
>
> The breakage stems from a number of problems:
>
> 1) If you have TEMPLATE_STRING_IF_INVALID='foo', and your template
> contains an {%if key%}, and the context does _not_ contain 'key', the
> 'if' will still return True, because the resolving process says 'key'
> exists - with the value 'foo'.
This is ticket #1338, by the way.
Glad you're picking this up, because every time I look at it, I keep
wandering down the path you have and thinking "this is messy."
> 2) Use of TEMPLATE_STRING_IF_INVALID is not disabled if
> TEMPLATE_DEBUG=False. As a result, if you have such a string defined
> in the settings file you use to run the unit tests, 18 of the template
> tests fail - even though the template tests specifically disable
> template debug to ensure consistent results.
I thought this was kind of the point of the string: there was a way,
even in production, so that you could replace a missing variable with a
default value.
> 3) Ultimately, TEMPLATE_STRING_IF_INVALID isn't particularly useful
> for diagnosing problems; it is a catch-all warning message, which
> indicates any one of a large number of problems, but doesn't provide
> any additional detail. In some cases ( e.g., when ObjectDoesNotExist
> is thrown), the string is used as an intentional silent failure
> message.
Agreed. I have never used TEMPLATE_STRING_IF_INVALID (that I know of),
so I don't feel able to speak to its merits. But I think you make a good
case why it's not instantly useful as a debugging tool. I'm not
convinced it is useful as a production tool (what string other than the
empty string would I always want to put in my templates in those
cases?), so we arrive at your conclusions.
Others may have better experiences, though.
> I would like to propose the following three changes:
>
> 1) Modify {%if %} statements to assume a value of False if context
> doesn't contain a key, regardless of debug settings.
Strongly agree.
>
> 2) Remove TEMPLATE_STRING_IF_INVALID as a configurable setting, and
> replace it with a set of fixed messages that more accurately describe
> the template problem; for example: "(UNKNOWN CONTEXT: '%s')",
> "(INVALID NUMBER: '%s')", etc. These messages would be short, context
> sensitive, and unique to each type of template problem.
>
> 3) Only display the template error messages if TEMPLATE_DEBUG is
> enabled.
These two kind of go together: you are proposing making
son-of-TEMPLATE_STRING_IF_INVALID more of a debugging aid than a
production tool. I can see the benefits here, but, as I said, I'm not a
big user of the current functionality, so I might be missing something.
Regards,
Malcolm
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---
Re: My patch is gathering dust
On Mon, 2006-07-03 at 21:04 -0700, jws wrote: > I submitted a patch to the tracker some time ago that was never merged. > I'm unsure if there's some other community protocol for getting things > accepted that I need to follow. > > I've described the problem, submitted a complete, working patch, got > buy-in from Adrian in principle, and had at least one person comment on > the need for such functionality. > > Can I get someone to look at this again? > > http://code.djangoproject.com/ticket/470 Speaking only for myself here... I'm not against the patch in principle, but I haven't applied it when I've been doing runs through outstanding patches for a few reasons: (1) It's been closed twice; once each by Jacob and Adrian. Closing it or applying both make some sense to me, so I'm not about to go any further without some clarification. (2) It is in no way a showstopper. You can always work around this by using ALTER TABLE statements to add the DEFAULT clause if you really want it in the "initial SQL" files. So even the final comment is possible to set up (and in fact, that commenter was informed of the fact on #django). (3) The patch still needs some work. There is neither documentation nor test cases with it, so if I was to sit down and try to resolve this, it's still going to take a bit of time (not sure if documentation is required or not, but that will take some thinking to work out, too). Again, my personal way of working is that I tend to prioritise patches in my head with the ones that don't have workarounds or are core functionality or very, very common use cases near the top of the list. Easy or obviously correct fixes are next. Ticket #470 doesn't meet any of those criteria for me. I'm sure we'll get to it eventually, but for now not applying doesn't really have a huge impact on usability. Trac doesn't lose patches, so it's still there for review. And re-raising it isn't a bad thing either, since you might get some more feedback. But there are just under 100 patches in Trac at the moment, of which about 65 - 70 are potential "evaluate and resolve" cases (and that's not counting genuine showstopper bugs that don't have a proposed solution yet, which may justifiably jump the queue sometimes). We are going through them steadily (the number is well down in the past few weeks) and it has gotten a bit out of hand. But realise it does take more than just five minutes to evaluate a patch, even the most trivial ones, so please don't think you're being intentionally ignored. Best wishes, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: My patch is gathering dust
On Mon, 2006-07-03 at 23:33 -0700, jws wrote: > > (1) It's been closed twice; once each by Jacob and Adrian. Closing it or > > applying both make some sense to me, so I'm not about to go any further > > without some clarification. > > The discussion regarding the closure and subsequent re-opening are > documented here- > > http://groups.google.com/group/django-developers/browse_thread/thread/741ac793a9bf8459/d9bf516943a2e997?q=jws&rnum=1#d9bf516943a2e997 No need to retype all the information, but that link should be attached to the ticket, since it provides relevant background. Without that, the ticket reads very differently. > > (3) The patch still needs some work. There is neither documentation nor > > test cases with it, so if I was to sit down and try to resolve this, > > it's still going to take a bit of time (not sure if documentation is > > required or not, but that will take some thinking to work out, too). > > I think the previous discussion is comprehensive. Would summarizing the > discussion in the ticket be more helpful? Does it need any additions to the API documentation was more what I was wondering about. Maybe it doesn't, but it's always something that needs considering. > The lack of test cases is more convincing. They should be added, since we are bound to get some edge cases wrong (c.f. initial SQL parsing) and having regression tests will help there. If you could add those, that would be great. Testing both sorts of quotes, backslashes, semicolons and percentage signs would be my suggestion for a start. What other characters might we screw up? Anything that is dangerous for any database or that we might be using as a replacement hook is a candidate. A couple of other things I noticed just reading it through quickly: (1) Python traditionally uses "!=" rather than "<>" for "not equals". (2) NotImplemented is a type, not an exception. You want to catch the NotImplementedError exception (although I can't see it being raised anywhere by your additions; is that an omission?). (3) You could combine the rawchars and cookedchars lists into a single list of pairs and then just do "for before, after in new_list:" . That will prevent inadvertently getting the two lists out of sequence (and make the code quite a bit shorter). (4) Maybe you should just write escapechars() once and pass in the list of things to escape. Currently, any change in one has to be changed in four places. Is that possible without getting into import loops? Some of these may have simple answers, I don't know. I'm working on other things at the moment, so I don't want to stop completely for this. But since you've brought this up, we might as well keep the momentum going to help you get it to the point of being easy to apply. Best wishes, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Manual settings and the user's environment
I was having a discussion today with a client who are using Django templates inside their own code, without using the ORM or views. It initially started out because they were using the old os.environ['DJANGO_SETTINGS_MODULE'] = '__main__' hack and had discovered that the timezone was being changed on them throughout. After I pointed out there is manual configuration now, they rediscovered a bug that I had forgotten to file or fix (yeah, I suck. I wasn't near a network when I noticed it): we don't set os.environ['TZ'] when you use manual settings. This is good for this client, since they want it left alone, but it's technically a bug, since it means the two config methods behave differently in an undocumented way. Two possibilities: (1) We add a 'no_environ' parameter to settings.configure(). It defaults to True. When False, we do not put anything into the environment when installing the settings (for now, that just means TIME_ZONE, but it may be other things in the future, I don't know). (2) We leave things exactly as they are now on the grounds that somebody using the manual configuration, on the balance, probably already has their environment configured nicely and we shouldn't start playing with it under the covers. We need to fix the docs in this case. Either of these options seems reasonable to me. Do people have a strong preference one way or the other? I am not advocating changing the behaviour when using Django as a full-up framework. There, setting the timezone does make sense, since we can get it right once in one place, rather than asking everybody to understand how os.environ behaves. This is only for the standalone case (whether that be standalone templates, views, models or "other") when somebody is integrating a part of Django into another system. Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: My patch is gathering dust
On Tue, 2006-07-04 at 08:05 -0700, jws wrote: > > They should be added, since we are bound to get some edge cases wrong > > (c.f. initial SQL parsing) and having regression tests will help there. > > If you could add those, that would be great. Testing both sorts of > > quotes, backslashes, semicolons and percentage signs would be my > > suggestion for a start. What other characters might we screw up? > > Anything that is dangerous for any database or that we might be using as > > a replacement hook is a candidate. > > There doesn't seem to be any document regarding how to write > appropriate and useful tests. That would be helpful to a casual > contributor like me. My 'testing' framework is pure duct-tape. I can > root through the existing code, but some guidance would make it easier > to do the right thing. The existing setup is simple enough that following existing examples is really the way to go. If your tests can provide information as examples (such as, how this might be used in practice), then they should go under tests/modeltests/ (pick an appropriate directory... probably will be able to fit under something already there). Just add command line examples of usage and docstring that explain it. Bear in mind that things under this directory make it to the model examples page on the website, so it should be clear what is going on. If the tests are purely "regression test" in nature -- adding no real clarification -- then tests/regressiontests/ is a good location. These tests do not get used as examples and can be a bit more boring and trying out corner cases. If you need to create a new directory under regressiontests/, just be aware that it cannot have the same name as any directory under modeltests/. It might even be that we need things in both areas: an example or two and then some regression cases. Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Manual settings and the user's environment
On Tue, 2006-07-04 at 19:08 +1000, Malcolm Tredinnick wrote: > I was having a discussion today with a client who are using Django > templates inside their own code, without using the ORM or views. > > It initially started out because they were using the old > > os.environ['DJANGO_SETTINGS_MODULE'] = '__main__' > > hack and had discovered that the timezone was being changed on them > throughout. After I pointed out there is manual configuration now, they > rediscovered a bug that I had forgotten to file or fix (yeah, I suck. I > wasn't near a network when I noticed it): we don't set os.environ['TZ'] > when you use manual settings. This is good for this client, since they > want it left alone, but it's technically a bug, since it means the two > config methods behave differently in an undocumented way. > > Two possibilities: > > (1) We add a 'no_environ' parameter to settings.configure(). It defaults > to True. When False, we do not put anything into the environment when > installing the settings (for now, that just means TIME_ZONE, but it may > be other things in the future, I don't know). > > (2) We leave things exactly as they are now on the grounds that somebody > using the manual configuration, on the balance, probably already has > their environment configured nicely and we shouldn't start playing with > it under the covers. We need to fix the docs in this case. Anybody have thoughts about this? Thinking about it further, I am more and more inclined to go with option #2: don't mess with the environment when running Django components embedded in another application. But I may be insane (the phrase has certainly been used in my presence before. :-( ). Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Manual settings and the user's environment
On Thu, 2006-07-06 at 10:11 +0800, Russell Keith-Magee wrote: > > > On 7/6/06, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote: > > Anybody have thoughts about this? > > Thinking about it further, I am more and more inclined to go > with option > #2: don't mess with the environment when running Django > components > embedded in another application. But I may be insane (the > phrase has > certainly been used in my presence before. :-( ). > > Well... if the shoe fits... :-) > > Seriously - I'm inclined to agree with #2 as well. In a pure Django > app, we know what needs to be/doesn't need to be in the environment. > If you are using manual settings, all bets are off in that regard. > Document the need for TZ if manually configuring, and leave things the > way they are. > > Truth be told, I'm even uncomfortable with the way Django sets TZ when > settings are loaded. To me, TZ should be documented as an external > environment requirement, rather than being a Django setting that > affects the environment. Sometimes I think you're right about this and then some days not so much. I'm not sure there's a perfect answer to this one (remote database servers and a couple of assumptions in the code about UTC are usually the times when I decide that timezones are evil). The drawback with not setting it is that it's a pain in the hindquarters to get it right in all circumstances on your own. You need to set it very early so that the whole app is in the same timezone. So "manage.py runserver" will be slightly different from a mod_python setup, etc. We already spend enough time helping with people with basic Apache configuration; many of us would get it right, but they aren't the ones who will be posting to the mailing lists. Do we need the extra inconvenience? One idea that was bounced around in my conversation with the client was that TIME_ZONE defaults to empty (or None, preferably) and if, not set, does not get inserted into the environment. Personally, I don't really care too much about this, since it's such a trivial thing to set. > At the very least, Django should check if there is an existing TZ > setting before overriding it. I really don't have strong feelings either way on this. I gather the reason we do the current thing is for things like shared hosting where you don't have control over the machine's timezone. How often are we going to have TZ set in the environment when running a full-up Django, though? Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Proposal: default escaping (and branch request)
On Wed, 2006-07-05 at 18:50 -0700, SmileyChris wrote:
[...]
> My proposal is that we don't use a {% autoescape on/off %} block tag or
> a new |raw filter in the template source at all, but rather always use
> the view to set it the auto-escaping status.
> The developer wanting to use autoescaping can simply mark any variables
> which should be raw using markescaped() in the view.
One ongoing theme throughout this endless discussion has been trying to
avoid templates working differently in different contexts. Your proposal
again leads to a case where looking at the template does not provide
information about what is escaped or not. And, in fact, the answer to
that question is different depending upon which view is used to render
the template. That seems problematic.
Malcolm
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---
Re: *_set (ForeignKey accessors) broken?
On Wed, 2006-07-05 at 22:50 -0600, m h wrote: > I'll just post the contents of my bug report. Has anyone else ran > into this? Any hints, suggestions? > > > http://code.djangoproject.com/ticket/2297 This isn't a bug. You are using foo_set incorrectly (it is not an attribute on the Bar model, as the error indicates). Either bar.foo or foo_instance.foo_set is what you want to use. Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: urlify.js blocks out non-English chars - 2nd try?
On Thu, 2006-07-06 at 10:57 +0200, David Larlet wrote: > Hi all, > > I've recently added an enhancement (ticket #2282) about urlify without > checking for duplicate and there is already a proposal (my mistake) > and a discussion on this mailing-list which were unfortunatly closed > now: > http://groups.google.com/group/django-developers/browse_thread/thread/cecdf42cb3430601/1a53ee84c1742b1e > > I'd like to know if it's possible to do something about it? What are > previous conclusions and facts since the last discussion? I'm new in > Django and I may help in Python but not in js so I need your help ;). There was reasonable consensus in one of the threads about doing something similar (but a bit smaller) than what Wordpress does. Now it's a case of "patches gratefully accepted". A lot of people say this is a big issue for them, so it's something that will be fixed one day, but nobody has put in a reasonable patch yet. When that happens, we can progress. Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: urlify.js blocks out non-English chars - 2nd try?
Hi Bill, On Fri, 2006-07-07 at 10:06 +0100, Bill de hÓra wrote: > Malcolm Tredinnick wrote: > > > There was reasonable consensus in one of the threads about doing > > something similar (but a bit smaller) than what Wordpress does. Now it's > > a case of "patches gratefully accepted". A lot of people say this is a > > big issue for them, so it's something that will be fixed one day, but > > nobody has put in a reasonable patch yet. When that happens, we can > > progress. > > What's the expected scope of the downcoding? Would it be throwing a few > dicts together in the admin js, or a callback to unicodedata.normalize? I thought there was some sort of consensus; I didn't claim all the details had been settled. Personally, I was kind of hoping whoever wrote the patch might think this sort of thing through and give us a concrete target to throw ideas at. :-) My own misguided thoughts (I *really* don't want to have write this patch): I thought the original design wish was "something that read sensibly" here, since slugifying is already a lossy process. If I had to write it today, I would do the "dictionary mapping on the client side" version. But you're more of an expert here: what does normalization gain us without having to move to fully internationalised URLs, which still seem to be a phishing vector: if we allow fully international URLs, then doing everything properly would make sense. However, is it universally supported as "not a security risk" in all common browsers yet? Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Enabling Row Level Permissions
On Sat, 2006-07-08 at 16:54 +, Chris L wrote:
> Hi,
>
> I've currently set up enabling row level permissions using the meta
> class, e.g. to enable row level permissions for the Mineral model you
> would have:
>
> class Mineral(models.Model):
> name = models.CharField(maxlength=150)
> hardness = models.PositiveSmallIntegerField()
>
> class Admin:
> pass
>
> class Meta:
> unique_together = (('name', 'hardness'),)
> row_lvl_perms = True
The word "level" has more "e"s in it. :-)
Or do you want to be a PPC developer ([1])? There are counter-measures
we can take for that (see [2]).
Seriously, there seems to be a (good) trend in Django attributes to
spell the word out fully (I'm going to duck the question of whether
"perms" should be expanded, but I wouldn't fight it if it was).
[1] http://lkml.org/lkml/2001/10/12/91
[2] http://lkml.org/lkml/2004/1/21/8
> >From then on, you can access all row level permissions related to an
> instance of Mineral by using the attribute row_lvl_perms (e.g.
> mineral.row_lvl_perms.all() ).
>
> The way I am adding the row_lvl_perms attribute is using the following
> code in ModelBase (located in django.db.models.base):
>
> if getattr(new_class._meta, 'row_lvl_perms', None):
> from django.contrib.auth.models import RowLevelPermission
> gen_rel =
> django.db.models.GenericRelation(RowLevelPermission,
> object_id_field="type_id", content_type_field="type_ct")
> new_class.add_to_class("row_lvl_perms", gen_rel)
>
> Wanted to get an opinion from the other developers, do you think this
> is the best way of enabling row level permissions?
So the idea here is that the each row in the RowLevelPermission's
database table holds the content type of the model it refers to and thus
you don't have to change the "permissioned" model's table at all? Is
that right? Sorry if this a dumb question; just trying to get the
ordering right in my head.
In case you were wondering (or it occurs to you later), one thing you
can't do here is call loading.get_model('auth', 'rowlevelpersmission')
-- which is what we do in some places. Because there is no guarantee
that the model cache is fully initialised at that point and it is wrong
(trust me!) to initialise the model cache inside get_model(). And you
can't import RowLevelPermissions at the head of base.py because, I
suspect, it will cause problems when trying to run ModelBase.__new__()
on itself.
> I thought about enabling them by default and having to specify if you
> want them disabled, but I'm not sure if that would be the best route,
> especially since general permissions have to be enabled to work. Any
> thoughts?
What I'm not sure I understand is how does the checking process work: if
I am trying to extract a particular instance from the database, does it
look up the row table for that content type, see if there's a matching
value for the primary key from my main model and then do the check?
If so, that's at least one extra database query per extraction (I think
it might be two extras at the moment, but we can cache the content-type
with something like ticket #1717, although that will have to wait until
Adrian's finished playing inside django.db.models now). So, personally,
I would prefer it to be something that isn't on by default.
Looks like progress here. Nice. :-)
Regards,
Malcolm
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---
Re: wontfix
On Sat, 2006-07-08 at 21:37 -0700, bradford wrote: > There are many tickets out there marked as wontfix. It would be nice > to see an explaination of why the ticket was marked as wontfix on all > tickets marked as wontfix. > > Just a brief description would do. Most of them are so annotated, in my experience. We obviously won't be going back and recommenting on any that are already closed, so is there a particular ticket you had a query about? Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Enabling Row Level Permissions
On Sun, 2006-07-09 at 05:37 +, Chris L wrote:
[...]
> Each row in the RLP(Row Level Permission) table contains:
> - The RLP id
> - The content type for the "owner" (e.g. user, group)
> - The id of the owner
> - The content type of the object
> - The id of the object
> - The permission id
> - If the RLP is negative or positive (e.g. user A can edit article 2 or
> user A can NOT edit article 2)
>
> No modifications of the permissions table at all. Trying to keep the
> current permission system the way it is, so if you don't want to use
> RLP, no need to make any changes.
No modification to the original model either, which is more what I was
asking about. Nice.
>
> > In case you were wondering (or it occurs to you later), one thing you
> > can't do here is call loading.get_model('auth', 'rowlevelpersmission')
> > -- which is what we do in some places. Because there is no guarantee
> > that the model cache is fully initialised at that point and it is wrong
> > (trust me!) to initialise the model cache inside get_model(). And you
> > can't import RowLevelPermissions at the head of base.py because, I
> > suspect, it will cause problems when trying to run ModelBase.__new__()
> > on itself.
>
> This could explain some problems I've been running into. Do you think
> my current location for importing the RLP ok?
It looks okay and you are protecting with a conditional test that checks
for the right attribute being present. What problems are you seeing?
It's pretty easy to get into a knot in the __new__() method there, but
from the fragment you posted you seem to have avoided the obvious
problems.
Just be aware (I guess you already are) that __new__() is run at import
time, not when you first create the model. And creating models with
relations to other models normally results in importing every model from
every installed app you have (because of some features in the Options
class implementation). I wrote up a somewhat detailed description of how
all this works in the wiki --
http://code.djangoproject.com/wiki/DevModelCreation -- which may be of
use if you haven't already worked out what is going on.
Difficult to guess what is happening without some details, but feel free
to post those if you are really stuck.
>
> > What I'm not sure I understand is how does the checking process work: if
> > I am trying to extract a particular instance from the database, does it
> > look up the row table for that content type, see if there's a matching
> > value for the primary key from my main model and then do the check?
> >
> > If so, that's at least one extra database query per extraction (I think
> > it might be two extras at the moment, but we can cache the content-type
> > with something like ticket #1717, although that will have to wait until
> > Adrian's finished playing inside django.db.models now). So, personally,
> > I would prefer it to be something that isn't on by default.
>
> It would have to look up the content type and use the content type and
> object id to determine the row level permissions that exist, or from
> another angle, the owner content type and the owner id. Never even
> thought about the extra database queries required. I think that's
> another good argument to have it disabled by default.
Just to be clear: looking up the content type is what ticket #1717 is
designed to optimise. It will cache the value after looking it up once.
It is, after all, a property of the model, not a property of each
instance of the model, so we can do it once quite safely. The only
reason it hasn't been committed yet is that I've been an old woman when
it comes to making a commitment on the API. I should just make a
decision (or hope that somebody else will!). So that query should be
essentially zero cost over the lifetime of a model. The other query to
determine the permissions is the one I was thinking about.
Regards,
Malcolm
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---
Re: multi-auth imports auth backend on each request?
On Mon, 2006-07-10 at 14:40 +0200, Michael Radziej wrote: > Hi, > > Looking into django.contrib.auth.__init__, it seems that > load_backend() is called for each request when you need to access > the request.user object. Then, the backend module is imported via > __import__. Isn't that a little much of an overhead? The __import__() call does not reimport the module if it already exists in sys.modules. It works just like the "import" statement in Python (and reload() works like imp.load_module()). So, aside from a single dictionary lookup, there is no overhead to calling it multiple times. Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: multi-auth imports auth backend on each request?
On Mon, 2006-07-10 at 15:01 +0200, Michael Radziej wrote: > Malcolm Tredinnick wrote: > > The __import__() call does not reimport the module if it already exists > > in sys.modules. It works just like the "import" statement in Python (and > > reload() works like imp.load_module()). So, aside from a single > > dictionary lookup, there is no overhead to calling it multiple times. > > Ahhh! Thanks, Malcolm, and sorry for bothering. Asking questions is never harmful. :-) Best wishes, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: a Caching model.manager
Hey Ian,
On Tue, 2006-07-11 at 11:59 +1000, Ian Holsman wrote:
> this is what I was thinking of
> from django.db import models
> from django.db.models.query import QuerySet
>
> class SmallQuerySet(QuerySet):
> cache={}
> def get(self, *args, **kwargs):
> cachekey = "%s:%s:%s" % (self.model, str(args), str(kwargs ) )
> if self.cache.has_key( cachekey ) :
> return self.cache[cachekey]
> x= super(SmallQuerySet, self).get(*args,**kwargs)
> self.cache[ cachekey ] = x
> return x
>
> class SmallManager(models.Manager):
> def __init__(self):
> super(SmallManager, self).__init__()
>
> def get_query_set(self):
> """Returns a new QuerySet object. Subclasses can override
> this method
> to easily customise the behaviour of the Manager.
> """
> return SmallQuerySet(self.model)
>
> the aim is to then use this model manager on Site and ContentType.
>
> this of course would mean you would need to 'bounce' your webserver
> if/when you change records in models which use this class.
> but that shouldn't be too often.
>
> is this totally out of line?
> or should I just be using the 'regular' cache here.
Tricky implementation ("tricky" != "bad"), but it looks like it should
work more or less. Don't you need to override every public method of
QuerySet, not just get() (e.g. filter(), order_by()...)?
In a long running process, memory usage is going to be an issue. You are
essentially caching the results of every single query during the runtime
of the process, aren't you? So if I retrieve something that temporarily
sucks in a couple of thousand lines of data and I only need it once, I
pay the memory penalty forever. A "sorted dict" structure that allows
you to remove the oldest entries easily would work around this somewhat
(managing total memory usage wouldn't be easy, but you could manage the
number of cached querysets). I'm a little concerned about whether an
"active" application is just going to end up thrashing the cache a lot
without seeing any real benefit, but the overhead of that should not be
too much. And it's testable once we have an implementation anyway.
I think your idea is probably worthwhile.
Regards,
Malcolm
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---
Re: Multi-db branch feedback/questions
I agree with most of your points (the settings one is actually kind of obvious that it may be backwards now that you mention it). Inter-database relations would be nice to have at some point, but it could be a post-first-release kind of feature. On Mon, 2006-07-10 at 21:38 -0500, Jacob Kaplan-Moss wrote: [...] > Finally: will there be syntax to use a specific connection for some > operations even if that model's set up to use a different backend? > I'd hope so, but I'm not sure I can come up with a good syntax... What sort of use cases do you have in mind for this? Constructing inter-db relations by hand? Thinking of some of his earlier mails, I think Jason already had in mind some way of getting the specific connections via db.backend, since we would need that for manually constructing SQL in any case. But the way you phrased the above query makes me think you have some trickier use in mind. Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Multi-db branch feedback/questions
On Mon, 2006-07-10 at 22:09 -0500, jason pellerin wrote:
> Hi all,
>
> Thanks, Jacob. And yes please, more eyes are badly needed. Aside from
> dropping tables, the branch is fully functional -- please try it out!
>
> > I'd like to see this done the other way around, which the DATABASES
> > settings defining which *models* are used. I'm picturing something
> > like this::
> >
> > DATABASES = {
> > 'a' : {
> > 'DATABASE_ENGINE' : 'sqlite3',
> > 'DATABASE_NAME': '/whatever/foo.db',
> > 'APPS' : ['myproject', 'yourproject.widget',
> > 'someotherapp']
> > }
> > }
> >
> > So each member of APPS is either an app specifier (as you'd pass to
> > models.get_app) or a model specifier (as you'd pass to
> > models.get_model). This way each site can specify different
> > databases for different apps or even individual models.
>
> This is interesting. How would it work -- look up the APPS in each
> settings entry at model class creation time, and fill in
> opts.db_connection that way? Or do you want to get rid of
> opts.db_connection entirely? I'd prefer to keep it -- for one thing,
> without that attribute around, the work to find the proper connection
> will have to be repeated every time the connection is requested. (If
> it's understood to be more of a connection-name cache rather than the
> prefered point of configuration, that's ok with me.)
>
> I'm not sure that tucking APPS into the db-definition dict is the
> right thing, though. This may be too "magical", but how about: the
> keys in DATABASES could be app or model names, and in that case,
> matching apps/models will use the connection for that key. Example:
>
> DATABASES = {
> # symbolic names are still ok, but you'd have to set them in the model
> 'a': { ... },
> # but specify an app or model and that app or model will use this
> connection
> 'myproj.myapp': { ... },
> 'thatproj.thatapp.ThatModel': { }
> }
Under Jacob's proposal, if I have five applications and all the models
in each application use the same connection, I need to write out (at
most), DATABASE_ENGINE and DATABASE_NAME once for each engine/name pair
and I need write the application names once.
Under your proposal, I need to write DATABASE_ENGINE and DATABASE_NAME
once for each application, because there is no way to group the
information is even one of my applications needs a different database.
That seems like a lot more configuration (longer config files are harder
to debug, too).
I would prefer to get rid of the option of putting it in models
altogether, having read Jacob's reasoning: having multiple ways to do
something seems unnecessary here (is it necessary for some reason I
haven't thought of?) and it is encouraging non-portable, difficult to
debug problems when moving applications around. What advantages does it
add that you see?
Jacob's approach still leaves you with a natural key to use when trying
to refer to the connection directly (so keeping DATABASES as a
dictionary, rather than just a list of dictionaries is worthwhile).
>
> That seems pretty natural and clear to me, at least.
I am unconvinced. :-)
Best wishes,
Malcolm
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---
Re: would is_loggedin be better than is_anonymous?
On Mon, 2006-07-10 at 20:05 -0700, Gary Wilson wrote: > Proposing an is_loggedin method for User and AnonymousUser models. > > Templates treat non-existing variables as False; therefore; a > request.user somehow failing to get assigned would result in the > template treating the user as they are authenticated (is_anonymous > evaluates False). An is_loggedin would be a more secure option because > if request.user failed to get assigned, the template would treat the > user as anonymous. > > I would much rather mistakenly treat an authenticated user as anonymous > than mistakenly treat an anonymous user as authenticated. > > is_anonymous could be changed to return the negation of is_loggedin. I like this. :-) I would prefer to declare is_anonymous as deprecated, though, rather than just make it work transparently again. Then we can consider removing it in some weeks/months (prior to 1.0). It will always be dangerous if your above logic is correct -- just tying it to is_logged_in (is_authenticated?) does not change that -- so we should force people to use the right method. Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Multi-db branch feedback/questions
On Mon, 2006-07-10 at 22:24 -0500, Jacob Kaplan-Moss wrote: > On Jul 10, 2006, at 10:09 PM, jason pellerin wrote: [...] > >> Finally: will there be syntax to use a specific connection for some > >> operations even if that model's set up to use a different backend? > >> I'd hope so, but I'm not sure I can come up with a good syntax... > > > > Well, you can always set Model._meta.db_connection = 'foo' at any > > time. Or create a manager that pulls the connection to use from > > someplace other than self.model._meta.connection, though that would be > > difficult without some additional abstraction. I'm not quite sure what > > the use of this would be, but that may mean there's a need that I just > > can't see. > > I posted a couple of example use-cases in response to Malcolm. > Manually setting ``db_connection`` is ugly but workable; I'll think > some more about a more elegant way of doing it. One possibility based on what you (Jacob) suggested earlier: if the db connection is a property of the model manager, then it is something you can override by using a different manager. We already have get_query_set() as the "default query set" for a manager. We could have get_db_connection() as the way to get the db connection (or make it a property or an attribute or whatever; the concept is the thing here). Then you can use various managers to get data from different databases. True, you could end having code that looks like this: if preferred_connection = 'first_db': manager = 'first' else: manager = 'second' data = getattr(MyModel, manager).filter(...) which could get old after the first two or three times, although wrapping that in a convenience function (MyModel.get_manager(manager).filter(...)) is not crazy in those cases. In passing, storing the *default* db connection in the Option class doesn't feel too ugly to me: it's a place to store model-specific (as opposed to instance-specific) information. We just happen to populate it mostly from Meta (I'm rewriting history saying it this way, I realise, but does that sound like a reasoning that makes sense). Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Manipulators going away?
On Tue, 2006-07-11 at 03:15 -0700, Bjørn Stabell wrote: > Adrian Holovaty wrote: > > You're right -- models will gain validation, and there will be a > > helper function/method that returns a data structure that encapsulates > > form validation/redisplay (like a Manipulator currently does). That'll > > be replacing the current "automatic" manipulators, which I've never > > really liked; they were originally intended for use only by the admin > > site, not by end users, which explains the problems such as "Why does > > file upload not work for the automatic manipulators" and "How can I > > use edit inline with the automatic manipulators." > > Sounds cool. I also thought manipulators were a strange abstraction. > Any timeline? :) It's been said "before 1.0" on this list previously. > > Would this also take care of the issue I'm having right now that > stand-alone validators (not defined inside of a manipulator) only has > access to the form data, not the the actual object? "Models will gain validation" does cover that. It is probably worth looking back to earlier threads in the archives about this sort of thing, particularly where the early use of validate() was discussed (around Feb 22 this year was a thread that should cover a lot of the concepts Adrian was discussing at the time). A lot of that got put off during the push to finish the magic-removal merge. > > I'm trying to detect and prevent cyclic relationships between objects > linked together using ForeignKeys, so I need to make sure the object > being changed isn't already part of some other list of objects. That's biting a few people at the moment. Moving a lot of the model specific validation inside the model itself will help this. Lack of access to "self" is currently a pain sometimes. Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Templates and user.is_anonymous
On Tue, 2006-07-11 at 15:52 -0700, SmileyChris wrote: > When using templates and the user interface, the logic seems backwards > to check for an anonymous user as opposed to a logged in user. > > By using user.is_anonymous in a template, you assume that if it's false > then the user is logged in, where in actual fact "user" may just not be > getting passed to context. It seems a pretty dangerous assumption to > make. Gary Wilson posted about this yesterday. Could we endeavour to keep that conversation in a single thread, rather than having to track two different threads? Thanks, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Suggestion for #1602, urlify.js blocks out non-English chars [was urlify.js blocks out...]
On Wed, 2006-07-12 at 02:54 +0100, Bill de hÓra wrote:
> Malcolm Tredinnick wrote:
>
> > Personally, I was kind of hoping whoever wrote
> > the patch might think this sort of thing through and give us a concrete
> > target to throw ideas at. :-)
>
> Hi Malcolm,
>
> Here we go:
aah ... batter up! :-)
>
> [[[urlify.js:
>
> var LATIN_MAP =
> {
> 'À':'A',
> 'Á':'A',
> 'à':'a',
> 'á':'a',
> '©':'c'
> } ;
> var LATIN_SYMBOLS_MAP =
> {
> }
> var GREEK_MAP =
> {
> }
> var TURKISH_MAP =
> {
> }
> var RUSSIAN_MAP =
> {
> }
>
> var ALL_DOWNCODE_MAPS=new Array()
> ALL_DOWNCODE_MAPS[0]=LATIN_MAP
> ALL_DOWNCODE_MAPS[1]=LATIN_SYMBOLS_MAP
> ALL_DOWNCODE_MAPS[2]=GREEK_MAP
> ALL_DOWNCODE_MAPS[3]=TURKISH_MAP
> ALL_DOWNCODE_MAPS[4]=RUSSIAN_MAP
>
> var Downcoder = new Object() ;
>
> Downcoder.Initialize = function()
> {
> if (Downcoder.map) // already made
> return ;
> Downcoder.map ={}
> Downcoder.chars = '' ;
> for(var i in ALL_DOWNCODE_MAPS)
> {
> var lookup = ALL_DOWNCODE_MAPS[i]
> for (var c in lookup)
> {
> Downcoder.map[c] = lookup[c] ;
> Downcoder.chars += c ;
> }
> }
> Downcoder.regex = new RegExp('[' + Downcoder.chars + ']|[^' +
> Downcoder.chars + ']+','g') ;
> }
>
> downcode= function( slug )
> {
> Downcoder.Initialize() ;
> var downcoded =""
> var pieces = str.match(Downcoder.regex);
> if(pieces)
> {
> for (var i = 0 ; i < pieces.length ; i++)
> {
> if (pieces[i].length == 1)
> {
> var mapped = Downcoder.map[pieces[i]] ;
> if (mapped != null)
> {
> downcoded+=mapped;
> continue ;
> }
> }
> else
> {
> downcoded+=pieces[i];
>}
> }
> }
> else
> {
> downcoded = slug;
> }
> return downcoded;
> }
>
> function URLify(s, num_chars) {
> s = downcode(s);
> removelist = ["a", "an", "as", "at", "before", "but", "by", "for",
> "from",
>"is", "in", "into", "like", "of", "off", "on",
> "onto", "per",
>"since", "than", "the", "this", "that", "to", "up",
> "via",
>"with"];
> r = new RegExp('\\b(' + removelist.join('|') + ')\\b', 'gi');
> s = s.replace(r, '');
> // if downcode fails, the char will be stripped here
> s = s.replace(/[^-A-Z0-9\s]/gi, '');
> s = s.replace(/^\s+|\s+$/g, ''); // trim leading/trailing spaces
> s = s.replace(/[-\s]+/g, '-'); // convert spaces to hyphens
> s = s.toLowerCase(); // convert to lowercase
> return s.substring(0, num_chars);// trim to first num_chars chars
> return s.substring(0, num_chars);// trim to first num_chars chars
> }
> ]]]
Probably only one of the last two lines is necessary. :-)
I am the about the worst guy in the world to review Javascript code, but
I agree it seems to behave logically. I'm just not familiar enough with
the ins and outs enough to spot any sneaky problems that might creep
into code like this. Fortunately, the audience is full of people with
real clues.
> I need to test this properly and fill in the mappings, but the gist of
> the approach should be clear. When that's done, unless someone has an
> objection, I'll file a patch against
>
>http://code.djangoproject.com/ticket/1602
>(also 2282)
> .
In the interests of economy I just closed #1602 as a dupe of #2282 (the
latter has more links to mailing list threads). It's too hard to track
all the dupes and semi-related bugs.
Anyway, thanks for doing the work, Bill. This looks like something that
is a great start and that we can tweak as people find problems in their
own locales.
Thanks,
Malcolm
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---
Re: would is_loggedin be better than is_anonymous?
On Tue, 2006-07-11 at 21:34 -0700, Gary Wilson wrote: > Thinking of this some more, I'm wondering about the names is_logged_in > or is_authenticated. They could be a little misleading since they > aren't really checking if the user is logged in or not. One might be > led to believe that they could do something like logged_in_users = > [user.is_logged_in() for user in User.objects.all()] or something like > user = User.objects.get(username='fred'); if user.is_logged_in(): ... > > It seems that maybe the is_anonymous function is just a way for the > object to say it's either an AnonymousUser object or it's not an > AnonymousUser object. Hmm ... good points. We document is_anonymous() the way you describe it in the last paragraph. In the same document, we use it in the slightly accident-prone fashion your original post (and the thread on django-users) described. Considering alternatives in order of decreasing "stuipdness": one fix is to make improvement in the documentation: "If you are going to put a "members only" block of text up, you should check that user.id evaluates as True." It is guaranteed that id == None on AnonymousUser objects. Another possibility is to call the name "not_anonymous" (but not really that name, since it's a pretty stupid name). Does "registered" work as a name? I guess I prefer finding a better name, rather than hoping for "success through documentation". Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Test framework for end-user applications
On Wed, 2006-07-12 at 15:38 +0800, Russell Keith-Magee wrote: > > Hi all, > > I've got a project at work that requires a formal test harness, so I > thought I would kill two birds with one stone and spend some time > adding a formal test framework for all Django applications. Hmmm .. deja vu. When Adrian said he was going to refactor db/models/ bits and pieces, I put the m-i work on hold last weekend and was thinking about Simon's post as well. I had a need to rewrite some templates and wanted some regression tests for them. I approached things from a different direction, though, more concentrating on how to unit test templates and views (as in not requiring a full end-to-end application setup in order to test them). So it's actually a orthogonal to what you are proposing here and fits in nicely. I'll write up what I was thinking about in another mail; I started to post it in response to this, but it adds no value to your proposal. > This came up a few weeks ago [1], and although I didn't comment at the > time, it struck me as a good idea. Plus, as Simon pointed out, it is a > feature that Rails has but Django does not, and we can't let those > dirty Rails-using savages have something that we don't :-) I have > created ticket #2333 to track the feature. > > As a first pass, I have been concentrating on adding the framework to > find and run unittests and doctests within a Django application. This > is essentially what tests/runtests.py already does for the Django > tests, except that I am using the unittest framework to find and > execute tests. > > In my sandbox, I have made modifications such that: > - Any classes in models.py that extend unittest.TestCase will be > identified as a test case. > - Any doctests in models.py will be extracted as test cases (using > normal doctest extraction strategies - any module, class, or method > level doctests, plus anything listed in a __test__ attribute) > - If a mysite.myapp.tests module exists, it will be searched for > unittests and doctests in the same way as models.py We are going to see more and more cases of a models/ sub-directory, not just models.py. So hopefully we can search through those as well eventually. Putting all models in one file leads to very large files. On the other hand, I'm pretty unlikely to put my tests in the model files anyway, since I prefer them in a separate directory (again, to keep the file sizes manageable). So if I'm being selfish, I don't care what you do with models.py. :-) However, if I was to be professional about it, we should maybe consider that case. > I have also added two targets to django-admin: > - 'test' which runs all the unit tests for all installed > applications. > - 'testapp' which runs the unit tests for all applications named on > the command line It seems that Django tradition, as handed down from the ancients across time, is that if a command is given arguments, we run the command against those apps. Otherwise it runs against all applications. So you could just add "test" and it either takes the applications to test as arguments or nothing (=> test all). > With this new framework in place, it is possible to refactor the > django tests; to this end, I have moved 'othertests' into the > 'regressiontests' package, so that it looks like an 'application' > within the regressiontests 'site'. > > When a test targets is executed, django-admin: > 1) creates a test database named 'test_' + settings.DATABASE_NAME > 2) Installs all the applications in INSTALLED_APPS > 3) Constructs a test suite from the test cases found in the requested > apps > 4) Runs the suite > 5) Destroys the test database I wonder how much debate this will cause. :-) This makes the tests order-dependent and possibly hard to run in isolation. Ideally, flushing and restoring the database between each test (i.e. it's a part of setup and tearDown, or maybe just a pre- and post-function on each TestCase-derived class) will keep things "clean". It will be mean the tests will take ages if you are using a disk-based database, though. I don't know if there's a right answer here, or even what the preferred choice would be. > Note that at the moment, I'm just looking at getting a basic testing > framework in place - after that, I will be looking at other problems > such as: > - installation of 'fixtures' (YAML/JSON/Python data for testing > purposes). If you haven't played with it yet, the "python" serialisation target in the stuff Jacob recently committed is really useful here. > - integration with selenium type tests that require a running server This is where you start brushing up against the areas I was thinking about. Although I was looking at it from isolated unit testing perspectives. > I figure that once we have a generic testing framework, we can wrap > just about anything else into it (for example, using purpose-specific > subclasses of unittest.TestCase for different test setup/teardown > sequences). > > I'm almost read
Re: Manual settings and the user's environment
On Thu, 2006-07-06 at 11:20 +0800, Russell Keith-Magee wrote: > On 7/6/06, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote: [...] > One idea that was bounced around in my conversation with the > client was > that TIME_ZONE defaults to empty (or None, preferably) and if, > not set, > does not get inserted into the environment. Personally, I > don't really > care too much about this, since it's such a trivial thing to > set. > > This seems like a reasonable idea to me. It has always slightly bugged > me that the default timezone is America/Chicago, and I have to modify > it on every project. Yes, it is a trivial thing to set, but 'system > clock/no specific timezone' would be a better default than > 'Chicago' (for everyone except Adrian, that is :-). The argument in favour of this is gathering strength: I just tried to work out #2315 and, essentially, we're playing with fire trying to set TZ on Windows systems. At least giving those people the option of setting TIME_ZONE = None so that we don't mess up their existing settings would be nicer than our current slightly misguided attempts. Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Test framework for end-user applications
On Wed, 2006-07-12 at 17:13 +0300, Jyrki Pulliainen wrote: > On 7/12/06, Russell Keith-Magee <[EMAIL PROTECTED]> wrote: [...] > > When a test targets is executed, django-admin: > > 1) creates a test database named 'test_' + settings.DATABASE_NAME > > This is created to the database server being used? Have you thought of > spawning a new sqlite database to the memory and using that. That way > you could be sure that no extra data is left behind (when you > terminate the application, the sqlite goes down too). This would also > enable testing in enviroinments where there is only one postgres db > available for some reason. If one wants to run the tests against in-memory SQLite, then you can use a settings file with the appropriate settings (there is nothing that says tests and "main" need to use the same settings file, in fact multiple settings files are easy and to be encouraged). However, any framework have to be able to test against all possible database configurations because of the need to test things like referential integrity, trigger behaviour, stored procedure actions, portable SQL, etc. Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Test framework for end-user applications
OK, here's a brief summary of what I was thinking about. It dovetails nicely with Simon's thinking, so I don't have to rewrite all my notes. This is a brain dump and can probably be safely ignored if it sounds like rubbish, for reasons I mention at the end. On Thu, 2006-07-13 at 01:41 -0700, Simon Willison wrote: > > On 12 Jul 2006, at 02:28, Malcolm Tredinnick wrote: > > > I approached things from a different direction, though, more > > concentrating on how to unit test templates and views (as in not > > requiring a full end-to-end application setup in order to test > > them). So > > it's actually a orthogonal to what you are proposing here and fits in > > nicely. > > I've been thinking about unit testing templates and views on-and-off > for the past few months. Here's the bulk of my thinking. > > The best way of running tests against a Django application, to my > mind, is to run a "fake" web server (i.e. one that doesn't actually > bind to a port) around the application. This fake server acts as a > test harness. Tests can then create HttpRequest objects (probably a > subclass called something like TestHttpRequest), pass them to the > Django application, get back an HttpResponse object and run > assertions against that. > > There's just one problem with this approach: while running assertions > against an HttpResponse object is adequate, the ideal situation would > be to run assertions against the template context as well. If you > just run assertions against the HttpResponse object you end up having > to process the HTML in some way to work out if the test has passed. Agreed. I was working out how to divorce each of the three components (M/V/T) from each other so that I could unit test each in isolation. Part of the logic here is to provide a way for template designers to test invariants of the template without having to worry about views. Similarly, view refactoring should not require delving deep inside the templates to work out what happens: that makes the test results too fragile in the face of any template changes. If it takes too long to maintain the tests, people don't do it, because it's just a chore, rather than pure benefit. Views first: we can make a TestHttpRequest style object and some utility functions to make simulating a request simple. We pass this plus canned parameters to the view function we are testing. We can override the template class (create a TestTemplate class) that does not actually render to a string (although it might need to behave string-like). This way we can test this object to verify that it was constructed from the right source and given the right context. We can also validate that the right properties were set on the HttpResponse object that is returned. Testing whether or not the template would have generated the right result is not part of "view" testing, that is "template" testing. In view testing we are verifying that the right input returns the right result. Initially, I think it will be hard to have the views entirely divorced from the model framework. To do that (separate them for testing purposes) would require making it very easy to creat mock objects. This isn't too painful, but capturing all the necessary data via all the function calls a view can take can be tricky. I have a sketch of how to capture this via the profiling hooks in Python (as a dev server runtime option) for seeding tests, but it's not implemented at the moment, so there are no doubt hidden problems (I don't think we can use signal dispatches here as Simon wants to for the view/template interaction, because we would need to add so many of them). So figure that we probably, unfortunately, need a seeded database and model setup for this to work initially. It's still useful, though, since now when we are working on views, we can check that we are still passing the necessary information into the template layer and back to the client and we can do this in a way that is independent of the particular way the template has been written. For template testing, I was thinking about passing in an object that supported the necessary attribute accesses (the beauty of everything being accessed as "foo.bar.baz" is that faking it is reasonably straightforward). The test takes the template source filename (or source string), the canned input data and we can run assertions against the result. The hard bit, that I haven't worked out a solution for -- and it may not exist nicely -- is how to handle custom template tags. We could test them as opaque, mock objects that return a canned response for testing purposes, but that only provides a way to test the templates they live in. How do you test the tags themselves? Maybe template tags can be test
Do we need an escaped version of unordered_list?
I'm doing a run through the admin interface, looking for non-escaping
problems.
One problem surfaced more or less immediately, though: the
"unordered_list" filter. Wrapping that in an escaped block or passing
the results through the "escape" filter obviously doesn't work. We
*want* the elements to have meaning. However, {{ obj_bag|
unordered_list }} is error-prone if any of the model __str__ methods
returns strings containing characters that should be escaped.
Is there any alternative to creating an escaped_unordered_list tag? (Any
better name for this tag? It's too long)
[Note: useful responses to this question do not include "auto-escaping
would be nice". That is not the issue here and it gets tiring reading
it.]
Thanks,
Malcolm
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---
Model verbose name restrictions?
A question for the maintainers...
The easiest way to screw up admin page displays at the moment is to put
a '<' in your model's verbose name or plural verbose name.
The admin interface handles this inconsistently: sometimes spitting out
just the raw string, and sometimes presenting the string via
{{ opt.verbose_name|striptags }}. This last one makes me wonder about
the intent here.
Historically, was there a wish/need to put interpretable HTML in the
model verbose names? I am happy to go through and add escaping
everywhere they are used in admin (and other places in contrib/), but if
they are needed for some reason not known to me, that will break
something. The alternative is to document that verbose_name and
verbose_name_plural must be HTML safe, but that doesn't feel right.
Thanks,
Malcolm
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---
Re: basehttp.py -- log_message
On Fri, 2006-07-14 at 02:25 +, Omar El-Domeiri wrote: > Hello, > > I would like to make the basehttp.py log_message() function log more > than it currently does. It seems if I want more things logged, I will > just have to pass them to the args argument. > > The problem is that I don't understand how this method is ever called. > I've grep'd for it by name and only found the functions definition. > Can anyone clue me in? The WSGIRequestHandler class overrides certain methods in BaseHTTPServer.BaseHTTPRequestHandler (from the standard Python library). The log_message() method is called from the base class. Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Model verbose name restrictions?
On Thu, 2006-07-13 at 22:06 -0500, Adrian Holovaty wrote: > On 7/13/06, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote: > > Historically, was there a wish/need to put interpretable HTML in the > > model verbose names? I am happy to go through and add escaping > > everywhere they are used in admin (and other places in contrib/), but if > > they are needed for some reason not known to me, that will break > > something. The alternative is to document that verbose_name and > > verbose_name_plural must be HTML safe, but that doesn't feel right. > > I think we must have had a need for interpretable in HTML in > verbose_names at some point, but I'm cool with changing the admin site > to escape the verbose_names rather than strip tags...Maybe Jacob has > additional insights, as he's the core maintainer of Ellington (and > pre-open-source-Django code) nowadays. OK, thanks. I'll leave this for a day so that Jacob can respond before doing anything damaging. Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Model verbose name restrictions?
On Fri, 2006-07-14 at 13:11 +1000, Malcolm Tredinnick wrote: > On Thu, 2006-07-13 at 22:06 -0500, Adrian Holovaty wrote: > > On 7/13/06, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote: > > > Historically, was there a wish/need to put interpretable HTML in the > > > model verbose names? I am happy to go through and add escaping > > > everywhere they are used in admin (and other places in contrib/), but if > > > they are needed for some reason not known to me, that will break > > > something. The alternative is to document that verbose_name and > > > verbose_name_plural must be HTML safe, but that doesn't feel right. > > > > I think we must have had a need for interpretable in HTML in > > verbose_names at some point, but I'm cool with changing the admin site > > to escape the verbose_names rather than strip tags...Maybe Jacob has > > additional insights, as he's the core maintainer of Ellington (and > > pre-open-source-Django code) nowadays. > > OK, thanks. I'll leave this for a day so that Jacob can respond before > doing anything damaging. Sorry, that sounded demanding and deadline-setting. I meant to say "a day or so...". Until I see something from Jacob, when he has time. Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: TreeField/ TreeWidget
Bjørn Stabell wrote: > I was wondering if anyone has made a field for managing hierarchical > data in Django? > > For example, a TreeField that manages "parent" relationships and gives > the model parent and children object managers: > > class Category(model.Model): > tree = TreeField() > > >>> a = Category() > >>> b = Category() > >>> c = Category() > >>> b.tree.parent = a > >>> c.tree.parent = a > >>> a.tree.children.all() > [b, c] Creating new classes derived from Field() is a little painful in Django, because you need to modify the creation and querying mappings in the database backends. If your field does not map smoothly onto an SQL field, you have even more problems to solve. Fortunately, however, this sort of class can be done easily as a model that you can refer to via a ManyToManyField. The standard tree structure with name and a foreign key to "self" works and then custom manager methods to provide access to the children (if you call the custom manager "children", the API will be exactly as you write above). You can go further and add extra fields to make the encoding a bit more complex in order to speed up look-ups (Vadim Tropashko has done a lot of work on tree representation and manipulation in SQL and his papers -- particularly the recent ones -- can be a little complex in places, but his methods very efficient). That is an internal implementation detail, though, rather than functionality of the model itself. It's not quite possible to do perfect validation on such data structures at the moment, because the default manipulators are not model-aware (and you want to validate that a re-parenting still maintains the tree structure). That will be fixed soon, though. If you want an example, there is some example code for the class (but not the manager) in the Wiki. There is an example with the managers in my blog source code ([1]) as well (and that will be the last time today I post that link; apologies to those on both lists). [1] http://www.pointy-stick.com/software/website-latest.tar.gz Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Do we need an escaped version of unordered_list?
On Fri, 2006-07-14 at 16:45 +0100, Simon Willison wrote: > On 14 Jul 2006, at 03:05, Malcolm Tredinnick wrote: > > > Is there any alternative to creating an escaped_unordered_list tag? > > (Any > > better name for this tag? It's too long) > > Yes. Implementing the auto escape proposal. > > > [Note: useful responses to this question do not include "auto-escaping > > would be nice". That is not the issue here and it gets tiring reading > > it.] > > I would argue that it is the issue here - more specifically, the > current auto_escape proposal is designed to tackle this exact > problem, among others. Simon, Take it as given that I agree with your proposal. I've read it a number of times, thought about it, think it's great. No arguments from me. I considered this a separate issue from auto-escaping because we need something *now* (admin has inadvertent security holes by default at the moment) and I thought there wasn't a real consensus around your proposal, so another response along these lines was not moving us any further from "problem" to "solution". However, in the process of checking the older threads before responding this morning, I realised there was a fair consensus and the only real sticking point was "on by default" or "off by default", which is not a showstopper at the moment. So, I've spent the afternoon implementing your proposal. I've ported about two thirds of the filters and a few of the tags so far, adding tests as I go. I'll stick the patch in a ticket when it's mostly done, which should be later this evening (Sydney time). Just posting this now in case you decided to get keen and start working on this yourself this weekend. I'm not meaning to tread on any toes, but I want to get some of these problems fixed and leaving it for "somebody else" wasn't working. Your response was basically saying stuff I already understood, but thanks for the boot in the butt anyway; it annoyed me enough to get me to work on the bigger problem. Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Auto-escaping patch
I have put an initial version of the auto-escaping patch I mentioned
yesterday into ticket #2359. I'll briefly describe what it does below.
The patch includes changes to the core and a test suite for the
auto-escaping changes (which is about half the patch).
My reason for posting this first pass is that there are a few issues
that have come up that I would like to get some consensus on now, rather
than after I have ported the rest of the core over to use this stuff. If
I need to change some things, this is the easiest time to do it. There
is no documentation patch in this pass because, again, I don't really
feel like having to re-edit practically the whole doc change if we
decide to rename some things. This email acts as a documentation proxy
for the moment.
The whole implementation is very close to Simon Willison's original
proposal [1]. With one exception, I have only modified it where there
were technical requirements to do so. I'm going to assume familiarity
with that throughout. His proposal really is very good and it seems to
meet all the sensible requirements brought up in the three threads
listed at the bottom of that page (the two older threads are probably
more informative than the most recent one, for those wanting to get up
to speed here).
[1] http://code.djangoproject.com/wiki/AutoEscaping
A summary of the points I'd like opinions on is at the end of the email.
What does this add?
---
(1) An "autoescape" template tag that turns automatic escaping on or off
throughout its scope.
(2) A "noescape" filter that marks its result as safe for use without
further escaping (see the description of "safe strings" below).
(3) SafeContext and SafeRequestContext classes that act like Context and
RequestContext, except that they automatically enable auto-escaping in
the templates they are applied to (you can also set context.autoescape
on any Context-derived instance, just as in Simon's proposal).
(4) A "mark_safe()" method to mark strings as not requiring further
escaping.
How does it work?
-
When a variable is evaluated in a context in a template, it is
considered to be either "safe" or not (Simon used the term "escaped",
but that seemed less universally true than "safe"). By default, strings
are not marked as safe.
When automatic escaping is enabled, either because {% autoescape on %}
is in effect, or because a SafeContext class is being used, all strings
that are not marked as safe are escaped at rendering time (here,
"escaped" means "conservative HTML escaping": &, <, >, " and ' are
converted to entities always).
Any string marked as safe (or passed through the "noescape" filter) is
not automatically escaped.
When automatic escaping is disabled in the template, all variable
results are output without further escaping, unless the "escape" filter
is applied to them (this is the same behaviour as currently in Django).
Because some filters are designed to return raw markup, the mark_safe()
function exists so that the returned strings can be designated as safe.
For example, {{ var|markdown }} returns raw HTML and the result is not
subject to any further auto-escaping.
Some filters (e.g. unordered_list) wrap HTML around raw content. If
auto-escaping is enabled, these filters will escape the content before
wrapping it in the HTML tags (the returned result is a safe string in
all cases). So such filters are auto-escaping-aware.
Filters that accept text strings as input and return text strings are
marked (with the "is_safe" attribute) as to whether or not they return a
safe string whenever they are passed a safe string as input. This has
the effect of preserving "safeness" for those filters. Note that this
attribute is a *guarantee* of preserving safeness, so a filter like
"cut" has is_safe = False: {{ var|cut:"&" }} could turn an escaped
string into a monster. If a safe string is passed into a filter that is
not marked as safe and auto-escaping is enabled, the resulting string
will be escaped. If the is_safe attribute is not attached to a function,
it is assumed to be not safe.
Because of the is_safe attribute, it will be possible to change the
automatically generated documentation in the admin interface to annotate
each filter with it's "safeness" guarantee.
The "noescape" filter acts as a way to annotate the result of a filter
chain as safe. So, although "cut" is not a safe filter, we know that
cut:"x" is safe (it can't harm our HTML-escaped strings) and thus
{{ var|cut:"x"|noescape }} will prevent further escaping of the result,
even in auto-escaping-enabled situations. The "noescape" filter does
nothing except mark the result as safe -- the string output is identical
to the input.
Is it backwards compatible?
---
Mostly.
Auto-escaping is not turned on by default (Adrian made a statement in
[2] and I'm going with that preference at the moment). If you pass a
normal Context or Context-derived class into a template and do not use
the aut
Re: new field idea
On Sun, 2006-07-16 at 23:31 +0200, [EMAIL PROTECTED] wrote: > Hi django-developers, > > I had the need for a field type similar to USStateField: an > OlympicNationField which takes 3 uppercase letters to the official olympic > nation abbrevation. > > Maybe this is generic enough to go into django.db.models.fields ? > I attach my solution. A general comment: There are a number of open tickets for adding fields of various levels of usefulness and generality. It isn't too hard to make a small change in Django's model construction so that creating new field derivatives that map easily onto existing database fields becomes easy and maybe we should do that. I have some bullet points in my Django notes directory about how to do this. Basically, anything that is a restriction of a CharField is pretty simple, making it appear as an instance of whatever the developer wants (rather than a string). I have been avoiding finishing this because I haven't worked out a nice way to do non-derivative fields easily. This crowd is of the "give them an inch and they'll ask where the rest of the mile is" variety sometimes, so I wanted to have something that was mostly feature-complete. But, screw it... it can be done in stages. I'll write up what I have into a patch this week and submit it. That should make things like this easy to add whenever somebody wants them. Without passing judgement on this particular addition, it's a really slippery slope to start adding lots of these things to core. Although it's usually true >1 people can use them, it adds more and more to the maintenance burden. If we can make it easy to create external field additions, then people can put them in the Wiki or on their websites or wherever and those who want them can use them and maintain them. Best wishes, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Auto-escaping patch
On Sun, 2006-07-16 at 21:30 +0200, Michael Radziej wrote:
> Hi,
>
> I really appreciate your work, it goes all along my wishes--thanks a
> *lot*, Malcolm!
>
> I'll try to find some time in the next few days to test how my
> existing stuff would look using autoescape.
>
> I have looked in your patch only cursory, so my comments refer to the
> general approach and not the actual code bits. I will try to look
> into the details later.
>
> > (3) SafeContext and SafeRequestContext classes that act like
> > Context and
> > RequestContext, except that they automatically enable auto-escaping in
> > the templates they are applied to (you can also set context.autoescape
> > on any Context-derived instance, just as in Simon's proposal).
>
> My feeling about this is -1; I think there should be only one place
> to switch autoescape on or off.
That's the reasonable counter-argument to the current approach. I was
wondering how big an issue that would be. (FWIW, I probably agree with
you: having this done in one place if not on by default is more
consistent.)
[...]
> > (1) Any violent (or even just passionate) objections to using terms
> > like
> > "safe" and mark_safe()?
> > - Should we use Simon's original proposal of escaped and
> > mark_escaped()? I feel "safe" is a bit more consistent with
> > the
> > behaviour (an opposite-but-similar term to Perl's "tainted").
>
> I'm more for 'escaped' and 'raw', but not really violently. This is a
> minor issue, and I wouldn't like to get the work delayed by it.
> Also ... I volunteer to rewrite the docs if these terms change. But
> only once ;-)
"Escaped" strikes me as bogus because it's not really the case: we are
just saying this output can be dumped in without further escaping. I
thought about "raw" on Saturday and wondered if it would lead to
confusion: is a raw string "untreated" or "should not be treated
further" (we intend the latter). Whatever we do, we want to designation
to apply to strings we have marked as not requiring more escaping:
default strings should be "unsafe" or "not raw" by default for
auto-escaping purposes.
I don't really have a strong opinion here, so all the suggestions so far
work. But I'm also not entirely illogical, so it would be nice if I
could use the word in a sentence without sounding like I'm redefining
the normal English meaning.
> > (2) Is the new behaviour of "escape" reasonable (i.e. it does
> > nothing on
> > safe strings)?
>
> I think so. Is there a case for escaping two times? I don't see any,
> and one could still easily craft a custom filter that does escape two
> times.
Damn. Your phrasing tipped me off to a case we need this more: RSS feeds
and Atom content elements with type="html". :-(
We might need a "mark as unsafe" filter for these cases (so that {{ var|
escape|unsafe|escape }}) works (or just make "escape" not mark the
string as safe, but I suspect that will have unintended annoying
side-effects).
This comment highlights a hole that needs to be fixed. Excellent.
Malcolm
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---
Re: Auto-escaping patch
On Mon, 2006-07-17 at 03:30 -0700, SmileyChris wrote: > Great job on the patch, Malcom! > I posted this in the ticket, then felt guilty because you told me not > to. So I'll post here for discusion. > > A couple of points: > If a markup filter fails due to an import error, I don't think it > should be marked as safe. Why not? The returned result is the empty string in that case and there's certainly no danger of that being presented in the raw. "Safe" implies nothing beyond "does not require further HTML escaping" (and that is the quite reasonable argument for finding another name for it). If a filter is returning a safe (or whatever we end up calling it) string, it should *always* return a safe string. Otherwise the end users will be uncertain about whether the returned result is safe or not and will always have to wrap it in an "escape" filter, which they will forget to do. > >From a skim read of the patch, I'm missing the purpose of having an > .is_safe property on filters - can't you just check the outputted > string to see if it's SafeData? No. Take a SafeString instance, split it do something to it, run join on the result. What you have is now a string (not a SafeString). So rather than overloading every single string method on SafeString and every single Unicode method on SafeUnicode -- which will add quite a bit of function call overhead -- and rather than requiring filter writers to always have to do "if SafeData" tests around string operations (which they'll screw up), we mark the filter appropriately. If a filter is marked "is_safe" and a safe string is passed in, then no matter how much it gets put through the meat grinder internally, we can happily convert it back to a SafeString at the end. Ditto for SafeUnicode. This makes the code much shorter and adds a large measure of certainty to the process. For filters that are not universally safe (e.g. the pluralize case I mentioned), it is still possible in some cases to internally check internally if the munging would keep a safe string "safe" and then explicitly call mark_safe() on it. That is what I intend to do to pluralize(). Thanks for the feedback anyway. Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
