Inline formsets

2009-08-15 Thread mrts

At HTML level, a form is a set of fields that gets submitted when
a the form submit button is pressed.

However, this is not the case with model forms and inline formsets
(e.g. an admin page with inlines) -- inline formsets are
disparate from the model form.

This creates at least two problems:
1) it's impossible to display inline formsets in between ordinary
form fields (although they may logically belong to a particular
fieldset and not to the end of the form),
2) it's impossible to create nested (recursive) inlines.

There's more to it though. Inline formsets usually represent
composition, i.e. the inline objects are inseparable from the
main entity. The relation is stronger than a ForeignKey (from the
main entity to another entity), yet the latter is displayed as a
field of the form and can be easily manipulated -- but inlines
are not.

What I propose may at first sound odd: adding a *formset field*
to forms (keen readers notice the Composite pattern in work
here):

FormsetField(form=CustomFormClass, << other options inspired by
admin inlines and formset factories >>)

It would overcome both of the problems mentioned above and, being
object-based, would be generally more flexible than the factories
currently in use. Nested formsets can be trivially created:

class Foo(forms.Form):
   foo = forms.FormsetField(...)

class Bar(forms.Form):
   foos = forms.FormsetField(form=Foo, ...) # <-- nested formset

Rendering
-

In an ideal world, the would-be formset field would support all
the niceness present or coming to Django admin: add/delete
instances on the fly, collapsing and reordering. That means
coupling JS to forms layer, which seems to be frowned upon.
Leaving either abstractions/hooks in place to support that
functionality and/or document how to bind the JS would
probably suffice though.

Implications for contrib.admin
--

FormsetField would be the recommended way for displaying and
manipulating inlines.

In yet another ideal world, assembling your own contrib.admin
would be trivial. All the functionality might be nicely packaged
to cohesive components (generic views, Alex's django-filter etc)
-- contrib.admin would be just a thin wrapper that binds all
reusable bits into a harmonious whole.

A rich FormsetField would contribute to that, making working with
formsets less onerous.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Model.objects.raw() (#11863)

2009-10-02 Thread mrts

Wishful thinking follows.

It would be awesome if one could mix ordinary QuerySet methods
with raw() (or, rather, raw_extra(), see below for that).

Assuming the following models:

class Foo(models.Model):
name = models.CharField(max_length=255)

class FooDates(models.Model):
foo = models.ForeignKey(Foo)
start = models.DateField()
end = models.DateField()

I for one would be definitely struck with awe if
I could write something in the lines of:

RAW_ORDER_BY = """
ORDER BY (SELECT MIN("start")
FROM "myapp_foodates"
WHERE "myapp_foodates"."start" >= %s
AND "myapp_foo.id" = "myapp_foodates"."foo_id")
"""

q = Foo.objects.filter(
foodate__end__gte=datetime.date.now())\
.raw(RAW_ORDER_BY, params=(datetime.date.now(),))\
.distinct()

and q.query.as_sql() would look as follows:

SELECT DISTINCT ...
  FROM "myapp_foo"
  INNER JOIN "myapp_foodates"
  ON ("myapp_foo"."id" = "myapp_foodates"."foo_id")
  WHERE "myapp_foodates"."end" >= "2009-10-02"
  ORDER BY (
   SELECT MIN("start")
 FROM "myapp_foodates"
 WHERE "myapp_foodates"."start" >= "2009-10-02"
   AND "myapp_foo"."id" = "myapp_foodates"."foo_id");

However, I assume that achieving this would be extremely
hard with plain raw().

What probably would work, however, is an extra()-like
raw_extra() that would accept raw parametrized strings
for each of `select`, `where` and `order_by` and plainly
replace the corresponding part in the query builder.

I.e. the example above would read:

q = Foo.objects.filter(
foodate__end__gte=datetime.date.now())\
.raw_extra(order_by=RAW_ORDER_BY,
params=(datetime.date.now(),))\
.distinct()

Best,
Mart Sõmermaa
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Model.objects.raw() (#11863)

2009-10-02 Thread mrts

On Oct 2, 5:42 pm, "Sean O'Connor"  wrote:
> To be honest this seems like something which would be a lot of work with
> relatively little gain.  For this to work raw() would need to change from a
> relatively simple bit of code which doesn't need to touch all of the complex
> query code in the ORM to a complex bit of code which needs to deeply modify
> the behavior of the query code.

Perhaps I exposed my thinking process too much. Surely .raw() should
remain
as-is and raw_extra() *added*, being orthogonal to raw() both usage-
and
implementation-wise. And it should certainly be labeled as "only use
this
if you know what you're doing."

As for the usefulness, I believe I'm not the only one who builds
queries
dynamically a lot (e.g. according to GET/POST parameters:
if 'foo' in request.GET: q = q.filter(...)) before evaluating it. For
that
use case, raw() does not help -- I'd have to glue together SQL strings
to
build the query manually, like the ORM does, which is obviously
fragile
(think about tracking joins) and defies the purpose of an ORM. Mixing
.filter() with .raw_extra() would be a cleaner way to execute advanced
SQL.

As for the implementation, I took a superficial glance at
django/db/models/sql/query.py before posting and e.g. the ORDER BY
override
looked quite doable in as_sql(). Let me provide an oversimplified,
naive
example:

--- django/db/models/sql/query.py   (revision 11594)
+++ django/db/models/sql/query.py   (working copy)
@@ -446,7 +446,9 @@
 result.append('HAVING %s' % having)
 params.extend(h_params)

-if ordering:
+if self.raw_order_by:
+result.append(self.raw_order_by)
+elif ordering:
 result.append('ORDER BY %s' % ', '.join(ordering))

But let me stop right here. If it doesn't look sensible to RKM or you,
let it be, lengthy arguments in similar situations have been futile
(and e.g. in case of
http://code.djangoproject.com/ticket/10697#comment:4 and the
corresponding django-dev thread, brought me bad karma which
I obviously don't have any use of :) ).

---

Finally let me express my thanks that you've taken the trouble of
implementing raw(). The prospect of having it in Django is excellent
news.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Model.objects.raw() (#11863)

2009-10-05 Thread mrts

On Oct 2, 7:35 pm, mrts  wrote:
> But let me stop right here.

Contradicting myself, I've pursued this further and
implemented raw_override() for order_by in [1] as a proof of
concept. See [2] for what's different and tests [3] for
usage.

I'd say that for order_by, the implementation is simple and
straightforward and not that much more fragile than extra()
(see tests for the second model in [3] for a complex-ish
example).

As for the usefulness, complex ordering can probably be
solved with existing extra() by SELECTing the complex order
criteria, aliasing them and using that alias in order_by.
But perhaps someone likes or needs the idea and wants to
take it further for other parts of the query.

Once again, over and out on this matter,
Mart Sõmermaa

[1] branch: http://github.com/mrts/django/tree/raw_sql_override
[2] diff: 
http://github.com/mrts/django/commit/dd127debfed070bb8152f10cb5770b216baee3de
[3] tests: 
http://github.com/mrts/django/blob/raw_sql_override/tests/modeltests/raw_override/models.py
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Branchy Django developement on GitHub

2009-10-05 Thread mrts

Many core developers and the GSOC folks use GitHub for
Django development, so I hope discussing the best
practices is appropriate for django-dev.

Let me outline the desired workflow, please amend and
correct as you see fit as there are probably errors or
omissions. I expect various useful patterns and workarounds
for problems have emerged which could be shared with the
larger community. I'll create a proper guide in the wiki
once feedback has been collected.

Assumptions
---

You want to work on several fixes and features (tasks) and
 * keep the changes for a particular task selt-contained in
   a branch,
 * frequently update the codebase with upstream changes,
 * have a has-it-all branch for testing out how the
   different task branches interact.

You have created a fork of django/django on GitHub,
`git clone`d the work to local machine, added the upstream
repository with `git remote add upstream`.

Working on a task
-

Goal: keep the changes self-contained, create and update
patches suitable for attaching to tickets in Django trac.

 # branch from master
 $ git checkout -b ticket1234 master
 $ ... change files, commit often ...
 $ ... run tests ...
 # create the remote branch and push changes into it
 $ git push origin ticket1234
 # create a patch, attach it to a relevant ticket in trac
 $ git diff master > ticket1234.patch

After upstream has changed
--

Goal: bring in the changes in upstream to a task.

 # pull in upstream changes
 $ git pull upstream master
 $ git checkout ticket1234
 # merge, fix any conflicts
 $ git merge master
 # push changes to GitHub, remote branch assumed to exist
 $ git push
 # update the patch
 $ git diff master > ticket1234.patch

Testing everything in the has-it-all branch
---

Goal: a place to try out the code from all tasks together.
'master' can not be used for this as it is used for tracking
upstream. This is a temporary branch that can be removed
after testing is done.

 $ git checkout -b has-it-all master
 $ git merge ticket1234
 $ git merge ticket1235
 $ ... run tests ...
 # delete the branch
 $ git checkout master
 $ git branch -d has-it-all



This is just a (untested) skeleton. All help in improving it,
adding caveats and tricks most welcome.

Best,
Mart Sõmermaa
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Branchy Django developement on GitHub

2009-10-05 Thread mrts

There's http://code.djangoproject.com/wiki/CollaborateOnGithub
that I wrote some time ago, but it doesn't give guidelines for
branchy development.

I will update that page with these instructions, but it would be
excellent if others who have collaborated more intensively via
GitHub could amend the text if needed.

Best,
MS

On Oct 5, 6:46 pm, Simon Willison  wrote:
> Thanks for putting this together - that's a really useful summary of
> commands. Any chance you could save it on the wiki somewhere?
>
> Cheers,
>
> Simon
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Branchy Django developement on GitHub

2009-10-06 Thread mrts

On Oct 5, 7:08 pm, mrts  wrote:
> There'shttp://code.djangoproject.com/wiki/CollaborateOnGithub
> I will update that page with these instructions,

Done. Any comments and amendments should probably go to the wiki page
now.

Best,
MS
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



The this-needs-to-be-in-django angst

2009-10-19 Thread mrts

Let me suggest that there are many who have at
times felt frustrated how contributions or
suggestions are managed in Django. Some of them
seem to have walked away or just don't participate
in discussions any longer. The same applies to
several other large open source projects, the
Linux kernel and it's mailing list being a prime
example.

However, the frustration was mostly justified
*before* Django 1.0 came out and the the
semi-official DVCS mirrors took off. It was hard
to keep patches updated - the internal APIs were
in flux and Subversion was and is not flexible
enough (even with svnmerge.py) to make branchy
development painless. The frustration stemmed from
too much change and no tools for managing it.

In my opinion, neither of the problems apply as of
now. Maintaining your own branches on GitHub or
BitBucket off the corresponding Django SVN mirrors
is easy and effortless, so it's time to put the
grudges behind and happily fork and branch Django
on the DVCS sites whenever there's a need for
something missing from or broken in the official
trunk - and, what's perhaps even more important,
give back by shepherding the corresponding tickets
in Django trac (keeping them up to date, improving
them according to other's suggestions etc).

"Relax," as CouchDB puts it :)

Best,
Mart Sõmermaa
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: The this-needs-to-be-in-django angst

2009-10-19 Thread mrts

(Inspired by Yuri Baburov's criticism and RKM's response.)
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: The this-needs-to-be-in-django angst

2009-10-20 Thread mrts

Jacob, I'm afraid you totally misunderstood me.
My message was intended to encourage people to
scratch their own itches more now that it's so
much easier -- and, of course, give back --
instead of grumbling on the mailing list.

I fail to see how can "so it's time to put the
grudges behind and happily fork and branch Django
on the DVCS sites whenever there's a need for
something missing from or broken in the official
trunk - and, what's perhaps even more important,
give back by shepherding the corresponding tickets
in Django trac" considered to be hostile.

Baffled,
MS

P.S. I've worked on [1] exactly that way, is that
considered to be hostile?! If so, please elaborate.

[1] http://code.djangoproject.com/ticket/7028

On Oct 20, 5:23 pm, Jacob Kaplan-Moss  wrote:
> Hi Yuri, Mart --
>
> I feel that I need to make it clear that I'm not ignoring you, or this
> conversation. However, the tone is so hostile and unprofessional that
> it'd be a waste of my time to try to engage, so I'm simply going to
> stay out.
>
> Jacob
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: The this-needs-to-be-in-django angst

2009-10-21 Thread mrts

On Oct 20, 7:19 pm, Jacob Kaplan-Moss  wrote:
> What's frustrating, though, is hearing that you have a
> problem with our workflow without any concrete suggestions
> to improve it or offers of assistance. There's a general
> theme underlying most of this "angst" (as you call it):
> the tone implies that you're somehow entitled to our (core
> developers) time and energy, when we're just as stressed,
> harried, and busy as you.

That's perfectly understandable and well respected. When I
referred to "frustration on how contributions or suggestions
are managed in Django" and later on to the Linux kernel and
its mailing list, I intended to suggest that to an extent,
frustration in the community is inevitable - and only then
demonstrate the Noble Path Out Of It, for a dramatic effect
of sorts :).

As for assistance - let me humbly point out that I've
written
http://code.djangoproject.com/wiki/CollaborateOnGithub
precisely for helping people to overcome their git-fear and
get a feel how easy maintaining a patch is if done properly.

It's also inevitable that the core devs are stressed and
busy, torn between real-life needs and the immense energy
required to steer a community full of both excellent and -
luckily to a much lesser extent - misguided ideas. That
stress does sometimes (albeit rarely) result in loss of
empathy and gentleness. This is human, understandable and
happens in all open source projects. No big deal, but I do
share Vinay's sentiment regarding the door metaphor.
http://www.ubuntu.com/community/conduct is a good read for
any community manager.

I generally agree that the current trac + mailing list
process is mostly fine (especially if coupled with fleshing
out ideas in the wiki that seems to be becoming standard
practice now) and believe I'm in no position to suggest
improvements - but being invited to it, let me share a few
nevertheless (directly borrowed from Debian/Ubuntu, Linux
kernel and Unladen Swallow communities).

Django Bug Days
---

At regular intervals, say twice a month on Saturdays,
set aside 2-3 hours for IRC-based bug hunting sprints.

Devs list tickets that they want people to work on and
people are free to suggest their own.  Work is done over
longer periods, but the bug days provide an occasion to
get feedback from the devs.

The patches will perhaps not be integrated into SVN but to a
branch on a DVCS (see the next suggestion).

Added benefit: if nobody shows up, they have no right to
grumble on the mailing list :).

A DVCS mm-tree
--

Quoting
http://en.wikipedia.org/wiki/Andrew_Morton_(computer_programmer)

"He currently maintains a patchset known as the mm-tree,
which contains not yet sufficiently tested patches that
might later be accepted into the official Linux tree."

I.e. there would be a DVCS branch (avoiding the f-word now)
maintained either by a core dev or a designated community
member that would accept patches more liberally -- but still
only patches that are up to Django standards, i.e. tests and
docs remain mandatory.

This will probably be most effective in context of the bug
days, otherwise the management overhead may be too big.

Also, that branch would have a few buildbots running tests
on it.

Code review
---

Code reviews are super useful, because, in the end of the
day, we communicate in code.

I have no idea if integrating a code review tool with trac
is feasible, so if this looks attractive, the only realistic
way to adopt it would be a social-coding-site-centric
process - patches remain in trac tickets, but it's highly
recommended to maintain corresponding branches in GitHub or
BitBucket, so that community members can write code reviews
there.

Best,
Mart Sõmermaa

P.S. I won't re-animate the ticket classification argument
that created a lot of unintended controversy last year, but
I personally think that divide and conquer is the only way
to manage the 1700+ open lot.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: The this-needs-to-be-in-django angst

2009-10-22 Thread mrts

On Oct 22, 3:36 am, Russell Keith-Magee 
wrote:
> On the other hand, saying "I am going to" is extraordinarily helpful.
> Saying "I already have" is even better. You don't need the core's
> blessing to do anything.

Yes, I wholeheartedly agree and that was the point of my initial
message.

> Allow me to assure you that if you develop
> and maintain a resource that is useful, people - including the core -
> will use it. The hard part has never been coming up with ideas. The
> hard part is delivering.

As I said, I don't think I'm actually in a position to suggest
improvements to the workflow (being Nobody in that little story :)),
but I'd be most pleased to do the following if core thinks
this advances Django's cause:
* create a "mm-tree" on GitHub and share commit bits to a few
core-designated people,
* regularly review and apply pull requests to that branch,
* organize regular microsprints to get more tickets into the tree.

The first step would be to write a proper proposal that describes the
workflow in greater detail -- should I do it?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: The this-needs-to-be-in-django angst

2009-10-22 Thread mrts

> Another benefit of a merge-queue branch is testing and verifying that
> multiple patches play well together before actually hitting trunk.
> For multiple big branches this is even more important.

Currently blocking on
http://support.github.com/discussions/feature-requests/560-two-forks-of-a-single-repo

If this will not be resolved (quite likely) then there will
be less convenience and GitHub goodness (e.g. "send pull request")
-- as of now, manually adding django-mergequeue as a remote to an
existing fork looks like the only option.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Bug in django\template\__init__.py ??

2009-10-29 Thread mrts

On Oct 29, 4:50 pm, Russell Keith-Magee 
wrote:
> Django's dependence on DJANGO_SETTINGS_MODULE is an oft-lamented
> problem. Suggestions on how to address this constraint are most
> welcome.

With an explicit main (i.e. a single explicit entry point to code).
Would also fix the "where can I initialize stuff" problem for good.

> However, it isn't a simple problem to fix.

With the current design, no. I personally use the following
idiom for an entry-point template with Werkzeug/GAE.

Think of `fw` as `django` in the following.

---

from fw.conf import settings

from . import settings as proj_settings

settings.register(proj_settings)

# all kinds of interesting explicit overrides
# and optimizations are possible now,
# just a random example that would
# be messy with current code
if settings.MIDDLEWARE_CLASSES:
from fw.core.handlers import MiddlewareHandler as Handler
else:
from fw.core.handlers import BaseHandler as Handler

from fw.webapp import run_wsgi_app

from .urls import url_map
handler = Handler(url_map)

def main(): # <-- the entry point
run_wsgi_app(handler) # or whatever

---

Disclaimer: I think that the Django way of doing things is
mostly fine and the effort to change the status quo is not
justified.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Inline Formsets Again

2009-10-29 Thread mrts

On Oct 28, 9:43 am, John Debs  wrote:
> I've written a creaky hack to work around the issue in my own projects
> but after seeing this thread I'm going to take a shot at implementing
> FormSetField (or something like it) myself. If anyone has any code -
> or more arguments for or against that haven't been covered - I'd love
> to hear them before diving in.

I have to admit that I haven't got time for working on this right now,
but I'd be most willing to collaborate - e.g. review code. Perhaps we
should continue off-list - as this hasn't been officially accepted -
until
we've fleshed out a proper proposal and some code.

Best,
Mart Sõmermaa
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Regularly-scheduled Django sprints, first December 11 - 13

2009-11-10 Thread mrts

Great news! Have you discussed the workflow yet?
I.e. will a DVCS be involved, and if yes, will there be
a central repo for coordinating the effort?

A special repo on Github would otherwise be perfect,
but as mentioned before, we have a problem with re-forking:
http://support.github.com/discussions/feature-requests/560-two-forks-of-a-single-repo

Whatever workflow and tools have been or will be
picked, may I suggest that clear instructions be written
for participants -- e.g. something in the lines of [1]?

[1] http://code.djangoproject.com/wiki/CollaborateOnGithub
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Regularly-scheduled Django sprints, first December 11 - 13

2009-11-11 Thread mrts

On Nov 11, 1:57 pm, Russell Keith-Magee 
wrote:
> Also - I know you're very enthused about Git and GitHub, but Django
> uses SVN as the canonical store for trunk and branches, and this isn't
> likely to change in the near future.

Thank you, but no, I'm not enthused about git and GitHub per
se :) -- I'm enthused that it's possible to maintain patches
and collaborate remotely with less agony by utilizing them.

Please let's be friendly and relaxed -- I was just asking
how do people plan to manage things. SVN is fine as well,
but it puts the burden of integrating work on core. It's
not the tool, but the goal that matters.

As far as I understand, that's one of the issues we are trying
to solve -- integrating contributors' work faster than core
is able to. Good for core and good for the contributors.

> As I have said to you in the past - if you want to make your
> contribution to Django to be a DVCS  repository that acts as a staging
> area for trunk-ready patches, that would be a very helpful
> contribution. This is doubly true during a sprint - sprints generate a
> lot of activity, so having good lieutenants to triage contributions is
> extremely useful.

Done, http://github.com/django-mq/django-mq .

I'm willing to regularly review pull requests and add
collaborators who are willing to do the same.

But there's the problem that I'm not Andy Morton --
considering my karma, I'm in no position in regard
to my "social capital" to lieutenant this.

> The important part is *not* the documentation of this process. The
> important part is *doing the work*.
>
> Key phrase: Do it. Not write an essay about how you would do it, and
> then enter into a debate about whether the workflow that you have
> proposed is optimal. Do the work, then tell a core dev that you have
> done the work. Fix the patch if required. Rinse. Repeat.

Wasn't the fact that it isn't and can't be that easy the
main problem we discussed in the angst-thread? That core is
too overburdened and small in numbers to oversee >1800 open
tickets?

Doing the work is usually easy, integration and avoiding
staleness is not.

As for "don't write essays" -- some traction is needed
behind the initiative and good coordination with core so
that the contributions start eventually flowing from the
repo back to the trunk. Few or no people will use the repo
unless there's a general consensus that it's a good thing.
That consensus can only come from clear understanding of
it's purpose and workflow (the "essays") -- only then can
people start contributing. Otherwise they will feel that
it's a waste of their time on an obscure fringe thing.

All in all, I think that we are not really on different positions
regarding this, it's just the karma issue that spoils the soup :)
So be it, I'll just try to keep my mouth shut.

Best,
Mart Sõmermaa
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Sprint issue tracking / triaging

2009-12-08 Thread mrts
On Dec 8, 5:32 am, Russell Keith-Magee  wrote:
> On Tue, Dec 8, 2009 at 11:11 AM, Tobias McNulty  
> wrote:
> > On Mon, Dec 7, 2009 at 6:31 PM, Russell Keith-Magee
> >  wrote:
> >> I was thinking more of having one person at the sprint to take the
> >> role of integrator - that is, the patches still go up on trac, but one
> >> trusted person at the sprint takes the role of lieutenant for the
> >> sprint, and selects patches that they consider trunk-ready. The end
> >> deliverable is a branch that a core developer could theoretically just
> >> git merge and push to SVN. In practice, there will probably be further
> >> review by the core, but the idea is to provide a single rich vein of
> >> high quality patches.
>
> > Yes, personally I don't see the need to be overly organized about
> > branching for each feature developed at the sprint.  There's no need
> > to dictate what version control system, if any, participants use to
> > develop code for Django.  Some prefer git, some hg, and some small
> > tickets might not need a branch at all.
>
> > The role of integrator does sound like a good one, though, both in
> > terms of making sure things work (and work together) and also for
> > keeping track of what actually gets done at the sprint.  Simply
> > merging that branch directly into trunk sounds a little dicey,
> > however, as commit granularity in terms of features (even small bug
> > fixes) is usually a Good Thing.
>
> I should clarify - I wasn't talking about an SVN style merge where N
> branch commits become 1 trunk commit. I meant a git/hg style merge,
> where each commit on the branch becomes a commit on the trunk.

I have thought about the process a bit and even wrote some
helper code.

Unfortunately I fell ill and haven't fully recovered (and am
therefore horribly off-schedule with my work), so I haven't
had the chance to continue with the effort.
*Notoriously* bad timing, considering we had a similar
discussion before and that the sprint is coming up soon.

Unfortunately only the less important general guidelines
are up as of now at
http://wiki.github.com/django-mq/django-mq

What I had in mind was to write proper guidelines so
that people can hop on easily and to keep things
properly organized. I also planned to run an example
sprint by myself to demonstrate how things actually
work.

Alas, I can not do that presently.
I'll get to it eventually though.

Best,
MS

--

You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.




Re: Sprint issue tracking / triaging

2009-12-08 Thread mrts
> On Tue, Dec 8, 2009 at 12:35 PM, Jeremy Dunck  wrote:
> > Sorry to hear that.  The document you linked is a start, but I'm a bit
> > concerned by this goal:
> >  "to keep the main integration branch as stable as the official trunk
> > so that it can be used in actual deployments"
>
> > My concern here would be that people become dependent on your
> > selections for prod, and some of these changes never get merged to
> > mainline, so that there's functionally a fork in terms of support and
> > community.

I would really, really want to avoid that. The merge branch would
let people test things out properly, perhaps even in production, and
run a buildbot off it. But the goal of the effort as a whole is just
getting
high-quality reviewed, tested and integrated patches back to Django
trac.

The wording is just a sketch, seems I have to re-iterate that no
forking in the "bad" sense of the word will occur. The work is meant
for merging back and the main value lies in ticket branches, not in
the merge branch.

What I envision is the following:
* there's a specific goal and deadline (i.e. fix 10 bugs in 2 weeks),
* sprinters claim tickets in trac,
* sprinters create ticket branches in their corresponding forks of
the Django SVN mirror and work off their branches,
* when the work is ready, it will be reviewed and merged into the
mq by sprint lieutenants,
* buildbots run tests off the mq,
* if all is well, the sprinter uploads the final patch to the
corresponding ticket,
* when the deadline is reached, the tickets that were properly
fixed during the spring are presented to core and will be eventually
merged to SVN trunk simply via the patches in trac.

As far as I can understand, core has been opposed to taking
more work upon themselves, so the last step should be
effortless, i.e. core can assume that the patch is
well reviewed, tested and non-controversial.

This workflow is suitable for janitor-type work, not for
epic or controversial changes.

On Dec 8, 7:40 pm, Alex Gaynor  wrote:
> Because of this work to review tickets, provide feedback, and get them
> into an RFC state is far more important IMO.

Exactly. However, there needs to be a coordinated effort and a
deadline to rally the masses :). The Django community is vast
and certainly full of excellent people who are more than able
to fix a couple of the 1700 open tickets. However, people who
provide patches are few and the patches are mostly
proof-of-concept quality.

I myself am used to providing somewhat proof-of-concept
quality patches.

Analyzing the reasons, I noticed that I semi-consciously
am thinking that the trunk changes too much to make
polishing feasible and that a core reviewer "knows better"
and will rewrite it anyway. This attitude doesn't work
within the merge queue workflow, where most of the work
is quickly integrated and thus the contributor needs to pay
more attention to polish.

So, there's a psychological benefit of the mq besides
the real one.

Over and out for today,
best,
MS

--

You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.




Re: Finalizing model-validation: ComplexValidator

2010-01-04 Thread mrts
Malcolm had some good comments when all of this was discussed
previously:

http://groups.google.com/group/django-developers/browse_thread/thread/436307e426844ae0/f2fa0647b97569ad

He also seemed to have further ideas and code, perhaps he can be
pinged for sharing them.

Best,
MS

--

You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.




Re: What The Enterprise wants from Django

2010-01-19 Thread mrts
On Jan 20, 5:59 am, David Cramer  wrote:
> The first three have been huges ones with us. We're just now running
> into the settings issue, but would love to see what people can come up
> with for solutions (we dont have a good one).

The override_settings() idea that Eric Florenzano describes at
http://djangodose.com/articles/2009/09/handling-development-staging-and-production-enviro/
with some improvements and fixes as described in the following
http://djangodose.com/articles/2009/09/handling-development-staging-and-production-enviro/#comment-16302200
have made my life considerably easier.

Hope the following helps others as well.

# 1.
# settings is a directory, containing host-specific overrides
# and the generic override logic

$ ls settings/
__init__.py   override.py   host1.py   host2.py

# 2.
# __init__.py has the same contents that settings.py
# usually has, except it automatically imports host-specific
# settings in the end

$ tail -2 settings/__init__.py
override.override_settings('projectname', platform.node().split('.')
[0],
sys.modules[__name__])

# 3.
# The generic override logic is as follows

$ cat settings/override.py
from django.utils.importlib import import_module
import warnings

def override_settings(package, name, settings_module):
rel_path = 'settings.' + name
try:
override_module = import_module(rel_path, package=package)
except ImportError:
warnings.warn("Failed to import settings from '%s.%s'" %
(package, rel_path))
else:
for key in dir(override_module):
if key.isupper() and not key.startswith('__'):
setattr(settings_module, key, getattr(override_module,
key))
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.




Re: Status of branch soc2009/http-wsgi-improvements? (and ticket 2131)

2010-02-10 Thread mrts


On Feb 10, 5:24 pm, Tom Evans  wrote:
> On Wed, Feb 10, 2010 at 3:09 PM, Jari Pennanen  
> wrote:
> > Hi!
>
> > I was wondering what is the status of branch branches/soc2009/http-
> > wsgi-improvements 
> > (http://github.com/django/django/tree/soc2009/http-wsgi-improvements
> > )? I'm personally interested one bug it fixes, mainly ticket #2131
> > (http://code.djangoproject.com/ticket/2131)

You don't need that fix to use efficient file serving.

Just use an empty response and set the X-Sendfile header manually if
using
Apache. If not, check your server documentation for the right header
name.

E.g.:

response = HttpResponse()
response['X-Sendfile'] = '/full/path/to/file'
response['Content-Type'] = 'some/content_type'
response['Content-Length'] = os.stat('/full/path/to/file').st_size
response['Content-Disposition'] = 'attachment; filename="foo"'
return response

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: Deprecating cmemcache, adding pylibmc

2010-03-16 Thread mrts
On Mar 1, 5:27 pm, Jacob Kaplan-Moss  wrote:
> Also, a step 2.5, if you'd like, would be to write a tiny app on pypi
> that enabled the use of pylibmc via an external cache backend. We
> could point to it in the deprecation notes when we explain why
> cmemcache is being deprecated.

See http://gist.github.com/334682

Best,
MS

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Threading review wiki page removal

2010-04-06 Thread mrts
James has replaced the content of

http://code.djangoproject.com/wiki/DjangoSpecifications/Core/Threading

with the following disclaimer:

"This page and several others were created by a
wiki user who was not and is not a member of the
Django core team. Previous contents of this and
other similar pages are not and should not be confused
with Django's own documentation, which remains
the sole source of official documentation for the Django project."

I personally think that the information on that page [1]
was both useful and mostly correct, albeit incomplete.
As such I fail to see why it should be removed --
the disclaimer given by James pertains to the majority
of pages in Django wiki; following that reasoning,
all of them should be removed :).
(Moreover, the disclaimer is implicit in most wikis' content.)

That doesn't necessarily mean that the page should be
restored as-is. Something seems to disturb James,
so the contents should perhaps be updated, amended
and relocated to a better URL so that he and other members
of the core would be happy with it instead.

Thoughts?

Best,
Mart Sõmermaa

[1] Original content:
http://code.djangoproject.com/wiki/DjangoSpecifications/Core/Threading?version=80

P.S. I'm the author of that page, but that's not really important.
What is important -- there was information on a core
aspect of Django not available anywhere else which helps
others to understand it better and write better patches for it.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: Threading review wiki page removal

2010-04-06 Thread mrts
On Apr 7, 6:21 pm, Russell Keith-Magee  wrote:
> James spoke with me about this decision at the time, and I completely
> agree with and endorse his actions.

So be it then.

> While it is true that wikis contain all sorts of information, often
> unofficial, the naming of the wiki pages in question --
> DjangoSpecifications/Core -- conveyed a *very* strong signal that the
> information contained was official in some capacity. Regardless of the
> merits of the information on that page, the simple fact remains that
> it *isn't* official, and it *wasn't* vetted or edited by anyone in the
> core.

The naming was an unfortunate mistake that dates back two
years-ish when I was trying to advocate supplementing the
mailing list discussions (that tended to be much lengthier
and more diffuse as Django was still much in flux back then)
with specification pages on the wiki (that usually tend to
coalesce to clear, to-the-point documents faster) --
inspired by Python's PEP process. Although I failed horribly
back then, I'm glad that everybody writes them nowadays
without much bravado and high ritual ([1], [2], [3], ...).
That's clearly the best way to do it.

The ill-named location doesn't render the information less
valuable in my opinion. I've referred to the review on
several occasions myself, also, Graham links to that page
from
http://code.google.com/p/modwsgi/wiki/IntegrationWithDjango
. But, evidently, I may be biased (being the "author").

> If you think there is some valuable information in the wiki pages that
> were deleted, then you need to turn that information into either
> tickets in Trac, or a draft for addition to the official
> documentation. If you think there is some information on that page
> that doesn't fit into the official locations that Django currently
> provides, then we need to have a meta-discussion around the right
> place to house information of that sort.

Although I personally would like to see a wiki section that
collects all the fine technical documents already referred
to above ([1], [2], [3], ...) + unofficial "internals" docs
in the vein of the threading review, I believe no-one will
step up to compile one. So let it be.

Best,
MS

P.S. What about
http://code.djangoproject.com/wiki/CollaborateOnGithub
that I have been maintaining? Should that also be maintained
outside of Django wiki?

[1] http://code.djangoproject.com/wiki/Signing
[2] http://code.djangoproject.com/wiki/CsrfProtection
[3] http://code.djangoproject.com/wiki/ReplacingGetAbsoluteUrl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



i18n improvements

2008-06-10 Thread mrts

Please give you thougths on the following.

 * contrib.localflavor should be deprecated and transferred to a
separate project or contributed to Babel.

   It is incomplete by nature as covering all regions/postal codes in
the world takes a lot of effort.
   That effort is out of Django's scope.
   A dedicated maintainer should use the ISO country database to
retrieve that information in a more
   complete, standardized manner (a generic import script for that
project or guidelines in the Django
   docs should be provided perhaps).
   The localflavor problems have been discussed before, the main point
was that there will eventually
   be massive amounts of data that cannot be easily stored in memory,
see
   
http://groups.google.com/group/django-developers/browse_thread/thread/18de17236ae98914/902efbbd4b867859
   As an aside, localflavor creates confusion for translators: leaving
the region names untranslated means that the
   translation is incomplete, but again, full translation takes a lot
of effort (see e.g. 
http://groups.google.com/group/Django-I18N/browse_thread/thread/887e11c89dda748e
).

 *  DRY in regard to Babel.

A lot of baseline effort is duplicated: date/time/currency/number
formats,
date etc parsing, weekday/month names etc. The options are as
follows:
* use an automated script to import and convert format strings
* add dependency to Babel
* bundle Babel

A decision should be made before 1.0.

Regards,
Mart Sõmermaa
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: i18n improvements

2008-06-11 Thread mrts

Could someone of the core devs give his opinion on this?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: RFC: Django 1.0 roadmap and timeline

2008-06-12 Thread mrts

I'd like to bring up trac milestones again.

Currently Django has over 1000 active tickets. Some of them are
relevant to oldforms-admin, some are from pre-qsrf merge, some are
features for 1.1. But quite a few are small bugs that should be fixed
before 1.0.

Putting up milestones in trac would spark a community effort to sort
them properly. Assuming one minute per ticket, the sorting alone takes
17 hours from a single person. Hence community effort is clearly
required. Having a defined ticket set for 1.0 and all the intermediate
alphas is very important IMHO and using a single VersionOneFeatures
wiki page for that is practically impossible. That page is great for a
general vision and major issues that span tickets, but as I said, we
have to deal with the multitude of small issues as well.

Milestones should be set up according to Jacob's post (+ a legacy
milestone, e.g. 0.96-legacy for capturing tickets that are made
irrelevant after nfa merge etc).
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Application Admin Index

2008-06-18 Thread mrts

Please see

http://code.djangoproject.com/ticket/3591
http://code.djangoproject.com/wiki/DjangoSpecifications/NfAdmin/FlexibleAppHandling
http://code.djangoproject.com/wiki/InstalledAppsRevision

Will 
http://code.djangoproject.com/wiki/DjangoSpecifications/NfAdmin/FlexibleAppHandling
do what you need? If not, feel free to add your ideas to that page and
lets bring it back to this list once we have a solid feature list.

On Jun 18, 6:25 pm, jurian <[EMAIL PROTECTED]> wrote:
> Not really no.
>
> With this solution I would override the entire admin index. Meaning
> other applications I use for the same project will not have the luxury
> of having their own way with the admin index page.
>
> My understanding is that applications should plug into projects
> without overriding each other's functionality.
>
> Please let me know if I am at fault here.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: GSOC: More backends for djangosearch

2008-06-21 Thread mrts

On Jun 20, 12:47 am, Ben Firshman <[EMAIL PROTECTED]> wrote:
> I'm still in exams unfortunately, so this will be brief, but I'd like  
> to hear some thoughts.
>
> Imagine a small site without a great deal of traffic or data, and they  
> wish to add search support for a model. Lucene or Xapian are probably  
> overkill, and they may not want to invest time into getting it  
> running. tsearch2 or MySQL full text indexes seem the obvious  
> solution, and I would like to try and implement them, but what about a  
> simple built in index that would be backend independent? Much like the  
> django-search "simple" backend:

An eager +1 for that. Have to admit being biased though as I've
proposed this before. It is in the general spirit of Django -- scale
only if needed but get running with the right API from the start
(think caching).
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



django.contrib.sessions problems

2008-07-07 Thread mrts

I'd like to get some feedback for the following major tickets
regarding sessions, all of which are in scope for 1.0.

1) Session key collisions: http://code.djangoproject.com/ticket/1180

Due to the birthday paradox, sqrt(n) is roughly the number you need to
have a 50% collision chance when picking items at random from an
inexhaustible set of n items. (0, sys.maxint - 1) is currently the
random range. On 32-bit platforms the collision bound is thus quite
low as sqrt(2^31) is about 46 000 keys.

Although get_or_create() is used to guard against collisions, they can
still occur in threaded environments that have millions of sessions in
session store (multiple threads enter get_or_create(), collision
probability is very high when the number of stored sessions is, say,
10 times 46 000). This has been reported by at least one user.

There's no problem on 64-bit platforms as the collision bound is
sqrt(2^63) ~ 3,000,000,000.

Proposal: use 63 bits of randomness regardless of architecture.
(Trivial) patch attached to #1180. According to reports, this fixes
the problem on 32-bit systems.

See analysis at http://code.djangoproject.com/ticket/1180#comment:26

2) Reliable session clearing: http://code.djangoproject.com/ticket/7515

Clearing session data is a very common use case. Currently, the only
way to do it is to manually erase session keys in a loop.

The patch attached to #7515 fixes that by clearing the underlying
dictionary in one step. However, it is not exception-safe, see problem
statement at http://code.djangoproject.com/ticket/7515#comment:6 .

The right way to solve this should be discussed further.

3) Clear session on logout: http://code.djangoproject.com/ticket/6941

Currently sessions are not related to users. As discussed before, one-
way relation is useful, i.e. users are related to sessions and
sessions are cleared if they logout or if they login and a different
user was logged in previously. This depends on #7515.

---

There are also two less acute tickets, http://code.djangoproject.com/ticket/6984
and http://code.djangoproject.com/ticket/6791 . IMHO they are in scope
for 1.0 as well.

After resolving these issues, the session framework should be ready
for 1.0 feature freeze.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: django.contrib.sessions problems

2008-07-07 Thread mrts



On Jul 7, 5:00 pm, "Jacob Kaplan-Moss" <[EMAIL PROTECTED]> wrote:
> On Mon, Jul 7, 2008 at 8:48 AM, mrts <[EMAIL PROTECTED]> wrote:
> > I'd like to get some feedback for the following major tickets
> > regarding sessions, all of which are in scope for 1.0.
>
> I don't really know a nice way of saying this, so I'll trust you to
> understand that I don't mean to be a dick.
>
> You don't get to decide what's in scope for 1.0 and what isn't.

That was not my intention (and it appears that somehow many of my
posts create a bit of friction :) -- their relevance looks to be
mostly accepted but wording not... oh well). What I wanted to say is
that robust sessions are a vital component of a web framework and all
the issues above need some discussion and decisions before 1.0. So,
indeed, I wanted to argue, not demand something.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: django.contrib.sessions problems

2008-07-07 Thread mrts

> > Due to the birthday paradox, sqrt(n) is roughly the number you need to
> > have a 50% collision chance when picking items at random from an
> > inexhaustible set of n items. (0, sys.maxint - 1) is currently the
> > random range. On 32-bit platforms the collision bound is thus quite
> > low as sqrt(2^31) is about 46 000 keys.
>
> This isn't correct. The birthday paradox models a situation of choices
> *with replacement* from a set. That is, when multiple entities can have
> the same value. We don't have replacement. You have a set of X active
> sessions, all unique (since that's a database constraint) and somebody
> is trying to guess one of them at random. So the probability is X/N, not
> something approaching sqrt(N)/N.

random.randint(0, n) *is* a set with choices with replacements.
Multiple picks can, should and, as demonstrated at
http://code.djangoproject.com/ticket/1180#comment:26 , do create
collisions at around sqrt(n) picks and less.

I entirely agree that collisions in picks from the random value set
shouldn't directly manifest in collisions in the session key set as
1) time.time() is used as another varying input
2) collision is explicitly checked for in a loop until a unique value
is created

So, for a session key collision to occur, multiple threads should
simultaneously enter the function, get the same time value and a
colliding random value (and negative result from the uniqueness
check). Admittedly this looks very unlikely when random values collide
at around 46 000 picks.

However, according to the report 
http://code.djangoproject.com/ticket/1180#comment:31
, collisions do exist when random range is (0, 2^31) and do not exist
when it is (0, 2^63). So perhaps we miss something.

To properly solve the problem, the particular scenario that leads to
the collision should be pinpointed. Then we can make an informed
decision about the amount of randomness needed.

Until that, 63 bits can be used as it is safer than 31, does not cause
much performance penalty and has proven to be enough to fix the
problem.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: django.contrib.sessions problems

2008-07-07 Thread mrts



On Jul 7, 6:32 pm, Malcolm Tredinnick <[EMAIL PROTECTED]>
wrote:
>
> That comment has no bearing.
>
> (1) We pick a random session key.
> (2) We save it to the database, where it should be unique, otherwise an
> error is raised.
> (3) We use that session key to pass back to the user.
>
> At this point, the birthday paradox no longer applies. (3) should not
> happen before (2). If it does, it's a bug that should be fixed, but the
> database should be being used to guarantee uniqueness here. It's the
> same database across all threads/processes/machines.
>
> Your comment in that ticket avoids step (2), which is a necessary
> uniqueness constraint.

Agreed, but increasing the range from (0, 2^31) to (0, 2^63) should
have no effect on collisions then. The report at #1180 seems to
indicate otherwise (unless it is invalid).
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: django.contrib.sessions problems

2008-07-07 Thread mrts

Thus, looks there are two problems -- collisions happen too often on
32-bit machines (fixed by my patch) and db session backend doesn't
handle them properly (no fix so far).

As for the latter -- Malcolm, you were about to add a clear
distinction between INSERT and UPDATE to .save() in
http://groups.google.com/group/django-developers/browse_thread/thread/179ed55b3bf44af0
. That would help to fix

def save(self):
Session.objects.create(
session_key = self.session_key,
session_data = self.encode(self._session),
expire_date = self.get_expiry_date()
)

As far as I can see, create uses .save() and .save() does an UPDATE
when the object exists instead of unconditionally failing as it should.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: django.contrib.sessions problems

2008-07-08 Thread mrts

On Jul 8, 5:27 am, Malcolm Tredinnick <[EMAIL PROTECTED]>
wrote:
> Increasing
> some value from 32 bits to 64 bits is only changing some probabilities;
> it's not actually solving the problem, just moving the goalposts to make
> it harder to score an own goal. The rest of the conversation should
> proceed on the assumption that the bug about creating unique database
> entries will be fixed first.

Agreed that the uniqueness bug takes precedence. But the uniform 63-
bit randomness patch is also important. Once collisions happen, their
handling is quite expensive, so they should be better avoided in the
first place.

Moreover, 63 bits are already used on 64-bit machines and I don't see
any reason not to use it on 32-bit machines as well.

---

I would like to get some feedback on the reliable session clearing
problem next, http://code.djangoproject.com/ticket/7515 .

Clearing session data in one step is a trivial addition:

   def clear(self):
self._session.clear()
self.modified = True

Though better than manually erasing all session keys in a loop, this
is not robust enough. Consider this scenario:

* session with key X is created for user A
* A "logs out" and session data is cleared, session key X will not be
reset
* session db backend tries remove the empty session, but database
connection fails, exception is raised
* cookie with old session key X remains in browser
* user B accesses the site using the same browser, cookie with key X
will be sent to the site
* database has come online, session exists and A's sensitive
information will be happily served to B

Thus, reliable session clearing should assure that the session cookie
is removed or updated with a different session key even when
exceptions occur in the session storage backend.

No hurry in replying, but discussing this is important in my humble
opinion.

Best,
Mart Sõmermaa
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: django serializers are not thread safe?

2008-07-08 Thread mrts

I reviewed threading in Django and found a few problems, one of which
is serialization.
See 
http://code.djangoproject.com/wiki/DjangoSpecifications/Core/Threading#Problems
.

Fix was contributed to the most eminent one in #6950, but I never got
to fixing serialization.
The fix is easy though, just use a temporary as demonstrated in
"BETTER" at
http://code.djangoproject.com/wiki/DjangoSpecifications/Core/Threading#Errorsduetoincompleteinitialization

On Jul 8, 3:56 pm, Malcolm Tredinnick <[EMAIL PROTECTED]>
wrote:
> On Tue, 2008-07-08 at 05:49 -0700, sector119 wrote:
> > HI All!
>
> > I have to deserialize, serialize objects in multiple threads with
> > django serializers.
>
> > But when I run Deserializer =
> > serializers.get_deserializer(SERIALIZATION_FORMAT) in my threads (more
> > then one, with _one_ thread all works great) I get KeyError in
> > get_serializer function at `return _serializers[format].Deserializer`
> > line because _load_serializers does not load all data at _serializers
> > dict before another tread run get_deserializer at this another thread
> > and completely populate serializers dictionary with all, builtin and
> > user-defined serializers!
>
> Sounds like a possible explanation. Please open a ticket for it so this
> doesn't get forgotten.
>
> Regards,
> Malcolm
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: django serializers are not thread safe?

2008-07-08 Thread mrts

Patch uploaded to http://code.djangoproject.com/ticket/7676 , please
test.

On Jul 8, 4:21 pm, sector119 <[EMAIL PROTECTED]> wrote:
> Problems
>    2. django/core/serializers/__init__.py: _load_serializers() is
> unsafe, patch attached to #FIXME:
>
> Where can I found the patch? Where is "#FIXME:" ? ;)
>
> On 8 Лип, 16:07, mrts <[EMAIL PROTECTED]> wrote:
>
> > I reviewed threading in Django and found a few problems, one of which
> > is serialization.
> > Seehttp://code.djangoproject.com/wiki/DjangoSpecifications/Core/Threadin...
> > .
>
> > Fix was contributed to the most eminent one in #6950, but I never got
> > to fixing serialization.
> > The fix is easy though, just use a temporary as demonstrated in
> > "BETTER" 
> > athttp://code.djangoproject.com/wiki/DjangoSpecifications/Core/Threadin...
>
> > On Jul 8, 3:56 pm, Malcolm Tredinnick <[EMAIL PROTECTED]>
> > wrote:
>
> > > On Tue, 2008-07-08 at 05:49 -0700, sector119 wrote:
> > > > HI All!
>
> > > > I have to deserialize, serialize objects in multiple threads with
> > > > django serializers.
>
> > > > But when I run Deserializer =
> > > > serializers.get_deserializer(SERIALIZATION_FORMAT) in my threads (more
> > > > then one, with _one_ thread all works great) I get KeyError in
> > > > get_serializer function at `return _serializers[format].Deserializer`
> > > > line because _load_serializers does not load all data at _serializers
> > > > dict before another tread run get_deserializer at this another thread
> > > > and completely populate serializers dictionary with all, builtin and
> > > > user-defined serializers!
>
> > > Sounds like a possible explanation. Please open a ticket for it so this
> > > doesn't get forgotten.
>
> > > Regards,
> > > Malcolm
>
>
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Testing django-admin and manage.py

2008-07-08 Thread mrts

> Yes, the tests run on Windows now as well.  Only problem is two tests with
> file paths in their output, the forward slashes are backslashes on Windows
> so the output doesn't match what the tests are looking for:
>
> http://dpaste.com/61364/
>
> Not sure if/how you ordinarily get around that in testing?

self.assertOutput(out, os.path.join('django', 'contrib', 'auth',
'models.pyc'), ...)
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Localizing digits

2008-08-19 Thread mrts

I'm using http://babel.edgewall.org/ exactly for that purpose. Works
extremely well. There are even some Django-specific extensions at
http://babel.edgewall.org/wiki/BabelDjango (I don't use them though).

On Aug 18, 4:48 pm, Joost Cassee <[EMAIL PROTECTED]> wrote:
> Hi all,
>
> A Farsi-localized Django site I'm working on will need Farsi digits,
> so I'm looking for a solution for replacing digits with their
> localized counterparts.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Be explicit about which reverse lookup failed after r8211

2008-08-23 Thread mrts

It is quite common to be hit by the insufficiently verbose reporting
that #8221 and #7524 fix -- e.g. see the duplicates that have popped
up.

As I already said, #8221 is only needed because the patch I provided
in #8177 and that got commited fixed only the most burning issue I was
directly hit with. Now that I looked around in urlresolvers, I found
other places where similar improvements were needed. #8221 covers all
them and supersedes #8177.

I personally think both should make it into 1.0, but James seems to
oppose, so can we discuss this a bit further?

Regards,
Mart Sõmermaa
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Be explicit about which reverse lookup failed after r8211

2008-08-23 Thread mrts

http://code.djangoproject.com/ticket/7524 is tagged as post-1.0.
http://code.djangoproject.com/ticket/8221 was closed as duplicate of
#7524, which it is not.

On Aug 23, 9:40 pm, Malcolm Tredinnick <[EMAIL PROTECTED]>
wrote:
> On Sat, 2008-08-23 at 10:46 -0700, mrts wrote:
>
> [...]
>
> > I personally think both should make it into 1.0, but James seems to
> > oppose, so can we discuss this a bit further?
>
> The ticket is open. It will either be committed, postponed or closed as
> a dupe of something else. Let's leave it at that and get some other work
> done in the interim.
>
> Malcolm
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Be explicit about which reverse lookup failed after r8211

2008-08-24 Thread mrts



On Aug 24, 4:40 am, "James Bennett" <[EMAIL PROTECTED]> wrote:
> On Sat, Aug 23, 2008 at 2:43 PM, mrts <[EMAIL PROTECTED]> wrote:
> >http://code.djangoproject.com/ticket/8221 was closed as duplicate of
> > #7524, which it is not.
>
> In triaging, I'm trying to take the position that all of the various
> "Django looks for an exception and accidentally swallows some other
> exception" tickets are really the same issue and should all be dealt
> with together, rather than being attacked piecemeal.

That argument pertains to #7524 and to only *half* of #8221 -- the
except (ImportError, AttributeError): raise NoReverseMatch("Error
importing '%s': %s." % (viewname, e)) bits in there. A general
agreement is indeed needed (e.g. what to do with the original
backtrace), but IMHO this should be in scope for 1.0.

The other half is different, although in the same spirit of "try to
provide all available information that helps to solve the problem" --
raise NoReverseMatch("Reverse for '%s' with arguments '%s' and keyword
arguments '%s' not found." % (lookup_view, args, kwargs)). This helped
me to track down an invalid slug that had slipped into an article
database. The article index view was dying with NoReverseMatch and I
had no clue why (everything *looked* OK) -- found it only after
patching my Django checkout with this.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Django 1.0 and multithreading issues.

2008-08-29 Thread mrts

I am mrts (Mart Sõmermaa).

The page was created and updated by me and I'm quite certain the
anonymous updates are also mainly by me. So I am entirely to blame for
the confusing naming and for any incorrect information. But I do hope
that the information is mostly correct and relevant, it is just
somewhat incomplete -- that's the main reason I never got to
announcing it here. As Malcolm pointed out, it is somewhat sketchy and
should be amended. But it is intended to be shared eventually, that's
why it's in Django wiki. Regarding Malcolm's concerns on whether it
somehow "looks" official due to naming -- as I've already iterated
many times previously, I never intended that and I accept whatever the
core devs decide in regard of renaming.

I really think something in the lines of that page -- sans the
automatic review results -- should be part of the original docs (docs/
internals/threading.txt perhaps, of course post-1.0), so that people
who contribute can be aware how threading is done in Django.

As for thread safety itself, the automatic review helped me track down
and patch two bugs --  #6950 and #7676. There was one spot in django/
db/models/fields/related.py I didn't analyze completely, quoting
myself "append() in add_lazy_relation() can add duplicated values,
which may or may not confuse pop() in do_pending_lookups()". Otherwise
all looked good when I reviewed the code. As you can see from the
page, the review was automatic and thus inherently incomplete -- I
only grepped for globals and reviewed the code around them. Class
attributes are reviewed to a lesser extent as tracking them is more
complicated.

And regarding thread-safety in practice: I have used Django with
mod_wsgi and threaded Apache for over a year with no ill effects (with
thorough testing as we did run an important project on threaded
Django). mod_wsgi is such a breath of fresh air after the heavy weight
of mod_python, everything runs lighter and faster and daemon mode +
user separation + touch the wsgi wrapper for reloading Python code has
made my life *considerably* easier -- so IMHO we should switch to
recommending mod_wsgi in docs instead of mod_python as the default
deployment option after 1.0.

P.S. I still think community design (wiki) is as important as are code
contributions (tickets and SVN), see 
http://mpt.net.nz/archive/2008/08/01/free-software-usability
for some points. Discussions here are a nice starting point, but they
tend to *diffuse* instead of *converging*. Wikis are good for
converging ideas and it has not to be a high-ritual process that most
coders loathe :). Currently the core spec process is either mostly
hidden from community -- devs meet in-person during conferences etc,
or is already done in practice this way -- both qs-rf and fs-rf had
extensive spec/doc pages. Why not make it semi-official with a PEP-
like process after 1.0?

Random collection of things that need specs for post-1.0:
 * app handling -- there's lot to gain, think of e.g. extending
setuptools
 * Identity Map (#17)
 * multi-db support
 * object history

On Aug 29, 3:32 am, Malcolm Tredinnick <[EMAIL PROTECTED]>
wrote:
> On Thu, 2008-08-28 at 16:42 -0700, Graham Dumpleton wrote:
> > Can someone give an update as to where Django 1.0 may stand in respect
> > of multithreading issues.
>
> > I note that all the links to tickets referenced in:
>
> >  http://code.djangoproject.com/wiki/DjangoSpecifications/Core/Threading
>
> > seem to be crossed out, implying they are closed.
>
> > Does this mean we are closer to being confident in saying Django may
> > be multithread safe or are there other issues that still need to be
> > addressed?
>
> Keep in mind that that page isn't neither a "specification" nor "core".
> It's a scratch pad of ideas by a few people (hard to tell how many
> because it's been edited anonymously at times). The naming is
> unfortunate and something we'll probably change in the near future.
> Definitely a useful page in some aspects, but to be treated
> appropriately as with anything in the wiki: anyone can, and has edited
> it, and it's difficult to view the history of what's gone on. At some
> point we'll have to rename it because people have been confused by the
> name.
>
> To answer your actual question, we should be pretty much threadsafe
> across the board. All tickets reported of that nature have been
> addressed. There are some places where we do intentional sharing, but
> they are just that: intentional. Other places should be filed as bugs,
> because they most likely are. To the best of my knowledge (and I do have
> some about the code), using multiple threads of execution for the same
> virtual host / location config blog (i.e. one settings file) should
> work.
>
> There are 

New milestones for ticket triagers

2008-09-05 Thread mrts

Hopefully the devs are taking a well-deserved break now as 1.0 is
out :)

Once back online, a bit of release planning + corresponding new
milestones in Trac would come in handy for ticket triagers to
categorize tickets.

I'd propose something in the lines of the following (and I'd really
prefer no flames this time, this is just a proposal and the intent is
*not* to intrude and step on dev's toes but get a healthy discussion
going :) ):

 * a minor point release 1.0.1 -- minor fixes for things that have
popped up since 1.0, e.g. #8882. In the agile spirit, the minor point
releases should be released rather quickly -- e.g. in weeks or a month

 * a major point release 1.1 -- features that don't break 1.0-
compatibility, but bring in larger internals fixes, e.g. de-cruftify
template tag loading etc. Release timeframe spans several months, e.g.
a release in every 2 months.

 * a major release 2.0 -- features that have larger impact to the
codebase and may or may not break 1.0-compatibility: #17, model
validation, app objects etc. The "official" development branch, no
release trimeframe, release when it's ready
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



I want a pony: Django Cheeseshop

2008-09-10 Thread mrts

This is for Django 2.0.

Rationale:

 a) useful (and less-useful) Django apps are proliferating on Google
Code, djangoplugables.com and Django resources wiki page,
 b) apps need refactoring (#3591,
http://code.djangoproject.com/wiki/VersionOneFeatures#INSTALLED_APPSrefactoring
)
 c) reusable apps are immensely useful and may be one of Djangos great
selling points once they are easy to deploy and some order is enforced
upon the current chaos.

Proposal:
===

Infrastructure
---
 * create a central app index à la Cheeseshop
 * create an automated system similar to easy_install for installing
apps from
   o that central repository
   o or a custom repository/URL
 to
   o either globally to Python packages -- *but under django namespace!
*
   o or locally into a concrete project
 * provide app dependency handling like setuptools does for
   o python package dependencies (identical to setuptools 'depends')
   o Django app dependencies (e.g. 'django_app_depends')
 * bundle the install tool either as a command for manage.py or a
separate utility in bin
 * create the solution so that it can be adopted by other projects by
clearly decoupling Django-specific functionality (i.e. engage more
brainpower and devs)

Apps
---
 * create rich app objects that contain at least the fields available
in setuptools setup files (i.e. app metadata),
 * extend them with Django-specific functionality to
   o lessen magic: explicitly specify files containing models,
templates and admin bits, i.e. obsolete all forms of autodiscovery and
path hacking
   o find a way to expose media
   o handle all issues outlined in #3591

etc.

I see that spec pages are popping up in wiki, e.g.
http://code.djangoproject.com/wiki/ReplacingGetAbsoluteUrl -- thanks,
Simon! This should eventually land in a spec page as well (unless
there will be a unanimous "No pony for you!" reply).

Regards,
Mart Sõmermaa
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: I want a pony: Django Cheeseshop

2008-09-10 Thread mrts

On Sep 10, 6:57 pm, "James Bennett" <[EMAIL PROTECTED]> wrote:
> Have I made my point clear enough yet?

Your point is clear but is likely to bring less order to the current
chaos. And it doesn't handle project-local installation (arguably this
can be done with virtualenv, but that will just clutter user-specific
"app-space" instead of the global one). Most of the apps are really
Django-specific and don't conceptually belong in global Python
namespace.

If setuptools remains the recommended way of packaging apps (I don't
necessarily oppose this, but I'd like to hear other opinions; what I
don't like is littering CheeseShop with stuff that is unusable in
general Python context), at least a fixed policy is needed:
 * designate a global namespace where all apps should live (e.g.
django.apps.mycompany.fooapp or just django.apps.fooapp)
 * find a way for avoiding name collisions there (mycompany would deal
with that)
 * document all the above and the steps and requirements for creating
and publishing a reusable app.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: I want a pony: Django Cheeseshop

2008-09-10 Thread mrts

On Sep 10, 7:12 pm, "Eric Holscher" <[EMAIL PROTECTED]> wrote:
> your django apps. There is no reason to reinvent the wheel here, especially
> after what Mark talked about at Djangocon (Django being considered seperate
> from the Python community).

Although I don't know anything about the talk, that's exactly what I
wanted to say: as Django and Python communities are different,
overflooding CheeseShop with Django apps does not serve Python
community at all.

And I never had re-implementing setuptools in mind -- what I meant was
to extend and build upon setuptools and easy_install.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: I want a pony: Django Cheeseshop

2008-09-10 Thread mrts

On Sep 10, 8:00 pm, Steve Holden <[EMAIL PROTECTED]> wrote:
> I don't see why Django can't be just as much a part of the Python
> community as, say, Zope, who frequently distribute code through PyPi. I
> don't see the advantage of fragmenting the infrastructure. That's what
> it's there for - supporting Python users.

Right, be it the CheeseShop then if we have the blessing of Steve
Holden.

Integration-wise, Django is more like Zope (apps are coupled to the
framework) than TurboGears (apps are loosely coupled), so namespace
packages may make sense. A namespace should be agreed upon anyhow
(i.e. even if namespace packages will not be used) though. And
packaging guidelines are direly needed.

And what's with Django itself? Now that 1.0 is out it's high time for
`easy_install django` to work (James?).
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: I want a pony: Django Cheeseshop

2008-09-10 Thread mrts

I think we largely have a consensus now (unless someone speaks up) --
CheeseShop and the global (or virtualenv) package space is the way to
go for all apps when packaging rules have been documented.

If someone wants to take this further, a ticket + a documentation
patch that outlines how to package apps would be most welcome.
Namespace issues still need to be resolved though. Looks like there
should be two topics
 * "How do I install external packages?" -- outline the two
strategies, global and virtualenv (for self-contained project
environment), how to find packages etc;
 * "How should I package and distribute a reusable Django app?" --
what namespace to use, uploading to CheeseShop, registering in
djangoplugables (i.e. there should be a semi-official richer
environment that lists only Django apps, djangoplugables looks good),
dependencies, media etc.

On Sep 10, 9:00 pm, "Brett Hoerner" <[EMAIL PROTECTED]> wrote:
> On Wed, Sep 10, 2008 at 11:20 AM, mrts <[EMAIL PROTECTED]> wrote:
> > And it doesn't handle project-local installation (arguably this
> > can be done with virtualenv, but that will just clutter user-specific
> > "app-space" instead of the global one).
>
> At some point the Django app you're trying to installed has to go
> somewhere.  If it doesn't go in global or user's local, where do you
> want it to go?

Into project dir of course. Most of the existing apps can be trivially
installed by cd myproject; wget somewhere/app.tgz; tar zxf app.tgz.

> With virtualenv you can (and I would encourage that you) have an env
> for each Django project, so any installing you do is only 'cluttering'
> that project, and at that point it isn't even 'clutter'.

That holds only if you don't live on shared hosting and can afford
direct project <-> user (evironment) mapping.

If you can't afford that, installing stuff either globally or into
virtualenv can create trouble, e.g. project X is written against app Z
version 1, project Y is written against (incompatible) version 2 of
the same app, both X and Y have to run under the same user (shared
hosting) -- the only way to run them is to have a Y-project-local copy
of Z.

This is a minor, uninteresting problem though, so no need to discuss
this further.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: I want a pony: Django Cheeseshop

2008-09-16 Thread mrts

On Sep 16, 2:10 pm, Malcolm Tredinnick <[EMAIL PROTECTED]>
wrote:
> On Tue, 2008-09-16 at 00:16 -0700, mrts wrote:
> > Let me try to wrap this up:
>
> > * there seems to be a general consensus that amending setuptools to
> > create Django-specific extensions is not required
> > * a separate repository is not required, apps should be published to
> > PyPi Famework :: Django category but a central site that tracks the
> > apps there to further classify, rate and enhance them would be useful
> > (like djangoplugables.com)
> > * app naming and namespace issues should be resolved (should all apps
> > have a django-prefix, the de facto standard? should they live in
> > django.apps namespace?)
>
> As others have pointed out, this isn't necessary and would be bad
> practice. Everybody should feel free to use whatever namespace they like
> to avoid clashes and so that we avoid lowering the quality of the Django
> brand (as soon as somebody installs a few low-quality "packages" and
> they misbehave in any way, it's going to be "Django is a pile of
> rubbish" because it's all coming from the Django namespace).

Another issue I have with this: we grab generic toplevel names like
"tagging", "registration" etc for apps that are coupled to Django.
Semantic names that clearly state the purpose of a package help to
make code more readable are of course good. But what if TurboGears
people want to name their tagging app "tagging" as well (instead of
Tasty as it's named now)? Ideally, I'd like to see code like this:

from djc import tagging # or `from django.apps import tagging`

def foo():
tagging.do_bar() # OK, at a glance I can see that this line
relates to tagging

instead of

import tasty # `import tasty as tagging` *can* be done, but oh well...

def foo():
tasty.do_bar() # What is tasty?! *Waste time for looking it up*
Aha, this line is for tagging


Also, there may be differences in toplevel generic `frobnicate`
(usable in Python in general) and `django.apps.frobnicate` (coupled to
Django and packaged as django-frobnicate) that adapts the former to
Django.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Proposal: app objects

2008-09-17 Thread mrts

I just copy-pasted the app objects requirement as listed and discussed
in http://code.djangoproject.com/wiki/InstalledAppsRevision . I
personally have no use case for it.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Proposal: app objects

2008-09-17 Thread mrts

On Sep 17, 11:13 am, mrts <[EMAIL PROTECTED]> wrote:
> I just copy-pasted the app objects requirement as listed and discussed
> inhttp://code.djangoproject.com/wiki/InstalledAppsRevision. I
> personally have no use case for it.

Doh, that should have been *multiple* app objects above.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Proposal: app objects

2008-09-17 Thread mrts



On Sep 18, 2:56 am, zvoase <[EMAIL PROTECTED]> wrote:
> I think the app object thing is a really good idea, but I have to say
> one thing; why not see if some middle ground can be met between the
> Django cheeseshop idea (going on in another thread in this group) and
> this.

That's the point. Both threads are interrelated, although the scopes
are a bit different. A holistic solution that tackles both how to
seamlessly add apps and how to easily access the data and logic they
encapsulate would be much welcome.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Call for comments: CommonMiddleware (and others) break streaming HttpResponse objects.

2008-09-22 Thread mrts

On Jul 4, 2:26 am, Tai Lee <[EMAIL PROTECTED]> wrote:
> > The only thing that might be worth doing in this are is adding a way to
> > say "middleware should never be called on this response" and then
> > somebody can write their own HttpResponse subclass and be in complete
> > control of their destiny.
>
> Would this disable ALL middleware from running? Or only the
> process_response methods on middleware? Either way, this is not ideal
> as there are useful middleware that CAN still work with on-demand
> content (e.g. the updated GZipMiddleware from the patch).

+1 for adding a way to say "process_response should never be called on
this response".

Taking a quick look at the source, HttpResponse seems to support
iteration already:

def __iter__(self):
self._iterator = iter(self._container)
return self

def next(self):
chunk = self._iterator.next()
if isinstance(chunk, unicode):
chunk = chunk.encode(self._charset)
return str(chunk)

I'd expect this is actually used in returning content, i.e. if no
middleware kicks in, passing an iterator to HttpResponse should just
work for streaming?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Call for comments: CommonMiddleware (and others) break streaming HttpResponse objects.

2008-09-23 Thread mrts

On Jul 1, 1:02 pm, Ivan Sagalaev <[EMAIL PROTECTED]> wrote:
> I've implemented iterable HttpResponse
> in the first place out of purely practical reason: web server times out
> if a particularly long file was passed into an HttpResponse. And also it
> was really slow and was consuming all the memory.

Ivan, does this imply that iteration does not work out of the box,
i.e. ordinary HttpResponse __iter__() and next() are not used in the
expected way? Or were you just using some middleware that consumed the
response and customization was required because of that?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: #6845 - model validation DDN

2008-09-25 Thread mrts

Hi Honza, thank you for your work on model validation!

What's the status? Getting model validation in (if the powers that be
agree) would help to fix a couple of annoying bugs listed below that
are present in 1.0.

http://code.djangoproject.com/ticket/9039
http://code.djangoproject.com/ticket/8882

All tickets tagged as model-validation:
http://code.djangoproject.com/query?status=new&status=assigned&status=reopened&keywords=~model-validation&order=priority

On Aug 12, 10:14 pm, "Honza Král" <[EMAIL PROTECTED]> wrote:
> Hey all,
> thanks for the feedback, I am still on vacation (haven't opened my
> notebook for days, what a feeling ;) ). I will be back in a couple of
> days and this thread is right at the top of my ToDo list.
>
> I plan to see this thing through unless somebody beats me to it or the
> design decisions go exactly opposite to my ideas (unlikely, even after
> reading this) and have set aside some time to do it next week.
>
> Thanks again for the feedback and patience...
>
> Honza Král
> E-Mail: [EMAIL PROTECTED]
> ICQ#: 107471613
> Phone: +420 606 678585
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Re-thinking model validation

2008-10-05 Thread mrts

mrts wrote:
> Looking at the path Honza has taken [1], it looks that it both

And the missing reference is
[1] http://code.djangoproject.com/ticket/6845
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re-thinking model validation

2008-10-05 Thread mrts

Looking at the path Honza has taken [1], it looks that it both
complicates things and causes overhead -- for every constraint on
an model object, an extra db call for checking it is made,
instead of relying on the constraint checks enforced by the db
backend and corresponding exceptions during `save()`
(e.g. for uniqueness validation, one db call is needed to check
the uniqueness constraint and a second one to actually save the
object).

The problem is conceptual: how aware should validation level be
of storage level. Python is known for the "It's easier to ask
forgiveness than permission" motto, the proposal is ideologically
based on that.

In short, db validation is shifted to `ModelForm.save()`. Outside of
ModelForms, model validation has to be done manually as
presently. The changes are less disruptive and fewer lines of
code need to be updated.

Here's the proposed idiom in code, explanation follows:

class FooForm(ModelForm):
class Meta:
model = Foo # model with unique constraints

def view_foo(request):
if request.method == 'POST':
form = FooForm(request.POST)
if form.is_valid():
try:
form.save()
return render_to_response('done.html')
except DBValidationError:
pass
else:
form = FooForm()

return render_to_response('form.html', { 'form' : form })

`ModelForm.is_valid()` remains unaware of db constraints.

`ModelForm.save()`
 * examines all constraints the model has
 * wraps Model.save() in a try/except block to catch all know
   constraints violations
 * if an exception is caught,
   + _errors is updated with relevant information (generally
 just _errors[field] = exception.message)
   + a DBValidationError subclass is re-raised

As a result, db state remains unchanged and the form contains all
the necessary error infromation for the user.

What needs to be done:
 * add DBValidationError and corresponding exception hierarchy
(UniqueValidationError, UniqueTogetherValidationError etc)
 * update model forms and formsets .save() as outlined above
 * update contrib.admin to take advantage of the

The current constraints are the following:
 * `unique`, `unique_together` and `unique_for_*`
 * `null=False`
 * `max_length` and `max_digits`
 * `choices`

Out of these, `null`, `max_length`, `max_digits` and `choices`
can be checked in `is_valid()`. It follows that only uniqueness
constraints remain in scope for this.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Re-thinking model validation

2008-10-06 Thread mrts

On Oct 6, 2:03 am, Malcolm Tredinnick <[EMAIL PROTECTED]>
wrote:
> On Sun, 2008-10-05 at 14:11 -0700, mrts wrote:
> > Looking at the path Honza has taken [1], it looks that it both
> > complicates things and causes overhead -- for every constraint on
> > an model object, an extra db call for checking it is made,
> > instead of relying on the constraint checks enforced by the db
> > backend and corresponding exceptions during `save()`
> > (e.g. for uniqueness validation, one db call is needed to check
> > the uniqueness constraint and a second one to actually save the
> > object).
>
> Except that very few validation requirements correspond to database
> constraints (for example any reg-exp matching field, or limits on an
> integer, etc). We aren't about to require database check constraints for
> all of those. So it's not really a one-to-one in practice. You've
> misread the patch quite badly,

Now that I re-read the first section of my post I do see that I made
the mistake of being over-emphatic and made an incorrect
generalization ("for every constraint on an model object, an extra db
call for checking it is made") -- sorry for that. My point was that
validating non-db constraints (like regex matching of an EmailField)
could remain the responsibility of form validation as it is now. So,
as presently, forms layer could take care of everything that can be
taken care of without touching the database (including max_length, not
null and choices) and model forms take additionally care of
IntegrityErrors by catching them, augmenting _errors[] with relevant
data and re-throwing them. And as IntegrityErrors can only be reliably
caught during save(), model forms have to be handled differently.

I had mostly backwards-compatibility and simplicity in fixing e.g.
#8882 in mind (as validate_unique() in forms/models.py lets
IntegrityErrors through anyway as of 1.0 and can not be relied on).
But if you think this is a bad idea, so be it.

- copied from below: -

> There needs to be a clear phase prior to saving when
> you can detect validation errors so that they can be correctly presented
> back to the user. You see this already with forms where we check for
> validity and, if it's not valid, we present the errors. If it is valid,
> we move onto doing whatever we want with the data (saving it or
> whatever).

General model validation is of course conceptually correct and good --
if the responsibility assignment between form and model validation is
sorted out in non-duplicated and elegant manner, unique checks handled
efficiently and if the code lands in a month or two timeframe, then
all is well. Meanwhile, people who want to use inline formsets with
unique validation are stuck on #8882 (as it looks that it takes quite
an effort to fix that with the current codebase -- and I'd like to be
wrong here).

> it sounds like: only the unique
> requirements have to check the database (and a pre-emptive check is
> reasonable there, since it's early error detection and there's only a
> small class of applications that are going to have such highly
> concurrent overlapping write styles that they will pass that test and
> fail at the save time).

- copied from below: -

> The only time there's any kind of overlap is when there's a database
> constraint such as uniqueness which we cannot guarantee will remain true
> between the validation step and the saving step. So there's a chance
> that save() will raise some kind of database integrity error. But that's
> actually the edge-case and in a wide number of use-cases it's
> practically zero since you know that your application is the only thing
> working with the database.

And your take is just to ignore that case and actually let e.g. admin
display 500 pages? It's an ordinary race condition (some of which are
indeed rare in practice but nevertheless usually taken care of) and
you yourself have generally advocated pushing this kind of problems to
the relevant backends -- which is nice and proper. Applying the same
principle to this case as well (and I can't think of anything other
but try save/catch IntegrityError) seems to me the only viable way to
avoid the race.

And I don't buy the "your application is the only thing working with
the database" argument -- it is, but it happens to be multi-process or
-thread. How am I supposed to avoid that thread or process A and B do
a validate_unique() for some x concurrently, finding both out that it
doesn't exist and proceed happily with saving, so that one of them
chokes on an IntegrityError?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Re-thinking model validation

2008-10-07 Thread mrts

On Oct 7, 12:08 pm, Malcolm Tredinnick <[EMAIL PROTECTED]>
wrote:
> On Mon, 2008-10-06 at 15:27 -0700, mrts wrote:
> resolution on updates. The multiple simultaneous writes to the exact
> same piece of data is only one corner of the full domain space. We plan
> for the worst, but often optimise for other cases. Again, I'm not
> dismissing anything.

Ack, sounds good. Over and out for this, thanks for elaborating.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Distutils/packaging sprint this weekend

2008-10-10 Thread mrts

In context of the Django community apps and app object discussions we
had some time ago, there's a sprint upcoming tomorrow:

http://mail.python.org/pipermail/python-dev/2008-October/082971.html
http://www.openplans.org/projects/plone-conference-2008-dc/distribute

This is a good occasion to speak up and influence things in case
something has been discussed internally and needs to be conveyed to
distutils maintainers.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



"OperationalError: database is locked" with Python 2.6 multiprocessing and SQLite backend

2008-10-21 Thread mrts

It seems that current DB lock management doesn't play nice with the
new Python 2.6 multiprocessing package and SQLite. See [1]. The same
error also popped up in Google search under mod_python [2].

I wasn't able to reproduce this with MySQL.

[1] http://code.djangoproject.com/ticket/9409
[2] 
http://209.85.135.104/search?q=cache:kEMOo-HuvzgJ:www.rkblog.rk.edu.pl/w/p/django-nginx/+django+OperationalError:+database+is+locked

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: "OperationalError: database is locked" with Python 2.6 multiprocessing and SQLite backend

2008-10-27 Thread mrts

On Oct 27, 5:16 pm, Brian Beck <[EMAIL PROTECTED]> wrote:
>
> As others have pointed out, this isn't an issue with Django.  The
> easiest solution is to make this error less common with a higher
> timeout.  In settings.py:
>
> DATABASE_OPTIONS = {'timeout': 30}

Thanks Brian, increasing the timeout fixed the problem.

IMHO this should be documented, so I reopened 
http://code.djangoproject.com/ticket/9409
and changed the component to Documentation.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: "OperationalError: database is locked" with Python 2.6 multiprocessing and SQLite backend

2008-10-27 Thread mrts

On Oct 27, 6:57 pm, mrts <[EMAIL PROTECTED]> wrote:
> On Oct 27, 5:16 pm, Brian Beck <[EMAIL PROTECTED]> wrote:
> > As others have pointed out, this isn't an issue with Django.  The
> > easiest solution is to make this error less common with a higher
> > timeout.  In settings.py:
>
> > DATABASE_OPTIONS = {'timeout': 30}
>
> Thanks Brian, increasing the timeout fixed the problem.
>
> IMHO this should be documented, so I 
> reopenedhttp://code.djangoproject.com/ticket/9409
> and changed the component to Documentation.

I've attached the explanation to
http://code.djangoproject.com/attachment/ticket/9409/database_is_locked_docs.diff
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



1.0.1 release blockers?

2008-10-31 Thread mrts

There has been much reluctancy in letting triagers tag and prioritize
1.0.1 milestone tickets. Now that 1.0.1 is really close, can we
perhaps discuss what are the things that really should be fixed before
it is released?

The only major issue I have encountered is 
http://code.djangoproject.com/ticket/8882
that makes inline formsets that have unique fields (that is, pretty
much every other use case for them) unusable. Looks like brosner is
already working on it -- thanks! -- and it would be perhaps wasteful
if 1.0.1 is released before he has finished.

I believe this does not fall to the often-quoted "this is your
personal issue, go write a patch" category (I have worked around it
and brosner is already on it) -- it is just a common use case that
doesn't work as advertised. There are probably other major bugs and it
would be nice if people brought them up. Also, it would be nice if
some restraint is exercised in regard the attitude-laden remarks that
are quick to arise in this type of discussions.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



String freeze for 1.0.1?

2008-11-03 Thread mrts

Shouldn't we have a string freeze before the 1.0.1 release (e.g. both
#8882 and #9493 bring in new error strings)?

Otherwise translators have to either continuously update the
translations as new strings land or accept that 1.0.1 is released
without complete translations.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: 1.0.1 release blockers?

2008-11-03 Thread mrts

On Oct 31, 5:19 pm, "Joey Wilhelm" <[EMAIL PROTECTED]> wrote:
> I would like to suggest the following:
> http://code.djangoproject.com/ticket/9245
> http://code.djangoproject.com/ticket/6710
>
> They both have fully functional patches... although granted the second has
> no tests.

IMHO http://code.djangoproject.com/ticket/8193 is also important
enough to justify inclusion before 1.0.1 (see aslo discussion
http://groups.google.com/group/django-developers/browse_thread/thread/d27261561bc36d96
). Quoting James's signature: Django should be "technically correct --
the best kind of correct." Apart from being an overall improvement in
correctness, see the problem reports at 
http://code.djangoproject.com/ticket/8193#comment:12
this would address. I'll update the patch shortly.

Additionally, some bits from the already semi-fixed
http://code.djangoproject.com/ticket/8882 that initially started this
thread have been shifted to http://code.djangoproject.com/ticket/9493
, which should also be integrated to close remaining inline formset
validation issues.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: 1.0.1 release blockers?

2008-11-04 Thread mrts

On Nov 4, 2:33 am, "Russell Keith-Magee" <[EMAIL PROTECTED]>
wrote:
> Yes, there is a reason, and it has been given several times in recent
> history. The v1.0.1 milestone has not bee created in Trac because it
> will not in any way help us deliver the v1.0.1 release. There is no
> difference between the "list of all bugs' and the "list of bugs that
> we want to close for v1.0.1". We may not be successful in meeting this
> goal, but that doesn't change the underlying goal.
>
> In understand that twiddling milestone flags on tickets apparently
> gives some people the warm, satisfying feeling that they are helping.
> However, speaking as one of the core developers, it would be much more
> helpful for that effort to be directed towards actually triaging new
> tickets (validating that a bug exists, finding duplicates, correctly
> classifying tickets etc), writing patches, and testing those patches.
> These are the jobs that are hard to do, and aren't assisted by having
> a new milestone flag to fiddle with in Trac.

Except that most of the tickets that have been brought up in this
discussion already have patches, they just don't get the needed
attention from core devs. Which is of course understandable as Django
is a volunteer project. But the goal of fixing *all* bugs for 1.0.1 is
not realistic for exactly that reason, which is exemplified in
timeline (look at check-ins) -- apart from brosner's good work on
fixing the formset unique foreign key issues, there are only doc,
translation and CSS fixes in last 7 days. In last month, same pattern:
very few core and a lot of doc fixes. (Note that I'm not blaming
anyone here, it's quite understandable that devs are busy outside
Django -- it's just not fair to slap people with the standard "just
contribute patches to all bugs" answer in this light, especially if
they have already done so for bugs that they care for.)

Most other projects are managed by a priority queue and clear target
set for releases ("this has to go into 1.0.1, this can wait until
1.0.2"). No problem if discussions on the mailing list are the
preferred way of doing it instead of Trac tools -- until things really
get done.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: 1.0.1 release blockers?

2008-11-05 Thread mrts

On Nov 5, 1:35 am, "Russell Keith-Magee" <[EMAIL PROTECTED]>
wrote:
> On Tue, Nov 4, 2008 at 11:04 PM, mrts <[EMAIL PROTECTED]> wrote:
>
> > Most other projects are managed by a priority queue and clear target
> > set for releases ("this has to go into 1.0.1, this can wait until
> > 1.0.2"). No problem if discussions on the mailing list are the
> > preferred way of doing it instead of Trac tools -- until things really
> > get done.
>
> "Most other projects" is an ad hominem, so I won't respond to that.

Yeah, that was imprecise and overgeneralized, sorry for that. I had
mainly Python and Debian/Ubuntu in mind, but I believe Gnome and
Firefox have a similar workflow. Python has release blockers (that may
become deferred blockers), Ubuntu and Debian have critical bugs that
are usually fixed before a release. There is a general sense of what
is important and what not in the bug database.

> Let me lay out the alternatives to you. Let us assume that we have a
> milestone, and tickets get assigned to it:
>
> Option 1: We don't release until all tickets on the milestone are complete.

Tickets in a milestone are prioritized. We don't release until all
_critical_ bugs on the milestone are fixed as other projects listed
above do. Final decision on whether something is critical or not is
made by core devs. There is a set timeline with a bit of flex to cater
for delays in getting critical fixes in. Even before minor releases
there is a string freeze so that translators have a known point in
time for contributing translations.

Mailing list discussions are good, but in an ideal world they would be
complemented by a severity tag in Trac to communicate priorities and a
classifier tag to separate bugs and features.

> Post v1.0, our only goal is "zarro boogs", delivered on a timely
> schedule - again, we don't need milestones to keep track of this goal,
> because every open ticket is a target. What we _do_ need is a
> community that works on triage and bug fixes, and draws the attention
> of the core devs to particularly annoying or confusing bugs.

You have that community. 712 out of 1249 open tickets have patches, I
personally usually write a patch if something disturbs me. But as of
now the list is just an unordered soup of trivial fixes, "I want a
pony" and larger issues. I really doubt anyone has a general overview
of all of it. I myself feel apprehensive at the bug mass -- how do I
know that I will not hit into something important hidden there, but
wading through the list to pick important bits apart from feature
requests is quite an effort.

All in all, I like Django itself a lot. The code and conceptual
structure is mostly a pleasure to look at, you all have done an
excellent job and deserve a lot of respect. So in no way do I want to
say that things are bad as they are. But the defensive attitude and
constant fighting (that has gone on for years) on the same issues has
caused a lot of bitterness (e.g. google django hate: a whopping
426,000 results, hopefully no more than a couple of first pages are
relevant *chuckle*) -- the main issue being that people do contribute
but they don't see it going anywhere. This is where Trac helps a
little by a) milestones: assuring that there is a known date when a
given ticket will be looked at b) priorities: assuring that when I
report something important it will not get buried under a ton of "I
want a pony" requests, also for effectively finding bits that need
attention and may disturb project development. A good process does not
work against people and even tries to take their psychology into
account, no?

Maybe I'm too outspoken as James suggests and I see some value in the
mailing list process, it is just that Trac would support it.

Over and out on this theme until it pops up again,
MS
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: 1.0.1 release blockers?

2008-11-05 Thread mrts

On Nov 5, 2:31 pm, mrts <[EMAIL PROTECTED]> wrote:
> little by a) milestones: assuring that there is a known date when a
> given ticket will be looked at

Let me clarify a little: "looked at" doesn't necessarily mean
"integrated", it may be deferred to the next or a future milestone,
i.e. it does not mean that core devs are somehow forced to give more
input to the project.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Multi-Threaded Dev Server

2008-11-18 Thread mrts

On Nov 17, 6:54 pm, Ludvig Ericson <[EMAIL PROTECTED]> wrote:
> On Nov 16, 2008, at 07:26, Chris wrote:
>
> > 3. Fear of multi-threading bugs shouldn't be a reason to avoid multi-
> > threading, especially when it could be very useful. We don't even know
> > if there are multi-threading bugs. And even if there are, they can be
> > fixed.
>
> There are bugs. Django isn't thread-safe, and we know that.

Which bugs in particular? By all means, if you find one, report it. I
have run Django with mod_wsgi threaded daemon mode with no ill effects
for a long time. See 
http://code.djangoproject.com/wiki/DjangoSpecifications/Core/Threading
, all bugs found during the review were fixed a long time ago (and
only one of them, #6950, was serious-ish).
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Explicit default managers?

2008-11-20 Thread mrts

Currently, "the first manager declared is the default manager".

However, given the following hierarchy:

---
class PublishedObjectManager(models.Manager):
def get_query_set(self):
return super(PublishedObjectManager, self).get_query_set()\
.filter(is_published=True)

class PublicationBase(models.Model):
...
objects = models.Manager()
published_objects = PublishedObjectManager()

class Meta:
abstract = True

class ToplevelPageManager(PublishedObjectManager):
def get_query_set(self):
return super(ToplevelPageManager, self).get_query_set()\
.filter(parent=None)

class PageBase(PublicationBase):
...
toplevel_pages = ToplevelPageManager()

class Meta:
abstract = True
---

all classes inheriting from `PageBase` get `toplevel_pages` as their
default manager.
To make `objects` default, repetition is necessary:

---
class PageBase(PublicationBase):
...
objects = models.Manager()
toplevel_pages = ToplevelPageManager()

class Meta:
abstract = True
---

only to specify the default manager. IMHO, a way to explicitly say
"this is the default manager that should be honoured throughout the
inheritance hierarchy" would be appropriate. Also, the '_' in front of
_default_manager implies that it is an implementation detail and
accessing default managers is not documented. But all reusable app
writers should really use model._default_manager instead of
model.objects for any foreign models they must handle.

Generally, there should be an explicit, documented way to set and get
the default manager.

Thoughts?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Explicit default managers?

2008-11-24 Thread mrts

http://code.djangoproject.com/ticket/9676 created to track this.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Dropping Python 2.3 compatibility for Django 1.1

2008-11-26 Thread mrts

On Nov 26, 3:20 am, Julien Phalip <[EMAIL PROTECTED]> wrote:
> On Nov 26, 11:43 am, "Russell Keith-Magee" <[EMAIL PROTECTED]>
> wrote:
>
>
> > On Wed, Nov 26, 2008 at 2:08 AM, Jacob Kaplan-Moss
>
> > <[EMAIL PROTECTED]> wrote:
>
> > > Hi folks --
>
> > > I'd like to officially drop Python 2.3 support in Django 1.1. Discuss.
>
> > I'm going to be the stick in the mud and say -0.
>
> > I don't have any particular love of or need for Python 2.3, but it has
> > taken us a lot of effort to get and maintain Python 2.3 compatibility.
> > I know maintaining this support is a pain, but in the grand scheme of
> > things it doesn't bite us that often.
>
> > I know the GIS stuff is bound to 2.4+, but other than this, is there
> > any particularly compelling reason to drop 2.3 support other than the
> > annoyance factor for 1.1? I'm just not convinced that the first minor
> > release after a major 1.0 release is the right time to do it.
>
> > Russ %-)
>
> Maybe the best approach would be to warn people one or two releases in
> advance. For example: "Python 2.3 support will be dropped in Django
> 1.3, so be warned and get ready for it."

Django 1.0.X is a solid base that everybody who still uses Python 2.3
can
rely on.

Django 1.1 should drop Python 2.3 support for the following reasons
not
mentioned or elaborated above:

 * Python 2.3 is officially not supported by Python developers since
2.3.5; it
   doesn't even receive security patches -- so, effectively, everybody
should
   avoid using it (the same is true for 2.4, 2.4.5 is supposedly the
last release
   in the series). It doesn't make sense to support something that is
   deprecated upstream.

 * as opposed to decorators that are just syntactic sugar, generator
   expressions provide a way to avoid using list comprehension (and
   thus building the full list where it is actually not needed)
throughout.
   Considerable memory savings are possible by using the former,
   see PEP 289.

 * there are many minor things, e.g. rsplit and key in cmp, that make
code
   considerably more efficient. For the quite common idiom
   "extract the last chunk from string separated by some separator":

>>> from django.contrib.webdesign.lorem_ipsum import words
>>> WORDS = words(10) # for larger strings the gain is more dramatic
>>> timeit.timeit('WORDS.rsplit(" ", 1)[-1]', 'from __main__ import WORDS')
0.84617710113525391
>>> timeit.timeit('WORDS.split(" ")[-1]', 'from __main__ import WORDS')
1.7152390480041504

   Also, urandom, getrandbits, and threading.local.

---

I'd like to see someone stand up and declare "I'm *planning* to use
Python 2.3
for my next large scale project that will be based on the upcoming
Django 1.1 because of X and therefore I'd like to see 2.3 supported".

If nobody steps up with the rationale X (apart from "because we can"),
I really don't see why 2.3 should be dragged along and keep hindering
efficiency.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Making Django 1.0 work well under Google App Engine

2008-12-05 Thread mrts

On Aug 11, 9:11 pm, Guido van Rossum <[EMAIL PROTECTED]> wrote:
> I'd write more but I'll first wait for responses, file some tickets,
> and think about what else Django could do to improve its cooperation
> with App Engine... because Django is App Engine's favorite framework!

Shouldn't easier integration with App Engine be also on 1.1 roadmap? I
did get the impression that adding the machinery to support
google.appengine.ext.db backend was on Malcolm's TODO list
(which would make models and admin work out of the box AFAICS), but
there's no corresponding ticket in trac. It would make sense to have a
page
that lists what's working and what's not with a list of corresponding
issues in Django trac.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Default manager

2008-12-15 Thread mrts

See also http://code.djangoproject.com/ticket/9676

On Dec 15, 12:51 pm, "Alex Rades"  wrote:
> Hi,
> my understanding about custom managers is that if you want to define a
> custom manager, you also HAVE to remember to define the "objects"
> manager first, otherwise some parts of django (eg. admin) will not
> work.
>
> I find this suboptimal.
>
> I think that "objects" should always be the default manager, always
> present and working. If you want to define a new one, this should not
> interfere with "objects" being present and working. Of course you can
> overwrite "objects" if you need to (eg. when you want to customize the
> initial queryset).
>
> The downside of this solution is that you cannot use a field named
> "objects", but this is a feature for me, as it improves readability.
>
> What do you think?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Some tickets that should perhaps get looked at before 1.1

2009-01-06 Thread mrts

As the list seems to be resuming from holiday hibernation, I risk
causing another "don't you dare to push us" flame-bombing :) by
proposing that the following get looked at before 1.1:

* `__import__(mod, {}, {}, [''])` causes double import of modules [1]

After discussions on python-dev, an idiom emerged for a clean fix.
Patch updated recently, Malcom had some concerns before, perhaps they
should be discussed further here. Backwards-compatible, in scope for
1.0.X and 1.1.

* Wrap help_text in "" [2]

Trivial patch, easy to review and commit. In scope for 1.1, but breaks
backwards-compatibility a bit. General consensus seems to be that this
is needed (as per JKM).

* Cache miss (locmem cache backend is not thread-safe) [3]

Again, the problem is clear and patch is trivial. In scope for 1.0.X
and 1.1.

* Formsets with data that violates unique constraints across forms are
not marked invalid [4]

Follow-up to http://code.djangoproject.com/ticket/8882 . In scope for
1.0.X, hopefully model validation will address this and similar issues
in 1.1.

http://code.djangoproject.com/ticket/8193
http://code.djangoproject.com/ticket/8426
http://code.djangoproject.com/ticket/9644
http://code.djangoproject.com/ticket/9493

Maybe we should arrange an online sprint ASAP, e.g. I myself am able
to contribute time for fixing other things that need attention as
well, but coordinating the effort would be nice.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Ticket 8764 (Mixing args and **kwargs in reverse() function)

2009-01-07 Thread mrts

On Jan 7, 3:43 am, Malcolm Tredinnick 
wrote:
> On Tue, 2009-01-06 at 15:38 -0800, Killarny wrote:
> > There are many instances where, in a complicated implementation of
> > views, one might want to have a combination of required args and
> > optional kwargs, and the inability to mix them introduces all sorts of
> > complexities to the logic of the views that shouldn't have to be dealt
> > with.
>
> I'll disagree with this. Whilst it's easy when one is faced with a
> particular problem to imagine that it must be a common case, in reality
> there aren't really that many instances where mixing is required (in
> fact, I can't think of *any* where it could be compulsory -- it's purely
> an alternative representation, so rewrites are always possible). There
> are cases where it can be used, as you witness, but it's not *required*.

E.g. Werkzeug Routes takes this further and handles *only* kwargs.
Less code, less complexity, less bugs, no problems whatsoever.

It goes as follows (simplified look at request-resolve-response
cycle):

def __call__(self, environ, start_response):
resolver = self.url_map.bind_to_environ(environ)
view, kwargs = resolver.match()
response = view(request, **kwargs)
return response(environ, start_response)

+1 to current behaviour or dropping supporting positional args tuple
passing altogether to get rid of the complexity.

Kilarny, which argument handling problems you have remain unsolved in
following examples?

>>> def foo(a, b, c=None): # 2 required and 1 optional arg
... pass
...
>>> foo(**{'a' : 1, 'b' : 2})
>>> foo(**{'a' : 1 })
Traceback (most recent call last):
  File "", line 1, in 
TypeError: foo() takes at least 2 non-keyword arguments (1 given)
>>> foo(**{'a' : 1, 'd' : 3 })
Traceback (most recent call last):
  File "", line 1, in 
TypeError: foo() got an unexpected keyword argument 'd'

>>> def foo(a, b, **kwargs): # 2 required and any optional args
... pass
...
>>> foo(**{'a' : 1, 'b' : 1, 'd' : 3 })
>>> foo(**{'a' : 1, 'd' : 3 })
Traceback (most recent call last):
  File "", line 1, in 
TypeError: foo() takes exactly 2 non-keyword arguments (1 given)
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Some tickets that should perhaps get looked at before 1.1

2009-01-07 Thread mrts

On Jan 7, 11:56 am, "Alex Gaynor"  wrote:
> I'd like to remind everyone that the policy for bugs is to fix them, they
> don't all need to be listed, because the goal is to fix them all.

Let me remind everybody that 1.1 is officially only 8 days away. The
goal is clearly not achievable in that time :), so a selection has to
be made. I'd say bringing up trivial, non-controversial, fully
documented and tested bugfixes like I and Tai Lee did, helps in that
selection and makes 1.1 a better release. The total ticket mass is
just unmanageable and picking ripe fruits for the devs to review and
commit should assist, not hinder in achieving that goal.

---

Which brings me to a related matter: as there have been no news about
Honza and his progress on model validation, I'll semi-champion it (at
least until he pops up again OR a core dev steps up to take it upon
him/herself OR someone opposes this). I've created a fork of the
unofficial git mirror for that purpose at 
http://github.com/mrts/django/tree/master
. Everybody is most welcome to contribute.

By semi-champion I mean that I'll contribute as much as I can beside
daily job tasks by directly writing code, hanging in #django-dev to
discuss things with people interested in helping out and merging
other's patches and upstream changes ASAP. Unfortunately I can not
warrant that this will be completed it in the short time remaining.
Design decisions will eventually appear in the wiki: 
http://wiki.github.com/mrts/django

As of now, I've only merged the latest patch by Honza, so the branch
should be rather unusable (trunk has changed considerably in some
areas -- neither did the patch apply cleanly nor was it always
applicable where it did).
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Some tickets that should perhaps get looked at before 1.1

2009-01-07 Thread mrts

On Jan 7, 9:59 pm, "Alex Gaynor"  wrote:
> 1.1 is not 8 days away, the 1.1 major feature freeze, this means items that
> are new features on the 1.1 features list.

/me puts on a brown paper bag. Right, feature freeze and bug fixing
are different things.

Can we perhaps agree on a set time when people can start shouting
"look at ticket x" (preferably in conjunction with a sprint)?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Distributed workflow and the woes of slow testsuite

2009-01-12 Thread mrts

A proper "agile" workflow for distributed collaboration on a largeish
chunk of functionality should be as follows:

Feature developer:
* implement tests for the features to be added
* commit locally
* implement the features
* commit locally
* run the test suite iteratively, fixing whatever the tests indicate
needs fixing
* commit locally
* push your changes to GitHub or whatever other public code hosting
service is used for collaboration

Lieutenant:
* review changes, expecting that the tests pass
* pull changes from feature developer's repo to the "central" repo
* run the test suite
* push changes to the public code hosting service

Nice and clean -- in an ideal world. In real life, the humongous time
wasted on running the test suite causes a tiny impulse of frustration
to be transmitted between the neurons in the developer's brain on
every line where you read "run the test suite".

Let me bet -- every single Django developer, who runs the test suite
regularly, has felt discomfort thinking of the time he has to wait for
the suite to complete. Which means slight aversion regards running the
tests, which in turn, brings about that tests are run as a last resort
("doh, I have to run the tests"), not as the happy default path ("yay,
time to run the tests!").

What if we try to be nice to ourselves and get #8138 and something in
the lines of http://oebfare.com/blog/2008/mar/25/faster-django-test-suite/
into trunk ASAP, especially now that the dynamic workflow is really
gaining ground (both aggregates and model-validation is managed this
way)?

(Lessons learned while collaborating on model-validation with Honza.
Given a dual-core T5500 CPU, 2 GB RAM, Ubuntu and SQLite on a laptop,
the full test suite takes 1346.841s (22 minutes!) to complete on the
trunk.)

"Work with People’s Instincts, Not Against Them"-ly yours,
Mart Sõmermaa

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Distributed workflow and the woes of slow testsuite

2009-01-12 Thread mrts



On Jan 12, 3:45 pm, "Jacob Kaplan-Moss" 
wrote:
> On Mon, Jan 12, 2009 at 6:53 AM, mrts  wrote:
> > What if we try to be nice to ourselves and get #8138 and something in
> > the lines ofhttp://oebfare.com/blog/2008/mar/25/faster-django-test-suite/
> > into trunk ASAP
>
> #8138 is nearly done; there are still a couple of tricky issues to
> work out first. Those who've been paying attention to the ticket know
> this.
>
> Running a multiprocess/multithreaded version of the tests is a
> trickier issue: while in theory tests ought to be isolated from each
> other, at times subtle bugs only show up when the whole test suite is
> run en masse. So while I think the way Brian's going, there's never
> going to be a substitute for simply running the whole test suite.

As of now, I'll stop pursuing this further, but when I come back to
this,
would the approach outlined in http://dpaste.com/108140/ be
acceptable?

I.e. if
1) not specified explicitly (by the --multiprocessing option to test)
2) there's only a single core in the system
3) importing the multiprocessing module fails
run the ordinary single-process test runner.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Testing speedup checked in

2009-01-17 Thread mrts

Great thanks and praises to everybody involved!

Getting down from 1346 to 204 seconds is such a relief...
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Overview of the "GitHub process"

2009-01-17 Thread mrts

Feeling that an overview of the "GitHub process" would help people to
get more easily involved, I created 
http://code.djangoproject.com/wiki/CollaborateOnGithub
.

Opinions, criticisms and edits most welcome.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Model-validation: call for discussions

2009-01-17 Thread mrts

There are several problems in model validation as of now. As pouring
them out here would create a too long, ill-formatted post, I created a
separate page for it at

http://wiki.github.com/mrts/honza-django/form-and-model-validation

This is just "design gruntwork", a basic text body analyzing the
relations between components and problems and offering the most
evident but not necessarily most elegant solutions. I hope to spark a
proper discussion here that hopefully converges to a solid design.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Model-validation: call for discussions

2009-01-19 Thread mrts
n process have already been run.

I've hacked up an incomplete rough draft at
http://dpaste.com/110671/ to illustrate this. Unfortunately it does
not illustrate all the points, but the general idea should
hopefully come through.

Action plan for 1.1
===

The mixin approach is not in scope for 1.1. We go with what we
already have, factoring the bits in current model fields' to_python
to a separate django.utils.typeconverters library
(see 
http://github.com/mrts/honza-django/commit/a8239b063591acc367add0a01785181a91a37971
for the first steps in that direction) and using both
django.core.validators and django.utils.typeconverters throughout
model and form fields.

I'm not sure about Honza's plans in regard avoiding duplicate
validation, hopefully he comments on that himself.

Also, let me remind that model and form objects have not been
discussed in this post (and it's already more than 150 lines long),
only fields.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Model-validation: call for discussions

2009-01-19 Thread mrts

On Jan 19, 1:43 pm, Malcolm Tredinnick 
wrote:
> one. Short version: when the form field validation would match what the
> model field is going to do anyway, don't do anything at the form level.
> The model field validation is about to be called anyway.

[snip]

> The solution here might not be too difficult and doesn't need the
> requirement of communication from the form to the model about what it's
> already done.
>
> To wit, when a formfield is constructed automatically from the model
> field, if the default formfield doesn't do any validation/normalisation
> beyond what the normal model field does, the formfield() method on the
> Field subclass returns a formfield with no validators. Thus, when the
> form cleaning process is run, all the validation for that field is done
> by the model. Thus form fields returned for model fields could well be
> slightly modified (in that their validator list is different) from what
> you get directly from using forms.fields.*.
>
> That dovetails nicely with the reuqirements of custom form fields. When
> a custom form field is created, the form field validation won't be
> skipped (unless the creator wants to remove any validation from the form
> field), so any differences between the default model field validation
> and the custom form field validation aren't overlooked: they both get
> run (which is the correct behaviour).

Looks like a good action plan.

> Attempts to abstract away similar-but-not-the-same algorithms can often
> lead to more difficult code than writing the same thing out directly in
> the two places. It tends to show up in the number of parameters you pass
> into the abstract algorithm and/or the complexity of the return types.
> And, in this case, it's only two places, not fourteen or anything like
> that. So whether this is worth it tends to be implementation specific.

Yeah, I won't pursue this further, although I still think the result
would be cleaner, not uglier.

> As a style issue, there's a bit of an arbitrary split going on there.
> Type conversion is really part of what we're calling "validation" (since
> the type coercion is part of the normalisation part of the process,
> which we tend to lump into validation). I'd lump things a bit closer
> together in the source. There's not really an external use-case scenario
> for the converters and, even within Django, they don't have a lot of
> user outside of the validators (we can't do a lot of sharing with the
> database-specific backend converters, since they do tend to be
> backend-specific).
>
> Maybe (and looking at my notes, I might upgrade that to probably, given
> the number of times I've mentioned it), we need to make a directory for
> django.core.validators instead of a single file. That gives us room to
> split out converters from things that work with standard types. But
> definitely try to keep them close together in the source, rather than
> putting them in django.utils, which can sometimes be a dumping ground
> for stuff that better belongs elsewhere.

As seen in the commit,
http://github.com/mrts/honza-django/commit/a8239b063591acc367add0a01785181a91a37971
,
my initial quick solution  was to lump them into the validation
module (as there's lot's of intermingling between validators and
coercers,
exemplified by merging to_pyton() and validate() to clean() in forms).

The directory-based approach is best, I'll go with it -- but it's yet
uncertain
when as I have to handle pressing matters at work during daytime.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Model-validation: call for discussions

2009-01-22 Thread mrts

On Jan 19, 11:23 pm, mrts  wrote:
> The directory-based approach is best, I'll go with it -- but it's yet
> uncertain
> when as I have to handle pressing matters at work during daytime.

I've implemented some fundamental changes that need review. The commit
is at 
http://github.com/mrts/honza-django/commit/482086df8d24b99f466152c51d2badee6ee6147d
. The changes are not finished yet, but they should illustrate the
proposed approach. I may fail to see all the negative implications as
I've been mostly addressing fields, not the whole picture. Critique
most welcome.

Changes:

1. consolidate validators and coercers into django/core/validators/
{__init__,typeconverters}.py as suggested by Malcolm

2. make both forms and fields use similar logic in clean():

core.validators:

def clean(field_instance, value, all_values,
field_container_instance):
if value is not None:
value = field_instance.to_python(value)
field_instance.validate(value, all_values,
field_container_instance)
return value

models.fields:

class Field(object):
...
def clean(self, value, model_instance):
if model_instance is not None:
all_values = model_instance.__dict__.copy()
else:
all_values = {}
return validators.clean(self, value, all_values,
model_instance)

forms.fields:

class Field(object):
def clean(self, value, form_instance=None):
if form_instance is not None:
all_values = form_instance.cleaned_data
else:
all_values = {}
return validators.clean(self, value, all_values,
form_instance)

Rationale: make validators that need to use other fields (e.g.
RequiredIfOtherFieldEquals) work uniformly on model and form fields. A
side note: the `instance` attribute is not used in validator functions
and I can't see a clear use case for it, so it looks like it can be
removed -- prove me wrong please (I do see obscure corner cases where
it could be useful -- if one needs to distinguish between forms and
models in custom validators or do gettatr('something', instance), but
these should really be handled in clean() manually).

3. as seen from above, to_python(), validate() and validators are back
in form fields as this greatly simplifies code. clean() is still
overridable and mostly backwards-compatible. There are a few fields
that have been ported to the new logic in forms/fields.py. Form fields
accept custom validators in `validators` attribute.

4. typeconverters take additional params, mainly intended for custom/
localized date/time input handling, but also useful when overriding
to_python in custom fields:

DateField:

def to_python(self, value, params=None):
 typeconverters.to_date(value, params)

def to_date(value, params=None):
# TODO: use params for date format

This is not yet fully implemented in the current changeset.

Unfortunately I've been unable to contribute as much time as I would
have liked (the fact that the commit is from 2.30 AM my time should
say it all), but I hope to drive field validation to completion soon
after receiving feedback on the decisions above.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Model-validation: call for discussions

2009-01-22 Thread mrts

On Jan 23, 3:40 am, Malcolm Tredinnick 
wrote:
> On Thu, 2009-01-22 at 17:27 -0800, mrts wrote:
>
> []
>
> >  A
> > side note: the `instance` attribute is not used in validator functions
> > and I can't see a clear use case for it, so it looks like it can be
> > removed -- prove me wrong please (I do see obscure corner cases where
> > it could be useful -- if one needs to distinguish between forms and
> > models in custom validators or do gettatr('something', instance), but
> > these should really be handled in clean() manually).
>
> For models, the "instance" will the models "self" attribute. So that one
> can do checks based on other information available on the model
> instance. It's kind of the whole point behind the "model-aware" portion
> of model-aware validation.
>
> Asking that everything like that gets pushed to clean() is being
> awkward. clean() is for multi-field validation, the individual
> validators are for single field validation. That doesn't mean the latter
> cannot use other information available on the form.

As can be seen from the above, fields are already passed to validators
via all_values = model_instance.__dict__.copy() for multi-field
validation. But I agree that requiring to override clear() for
anything more advanced is too restrictive, so let the instance be part
of the signature.

> I'll try and make time to look at this, along with other recent work in
> this area, over the weekend.

That would be most welcome, perhaps you can pop by #django-dev for
more rapid idea exchange?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Model-validation: call for discussions

2009-01-23 Thread mrts

As the uniform all values approach has created a bit of confusion, let
me present a larger example:

Validators in core.validators should not be concerned with either
model or form internals if possible. This is currently straightforward
to achieve by passing all values always as a dict via
form.cleaned_data or model.__dict__.copy() (dict copying is cheap, the
original instance dict is unsafe to pass around).

This makes multi-field validators work uniformly for both models and
fields. Otherwise you have to always handle identical validation
differently for  forms and models -- this is extremely smelly in my
opinion.

And now the example:

class RequiredIfOtherFieldEquals(object):
   def __call__(self, value, all_values={}, instance=None):
   if self.other_field in all_values and all_values
[self.other_field] == self.other_value and not value:
   raise ValidationError(self.error_message, REQUIRED)

works both for forms and models with the proposed approach (all_values
is model_instance.__dict__.copy() in model field validation case).

Handling this specially for model instances would be really ugly:

   def __call__(self, value, all_values={}, instance=None):
   if instance is not None:
   if hasattr(self.other_field, instance) and getattr
(self.other_field, instance) == self.other_value and not value:
   raise ValidationError(self.error_message, REQUIRED)
   elif self.other_field in all_values and all_values
[self.other_field] == self.other_value and not value:
   raise ValidationError(self.error_message, REQUIRED)

and entirely unnecessary under the all_values approach.

Everybody is most welcome to prove me wrong :)
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Model-validation: call for discussions

2009-01-23 Thread mrts

After discussing with Honza, we agreed that the dichotomy between
forms and models that was present before will remain, i.e. instance
will always be a model instance if given and all_values will always be
form.cleaned_data.

Honza's rationale was that it's common to have properties in models
and therefore model_instance.__dict__ (or forms.models.model_to_dict)
may be too constraining. Thus, a generic value getter will be used in
validators to provide uniform access to dict values and instance
fields as follows:

def _get_value(name, instance, all_values):
if instance is not None and hasattr(instance, name):
return getattr(instance, name)
if name in all_values:
return all_values[name]
raise AttributeError('%s not found in forms values or model
instance')

class RequiredIfOtherFieldEquals(object):

def __call__(self, value, all_values={}, instance=None):
if _get_value(self.other_field, instance, all_values) ==
self.other_value and not value:
raise ValidationError(self.error_message, REQUIRED)
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Model-validation: call for discussions

2009-01-23 Thread mrts

On Jan 23, 4:38 pm, Marty Alchin  wrote:
> I haven't been following everything, but I do have a couple comments
> to make here.

Thanks, interesting points. The get_value approach looks simpler
though, so unless you or anybody else disagrees I'll implement this.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Model-validation: call for discussions

2009-01-24 Thread mrts

After several discussions with Honza, we are still on somewhat
different positions what the validator function signature should
be and how core validators should access the fields of a form or
a model instance.

In core validators, no special case handling of forms and models
is needed even in multi-value validators. All what is needed is
an abstraction of "the set of other values".
AlwaysMatchesOtherField, RequiredIfOtherFieldDoesNotEqual etc
will work with that abstraction, primary keys and other complex
model fields also work for same-type comparison.

My initial idea was to reduce the model instance to a dict (as
discussed previously), however, this is really too restrictive.
Passing the form or model instance and treating it in dict-like
manner provides much more flexibility.

That can be achieved by the following:

class Model(...):
...
def get_value(self, field_name, *args):
"Default is specified with the first positional argument"
if args:
return getattr(self, field_name, args[0])
return getattr(self, field_name)

class BaseForm(...)
...
def get_value(self, field_name, *args):
"Default is specified with the first positional argument"
if args:
return self.cleaned_data.get(field_name, args[0])
return self.cleaned_data[field_name]

(It's not possible to implement the __getitem__ protocol as it
already works differently in forms.)

In this case the form/model dichotomy is not required in
validator signature, simple validators work as follows:

def is_slug(value, instance=None):
(access only value)

and multi-value validators as follows:

class AlwaysMatchesOtherField(object):
...
def __call__(self, value, instance=None):
if value != instance.get_value(self.other):
raise ValidationError(...)

There are currently no model-specific complex validators. If a
need for them should arise, it's probably easiest to override
validate() in the corresponding class, calling super().validate()
and then perform any additional checks there.

Custom validators can either use the get_value() abstraction to
support both models and forms or be model/form-specific and use
the instance in whatever way they like. Checking whether you deal
with a model or form instance is generally not required -- if you
accidentally pass a custom form validator to a model, it will
visibly break and vice-versa.

Honza doesn't like this, mainly because it's too easy to shoot
yourself into foot with this (as outlined above, I disagree with
that) and as of now we are using the following instead to handle
model and form fields uniformly:

def _get_value(name, all_values, instance, do_rise=True):
if instance is not None and hasattr(instance, name):
return getattr(instance, name)
if name in all_values:
return all_values[name]
if do_raise:
raise AttributeError('%s not found in form values or model
instance'
% name)
return False

class AlwaysMatchesOtherField(object):
...
def __call__(self, value, all_values={}, instance=None):
if value != _get_value(self.other, all_values, instance):
raise ValidationError(...)

Honza's comments:
-

The problem is that I do not want Form class to be passed to
validators for several reasons:

Form shouldn't be anything except something that obtains data
from request and can validate them, passing it validators could
promote adding some methods and properties to be used in
validators, instead of in the form's clean() method.

Form and model treat data differently, that's just the way it is
and as it should be - pretending it's not true will  just create
opportunity for nasty surprises.

Validators wouldn't work outside forms and models (it's easy to
pass in a dictionary of all_values, but fabricating a class with
the same method is work).

I would still have to check whether I am dealing with model or
form when I would like to access some model's methods and
properties that cannot be retrieved this way.

I have more reasons, but these are the major ones, if it's not
enough, contact me and we can talk ;).

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



App settings

2009-02-02 Thread mrts

Whenever the app objects discussion is resuscitated, the following by
yours truly may be of relevance:

http://github.com/mrts/plugit/tree/master

As of now, settings handling is present. The SettingsUpdater class in
plugit/settingshandler.py provides a high-level API for updating
configuration files with Django settings.py format. The Django-
specific use case (the project is not Django-related generally) is as
follows.

---

Suppose the app 'foo' needs to be enabled. During enabling, the app
wants to register it's middleware and add a custom setting, 'FOO =
42'.

$ cat settings.py
MIDDLEWARE_CLASSES = (
'django.middleware.common.CommonMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
)

INSTALLED_APPS = (
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
)

# imagine that user calls ./manage.py enable foo, among other things
the following happens:

$ python
Python 2.5.2 (r252:60911, Oct  5 2008, 19:29:17)
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from plugit.settingshandler import SettingsUpdater
>>> updater = SettingsUpdater('settings.py')
>>> new_settings = {'FOO': 42}
>>> append_settings = {'MIDDLEWARE_CLASSES': 'foo.middleware.FooMiddleware',
... 'INSTALLED_APPS': 'foo'}
>>> updater.update(new_settings, append_settings)
>>> updater.save()
True

# and the result:

$ cat settings.py
MIDDLEWARE_CLASSES = (
'django.middleware.common.CommonMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'foo.middleware.FooMiddleware',
)

INSTALLED_APPS = (
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'foo',
)
FOO = 42

---

Currently it's just a proof of concept and a little foray into lib2to3
capabilities. Without careful dependency handling it's not much
useful.

So, this is one way of handling the app installation and corresponding
settings update problem. Hopefully this demonstrates that settings
updates are feasible and robust and perhaps sparks ideas that are
directly useful in Django context.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Model-validation: call for discussions

2009-02-18 Thread mrts

The last unsolved model-validation design issue is the error message
protocol (my work on fields is waiting on it). Let me present the
approach that looks sane to me.

The old Field.default_error_messages dict and corresponding
logic is no longer required (and removed in my branch) as default
error messages live in core/validators now.

In field construction, the custom error logic is plain:

   CharFied.__init__(self, ..., error_messages={}):
   ...
   self.error_messages = error_messages

And it is used as follows:

class Form(...):
   ...
   def full_clean(self):
   ...
   for name, field in self.fields.items():
   try:
   ...
   # clean() calls to_python() and validate(),
   # raising ValidationErrors
   value = field.clean(value)
   ...
   except ValidationError, e:
   e.custom_messages = field.error_messages
   self._errors[name] = self.error_class(e.messages)

The e.custom_messages = field.error_messages binds custom { code:
message } overrides to the ValidationError.
e.messages is a property that returns the list of error messages.

In reality, ValidationError contains only a single error message. The
list is required for aggregating several errors from the validator
collection associated with a field. However, as of now, the
ValidationError class tries to be both a single value and a
collection, resulting in quite a mess.

This is a classic case of the Composite pattern (http://
en.wikipedia.org/wiki/Composite_pattern ), so I propose that
 * ValidationErrors remain single values
 * core/validators gets an additional ValidationErrorComposite class
that derives from ValidationError and has the same interface

It will be used as follows (example for form field, model field
similar):

class Field(...):
   ...
   def clean(self, value, all_values={}):
   if value is not None:
   value = self.to_python(value)
   self.validate(value, all_values)
   return value

   def validate(self, value, all_values={}, form_instance=None):
   value_is_empty = value in validators.EMPTY_VALUES
   if self.required and value_is_empty:
   raise ValidationError(_("This field is required."),
   validators.REQUIRED)

   error_collection = ValidationErrorComposite()

   for validator in self.validators:
   if value_is_empty and not getattr(validator, 'always_test',
False):
   continue
   try:
   validator(value, all_values)
   except ValidationError, e:
   error_collection.add(e)

   if error_collection:
   raise error_collection

User code can remain oblivious of the composite, catching
ValidationErrors as before and using them as specified.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Model-validation: call for discussions

2009-02-18 Thread mrts

On Feb 18, 2:28 pm, mrts  wrote:
>    def validate(self, value, all_values={}, form_instance=None):

That should have been

def validate(self, value, all_values={}):
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Model-validation: call for discussions

2009-02-18 Thread mrts

On Feb 18, 8:03 pm, Honza Král  wrote:
> Hi,
> see inline text.
>
> On Wed, Feb 18, 2009 at 1:28 PM, mrts  wrote:
>
> > The last unsolved model-validation design issue is the error message
> > protocol (my work on fields is waiting on it). Let me present the
> > approach that looks sane to me.
>
> > The old Field.default_error_messages dict and corresponding
> > logic is no longer required (and removed in my branch) as default
> > error messages live in core/validators now.
>
> What if I want to use the same error_messages on multiple instances of
> a given field?
>
> Now I would subclass the FormField and override it's default_messages,
> with your approach it would require overriding __init__. I am not
> saying that that is a bad thing, just that it still has a reason.

Yup, design decision needed:

is the need for class-wide overrides of error messages common enough
to justify leaving it as-is (considering that this can still be done
via __init__)?


> Besides not all validation will be done inside validators, some will
> remain on the Field classes.

As of now, I've managed to weed out all validation from fields into
validators. The result is not entirely stable yet and it brings in
minor backwards-compatibility issues, so it remains to be seen if this
is entirely viable.

> > In field construction, the custom error logic is plain:
>
> >   CharFied.__init__(self, ..., error_messages={}):
> >       ...
> >       self.error_messages = error_messages
>
> > And it is used as follows:
>
> > class Form(...):
> >   ...
> >   def full_clean(self):
> >       ...
> >       for name, field in self.fields.items():
> >           try:
> >               ...
> >               # clean() calls to_python() and validate(),
> >               # raising ValidationErrors
> >               value = field.clean(value)
> >               ...
> >           except ValidationError, e:
> >               e.custom_messages = field.error_messages
> >               self._errors[name] = self.error_class(e.messages)
>
> > The e.custom_messages = field.error_messages binds custom { code:
> > message } overrides to the ValidationError.
> > e.messages is a property that returns the list of error messages.
>
> > In reality, ValidationError contains only a single error message. The
> > list is required for aggregating several errors from the validator
> > collection associated with a field. However, as of now, the
> > ValidationError class tries to be both a single value and a
> > collection, resulting in quite a mess.
>
> > This is a classic case of the Composite pattern (http://
> > en.wikipedia.org/wiki/Composite_pattern ), so I propose that
> >  * ValidationErrors remain single values
> >  * core/validators gets an additional ValidationErrorComposite class
> > that derives from ValidationError and has the same interface
>
> What if then validators want to raise multiple messages? Of FormFields
> for that matter?
> for example:
>
> "Must be greated than 1000."
> "If field Y is supplied, X must be greater than Y."
> "I don't like your shoes!"

In core/validators there are none that need to raise multiple error
messages.

> Would they then raise ValidationErrorComposite instead? So you just
> pushed the check to a different layer. I agree that it makes more
> sense in it's new location, but the distinction still has to be made.

Yes, if that need arises, they can throw a ValidationErrorComposite.

> And how would ValidationErrorComposite.add(exception) work then? It
> would have to check the exception type to know whether to append or
> extend it's message list.

I see no problem there.

def add(e):
if isinstance(e, ValidationErrorComposite):
add the collection of ValidationErrors to self.errors
else:
add the single e to self.errors

> I agree that handling of different datat types (string, list or dict)
> in ValidationError isn't pretty, but it IMHO greatly simplifies it's
> usage and handling.

Let's just leave it to core devs to decide.

> > It will be used as follows (example for form field, model field
> > similar):
>
> > class Field(...):
> >   ...
> >   def clean(self, value, all_values={}):
> >       if value is not None:
> >           value = self.to_python(value)
> >       self.validate(value, all_values)
> >       return value
>
> >   def validate(self, value, all_values={}, form_instance=None):
>
> unrelated to custom messages - you cannot pass all_values to field for
> validation, because at the time

Re: Model-validation: call for discussions

2009-02-19 Thread mrts

On Feb 19, 12:49 am, Malcolm Tredinnick 
wrote:
> On Wed, 2009-02-18 at 04:28 -0800, mrts wrote:
> > The old Field.default_error_messages dict and corresponding
> > logic is no longer required (and removed in my branch) as default
> > error messages live in core/validators now.
>
> It *is* still required. Firstly, it's a very useful way of specifying
> holistic error messages. Secondly, removing it would be backwards
> incompatible in a major way (many custom fields would no longer work
> properly, for example).

Agreed.

> When I get out from under some work stuff this week, I'm going to commit
> what I've been merging (it a merge of yours and Honza's git branches and
> a lot of editing and review changes) into a subversion branch so that
> more people can test it. I'm then happy to pull in and review changes
> from git branches if you guys want to work on it there, but we need to
> have this done in a more general way. We've all dropped the ball a bit
> -- I thought the problems that were creeping in were going to simple to
> fix up in review; I was wrong. It's the broken-windows problem, where
> one little variation from "great" leads to another and another, without
> realising it.

Sounds good. Keep us updated and let know if you need us to work on
something.

Best,
Mart
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Milestones and roadmap cleanup - thanks!

2009-03-08 Thread mrts

Jacob,

thanks for the great work on Trac cleanup! This makes the devs
intentions and goals so much more transparent. I especially like that
we have a somewhat official ticket-set-based roadmap now (http://
code.djangoproject.com/roadmap ) instead of the manual one (http://
code.djangoproject.com/wiki/Version1.1Features ).

Everybody, let's grow the green on http://code.djangoproject.com/milestone/1.1
!

Thanks and praises!
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



  1   2   >