Re: Proposal: JS library in admin and tools

[Andreas]

By the way, rails is going agnostic in 3.0... Says it all.

On Aug 18, 4:06 pm, diogobaeder  wrote:
> Hi there,
>
> After reading this 
> posthttp://groups.google.com/group/django-developers/browse_thread/thread...
> I decided to ask you guys about your opinions in choosing an official
> JavaScript library for Django - what would be the benefits and
> drawbacks of that -.
>
> I mean, Rails has an official JS library (Prototype/script.aculo.us),
> Symfony too (Prototype/script.aculo.us), and both can work with other
> libraries without too much rework.
>
> Well, as an Prototype ex-user, and as a currently jQuery heavy user, I
> much recommend jQuery, because of its ease of use and unobstrusive
> approach - it doesn't clash with other libraries or classes -.
>
> Also, as Django Developer Tools already uses jQuery, it would be great
> to adopt the same library for it and the admin site, if this extension
> gets to the Django trunk.
>
> Diogo
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Proposal: Better HTML4 support

[Andreas]

html4 is derecated.
xhtml is deprecated.


With google chrome frame out there theres absolutly no reason to not
begin with html5 already.
The longer we wait, the longer it will take before html5 becomes "the
it".



On Sep 27, 1:46 pm, Simon Willison  wrote:
> On Sep 27, 10:49 am, veena  wrote:
>
> > my bits to discussion about supporting various (X)HTML versions.
>
> > 1) Problem with (X)HTML in python code (in applications)
> > I discovered this python packagehttp://pypi.python.org/pypi/html/1.6
> > It allows you to write "python like HTML syntax" and generates HTML or
> > XHTML.
>
> Something like that might be an option for cleaning up the way markup
> is generated within the forms framework itself, but ultimately I think
> adding an entirely new Python-based generation method just to output a
> few strings wouldn't be worth the added dependency.
>
> > 2) Problem with (X)HTML in templates
> > I think there should be parser, which parse template just before
> > caching templates. Code could be messy HTML or (XHTML) or invalid HTML
> > (like undisclosed tags, attributes without quotations marks etc.) and
> > from this make pretty HTML or XHTML according to html coder settings
> > by {% doctype %}
>
> The performance overhead rules out this kind of approach for Django
> core - any post-processing run against the output of the templates
> would be executed on every single Django request, and HTML parsing and
> rewriting is a very expensive operation. That's not to say a third
> party module couldn't be written to do this kind of thing for people
> who really want to clean up their output (I remember there being an
> HTMLTidy middleware a few years back that did this) but it wouldn't be
> suitable for inclusion in core.
>
> Cheers,
>
> Simon
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: admin code is really really "not best practice"

[Andreas]

> I think you're misunderstanding. What Robert said was that the code
> that generates admin views is complex and hard to understand. It
> really has no effect on use of Django unless you plan to edit the
> admin-view code for some reason.

Don't you think it is a little worrying that an interface like the one
used in the admin view seems to be only achievable by, as Robert wrote,
"hacking around the framework"? If it is achievable within the limits
of the framwork, without workarounds -- who should be able to figure it
out if not the authors of the framework?

I don't think this is just an issue for those planning to edit the
admin-view code.

Andreas

Debugging Django

[Andreas]

I'd love to help with the Django development. If I encounter a
Traceback in the admin interface, how would you recommend I start
debugging? Running the webserver in Komodo's debug mode fails since
multi-thread debugging is not supported. Is there a way to "simulate" a
request for a page like http://localhost:8000/admin/myproject/tasks/
that allows setting breakpoints?

Re: RFC: django.template refactoring (#7806)

[Andreas]

What really annoys me is all the talk about how important it is that
its designed for non-programmers but if i recall corectly jacob said
on djangocon that they never aimed for making django a general purpose
webframework, they just made something that fit their needs. With that
said, how many django projects actually involves "template designers"?
Such a job position doesnt exists here in sweden. I bet my five cents
that in most cases the programmer also makes the templates with
semantic html, and the css and the js. Most developers today has
competence that spans all over these areas. So shouldnt we make
something that fit our needs instead of making something for someone
else? Else id like to see some "template designers" in this
discussion.

(Of course Its loose coupled and i can use anything i want, which i
actually do)

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Proposal: New transaction API with multiple databases

[Andreas]

I dont know if this has been covered in some of the mentioned previous
multi db support threads but how is it supposed to work with admin?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Proposal: New transaction API with multiple databases

[Andreas]

I guess this just proves there's too many multi db threads and that we
are many who're happy Adrian is making it happening. :)

On Apr 6, 11:04 pm, Alex Gaynor  wrote:
> On Mon, Apr 6, 2009 at 5:04 PM, Alex Gaynor  wrote:
>
> > On Mon, Apr 6, 2009 at 5:00 PM, Andreas  wrote:
>
> >> I dont know if this has been covered in some of the mentioned previous
> >> multi db support threads but how is it supposed to work with admin?
>
> > It seems rather orthagonal to the admin to me(at least as far as it can
> > be).  The Admin executes queries using the Django ORM, so all the hooks that
> > have been proposed can be fully utilised by any queries the admin does.  The
> > admin doesn't need special hooks, other than the queryset method which
> > already exists, and even if it did those are fully outside the scope of my
> > proposal.
>
> > Alex
>
> > --
> > "I disapprove of what you say, but I will defend to the death your right to
> > say it." --Voltaire
> > "The people's good is the highest law."--Cicero
>
> Wow, I'm a fantastic fool, somehow I thought this was my multi-db thread.
> Please ignore my previous message.
>
> Alex
>
> --
> "I disapprove of what you say, but I will defend to the death your right to
> say it." --Voltaire
> "The people's good is the highest law."--Cicero
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Schema Evolution

[Andreas]

I know everyone is fed up with discussions about schema evolution and
multi db support but when it comes to schema evolution I think it's
time to have a discussion about it again.

I think we all can agree that picking a schema evolution app for a
django project isnt one of those things everybody wants to do when
they begin developing with django. And if im not mistaken, it has been
said on conferences and google talks that django is about not making
the developer to do decisions like "which auth framework or template
engine do i need" when they start out. I think schema evolution is one
of those pieces aswell. Of course one shouldnt have to use a django
shipped one.

My suggestion...

I've seen more and more people involved in django core development
either as core developer or submitting patches recommending south
( http://south.aeracode.org/ ) on twitter and people talking about it
on irc. My suggestion is simply to just through south in. Of course I
can see developers of other schema evolution frameworks could become
disappointed with this but Ill think it's in their interrest aswell
getting something shipped with django, and they can help contribute to
make it fit their needs aswell which would make the "default choice"
even better for more people. And if it totaly doesnt fit their needs,
they already have their own stuff to continue with.

I hope the core developers can make a decision that doent include
stepping on a sore toe and I hope they are not afraid and have what it
takes to pick something already built and branded, there's no need to
build a schema evolutotion framework for django.contrib form scratch.

Of course the 1.1 release is to soon for something like this but give
it some thought and a decision could be made before the summer for the
1.2.

My 5 swedish ören because I dont have any cents.


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Schema Evolution

[andreas]

My point wasnt to pick a solution, in this case South and just throw
it into django.
What I mean is, there is no winner nor any loser. I was more looking
for taking it,
and changing it in a way making most the core devs happy, and
hopefully a majority
of the community happy. Just to fork any of all migration apps there
is is good enough,
there's absolutly no reason to start from scratch? Yes I mean fork,
and let it live it's
own life in django.contrib without any constrains about pulling
changes from the original.
It seems like its to difficult to make such a decision, I get the
feeling that the core team is
to afraid to get criticism of picking the wrong one. But as I said,
there is no wrong one.
People with other needs can always choose some of the other
solutions.

There's no way there will migrations in django ever which will fit all
django users in the world,
this is pretty clear after all discussions there have been.

Thats the nice thing about loose coupling.

I don't see why this is a terrible time for discussions, the whole
idea of discussions is to laborate with ideas. I don't see how that
makes it difficult for us to get 1.1 out.

On Apr 27, 1:45 pm, Andrew Godwin  wrote:
> > As Russ said, this is a terrible time for this discussion.  Further
> > even if 50 people come here and say that they like South that's not a
> > convincing reason to include it.  50 users is a tiny minority of
> > Django developers, so we need to look at real technical reasons, not
> > what people like, since we can probably get 100 people who like any
> > given migration system.
>
> I agree with this completely; the core devs I've spoken to don't yet
> want to roll in a migration solution, and I'm 100% behind that, as I
> still think there's no clear 
> winner.http://code.djangoproject.com/wiki/SchemaEvolutionlists six active
> migration solutions, and there's obviously a reason there's more than
> one; Django has always been about swappable components, and rolling one
> solution in might easily make people think that's the only choice. (In
> addition, it would slow down development progress into a 6 month release
> cycle, and that's something I personally think South in particular
> wouldn't benefit from at the moment)
>
> That said, I would like to see some mention of schema evolution in the
> Django docs at some point; it's a very common problem, and we could
> easily come up with a neutral page to point people at all their options,
> rather than leaving them to find that wiki page on their own.
>
> Andrew
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Schema Evolution

[andreas]

On Apr 27, 10:49 pm, Malcolm Tredinnick 
wrote:
>
> Permit me to correct that impression: you are mistaken.

Good! That's what I want'ed to be.

>
> That's also not a really valid assumption. There are still significant
> differences and different limitations and advantages with each approach
> to database migration. A bit more convergence and experience -- which is
> happening -- is already going on and will continue to do so. Look at all
> the active projects in this space. They are all changing their
> approaches as time goes by, based on feedback, maintenance experience
> and general whims.

This applies to django, webdevelopment and software in general aswell
and it's a good thing imo, "magic removal" in django
is a good example of how django not changed the values for what it
stands for
but slightly changed it's approach and became more explicit.



--

I will accept the fact that you guys don't want to discuss this now.
I didnt want to make anyone upset. I was kind of thinking about this
when I
mentioned that "everyone is probably fed up with this topic"
So Im hereby withdrawing from this thread.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

RequestContext not working?

[Andreas Neumeier]

I just created a new djangorpoject, with a moraless empty view, which
looks like that:

--- cut ---
from django.template import Context, RequestContext
from django.core import template_loader
from django.http import HttpResponse

def index(request):
  t = template_loader.get_template('portal/index.html')
  c = RequestContext(request)
  return HttpResponse(t.render(c))
--- cut ---

This is not working at all.

If I replace c = RequestContext(request) with c = Context(), the
rendered template appears...

This makes authentication impossible, I guess?

Im using django current, rev 2917.

Regards,

Andreas
http://lovebox.ath.cx


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: RequestContext not working?

[Andreas Neumeier]

Well, thats exactly my problem :)

There is no access.log entry, no error.log entry and a blank page
outputted... Debug is set to ON

It looks like RequestContext just dies silently, but I couldn't tell
you for sure.

I also figured out, there is another installation (rev. 2893 on Ubuntu
Dapper), which looks that it works perfectly. rev 2917 on Breezy seams
to fail for some reason i cannot describe. Where could I have a deeper
look? Does django depend on certain mod_python versions?

Adrian Holovaty schrieb:

> Hi Andreas,
>
> RequestContext is working perfectly for me. If you explained what part
> of it isn't working for you, somebody could help solve your problem.
>
> Adrian
> 
> -- 
> Adrian Holovaty
> holovaty.com | djangoproject.com


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: RequestContext not working?

[Andreas Neumeier]

Adrian Holovaty schrieb:

> Does the problem happen on both the Django development server and mod_python?
Actually, this didn't come to my mind before, but I tried it just now
and the development server is working

What could be the issue there? 
--
Andreas
lovebox.ath.cx


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: RequestContext not working?

[Andreas Neumeier]

[EMAIL PROTECTED] schrieb:

> Isn't this just a mysql libs in php vs libs in mysqldb issue?
> Died silently with a blank page until i switched to postgresql.
> Or recompile php correctly yourself.

I'm not quite sure about the meaning here: why should I compile php
_at_all_? :-)

--
Andreas
lovebox.ath.cx


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: RequestContext not working?

[Andreas Neumeier]

Did you overwrite TEMPLATE_CONTEXT_PROCESSORS ?

>I made the mistake once, because I didn't knew this variable is filled
>by default. You have to write something like:
>
>TEMPLATE_CONTEXT_PROCESSORS = >DEFAULT.TEMPLATE_CONTEXT_PROCESSORS +
>('my.preprocessor',)
>
>Took me some time to figure out why admin was not working anymore :)
>
>The difference of Context and RequestContext is not a deep one, so I
>don't think its a python issue here.

Actually, I did not overwrite TEMPLATE_CONTEXT_PROCESSORS, but still it
breaks with mod_python in apache2, where it works with the development
server.

I began tweaking the TEMPLATE_CONTEXT_PROCESSORS, and it seems, that
django.core.context_processors causes some problems, since the other
default processors (debug, request and i18n) seem to do their job.

I try to dig deeper into this, which is quite hard since there are no
errormessages at any point :(

I would be glad, if anyone could point out to any possible issue here

--
Andreas Neumeier
lovebox.ath.cx


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Django Book enhancement

[Andreas Eigenmann]

Hi all

The book looks already very good. Congratulation!
But there are some additional things that would be nice to see in the
book.

1. There should be a appendix about Testing like described in the docu
http://www.djangoproject.com/documentation/testing/ . (Why is this
document not linked in the documentation index ?)
Testing is a very important thing in a professional web application
and also part of every ruby on rails book.

2. A small chapter about Ajax and json would be nice. Ajax at least in
a small chapter is part of every new book about web development.

3. The chapter about legacy databases should be enhanced by a section
about extendig a consisting django application. A web application
isn't static and changes in the schema from time to time are
inevitable.

Thanks
Andreas


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

RESTful web services for Django

[Andreas Stuhlmüller]

As part of this year's Summer of Code, I proposed to implement an easy
way to make existing Django models available via RESTful APIs.  My
proposal [1] got approved, my mentor is Malcolm Tredinnick. I would be
glad to read some feedback from those of you who are interested in how
REST-based web services with Django should look like.

My work consists of two major parts:

a) Easily configured Create/Read/Update/Delete (CRUD) method
   access patterns for models.

and

b) Resources that don't correspond 1:1 to models.

I made both part of my proposal since I would like to see REST
implemented in a way that makes it easy to seamlessly use both model
resources and custom resources that may not necessarily be related to
individual models.

I know three projects that overlap with the first part of my proposal.

1. djangocollection
http://code.google.com/p/djangocollection/
The djcollection app provides a set of generic RESTful urls for all
the models of a project and a GenericCollection class which uses
Django's generic views (and therefore templates) to make models
CRUD-accessible.

2. django-crudapi
http://code.google.com/p/django-crudapi/
Provides a set of catch-all urls and CRUD functions that make the apps
of all models of a project CRUD-accessible, uses Django's serializers
to return json or xml.

3. django-restful-model-views
http://code.google.com/p/django-restful-model-views/
Adam Smith follows an approach similar to the one I proposed (an
abstract Resources class and a ModelResources class for model-bound
resources). He uses a lot of introspection to automagically create url
patterns for all models that should be available. Adam intends to use
Django's generic views (not implemented yet).

These projects look like a good point to start, but they don't seem to
pay much attention to security questions like which data should be
accessible, which CRUD actions possible and which model fields
visible. To my mind, these should be available as settings similar to
the urls.py example shown in my proposal.

Other questions that need to be resolved include:
- Should URLs be generated automatically?
- Do we make use of the existing generic views (as in 1. and 3.),
  use serializers (2.) or make both an option?
- How does the interaction between the API and authentication
  look like? If we want to allow different levels of access for
  authenticated and unauthenticated users, how should this be
  configured?

As far as part 2 of the proposal is concerned, the main question is
whether (and how) we can make creating these resources easier instead
of just changing the type of work that is required to implement them.

I will try to come up with a concrete example that allows me to
illustrate the different ideas I have, both regarding part 1 and part
2 of my proposal, and that makes it easier to discuss these ideas.

According to Google's timetable, I should start coding on May 28.
However, during the two weeks before the SoC midterm evaluations
on July 9 I am writing eight exams and therefore I don't expect to get
much coding done. I'll make a bit of an early start to make up for this.

Andreas

[1] www.aiplayground.org/artikel/improving-django/

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: SITE's -- an idea to make them transparent/easier to use

[Andreas Stuhlmüller]

On 11/23/05, Kevin <[EMAIL PROTECTED]> wrote:
>
> Ok, I jumped into Django's code and it seems that the where_constraints
> is instead an array of sql strings, not the keyword mapping.
>
> so instead of:
> where_constraints = {'online__exact: True}
> you would need:
> where_constraints = ['online = true']
>
> Unfortunately, this basically makes it impossible to retrieve any row
> in the database that doesn't match the clause.  So in my example, if
> you set the online field to false, you can never retrieve that article
> without resorting to altering the database outside of django.

Actually, you can.

>>> from django.models.article import articles
>>> original_constraints = articles.Klass._meta.where_constraints
>>> articles.Klass._meta.where_constraints = []
>>> articles.get_list(online__exact=False)
>>> articles.Klass._meta.where_constraints = original_constraints

But is this safe or will it eventually cause problems?

Andreas

Re: Magic - removal branch .. is the API stable yet?

[Andreas Stuhlmüller]

On 1/23/06, Jason Davies <[EMAIL PROTECTED]> wrote:
> I'm keen to get the "semantics of subtyping" stuff done.  If you have
> any implementation hints to help me along that would be really helpful,
> thanks.

Same here. I'd love to see a working subtyping implementation for
magic removal, the sooner the better. If there is anything I can do to
help just drop me a note.

Andreas

Re: magic-removal: where_constraints

[Andreas Stuhlmüller]

On 2/14/06, Ian Holsman <[EMAIL PROTECTED]> wrote:
> my first stumbling block I have found is that I was using
> 'where_constraints' to automatically set  the SITE_ID.
>
> how does one do this in magic_removal?

I replaced the where_constraints by overwriting the default query
sets. See 
http://code.djangoproject.com/wiki/RemovingTheMagic#YoucanoverridedefaultQuerySets.
Using python parameters instead of SQL is a nice improvement over the
where_constraints method.

Andreas

Re: Permissions

[Andreas Neumeier]

I'm not quite sure, it may be me: Is the permission system working at
all in the magig-removal branch?

I added something like that to my model just yesterday:
---
  class Meta:
permissions = (
  ("can_create", "Can create"),
  ("can_modify", "Can modify"),
  ("can_view", "Can view"),
  ("can_transfer", "Can transfer"),
  ("can_delete", "Can delete"),
)
---

and I actually expexted './manage.py sqlall model' to produce some SQL
to insert those permissions into the auth_* tables.

nothing happened.

how does it differ from the default branch?

Thanks for the hint

Andreas  -- http://andreas.neumeier.org


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Test reordering and TransactionTestCase cleanup

[Andreas Pelme]

I am trying to run my Django test suite with an alternative test runner 
(py.test), and found some issues with test isolation.

TransactionTestCase does currently not clean up after itself (i.e. flush the 
database), but instead assumes that the next test will flush the database. It 
is generally a good idea to restore the state after the test run, and let other 
tests start out with a known state.

Djangos default testrunner reorders the test suite to run all TestCase tests 
before TransactionTestCases, which avoids this problem. I cannot find this 
reordering documented anywhere, or even commented anywhere in the 
implementation (test/simple.py: reorder_suite and 
DjangoTestSuiteRunner.build_suite).

I propose to move the flush command out of _fixture_setup to _post_teardown in 
TransactionTestCase. This makes it possible for arbitrary execution order of 
the test suite. Performance could potentially be an issue, since it is "lazy" 
today and flushes the database just right before it is needed. That would 
however only affect the last test in the run (i.e. it will truncate the tables, 
and then the database itself will be teared down).

Are there any objections or anything that might break with such a change? Are 
there any specific reasons it works the way it does?

I will be happy to create a ticket and patch for this, I would get to input in 
case I miss something obvious.

Cheers
Andreas

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Test reordering and TransactionTestCase cleanup

[Andreas Pelme]

On Thursday 3 May 2012 at 21:17, Karen Tracey wrote:
> On Thu, May 3, 2012 at 12:29 PM, Andreas Pelme  (mailto:andr...@pelme.se)> wrote:
> > 
> > Djangos default testrunner reorders the test suite to run all TestCase 
> > tests before TransactionTestCases, which avoids this problem. I cannot find 
> > this reordering documented anywhere,
> 
> It is documented:
> https://docs.djangoproject.com/en/1.4/topics/testing/#django.test.TransactionTestCase
> 
> Mentioned in both the 3rd paragraph and the note.
Thanks -- I'm not sure how I missed that. I'll make sure my patch updates the 
docs too.

Andreas


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Test reordering and TransactionTestCase cleanup

[Andreas Pelme]

On Thursday 3 May 2012 at 22:14, Ramiro Morales wrote:
> On Thu, May 3, 2012 at 1:29 PM, Andreas Pelme  (mailto:andr...@pelme.se)> wrote:
> > I am trying to run my Django test suite with an alternative test runner 
> > (py.test), and found some issues with test isolation.
> > 
> > TransactionTestCase does currently not clean up after itself (i.e. flush 
> > the database), but instead assumes that the next test will flush the 
> > database.
> > [...]
> > 
> > Djangos default testrunner reorders the test suite to run all TestCase 
> > tests before TransactionTestCases, which avoids this problem.
> 
> Just a quick related note
> 
> There is a [1]proposal to extend this reordering to be:
> 
> * TestCases
> * doctests
> * TransactionTestCases
> 
> So doctests aren't affected either.
> 
> I intend to commit a fix for it soon.
> 
> -- 
> Ramiro Morales
> 
> 1. https://code.djangoproject.com/ticket/12408

This won't be an issue if TransactionTestCase resets the database after itself.

Andreas 


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Test reordering and TransactionTestCase cleanup

[Andreas Pelme]

On Thursday 3 May 2012 at 19:23, Anssi Kääriäinen wrote:
> On May 3, 7:29 pm, Andreas Pelme http://pelme.se)> wrote:
> > I am trying to run my Django test suite with an alternative test runner 
> > (py.test), and found some issues with test isolation.
> >  
> > TransactionTestCase does currently not clean up after itself (i.e. flush 
> > the database), but instead assumes that the next test will flush the 
> > database. It is generally a good idea to restore the state after the test 
> > run, and let other tests start out with a known state.
> >  
> > Djangos default testrunner reorders the test suite to run all TestCase 
> > tests before TransactionTestCases, which avoids this problem. I cannot find 
> > this reordering documented anywhere, or even commented anywhere in the 
> > implementation (test/simple.py: reorder_suite and 
> > DjangoTestSuiteRunner.build_suite).
> >  
> > I propose to move the flush command out of _fixture_setup to _post_teardown 
> > in TransactionTestCase. This makes it possible for arbitrary execution 
> > order of the test suite. Performance could potentially be an issue, since 
> > it is "lazy" today and flushes the database just right before it is needed. 
> > That would however only affect the last test in the run (i.e. it will 
> > truncate the tables, and then the database itself will be teared down).
> >  
> > Are there any objections or anything that might break with such a change? 
> > Are there any specific reasons it works the way it does?
>  
> I don't know if anything will break...
>  
> I think this is a good idea. In addition to the above issues this
> would allow the test ran tell the flushing which tables are dirty,
> which not. Currently the transaction_regress for example uses just a
> single table. But after each test, all tables will be truncated, and
> the contenttypes and permissions will be reloaded. I bet we could
> speed up the testing _a lot_ if the cleaning up could be made more
> intelligent. Making it robust is somewhat hard but should not be
> impossible. Of course, the "StateTrackingTestCase" should not be done
> in the same patch at all... just a possibility for future work.


There are indeed some work that can be done to speed up the test running. :)

Here's the ticket:
  
https://code.djangoproject.com/ticket/18271  

… and an initial patch:

https://github.com/django/django/pull/45

Is there any sane way to test this except to run the test suite itself?

Andreas


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Test reordering and TransactionTestCase cleanup

[Andreas Pelme]

Anssi, thanks a lot for the detailed feedback, it is much appreciated! My 
comments are inlined below:


On Friday 4 May 2012 at 10:03, Anssi Kääriäinen wrote:

> I marked the ticket DDN, there are three reasons:
> 1. If a test case screws up cleanup it will cause problems for
> itself currently, after the patch it will cause problems for the next
> test case which makes debugging much harder.

The same cleanup will still be done, it will just be moved from before the test 
run, to after.

The only way for a test case to screw up would be a bug in Django's 
_pre_setup/_post_teardown implementation, which would cause tricky bugs, no 
matter if the cleanup happens before or after the test.

> 2. Is this backwards incompatible? The behavior was documented.
See notes below.
> 3. Is there some reason why the flush must be done pre-test which
> we are overlooking. Why was it implemented that way originally?

That's a good question. Anyone who wrote to original 
TransactionTestCase/reordering implementation that wants to chime in? :-)
> Perhaps because of the first item?

The first item should get a newly created database, so it should not be a 
problem.  



> Each test case taking care of cleaning up after itself feels like the
> right way to go. There are a couple of reasons:
> - TestCase works already this way: it doesn't do cleanup before, it
> just makes sure it cleans after itself (which is easy as it doesn't
> leave any trash behind).
> - The database is treated specially. The tests trust other tests to
> clean up their state. Settings is an example.
> - As said upthread, this allows for special test case subclasses
> which do more intelligent cleanup. I don't think an intelligent
> TransactionTestCase subclass which doesn't run flush pre-tests is
> possible in the current implementation, as it would then see the trash
> of other TransactionTestCases.

Agreed.  
> But, as said, I left the ticket DDN. None of the issues mentioned
> above seem like a blocker. The backwards incompatibility is the most
> likely blocker - I can't see that changing the behavior is going to
> cause problems, but maybe there are users out there whose test cases
> somehow require pre-test flush?


I have run Djangos own test suite, and my current project's test suite with 
~500 tests and some TransactionTestCases without problems.

This is indeed a change in documented behavior, although I don't think it will 
affect peoples test. In order for a test case to be backward incompatible, a 
user would need to override 
_pre_setup/_fixture_setup/_post_teardown/_fixture_teardown (which could be 
considered private?). Unless that is done, I cannot see how tests can be 
affected by this, but there can certainly be something I am missing here!

I guess we need someone who wrote the initial TransactionTestCase or knows why 
it is the way it is to give his/hers opinion on this not to oversee backward 
incompatible changes.


Andreas

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Test reordering and TransactionTestCase cleanup

[Andreas Pelme]

On Friday 4 May 2012 at 13:30, Karen Tracey wrote:
> On Fri, May 4, 2012 at 4:46 AM, Andreas Pelme  (mailto:andr...@pelme.se)> wrote:
> > 
> > That's a good question. Anyone who wrote to original 
> > TransactionTestCase/reordering implementation that wants to chime in? :-)
> 
> I worked on the test speedups that introduced TransactionTestCase and
> I added the re-ordering, but the behavior of database flush being done
> at the beginning of a test existed before that work. The conversation
> that led to addition of the reordering is here:
> 
> http://groups.google.com/group/django-developers/browse_thread/thread/1e4f4c840b180895/
> 
> doctests (which we have no more?) play a prominent role in that
> discussion. While we have gotten rid of doctests in Django's own
> suite, we still support apps which may use doctests in their code, so
> anything we do to change when the DB is cleared needs to take that
> into account. On the table then was the idea of adding cleanup after
> doctests, possibly that would need to be re-considered if you want to
> move the database clearing to the end of everything rather than the
> beginning of TransactionTestCase.


#12408 is related here, which is caused by TransactionTestCase's not cleaning 
up after themselves. Rather than changing the test order to TestCase -> 
doctests -> TransactionTestCase I think the solution below is better.

Karens proposed a fix for this (#2) in the above thread [1], it basically says:

1) Make TransactionTestCase clean up after itself
2) Order the test suite to make sure doctests are always run *last*

That would run all unittest-style test cases first (the order of TestCase / 
TransactionTestCase will not be important), and last, all doctests.

A test suite that have doctests that *expects* state to be left from a 
TransactionTestCase before it would be broken by this change. In my opinion, a 
test suite with such expectations is already very broken and needs to be fixed 
any way, and should not be considered a blocker to this change (in that case, 
#12408 is a blocker too, since that effect will be the same).

I have updated my pull request to make unittest.TestCase subclasses to run 
first (i.e. doctests will run after all unit tests):
https://github.com/django/django/pull/45

Am I missing something? This seems like the most reliable way to solve this to 
me.

(I originally discovered this when I tried to run my tests with an alternative 
test runner, which does not order the tests in any particular way. I will still 
not be able to run doctests that does not clean up after themselves in 
arbitrary order, but as long as I document that's not supported with my test 
runner, I don't consider that to be a problem.)

Andreas

[1] https://groups.google.com/d/msg/django-developers/Hk9MhAsYCJU/fB4rj7F5SXEJ

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Test reordering and TransactionTestCase cleanup

[Andreas Pelme]

On Saturday 5 May 2012 at 00:39, Anssi Kääriäinen wrote:
> On May 4, 2:30 pm, Karen Tracey http://gmail.com)> wrote:
> Thanks for the link. While reading the previous threads I spotted at
> one blocker issue: the first TransactionTestCase will not start with
> zeroed database sequence values. To prevent this one would need to
> flush the DB before the first TransactionTestCase. In addition, if the
> goal is to get rid of test ordering, then one would need to flush
> before every test case which isn't good. Getting rid of the
> requirement to reset sequences between test cases would be good.
> Oracle doesn't reset the sequences reliably currently just for
> example. But I don't think we can change that for backwards
> compatibility reasons. While tests that rely on sequence values are
> IMHO broken, changing this would result in numerous broken tests for
> users upgrading Django. Notably normal TestCases do not do sequence
> resetting at all currently.
>  
> This is a blocker: why change the flushing from pre-test to post-test
> if the end result is that you can't run the tests in arbitrary order
> anyways? Or if you want to be able to run them in arbitrary order you
> will need to run at least the sequence resets pre-test anyways.
> Resetting sequences can be really expensive, so there goes any
> potential savings in speed.
>  
> Ideas?

Agreed, it is indeed backward incompatible. However, I think we can make the 
upgrade for those affected very anyways:

 * Document this change properly in the release notes and in the testing docs
 * Document that tests should not depend on hard coded primary key values (this 
is already hinted in the testing docs where TestCase/TransactionTestCase is 
described)
 * Add a reset_sequences attribute on TransactionTestCase, that can be applied 
to existing TransactionTestCase tests that already uses hard coded primary key 
value - which will reset sequences *before* the test run, just like before. 
(And those tests will also flush the database after they are run)

I have updated the pull request with the reset_sequences attribute and an 
initial draft for the release notes and documentation:
https://github.com/django/django/pull/45

1.1 introduced a far greater change to testing behavior (the TestCase with 
transaction/rollback), if that change was considered acceptable from a backward 
compatibility perspective, this change should be too. When that happened, the 
same problem appeared since TestCase does not reset sequences. (Yeah, just 
because it was done in the past does not make it OK now, but that should give 
some perspective).

I guess it boils down to where to draw the line for backward compatibility 
here. :-) I personally think it is acceptable since we can offer a very easy 
fix for the existing (already broken) tests out there.


Andreas


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Settings: lists or tuples?

[Andreas Kahnert]

Hi all,

I'm strongly against lists. Lists are mutable objects, their components can 
be changed in place. The settings are initialized at server start and after 
that changes on them arn't reflected. Therefore all settings should be 
tuples from my point of view. Using a custom list/tuple class for "easy" 
notation within the settings.py might be an option, but after server 
startup all of those should be immutable. So for the sake of logic, take 
tuples, please.

Andreas

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/abf9f995-9e2a-4a8a-a6a5-67dad84fdb69%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Settings: lists or tuples?

[Andreas Kahnert]

I'm not talking about modifications inside the settings.py but in other 
place. With lists, unexperienced devs might do things like: from 
django.conf import settings; settings.TEMPLATE_DIRS[3] = '/my_tpls'; and 
expect to work out good. (This doesn't violate the docs, since technicaly 
settings.TEMPLATE_DIRS is still the same list, means the settings variable 
itself hasn't been changed.) This is realy just a question of logic: lists 
are mutable - settings are immutable; see the conflict?

Am Montag, 19. Januar 2015 14:20:51 UTC+1 schrieb Florian Apolloner:
>
>
>
> On Monday, January 19, 2015 at 12:35:10 PM UTC+1, Andreas Kahnert wrote:
>>
>> I'm strongly against lists. Lists are mutable objects, their components 
>> can be changed in place. The settings are initialized at server start and 
>> after that changes on them arn't reflected. Therefore all settings should 
>> be tuples from my point of view. Using a custom list/tuple class for "easy" 
>> notation within the settings.py might be an option, but after server 
>> startup all of those should be immutable. So for the sake of logic, take 
>> tuples, please.
>>
>
> As long as  something like settings.MY_SETTING=(1,); 
> settings.MY_SETTING+=(1,2,3) works, that argument is kinda moot imo.
>
> Cheers,
> Florian
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/5b974048-0c9f-4ae2-8ff2-8db1fbd736f4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Settings: lists or tuples?

[Andreas Kahnert]

Test Code:
old_var = settings.TEMPLATE_DIRS
settings.TEMPLATE_DIRS += ('foo',) if isinstance(settings.TEMPLATE_DIRS, 
tuple) else ['foo']
old_var is settings.TEMPLATE_DIRS  # will return True if it was a list; 
will return False if it was a tuple, since a new tuple was assigned to 
settings.TEMPLATE_DIRS (violating the docs)

This was just a silly example, but if think of modularity and so on, this 
could get a disaster. The assignment could be somewhere in a 
helper-function where it's not obvious that the list originates from the 
settings. Another helper-function might use settings.TEMPLATE_DIRS to 
iterate over (maybe to validate all templates are reachable or something) 
yielding different results than the framework actual usage. Debuging this 
would be pretty hard (especially for those who arn't aware of the subtile 
differences between lists and tuples)


Am Montag, 19. Januar 2015 15:49:37 UTC+1 schrieb Marc Tamlyn:
>
> I think Florian's point would be that you can still do:
>
> from django.conf import settings
> settings.TEMPLATE_DIRS += ('foo',)
>
> if it is a tuple, so I don't really see how it being a tuple is making it 
> necessarily more safe than a list - is the above code really much different 
> to:
>
> from django.conf import settings
> settings.TEMPLATE_DIRS.append('foo')
>
> On 19 January 2015 at 14:45, Andreas Kahnert  > wrote:
>
>> I'm not talking about modifications inside the settings.py but in other 
>> place. With lists, unexperienced devs might do things like: from 
>> django.conf import settings; settings.TEMPLATE_DIRS[3] = '/my_tpls'; and 
>> expect to work out good. (This doesn't violate the docs, since technicaly 
>> settings.TEMPLATE_DIRS is still the same list, means the settings variable 
>> itself hasn't been changed.) This is realy just a question of logic: lists 
>> are mutable - settings are immutable; see the conflict?
>>
>> Am Montag, 19. Januar 2015 14:20:51 UTC+1 schrieb Florian Apolloner:
>>
>>>
>>>
>>> On Monday, January 19, 2015 at 12:35:10 PM UTC+1, Andreas Kahnert wrote:
>>>>
>>>> I'm strongly against lists. Lists are mutable objects, their components 
>>>> can be changed in place. The settings are initialized at server start and 
>>>> after that changes on them arn't reflected. Therefore all settings should 
>>>> be tuples from my point of view. Using a custom list/tuple class for 
>>>> "easy" 
>>>> notation within the settings.py might be an option, but after server 
>>>> startup all of those should be immutable. So for the sake of logic, take 
>>>> tuples, please.
>>>>
>>>
>>> As long as  something like settings.MY_SETTING=(1,); 
>>> settings.MY_SETTING+=(1,2,3) works, that argument is kinda moot imo.
>>>
>>> Cheers,
>>> Florian
>>>
>>  -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com .
>> To post to this group, send email to django-d...@googlegroups.com 
>> .
>> Visit this group at http://groups.google.com/group/django-developers.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/5b974048-0c9f-4ae2-8ff2-8db1fbd736f4%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/django-developers/5b974048-0c9f-4ae2-8ff2-8db1fbd736f4%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/ba609be2-1f20-4a77-9692-1b95fc679117%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Settings: lists or tuples?

[Andreas Kahnert]

Well, yep. You can't prevent programmers to do stupid things in python. But 
I'm kind of a theroretician and it hurts me if I see that exactly that what 
it should not be is proposed as the standard. And for the dicts: In my 
private code-base I use a frozendict c-package I wrote.

@Collin: The notation used within settings.py isn't that important. But I 
think in the moment the framework constructs the settings object all 
sequences (might be generators as well) should be turned into tuples.

For the trailing-comma-problem: I notate every (globally available) 
constant sequence in the pattern:
A = (
1,
2,
)
which is perfectly PEP-conformant and makes reodering elements manually 
also more easy because you can cut'n'paste whole lines.

Am Montag, 19. Januar 2015 17:35:44 UTC+1 schrieb Florian Apolloner:
>
> On Monday, January 19, 2015 at 3:45:18 PM UTC+1, Andreas Kahnert wrote:
>>
>> I'm not talking about modifications inside the settings.py but in other 
>> place. With lists, unexperienced devs might do things like: from 
>> django.conf import settings; settings.TEMPLATE_DIRS[3] = '/my_tpls'; and 
>> expect to work out good. 
>>
>
> And with tuples they just do settings.TEMPLATE_DIRS = 
> list(settings.TEMPLATE_DIRS) followed by your example. What I am trying to 
> say is that a developer ignoring the docs will always run into issues, no 
> matter how hard we try to prevent it. 
>
> This is realy just a question of logic: lists are mutable - settings are 
>> immutable; see the conflict?
>>
>
> Well, what is your suggestion for the dictionaries in the settings then ;) 
> In the end it just doesn't matter if it's a list or a tuple -- you can 
> "mutate" it anyways. For the functionality of Django it makes no difference 
> if you altered the setting in place or just replace it -- stuff __will__ 
> break. 
>
> Cheers,
> Florian
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/07fd597a-01d2-4f6c-845b-5587a19b3263%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Settings: lists or tuples?

[Andreas Kahnert]

PS: python should be able to access tuple members faster, because it can be 
implemented as array instead of double-linked-structs which are necessary 
mutable lists. But as far as I know it doesn't.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/5c8cacca-60d1-449a-b5bb-e1a07f4006ec%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Settings: lists or tuples?

[Andreas Kahnert]

Hi Loïc,

I agree that we should not discuss about the theoretical aspects too much 
(while I disagree on your distinction, the API difference is just their 
mutability, so unless you refer to python intern algorithms for sort- 
/lookup- optimization (if so, excuse me) your distinction is just your 
personal taste)

The main point is the accidential assignment to list members / extension of 
the list in places where it's not obvious that the list of subject is part 
of the settings. Passing the whole settings object through functions makes 
no sense since you can always get it by importing, which minimizes the risk 
of accidently assigning to it.

Am Montag, 19. Januar 2015 18:53:26 UTC+1 schrieb Loïc Bistuer:
>
> Hi Andreas, 
>
> As Florian pointed out, you can't mutate a tuple, but you can mutate the 
> settings object, so using a tuple or a frozendict won't buy you much. 
>
> Regarding the theoretical perspective, tuples aren't meant to be immutable 
> lists. Tuples are for heterogenous collections of fixed size (e.g. GPS 
> coordinates); whereas lists are for homogenous collections of variable 
> size, which suits perfectly our settings use-case. Of course Python being 
> Python you can use them however you like but let's not talk about theory in 
> that case. 
>
> Cheers 
>
> -- 
> Loïc 
>
>
> > On Jan 20, 2015, at 00:15, Andreas Kahnert  > wrote: 
> > 
> > Well, yep. You can't prevent programmers to do stupid things in python. 
> But I'm kind of a theroretician and it hurts me if I see that exactly that 
> what it should not be is proposed as the standard. And for the dicts: In my 
> private code-base I use a frozendict c-package I wrote. 
> > 
> > @Collin: The notation used within settings.py isn't that important. But 
> I think in the moment the framework constructs the settings object all 
> sequences (might be generators as well) should be turned into tuples. 
> > 
> > For the trailing-comma-problem: I notate every (globally available) 
> constant sequence in the pattern: 
> > A = ( 
> > 1, 
> > 2, 
> > ) 
> > which is perfectly PEP-conformant and makes reodering elements manually 
> also more easy because you can cut'n'paste whole lines. 
> > 
> > Am Montag, 19. Januar 2015 17:35:44 UTC+1 schrieb Florian Apolloner: 
> > On Monday, January 19, 2015 at 3:45:18 PM UTC+1, Andreas Kahnert wrote: 
> > I'm not talking about modifications inside the settings.py but in other 
> place. With lists, unexperienced devs might do things like: from 
> django.conf import settings; settings.TEMPLATE_DIRS[3] = '/my_tpls'; and 
> expect to work out good. 
> > 
> > And with tuples they just do settings.TEMPLATE_DIRS = 
> list(settings.TEMPLATE_DIRS) followed by your example. What I am trying to 
> say is that a developer ignoring the docs will always run into issues, no 
> matter how hard we try to prevent it. 
> > 
> > This is realy just a question of logic: lists are mutable - settings are 
> immutable; see the conflict? 
> > 
> > Well, what is your suggestion for the dictionaries in the settings then 
> ;) In the end it just doesn't matter if it's a list or a tuple -- you can 
> "mutate" it anyways. For the functionality of Django it makes no difference 
> if you altered the setting in place or just replace it -- stuff __will__ 
> break. 
> > 
> > Cheers, 
> > Florian 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups "Django developers (Contributions to Django itself)" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an email to django-develop...@googlegroups.com . 
> > To post to this group, send email to django-d...@googlegroups.com 
> . 
> > Visit this group at http://groups.google.com/group/django-developers. 
> > To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/07fd597a-01d2-4f6c-845b-5587a19b3263%40googlegroups.com.
>  
>
> > For more options, visit https://groups.google.com/d/optout. 
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/19ce444f-f676-460c-ba92-7033d4b6e25d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Settings: lists or tuples?

[Andreas Kahnert]

Interesting,  ... so excuse me.
The next point also clearifys my PS from above, lists are over-allocated 
dynamic sized arrays. (this explains why python is such an memory eater as 
well as why I experienced performance loss when using the mutability of 
lists extensivly)

Am Montag, 19. Januar 2015 19:55:59 UTC+1 schrieb Loïc Bistuer:
>
>
> Presumably more than just my personal taste: 
> https://docs.python.org/2/faq/design.html#why-are-there-separate-tuple-and-list-data-types.
>  
>
>
> -- 
> Loïc 
>
>
But coordinates have homogenous members (floats) as djangos setting 
sequences have (strings), both are of fixed size (those they have after you 
declared them),
So this still doesn't change my (theoretical) opinion.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/ca250479-c212-4552-97aa-6b757cf6fe3e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Settings: lists or tuples?

[Andreas Kahnert]

I advertise that strongly against lists, because we actually had that kind 
of issue in our company.
One colleague created a list with phrases for some verbose logging in the 
settings.py. In the view function he promoted this list together with the 
actual data, also a list which is used for storing the data afterwards, to 
another function. The other function was implemented by another colleague, 
he was aware that he should not modify the second list for not corrupting 
the data to be safed. So he appended the data components to the first list 
zipped it to an dict and called the logging function. The result was that 
every log entry contained all data from previous calls and the server loked 
down very quick. It took them a day to debug this.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/64cb1e61-8eb2-4459-ac77-6fa1c3837c98%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Settings: lists or tuples?

[Andreas Kahnert]

Just for completness: accidential assignment to the settings object itself 
could be easily prevented with a __setattr__ method on its class, since 
django yields on various other places about configuration problems it could 
not be wrong if the programmer gets noted about an illegal assignment. If 
everything works fine the method will only get called during startup, so 
there is no runtime overhead. Simplified example:
def __setattr__(self, key, val):
if self.configured:
raise Exception('settings can not be changed after server startup')
super(LazySettings, self).__setattr__(key, val)

@Carl Meyer: At the first hand you're right, a thing all programmers should 
know about (if they call themself so), but he assumed there existed some 
kind of copy-on-read semantic for the settings, because you get something 
different when imported from django.conf instead directly and because it's 
a "magical" lazy object.


But since you all seem to like lists that much, maybe a compromise would be 
to explicitly note in the docs that there is a danger in using lists which 
can be prevented by tuple usage.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/ecb152a6-5478-45f1-89b9-3cfc0db27f30%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Settings: lists or tuples?

[Andreas Kahnert]

Hi again,
Well, I can acknoledge that your reasons for list (beginner friendly) are 
as good as my reasons for tuples (seems to be more logical choice for 
things that are static). To say it in other words, my idea was simply: Use 
tuples and the programmer will know that these arn't ment to be altered at 
runtime.
But this reformulation made me thought of the docs comment I suggested. And 
probably the docs should clearly state (in 
https://docs.djangoproject.com/en/1.6/topics/settings/#creating-your-own-settings):
The settings are solely for configuration of django and its components / 
extensions, but not for initial/default values of your application, because 
the settings have to be imported in a special way (through django.conf; 
sidenote - it states there: "For settings that are sequences, Django itself 
uses tuples, rather than lists, but this is only a convention")
At the end, this was what the confusion between my colleages caused and 
what his true programming-theoretical mistake was. (nowadays all of our 
apps have a definitions.py module)

I hope this is something all could agree with and ends the discussion I 
caused.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/b243c053-667d-443a-8ec4-c586ed102d63%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Allow deferral of ModelSignal callback invocation until after transaction commit

[Andreas Pelme]

Hi,

> On 30 apr 2015, at 18:42, Carl Meyer  wrote:
> 
> transaction-hooks is actually fairly small and understandable too. And I
> don't think it's hard to use for this situation, either; you'd just need
> to use `connection.on_commit` in your signal handler if you wanted to
> delay some action until post-commit.
> 
>> Unless it's going to be super easy to port transaction-hooks over, I
>> think it might be nice to get this in and sealed (it's not adding too
>> much more complexity). If transaction-hooks would land in core well,
>> let's do that.
> 
> I don't think landing transaction-hooks in core for 1.9 would be hard at
> all, and no-one has objected to the idea (AFAIK). I (or anyone really)
> just need to get around to it. Motivation has been low so far mostly
> because it works fine as an external add-on.


I did an initial port of django-transaction-hooks, it was pretty 
straightforward. All the hard work has already been done. :-)

Here is the PR: https://github.com/django/django/pull/4593

The patch is not yet finished (there’s a todo-list at the PR with some missing 
pieces). Let me know what you think and I’ll be happy to continue working on a 
proper patch to get it into a merge-able state.

Cheers,
Andreas

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/5A9C5A84-A28C-4556-BD7F-7F1699A166AA%40pelme.se.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Allow deferral of ModelSignal callback invocation until after transaction commit

[Andreas Pelme]

> On 3 maj 2015, at 15:44, Christopher Adams  
> wrote:
> 
> So unless anyone has objections, I'm going to put my branch on hold for now. 
> If anyone still wants me to see if there's a way it can work I'm willing to 
> give it a bit more work, however I think it should probably be passed over 
> for the time being. Glad my post may have kicked off the conversation to get 
> a eventual solution in.
> 
> Next steps: is everyone agreed that attempting to merge Carl's 
> django-transaction-hooks is the way forward? Happy to help on that if you 
> guys need help. Pretty excited to get a post-commit hook into core in 1.9, no 
> matter how that ends up happening.

Christopher: Thank you very much for your efforts on this, I am also very 
excited to get some post-commit hook into Django. This has been a long standing 
issue for me personally, having run into this in different projects in 
different situations over the years. Lately I’ve been happily using 
django-transaction-hooks, but at the same time been sad that it wasn’t part of 
Django. You’re post made me jump in and try to port django-transaction-hooks.

There is some work/discussions/progress going on in the PR at 
https://github.com/django/django/pull/4593. Currently most things seems to work 
and the docs is written. The most tricky issue (related to sqlite autocommit 
semantics) has hopefully been solved.

Any review of the code/tests/docs and input on the open questions would be very 
welcome and helpful!

Cheers,
Andreas

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/10E04DB3-15D0-4BBE-9087-B0C3F00633F2%40pelme.se.
For more options, visit https://groups.google.com/d/optout.

Django Run in Aptana

[Andreas Rudischhauser]

Hello, 

i'm trying to get Django running in Aptana Studio. I've got PyDev installed and 
can create a new project. The point that's missing is the "run as". Does 
anybody know how to setup a run configuration.
(Im running it on mac, if that's important to know). Or is there a better dev 
environment to develop for django?

Regards
Andi

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Attach a technincal html response to email on server error

[Andreas Pelme]

On 10 nov 2011, at 22:03, Ric wrote:
> hi, i receive django email on 500 error.
> 
> with that kind of mail is difficult to debug, because i don't have a
> lot of informations.
> 
> it's much more simple to debug when i set DEBUG = True and i receive
> an html technical response.
> with that response i can read a lot of informations (like vars,
> request information, server settings and so on)
> 
> my idea is to attach an html file, that is the very same response you
> got on error with DEBUG = True.
> 
> this response has got no external dependencies (css and js are inline
> in the html), so it can be attached as a file in the email that is
> sended on server error.
> 
> this is very simple to archieve, and developers can open it with the
> browser and see a lot of information.



This is already possible since Django 1.3 with the logging framework and 
AdminEmailHandler:

https://docs.djangoproject.com/en/dev/topics/logging/#django.utils.log.AdminEmailHandler


-Andreas

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

RegexURLResolver.resolver can fall through and return None

[Andreas Klöckner]

Hey there,

I tried to submit this bug, but it wouldn't let me--it thought I'm a spammer. 
You should really work on that spam detection thing, it should not turn away 
real submissions.

Here's the bug:

RegexURLResolver.resolver can fall through and return None

If the "if match" condition at the top is not met, it will return None, 
causing this:

Traceback:
File 
"/sparc_soft/www.dam/htdocs/scicomp/pool/lib/python2.5/site-packages/django/core/handlers/base.py"
 
in get_response
  73. callback, callback_args, callback_kwargs = 
resolver.resolve(request.path)

Exception Type: TypeError at 
Exception Value: unpack non-sequence
--------

Andreas


signature.asc
Description: This is a digitally signed message part.

Re: RegexURLResolver.resolver can fall through and return None

[Andreas Klöckner]

On Donnerstag 26 Juni 2008, [EMAIL PROTECTED] wrote:
> To avoid spam detection sign up.

Well, then the web site should hint at this fact, too. I was fairly close to 
just abandoning the effort of trying to tell you about the issue I 
encountered.

Andreas


signature.asc
Description: This is a digitally signed message part.

Re: RegexURLResolver.resolver can fall through and return None

[Andreas Klöckner]

On Donnerstag 26 Juni 2008, Collin Grady wrote:
> Andreas Klöckner said the following:
> > Well, then the web site should hint at this fact, too. I was fairly close
> > to just abandoning the effort of trying to tell you about the issue I
> > encountered.
>
> It does. Quite clearly at the top of the 'new ticket' page, there are
> several notes about filing tickets, including this one:
>
> # If you're getting rejected by the spam filter, we apologize! The
> best way to avoid that is to register for an account and make sure
> you're logged in.

Ok, I missed that. I meant more that the advice to log in should be on the 'Go 
away, spammer' error page, where it's actually useful.

Andreas


signature.asc
Description: This is a digitally signed message part.

Re: RegexURLResolver.resolver can fall through and return None

[Andreas Klöckner]

This, with more elaborate explanation, is now issue 7550, at 
http://code.djangoproject.com/ticket/7550.

Andreas

On Donnerstag 26 Juni 2008, Andreas Klöckner wrote:
> Hey there,
>
> I tried to submit this bug, but it wouldn't let me--it thought I'm a
> spammer. You should really work on that spam detection thing, it should not
> turn away real submissions.
>
> Here's the bug:
> ---
>- RegexURLResolver.resolver can fall through and return None
>
> If the "if match" condition at the top is not met, it will return None,
> causing this:
>
> Traceback:
> File
> "/sparc_soft/www.dam/htdocs/scicomp/pool/lib/python2.5/site-packages/django
>/core/handlers/base.py" in get_response
>   73. callback, callback_args, callback_kwargs =
> resolver.resolve(request.path)
>
> Exception Type: TypeError at
> Exception Value: unpack non-sequence
> ---
>-
>
> Andreas




signature.asc
Description: This is a digitally signed message part.

Model fields in localflavor

[Andreas Pelme]

[This messages is related to
http://groups.google.com/group/django-developers/browse_thread/thread/64e4d42590145347
]

On Jul 27, 11:39 pm, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote:
> On Sun, 2008-07-27 at 16:39 +0200, Andreas Pelme wrote:
> > Hello
>
> > I have started writing a patch for adding Swedish (se) local flavor.
>
> > I looked at the code for other countries, and noticed that they only
> > contains form fields, not model fields.
>
> > Why is that? E.g. a postal code is very likely to be stored in a
> > database. Why not add default model fields (with appropriate
> > formfield(), validate() etc)?
>
> Well, right now, validation at the model field level isn't really
> supported. The old-style validators were only used by oldforms (and
> old-admin) and the new work that Honza Kral is doing isn't finished yet
> (although there'll probably be something done in time for the 1.0-beta.
>
> Even when that's done however, it won't require adding model fields for
> every type of localflavor. That would be overkill. Instead, it will be
> appropriate to add some validators for those particular types (and
> there'll be a fair bit of sharing with existing form validators).
>
> Realise that (a) almost everything is going to be stored in the database
> in a character field anyway, so different explicit Field subclasses
> doesn't add a lot and (b) you often aren't going to want a separate
> field column in the database for each locale. If somebody is entering an
> address, it will usually be stored in the same fields, regardless of
> whether they're entering an address in Sweden, Argentina or Australia.
>
> Regards,
> Malcolm

I finally got a ticket and patch created for the Swedish localflavor:

http://code.djangoproject.com/ticket/9289
http://code.djangoproject.com/attachment/ticket/9289/django_localflavor_se.diff

However, Swedish personal identity numbers are a little tricky, and can
be represented in a couple of different formats.

The appropriate thing would be to save the fields as their "long"
representation MMDD in the database. However, it is really
common to use them formatted as YYMMDD-, which creates some issues
with people older than 100 years. I wont go into more details about how
the numbers works, however, it would be really useful to do something
like this:

Having a model field with appropriate formfield(), that has the methods
long() and short() for outputting the identity number in the different
formats. You most likely want to store the data in the long format, and
mostly you are interested in the short format. It could also have
birth_day() that returns a datetime.date object.

My questions:
1) Would this be a good use case for introducing a model field in
localflavor(.se)?
2) If so, where should the tests live?
3) If not, with just the form field, the long(), short() and birth_day()
functions would still be very useful when dealing with these numbers.
Where should they live then? contrib.localflavor.se.utils?


Cheers
-- 
Andreas Pelme <[EMAIL PROTECTED]>
PGP/GPG key: 1024D/2F6D209F

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Swedish localflavor

[Andreas Pelme]

The ticket/patch for Swedish localflavor[1] needs review. It has been
discussed[2] on the Swedish user group and everyone seems to be happy
with the features and implementation. The patch contains tests and docs.
I have been using it on several production sites for the last couple of
months without any issues.

Is there any chance we could get this into 1.1? What is currently
keeping it from not being committed to trunk? If there is something I
can do to help getting it into trunk, please let me know.

Best regards
Andreas Pelme

[1] http://code.djangoproject.com/ticket/9289
[2]
http://groups.google.com/group/django-se/browse_thread/thread/bfa40d7637607f2c

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: App-loading reloaded - running code at startup

[Andreas Pelme]

Aymeric,

Your work is amazing and has improved Django so much. Your work on timezone
support, the transaction overhaul and now this app changes makes my and a
lot of other peoples experience with Django so much better. Thank you!

2013/12/30 Aymeric Augustin 

> ## Solution 2.a — Require users to call django.setup() before they use the
> ORM. setup() would configure the settings and populate the app registry.
>

I think this approach is the only proper solution. We have hit multiple
bugs within the import mechanism in the ORM with models not finding related
models (#20143 for instance). To workaround this, we always explicitly call
get_models() in the web app servers, celery workers and scripts before
anything else. Whenever we forget the get_models() call, things break in a
lot of interesting ways. :-)

Being explicit with an exception when trying to use the ORM in a
non-initialised state would be very very useful in situations like this,
and would have saved us many hours of confusion.

--
Andreas Pelme

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CA%2B8bEMHrH10OLxQ-50UeRM08YDpAiRhXGq6LhTpOG7JsRp_M7w%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Feature request: post/pre commit/rollback signals

[Andreas Pelme]

On 10 jan 2014, at 09:19, Jesús Espino  wrote:

> Hi folks!
> 
> The propose is add post and pre signals for commit and rollback actions (as 
> sqlalchemy orm events). This allows attach some code when a transaction is 
> committed or rolled back.
> 
> I have some problem, such as send email only when a transaction is committed 
> successfully, and it's can be done with simple monkey patching the db 
> backend. But it not appear to be an isolated problem, and would be awesome if 
> these signals are included in core.
> 
> If a purpose is accepted, I can take care of making the issue + pull-request.
> 
> Greetings.
> Jesus.


Hi,

This is a tricky issue that has been discussed earlier, you will probably want 
to check out ticket #14051:

https://code.djangoproject.com/ticket/14051

—
Andreas Pelme

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/948D5F1E-F86D-44A0-9EEA-F1E9A6A4A382%40pelme.se.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [GSOC] Shifting to Py.Test and Improving the Test Suite

[Andreas Pelme]

Hi Chris,

It looks like you invoke nosetests and not py.test, therefore you do not get 
the results one would expect with py.test:

On 7 apr 2014, at 11:52, Chris Wilson  wrote:
> OK, here is one.
> 
> chris@lap-x201:~/projects/2014/webassets$ .ve/bin/tox -e py27 -- 
> tests.test_filters:TestPyScss.test_search_path

...

> py27 runtests: commands[0] | nose tests 
> tests.test_filters:TestPyScss.test_search_path

...

>  File 
> "/home/chris/projects/2014/webassets/.tox/py27/local/lib/python2.7/site-packages/nose/case.py",
>  line 197, in runTest


This is what the output from a similar test with pytest would result in: 
(Screenshot at http://pelme.se/~andreas/private/pytest_assertion.png to give 
the correct colors/formatting.)

$ py.test test_foo.py
= test session starts 
==
platform darwin -- Python 2.7.5 -- pytest-2.5.2.dev1
plugins: xdist
collected 1 items

test_foo.py F

=== FAILURES 
===
 test_something 


def test_something():
foo = {'out.css': 'something'}
>   assert foo.get('out.css') == 'h1 {\n  color: #ff;\n}\na {\n color: 
> #ff8000;\n}\n'
E   assert 'something' == 'h1 {\n  color: #ff...\n color: #ff8000;\n}\n'
E - something
E + h1 {
E +   color: #ff;
E + }
E + a {
E +  color: #ff8000;
E + }

test_foo.py:4: AssertionError
======= 1 failed in 0.01 seconds 
===



Cheers,
Andreas

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/38638CE1-A189-4954-A412-FE77908436AA%40pelme.se.
For more options, visit https://groups.google.com/d/optout.

Re: PEP 484 type hinting in Django

[Andreas Galazis]

To me one approach would be to put a cut off for any merged code /PR  start 
inlining type hints/annotations for all new code. This seems to simple to 
be a solution but at the end of the day as code gets updated even 
bigger part of the codebase will have type hints. The question is whether 
partial type-hinting is actually useful, but at least it supports heading 
towards the right direction.

On Tuesday, 10 April 2018 15:24:19 UTC+3, dmoisset wrote:
>
> A long due update on this, given that the question popped up recently:
>
> I worked at some time in type annotations and published some for Django 
> 1.10 ; I'm currently not working on them, given that my current work has 
> not been very close to Django development
>
> After trying a few things, the best way to get this would be:
> 1) Add annotations upstream. Having to maintain a separate set of 
> annotations is probably not sustainable and will lead to errors. This of 
> course needs annotations accepted by the team.
> 1.1) Integrate typechecking into Django's build/CI process. This would 
> ensure that annotations are consistent with the implementation.
> 1.2) Annotations do not need to be exhaustive (and there are certainly 
> parts of Django which could benefit from them much more than others)
> 2) Generate stubs from the annotated source. This was not possible when I 
> wrote my stubs, but I've commited the mypy improvements with that feature. 
> 2.1)A few stubs may need to be overridden manually from the autogenerated 
> (I found a few examples of this when I wrote my stubs)
> 3) Distribute annotated files separately (not with typeshed, and probably 
> not with django) so people can install the version they need. Distribution 
> of stubs has been problematic, PEP 561 should solve it but an 
> implementation for it has been just merged last week into mypy, and 
> probably released soon.
>
> Before PEP561 and the ability to generate stubs, I don't believe it was 
> practical to make this project sustainable. At this time, it might be and 
> it can be a good time to restart this discussion.
>
> Best,
> D
>
> On 6 April 2018 at 12:19, Florian Apolloner  > wrote:
>
>> To the best of my knowledge JetBrains fundled the money to Django for 
>> that specific purpose -- so yes the funding should be here if needed. That 
>> said, there is no decission on a) whether we actually want type hints and 
>> b) if the should be inline or stubs. Those two points have to be cleared 
>> first.
>>
>> Cheers,
>> Florian
>>
>> On Friday, April 6, 2018 at 9:11:04 AM UTC+2, Eddy C wrote:
>>>
>>> Hi Tim, do you know if JetBrains still willing to fund the project as 
>>> the upcoming django 2.1 will only support python 3.5+ that pave the way for 
>>> inline annotations.
>>>
>>> On Wednesday, August 17, 2016 at 11:41:03 PM UTC+10, Tim Graham wrote:

 The JetBrains announcement that they want to fund the project isn't a 
 guarantee that it'll be implemented. The feature needs to go through the 
 normal feature acceptance process, which as Markus said, might involve a 
 DEP.

 Assuming the idea is accepted, my sense on timing would be to wait 
 until January when Django drops support for Python 2.7 and 3.4 in master. 
 Then we could use inline annotations rather than the stub files.

 Past discussions of type hinting:

 https://groups.google.com/d/topic/django-developers/z_P1TvJ6QG8/discussion

 https://groups.google.com/d/topic/django-developers/xOTmq93YZuQ/discussion

 On Wednesday, August 17, 2016 at 5:30:56 AM UTC-4, Florian Apolloner 
 wrote:
>
>
>
> On Wednesday, August 17, 2016 at 11:06:47 AM UTC+2, dmoisset wrote:
>>
>> @Florian
>> Would you care to ellaborate? I couldn't find the post you mention 
>> (although requests is one of the few 3rd party projects that have 
>> support 
>> at the official typeshed repository, 
>> https://github.com/python/typeshed )
>>
>
> https://lwn.net/Articles/643269/ and https://lwn.net/Articles/643399/ 
> -- might be that things changed by now.
>
 -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com .
>> To post to this group, send email to django-d...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/django-developers.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/6f2b5dd9-f8e6-4b67-8e25-585cbd802413%40googlegroups.com
>>  
>> 
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> -- 
> Daniel F. Moisset - UK Country Manager - Machinali

Re: PEP 484 type hinting in Django

[Andreas Galazis]

I agree with you, but at some point, we could combine solid annotated core 
with a cut off for non annotated code? Otherwise, this will end up being a 
loop.

On Wednesday, 11 April 2018 17:16:21 UTC+3, dmoisset wrote:
>
>
>
> On 11 April 2018 at 11:21, Andreas Galazis  > wrote:
>
>> To me one approach would be to put a cut off for any merged code /PR  
>> start inlining type hints/annotations for all new code. This seems to 
>> simple to be a solution but at the end of the day as code gets updated even 
>> bigger part of the codebase will have type hints. The question is whether 
>> partial type-hinting is actually useful, but at least it supports heading 
>> towards the right direction.
>>
>>
> I don't think that approach will work. Partial type hinting is useful and 
> viable, but not randomly you need to do it bottom up (covering basic 
> abstractions first). So the place to start is probable the parts of django 
> that change less and are more solid foundations (in my case I started with 
> requests, views and URL resolvers, etc)
>
>
>
> -- 
> Daniel F. Moisset - UK Country Manager - Machinalis Limited
> www.machinalis.co.uk <http://www.machinalis.com>
> Skype: @dmoisset T: + 44 7398 827139
>
> 1 Fore St, London, EC2Y 9DT
>
> Machinalis Limited is a company registered in England and Wales. 
> Registered number: 10574987.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/2d2a2ce3-23c7-4616-af48-52d2c1c6c866%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Pluggable secret key backend

[Andreas Pelme]

Hi,

settings.SECRET_KEY can be used for sessions, password resets, form wizards and
other cryptographic signatures via the signing APIs. Changing SECRET_KEY means
that all of those will be invalidated and the users will be affected in weird
ways without really knowing what happened. (Why am I logged out? Where did my
form submission go? Why does not this password reset link work?). This is
desirable in case the key is compromised and swift action must be taken.

There are other situations when it would be nice to change the SECRET_KEY when
this sudden invalidation is not desirable:

- When someone leaves a project/company that had access to the production
  system. After SSH keys/login credentials is revoked the developer could
  potentially have a copy of the secret key. It is essentially a backdoor with
  full remote access. It would be wise to rotate the key in those cases.

- Periodic and automatic rotations of keys to make it less useful in the
  future.

The current situation of a single SECRET_KEY makes key rotation impractical. If
you run a busy site with active users 24/7, there is never a nice time to
change the SECRET_KEY.

A solution for this problem would be sign new secrets with a new key while
still allow signatures made with the old key to be considered valid at the same
time. Changing keys and having a couple of hours of overlap where signatures
from both keys are accepted would mitigate most of the user facing problems
with invalidating sessions, password reset links and form wizard progress.

You could do this today by implementing your own session backend, message
storage backend and password reset token generator but that is cumbersome and
does not work across reusable apps that directly use low level Django signing
APIs unless they too provide hooks to provide your own secret.

I propose a pluggable project wide secret key backend
(settings.SECRET_KEY_BACKEND maybe?) with an API something like:

class SecretKeyBackend:
  def get_signing_key(self): …
  def get_verification_keys(self): ...

The default (and backward compatible) backend would then be implemented as
something like:

class SecretKeySettingsBackend:
  def get_signing_key(self):
return settings.SECRET_KEY
  def get_verification_keys(self):
return [settings.SECRET_KEY]

django.core.signing.Signer.{sign,unsign} would need to be updated to use this
backend instead of directly using settings.SECRET_KEY.

That would solve the problem project wide and work across any third party
application that uses django.core.signing directly.

This would open the door for third party secrets backend packages that
retrieves keys from systems such as Hashicorp Vault, AWS Secrets Manager,
Google Cloud KMS, Docker Secrets etc.

Having a method that retrieves the key would allow changes to secret key during
run time instead of relying on a hard coded setting would allow the key to
change without restarting the server process.

Would something like this be worth pursuing? Could it be designed in som other
way? I could not find any previous discussion/tickets on this and thought it
would be a good idea to discuss it here before opening a ticket or making an
attempt at a PR. :)

Cheers,

Andreas


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/20D8A2BD-BC9C-4F02-9038-044687165DE9%40pelme.se.
For more options, visit https://groups.google.com/d/optout.

Re: Pluggable secret key backend

[Andreas Pelme]

> On 10 Nov 2018, at 13:00, ludovic coues  wrote:
> 
> I don't see how this would work.
> 
> For example the session. You take the user cookie. You try to validate with 
> your secret key. That doesn't work because the current key is the new one.
> 
> With a custom cookie backend, you could check if the old secret could 
> validate the cookie. But you need to change your cookie backend to handle the 
> case of multiple secret key. And all third party session backend need to 
> update.


I propose that we make the low level django.core.signing aware of multiple 
keys. Everything that is already using django.core.signing such as signed 
cookies, sessions and password reset tokens would need *not* need to change.

Cheers,
Andreas

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/5ED0D5A2-AC77-4231-824C-2EDDD7F2A903%40pelme.se.
For more options, visit https://groups.google.com/d/optout.

Re: Pluggable secret key backend

[Andreas Pelme]

On 10 Nov 2018, at 13:29, Adam Johnson  wrote:
> 
> Hi Andreas
> 
> I like your proposal, moving to a backend is an elegant way of solving both 
> the immediate problem and opening up the other possibilities you mentioned.

Thanks Adam, I am glad you like the proposal. :)

> I think it would also be nice to have an "out of the box" way of rotating the 
> key, without needing to implement a custom backend. Perhaps a second setting 
> OLD_SECRET_KEYS that may contain a list of old keys that are returned for 
> verification too? Or we could allow SECRET_KEY to be a list/tuple, and if so, 
> sign with the first and verify with all of them.

Agreed, I will add something like that then! :)

Cheers,
Andreas

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/A16A11DF-1439-46EF-BF0D-85C483F53608%40pelme.se.
For more options, visit https://groups.google.com/d/optout.

Re: PEP 484 type hinting in Django

[Andreas Galazis]

Just stumbled on this blog post: 
https://www.willmcgugan.com/blog/tech/post/adding-type-hints-to-the-django-orm/.
 
An interesting but too theoretical suggestion: We could even get rid of 
field/ field configuration. But I guess that's far ahead of what we are 
trying to achieve xD.

On Wednesday, 12 December 2018 18:06:16 UTC+2, Carlton Gibson wrote:
>
> Hi all. 
>
>
> Where are we with this Type Hinting work? 
>
>
> I just closed https://code.djangoproject.com/ticket/30019 as needsinfo 
> pointing back to this thread. 
>
>
> As far as I can see: 
>
> * There's keenness for this. 
> * There's a number of people who are prepared to put in the effort. 
> * But we just need a strategy. 
>
> It looks then like a coordination problem. Do we just need to get said 
> people in a (virtual) room...? 
>
> Maybe one of you — anyone — opening a DEP to begin the conversation would 
> be enough.. 
> (That's just an idea.) 
>
> Looking here: 
> https://github.com/django/deps/blob/master/final/0001-dep-process.rst#dep-submission-workflow
> I'd guess "Forming the Team" is relevant. 
>
> Happy to help if I can. 
> (I recall Frank mentioned it no too long ago...) 
>
> Kind Regards,
>
> Carlton
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/30039f0c-6cbd-4ef4-954a-d0343b21e46d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: django.utils.dateparse

[Andreas Pelme]

On 4 Feb 2019, at 15:04, Giuseppe De Marco  wrote:
> 
> python3 -m timeit -s   "import sys, os; sys.path.append(os.getcwd()); from 
> datetime_heuristic_parser import datetime_heuristic_parser; 
> print(datetime_heuristic_parser('04/12/2018 09:7:4Z'))"


That command is not correct. timeit -s takes two arguments: setup code and 
benchmark code. This command just executes the setup code and does not run any 
code at all for the actual benchmark.

The correct command would be something like this (I did not run this command 
myself but you get the idea):
python3 -m timeit -s   "import sys, os; sys.path.append(os.getcwd()); from 
datetime_heuristic_parser import datetime_heuristic_parser” 
"datetime_heuristic_parser('04/12/2018 09:7:4Z’)"

Cheers,
Andreas

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/0DCE461D-F01F-48C6-B5DD-CBCEFE2895FC%40pelme.se.
For more options, visit https://groups.google.com/d/optout.

Re: Discussion related to ticket #26822 (new migrations, --keepdb and --parallel option)

[Andreas Pelme]

Hello,

> On 5 juli 2016, at 00:22, Romain Garrigues  
> wrote:
> After some investigation, I have seen that, in case of keepdb context, in 
> django/db/backends/*/creation.py, if the cloned databases already exist, we 
> don't touch them, which leads to this new field not created in cloned ones.
> 
> I have proposed in the PR to rebuild the cloned databases, even with keepdb 
> option, to be sure that we always have the cloned databases with the latest 
> migration state.
> 
> The problem with this method is that it will increase test database 
> initialization time, as we will now systematically copy all cloned databases, 
> even with --keepdb option (except the default one).

We’ve been doing similar things in pytest-django (with the --reuse-db option 
and pytest-xdist) and faces similar problems. Currently you have to 
force-recreate the databases and then all processes will run migrations and it 
is very slow.

I’ve been playing around with a solution to this: In my own project I create a 
template database and call it `test_myproject_`. 
Whenever a migration file changes (an existing file or a new migration file) - 
a new database is created and all clones are recreated.

Currently this lives as a hack in my own code base, but I would like to explore 
this further and it could be a way forward. Here is my scripts that calculates 
the hash and creates the databases:
https://gist.github.com/pelme/4b3dac475cd6b1dec4fd67d25d2e7cdc
https://gist.github.com/pelme/4a3ad3a62b6244068ff63736342f9509

This method could be refined: It is not necessary to create a database with a 
new name every time migrations change. I.e. we could create a private table 
with a single row that contains the hash.

This approach hashes only the migration files directly involved in migrations, 
if you are using a 3rd-party library that’s imported, that will not trigger a 
new migration run.

As an end user of this the experience is quite nice: You only experience the 
migration/cloning slowness whenever migrations actually changed, otherwise 
everything is fast. You don’t have to remember any special command line options.

Cheers,
Andreas

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/5A553E81-F6FF-4A85-ADB3-01D66AE072B9%40pelme.se.
For more options, visit https://groups.google.com/d/optout.

GSoC 2007 Status Update: Django REST interface

[Andreas Stuhlmüller]

This is the first weekly status update for my Summer of Code project,
a generic REST interface for Django. If you haven't seen it, take a
look at http://code.google.com/p/django-rest-interface/.

This week, I have started checking in code. I am always happy about
feedback. Add a few lines to your (non-production!) urls.py [1], make
your models available in XML or JSON format and tell me what works for
you and what doesn't.

Compared to my timetable, I have managed to start a few things I
planned to get done next week or the week after (create/update/delete
operations, basic url patterns generator) and I have yet to add the
option to format output with templates, something I wanted to get done
by today.

Besides templates, I am going to work on error handling, pagination,
the option to limit the model fields exposed by the API, a
_load_put_and_files equivalent to _load_post_and_files and the
inclusion of resource urls in serialized output until next Friday.

Regards,
Andreas

[1] Like this: 
http://django-rest-interface.googlecode.com/svn/trunk/django_restapi_tests/urls.py

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: GSoC 2007 Status Update: Django REST interface

[Andreas Stuhlmüller]

On 6/2/07, Amit Upadhyay <[EMAIL PROTECTED]> wrote:
> Django is a web development framework, and not model development. REST
> (XMLRPC/SOAP) API are created to expose webservices, which by definition
> could be anything, not just models. CRUD is myopic world view.

Thanks for your comment!

You are right in saying that making models CRUD-accessible is just a
small part of what should be possible with the REST API. That's why my
proposal includes two parts: Easily configured CRUD method access
patterns for models on the one hand and resources that don't
correspond 1:1 to models on the other hand. I will take a closer look
at the tickets you opened when I get to the second half of my
proposal, but note that what I am working on is not XMLRPC or SOAP.

Regards,
Andreas

On 6/2/07, Amit Upadhyay <[EMAIL PROTECTED]> wrote:
> On 6/1/07, Andreas Stuhlmüller <[EMAIL PROTECTED]> wrote:
> > This week, I have started checking in code. I am always happy about
> > feedback. Add a few lines to your (non-production!) urls.py [1], make
> > your models available in XML or JSON format and tell me what works for
> > you and what doesn't.
>
> I contributed patch for
> http://code.djangoproject.com/ticket/547, and tho did not
> object much when it was closed in favor of
> http://code.djangoproject.com/ticket/115, but I feel both
> your approach and #115 are wrong.
>
> Django is a web development framework, and not model development. REST
> (XMLRPC/SOAP) API are created to expose webservices, which by definition
> could be anything, not just models. CRUD is myopic world view. Have you seen
> the API's that Flickr/Delicious/FaceBook exposes? *None* of them are
> (strictly) CRUD related. What should you make easy is making any python
> function available through REST, and that function can in turn implement
> CRUD if you want. I am not against CRUD api, but thats a very small, and in
> my opinion unused subset of what REST etc would be used for.
>
> Developers, please relook at 547, and compare it with 115. 547 is closed by
> jacob saying " This is a subset of #115." which I feel is not the case, as
> #115 is "Models CRUD via web services".
>
> Related bug: http://code.djangoproject.com/ticket/552
>
> --
> Amit Upadhyay
> Vakow! http://www.vakow.com
>  +91-9820-295-512
>  >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

GSoC 2007 Status Update II: Django REST interface

[Andreas Stuhlmüller]

This is the second weekly status update for my Summer of Code project,
a generic REST interface for Django [1].

This week, I was mostly working on improving the architecture of the
code, in particular by splitting ModelResource into two classes,
Collection (for querysets) and Entry (for individual models). Other
improvements include appropriate HTTP status codes for POST/PUT
requests, the option to use templates for the response formatting
(TemplateResponder), basic pagination (works both with serialized and
templated output) and load_put_and_files, a function that populates
request.PUT and request.FILES via _load_post_and_files.

Compared to my initial timetable, I am in time. Looking at my last
status report, I notice that error handling isn't perfect yet (needs
to be unified) and the option to limit the model fields exposed by the
API and the inclusion of resource URIs in serialized output are not
done yet.

The main thing I will do next week is to think about and improve
automated URL generation.

I am still working on the first part of my proposal (resources that
correspond directly to models). If you have any thoughts on what
should or should not be possible with such a REST API or if you have
specific comments on the code, please let me know.

Regards,
Andreas

[1] http://code.google.com/p/django-rest-interface/

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

GSoC 2007 Status Update III: Django REST interface

[Andreas Stuhlmüller]

This is the third weekly status update for my Summer of Code project,
a generic REST interface for Django [1].

I spent the SoC-part of this week mostly thinking about automated URL
pattern generation, wrote some code and added a few tests that
demonstrate how to use this feature. Currently, you can:
1. Let the REST interface generate all your API URLs automatically.
2. Give a custom base URL and let the REST interface do the rest.
3. Subclass model_resource.Collection, overwrite Collection.get_entry
and (if you want to filter the data returned) Collection.read, and use
arbitrary custom URLs.

I am not completely happy with how this works yet. Using custom URLs
should be as intuitive as using automated URLs, and I don't think
that's the case yet. If you see how things could be improved, please
let me know.

Other changes I made this week include the option to restrict model
field access via expose_fields and better error handling (all
exceptions are caught in Collection.dispatch, the Responder class is
responsible for returning a nicely formatted error message).

Besides thinking about how custom RESTful URLs could be implemented in
a more intuitive way, my main task for next week is to think about the
relationship between the REST interface and authentication. It should
be possible to have different levels of access for authenticated and
unauthenticated users.

Criticism, ideas and suggestions are welcome!

Regards,
Andreas

[1] http://code.google.com/p/django-rest-interface/

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

GSoC 2007 Status Update IV: Django REST interface

[Andreas Stuhlmüller]

This is the fourth weekly status update for my Summer of Code project,
a generic REST interface for Django [1]. Issues I worked on this week:

1. Authentication. I added authentication hooks to the Collection
class, an implementation of HTTP Basic and Digest authentication
(stateless, header is sent with each request), authentication tests
and a function that connects HTTP Basic auth with Django's User model.

2. Continued work on URL generation. It is now possible to overwrite
all URLs by subclassing Collection. The distinction between methods
that return URLs and methods that return URL patterns got clearer. The
URL generation part of the REST interface is still going to change a
lot.

3. Appropriate error messages. I added methods to JSONResponder and
XMLResponder that return mimetype-specific error responses which
include a human readable error message, application-specific errors
(e.g. missing ) and a machine readable status code.

According to my initial timetable, next week is reserved for writing
unit tests and documentation. Although not complete, I have already
written quite a few tests along the way. The main task will be to fit
these tests into Russell's unit testing framework. Another issue that
needs to be resolved next week is that authentication should allow
different levels of access to the same resource for authenticated and
unauthenticated users.

As always, criticism, ideas and suggestions are welcome!

Regards,
Andreas

[1] http://code.google.com/p/django-rest-interface/

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

GSoC 2007 Status Update V: Django REST interface

[Andreas Stuhlmüller]

This is the fifth weekly status update for my Summer of Code project
[1]. Since it is the last update before I take two weeks off to study
for my exams (as arranged with my mentor), this is probably a good
time to sum up the current status of the project and to talk about
what's left to do.

The basic framework is done. If you feel experimental, give it a try
and take a look at the RESTful API for your Django models, the
serialized (JSON, XML) or templated output, the authentication
mechanisms and the automatically generated URL patterns. Of course, I
can't give any guarantee that by the end of the SoC project any
specific feature will still look like it does now -- and I am fairly
certain that it will be a lot easier to use custom URLs then than it
is now.

I am currently in the process of moving the tests to Django's unit
testing framework. This turns out not be quite as as straightforward
as I thought (e.g. there is no simple alternative to httplib2's
add_credentials method for authentication tests), therefore I will
continue working on that after my break.

Besides finishing up the model resource part (this includes URL
generation, unit tests and a few smaller issues), my main objective
for the remaining time will be to think about how we can make it
easier to write REST APIs that don't correspond 1:1 to models. If I
wanted to implement the Atom Publishing Protocol in Django, what would
I do? Is there code that needs to be written for almost every REST app
in some way or the other and that we can take care of?

Expect the next update around July 20th.

Regards,
Andreas Stuhlmüller

[1] http://code.google.com/p/django-rest-interface/

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: GSoC 2007 Status Update V: Django REST interface

[Andreas Stuhlmüller]

On 7/3/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
> I would love to try it out with one of my apps, but maybe a really
> quick HOWTO on how it's supposed to be set up with a project?

Thanks for the suggestion! I just added a short tutorial to the
project page [1]. The five steps contain only the most basic things
needed to get the REST interface running. If anything is unclear or
doesn't work as expected, please let me know.

You don't need to add anything to INSTALLED_APPS.

As long as the API definitions remain relatively simple, just use
urls.py. Once you inherit from Collection and write custom resource
methods, it might make sense to store your classes somewhere else in
the view layer and just import from there. How the long-term guideline
regarding where to store API code will look like depends on a) how the
URL generation feature changes and b) which insights we gain from the
work on the non-model-based part of the project. I will remember your
point when I write the documentation on how to use the API.

Regards,
Andreas

[1] http://code.google.com/p/django-rest-interface/

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: GSoC 2007 Status Update V: Django REST interface

[Andreas Stuhlmüller]

On 7/4/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
> Would it be possible to, instead of passing in collection_url_pattern
> to the Collection, pass in the name of a named url pattern, which
> internally would get reverse()'d?

Yes. After talking to Malcolm about how URL customization could be
made easier, I intend to add a view function, e.g. rest_view, that
handles detecting the method and dispatching based on that. Both
automatically generated and custom patterns would then look like this:

url('^/articles/(?P\w+)/$', rest_view,
 entry_method_map, "article-entry")

url('^/articles/$', rest_view,
 collection_method_map, "article-collection")

with

entry_method_map = {
 'GET' : get_view,
 'PUT' : put_view,
 'DELETE : delete_view
}

collection_method_map = {
 'GET' : get_view,
 'POST' : post_view
}

In order for automatic URL generation to work, you could pass the URL
pattern to the Collection() instantiation call. When the URL for an
individual model is needed, we just reverse() the entry URL pattern.
Note that in the the above example we might replace the url() call
with an api_url() call that saves us from repetitive definitions.

Regards,
Andreas

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: GSoC 2007 Status Update IV: Django REST interface

[Andreas Stuhlmüller]

On 7/13/07, David Larlet <[EMAIL PROTECTED]> wrote:
> * About users, how can I handle django users' permissions? John
> Sutherland had already done some work on django-crudapi [1] and it
> could be interesting to allow this access control.

Good question. I would subclass one of the authentication classes,
e.g. HttpBasicAuthentication, and add a has_perm() check dependent on
the request method to is_authenticated().

Currently, REST authentication is independent from Django's
authentication middleware. It might make sense to move most of the
code in HttpBasicAuthentication and HttpDigestAuthentication to
django.contrib.auth.backends and to let the middleware do most of the
work.

> I'm not fond of CRUD names and I prefer to keep an information on
> the HTTP verb in the function name but it's maybe personal.

I chose the CRUD names as they better describe what the methods
actually do, but if the consensus is that it is more important to
highlight the HTTP method, I will change this.

> * About dispatch, why don't you use the __call__ trick?

Fixed, thanks for the pointer.

> * About verbs, I know this is not really GSoC related but what is the
> recommended way to handle fake PUT and DELETE from a browser? The
> HttpMethodMiddleware[4]? Maybe a simple form/deletion in the polls
> application will be an interesting example for a real django case.

Yes, I recommend HttpMethodsMiddleware [4]. I just added a "delete"
button to the template example and verified that it works if you add
HttpMethodsMiddleware to your middleware classes.

Thanks for your comments!

Regards,
Andreas

[4] http://www.djangosnippets.org/snippets/174/

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

GSoC 2007 Status Update VI: Django REST interface

[Andreas Stuhlmüller]

This is the sixth status update for my Summer of Code project. You can
find more information as well as an updated timetable at [1]. During
the last two weeks, I mainly worked on unit tests and on simplifying
the relationship between the REST interface and URLs.

I added manage.py, settings.py and tests.py in order to enable tests
with Django's unit testing framework, converted the existing tests,
removed the old (now obsolete) test files, added fixtures and
restructured the HttpDigestAuthentication code in order to allow
testing without a web server and httplib2.

You now need to specify the URLs for collections and entries within
urlpatterns. This change is backwards-incompatible. See the five
examples [2] which replace the large urls.py example for more
information on how to use the interface. The URL generation logic now
uses reverse(). The class for collection entries can be specified in
the Collection() constructor call. Thanks to David Larlet for
recommending the the __call__ trick as a replacement for the
dispatch() methods.

I am currently reading "RESTful Web Services" by Leonard Richardson
and Sam Ruby. See [3] for a summary of the main points from the
chapter on best practices for REST-oriented architectures with short
notes on where the REST interface still needs improvement.

If you feel that there are important issues I am missing, please write
me or just open a ticket [4].

Regards,
Andreas Stuhlmüller

[1] http://code.google.com/p/django-rest-interface/
[2] 
http://django-rest-interface.googlecode.com/svn/trunk/django_restapi_tests/examples/
[3] http://code.google.com/p/django-rest-interface/wiki/BestPractices
[4] http://code.google.com/p/django-rest-interface/issues/list

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: GSoC 2007 Status Update VI: Django REST interface

[Andreas Stuhlmüller]

On 8/6/07, David Larlet <[EMAIL PROTECTED]> wrote:
> * What about additional pages like: create form, update form, delete
> confirmation and so on? I've just noticed the get_create_form at the
> end of the TemplateResponder but how can I redirect to this function?
> (I haven't seen how to do this in url generation.)
> * How can I deal with custom forms for a resource?

I extended the template example to include a create form and an update
form [1]. Note that you need something like HttpMethodsMiddleware [2]
in order to use the update form and that the forms are currently not
redisplayed after erroneous input. For any other forms and extensions,
just write views and add the URLs to urlpatterns (above the catch-all
resource URLs). If you feel that the REST interface should support
this kind of work and have ideas how this might look like, please let
me know.

> * Is it possible to specify the format at the end of an url or will it
> break the parsing?

Yes, this is possible if you specify the urlpatterns for collection
and entry individually. I just added such an example [3].

> And last but not least, what is the current stability?

If you can, wait at least until Fri, August 24th before using the REST
interface in any kind of production environment. By then I hope to
have done more thorough testing. If you stumble upon a bug, please let
me know.

Thanks for your feedback.

Regards,
Andreas Stuhlmüller

[1] 
http://django-rest-interface.googlecode.com/svn/trunk/django_restapi_tests/examples/template.py
[2] http://www.djangosnippets.org/snippets/174/
[3] 
http://django-rest-interface.googlecode.com/svn/trunk/django_restapi_tests/examples/fixedend_urls.py

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: GSoC 2007 Status Update VI: Django REST interface

[Andreas Stuhlmüller]

On 8/11/07, David Larlet <[EMAIL PROTECTED]> wrote:
> Here is my suggestion: the current forms from model or instance are
> great but you often need a custom form, what about an extra-argument
> to the collection (better because I already need a form in the
> collection in order to return a creation form on the list view, about
> that if you have better idea...) or the responder with the form class?

Forms seem to be relevant only if you use TemplateResponder, therefore
I am against adding form-specific code to ModelResource. You can
already specify extra_context for TemplateResponder. Might that be
sufficient?

> * in order to be consistent with template, the name should be related
> to elem._meta.module_name and not self.template_object_name

Fixed, thanks.

> About urls, how can I get a "reversed url". For example, is it planned
> to deal with {% url %} tag and permalink? I've tried to do this
> without any success.

This should definitely be possible, but I have not found an elegant
way to do this yet.

> So thank you for your awesome work and I will not hesitate to fill
> a bug (and maybe to help if GSoC allow that) if I find one.

Once the SoC final evaluation is over (Mon, August 20th), it will be a
lot easier (=possible) for me to work together with other people and
to accept code contributions. This strikes me a little strange since
SoC is supposed to encourage work within the open source community but
what you end up with is "work on your own, supported by your mentor"
(which, admittedly, works quite well).

Regards,
Andreas Stuhlmüller

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

GSoC 2007 Status Update VII: Django REST interface

[Andreas Stuhlmüller]

This is the seventh status update for my Summer of Code project, the
Django REST interface [1].

Last week, I extracted code from the ModelResource class that is not
directly related to models but useful for resources in general, moved
it to Resource and made ModelResource inherit from Resource. I added
an example of a non-model-based resource and corresponding tests. The
generic resource class now uses the same kind of authentication that
ModelResource uses.

You can now run "python manage.py syncdb; python manage.py runserver"
in the django_restapi_tests folder and get a testable environment that
includes some initial data.

I added code for forms to TemplateResponder (create_form,
update_form), an example that shows how to use this code and another
example that shows how to specify the data format at the end of your
URLs.

A few days ago, I started writing a short tutorial [2] that describes
how to provide an API for an existing Django application using the
REST interface. If you have any questions, ideas, can't get the
interface to work or want to contribute bugfixes and other
enhancements, please go ahead.

Regards,
Andreas Stuhlmüller

[1] http://code.google.com/p/django-rest-interface/
[2] http://code.google.com/p/django-rest-interface/wiki/RestifyDjango

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: GSoC 2007 Status Update VII: Django REST interface

[Andreas Stuhlmüller]

On 8/17/07, David Larlet <[EMAIL PROTECTED]> wrote:
> Just one (latest?) thought, it's a bit hard to debug because
> 400 errors are not really verbose, is it possible to find a
> way to make debug easier? For the moment, I have: if
> settings.DEBUG: print i.errors but I'm sure it can be better.

I made the 400 error template a bit more verbose. The responder
classes that use serializers include the model errors in their
responses anyway. Is this what you were looking for?

> In fact, the problem is that you use ResourceForm in
> model_resource and it could be interesting to use the given
> form_class instead.

You're right, it might be nice to be able to change the validation and
data cleanup behavior of resources. Check out the latest revision and
tell me if this is along the lines you were thinking.

Regards,
Andreas

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

GSoC 2007 Status Update VIII: Django REST interface

[Andreas Stuhlmüller]

 interesting extension at least for
your pet projects, please give it a try, test, test, test, create
tickets for any bugs you find and then, test some more.

Thank you, Malcolm Tredinnick, for great advice and great patience.
Next time, I will try to reduce the number of times I made your head
explode and your brain leak out of your nose a bit :-). I am also
grateful to anyone who asked questions or suggested features and, of
course, to the community of Django developers in general.

I hope you like and use the Django REST interface.

Cheers,
Andreas Stuhlmüller

[1] http://code.google.com/p/django-rest-interface/
[2] http://www.aiplayground.org/artikel/improving-django/
[3] 
http://django-rest-interface.googlecode.com/svn/trunk/django_restapi_tests/examples/
[4] http://code.google.com/p/django-rest-interface/issues/list
[5] http://groups.google.com/group/django-developers/msg/00481aec2ef71a38

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---