Re: Django's testing infrastructure
Great work! When it comes to hardware, I was kind of convinced that Django have been already using the snakebite network to access various platforms for builds testing (http://djangoninja.com/post/73799680/django-on- snakebite) Isn't their network compatible? (more about SnakeBite: http://www.snakebite.org/) Jakub On Feb 26, 4:54 am, Eric Holscher wrote: > Hey everyone, > > During the sprints, I worked to set up a hudson instance for Django. This is > hopefully going to be the way that we are going to go forward for now with > doing continuous integration for Django. I have a pretty good setup going > currently, and want make it really fantastic. At this point in time, what we > really need now is some more hardware to be able to run tests on. > > The current setup is hosted at:http://hudson.djangoproject.com/ > > Currently, I have tests running on the following architectures: > > Django trunk: > > Solaris: > Python 2.4-2.5 > Databases: sqlite, postgres, mysql > > Ubuntu: > Python 2.5-2.6 > Databases: sqlite, postgres, mysql > > Django 1.1.X: > > Solaris: > Python 2.5 > Databases: sqlite, postgres > > This gives us pretty good coverage currently, but the whole point of doing > CI is that we catch bugs on other platforms we wouldn't normally run on. > > What we need > === > > So I'm looking for people who can offer other boxes that they would like to > see tested. Currently the big ones we aren't covering are Windows boxes, and > the Oracle backends. There are lots of other permutations that we could be > testing against (Different python runtimes being a good example). > > However, we don't want to get in a situation where boxes that people have > set up just disappear. So, I'm only currently looking for machines that > people would be able to dedicate to the effort. We would require a > django-testing user account on the box, with our SSH key on it. There would > also be a team of trusted users, who would have access to this key and thus > your machine. > > We want the build farm to be stable and useful, and in the past we have had > too much trouble having machines just disappear. > > Requirements > === > > Currently the hudson requirements seem to be about <1GB of disk space, with > 512MB of ram. I'm also looking into some pony build/barn based alternatives > that would knock the memory requirements down pretty substantially. However, > for the current 1.2 release it looks like hudson is how we're going to make > it going forward. > > Note that for $20/mo a 512MB machine can be run on Rackspace cloud, so > another way that we might be able to get this going is to be able to have > donations to the DSF, and have them get some dedicated rackspace boxes. > However, for now, I'm hoping that we can cobble together enough stuff to get > 1.2 tested really well. > > Feedback > == > > If you have any thoughts on CI, or any advice, I would love to hear it. I'm > trying to make our Continuous Integration setup really kick ass, so feedback > is necessary to do this. I have some notes and ideas that I have been > recording while setting things up over at the pycon etherpad: > > http://pyconpads.net/django-testing > > Let me know if you have any thoughts, questions, or concerns. > > Cheers, > Eric -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: dbsettings, and user configurable app settings
have been hitting my head trying to come up with a better way of managing configs. I've worked with quite a lot of other systems, and one thing i like about django is how "simple" it "still" is. this simplicity is a strength, and having a registry makes things.. complicated. some wild ideas I have been thinking about: a) abstracting configuration to allow config backends. this might be an overkill, but being able to have settings in: a ldap backend, database backend, plain python file, or ini file. you choose. a simple, but powerful api allowing this might be nice. i work with hetergenous systems and being able to easily share certain settings between systems would be eaiser this way. b) easy way to register, override, configurations for apps. a settings.py in an app folder that is "magically" loaded the same way other things are would suffice, but there has to be a way to: -override and prioritize settings, as sometimes you have to override things. -mark defaults, get defaults from other apps, and mark some settings as obligatory. i have a django mingus blog now. when i upgrade mingus, and the many many many apps it has, how do i know without reading all the docs, if there is a new setting that i should set or change? that takes me to: c) configuration migrations and making them easy for users. django core can also benifit from this of course. an easy way to see what configurations have changed, which new ones are available, which ones have been deprecated... d) as mingus kevin points out, we need a conventional way of registering settings that are to be accessible via the admin and do not require an apache restart... damn, battery down, and stil want to write a new post. :D On Feb 26, 7:53 pm, Jared Forsyth wrote: > If you look at dbsettings (which again, might be dead - i don't know) it > solves a lot of those problems; settings are tied into the "sites" module, > and are presented to the user in a very friendly way. The problem of db > migration is still there, but I can envision an import/export to XML (or > something) functionality where you can get/transfer all settings (or just > for one/a few apps) to another installation. > I've forked my own copy of dbsettings for the project I'm currently working > on (as I said, it was last touched 2007, and out-of-the-box incompatible), > and improving, so I might try to get it polished and more integrated w/ > contrib.admin, and see how it works. > > Jared > > > > On Fri, Feb 26, 2010 at 11:00 AM, Wes Winham wrote: > > I'd love to see a better way of managing settings in the core of > > django. It's a real pain point sometimes for writing and using > > pluggable applications and there's a wide range of ways that > > application developers try to tackle it. Some have basically no > > settings, some plan on users reading the documentation and defining > > all required settings, some have a way of using defaults that can be > > overridden and some have hacked together ways of using the database > > for some settings (I think ReviewBoard does something like this, or > > did). The problem I see is that the *best* way hasn't really bubbled > > to the surface as far as I can tell. Kind of like how database > > migrations are a hard problem, settings management seems kind of > > tricky. > > > Are there things stopping someone from making a really good app for > > managing settings and letting that bubble up to the de facto solution > > like with South? If there are, maybe removing those things would be a > > good target before trying to build the actual solution in contrib. Of > > course, this is probably not the best time for discussing this since > > everyone is trying to get 1.2 out the door, but I'd love to see this > > area improved down the road. > > > My specific pain points: > > * With a given app, it's a crapshoot to find where they define their > > internal settings (some use app/conf/settings.py, some app/conf/ > > default.py, some app/default.py etc) > > * It's hard to tweak settings on deployment for different > > environments. Using a settings_local.py that gets included is a > > workable solution, but it requires you to maintain a monolithic file. > > Something like the conf.d/*.conf debian style of configuration might > > > Pinax is having a similar discussion right now. Maybe something will > > come out of that project that can be applied/adapted to some later > > version of django. That might be a place to apply some development if > > anyone is interested in working towards better settings management. > > >http://groups.google.com/group/pinax-core-dev/browse_thread/thread/d9... > > > -Wes > > > On Feb 26, 2:11 am, Jared Forsyth wrote: > > > I have been looking around for a way of managing user-configurable > > > application settings, and the only solution I have found is dbsettings, > > > which looks like it hasn't been touched in 3 years. > > > So, I would like to know: is dbsettings dead? Or is there a different > > > generally
Issues surrounding IDN validation and URLs in general
Validation of IDN (Internationalized Domain Names) was added in
[12474], but I noticed that the verify_exists option doesn't work when
you use an IDN. This is caused by urllib2 not supporting IDN and the
validation code using the original unicode version of the URL when
testing for existence.
The problem is within URLValidator and can be fixed relatively easily
by using the IDNA-encoded version of the domain when testing that the
URL exists. I have a patch worked up for this and will raise a ticket
shortly.
However, I would like to open a discussion on Django's handling of non-
ASCII URLs...
Should the clean method of forms.URLField return the unicode value as
entered, or an IDNA-encoded URL?
What happens if a non-ASCII character is used in another part of the
URL? For example, opening http://en.wikipedia.org/wiki/Café in a
browser will work, but Django will not validate this as a legal URL
(because it isn't). In reality, the browser is converting the URL to
http://en.wikipedia.org/wiki/Caf%C3%A9 and requesting that.
Perhaps Django should permit such URLs and perform the same encoding
as a browser. In which case, should the clean method of forms.URLField
return the unicode value as entered, or urlencoded UTF-8 version of
the URL?
I guess an approach to these URL "complexities" would be to introduce
a utility function within django.utils.http such as:
def safe_url(url):
scheme, netloc, path, query, fragment = urlparse.urlsplit(url)
netloc = netloc.encode('idna')
path = urlquote(path)
# TODO -- should query and fragement be escaped?
return urlparse.urlunsplit((scheme, netloc, path, query,
fragment))
This could then be used by URLValidator, but also anyone who needs to
deal with non-ASCII URLs.
It is probably overkill and overcomplicating things, but I had also
thought about suggesting a "URL" object that would be returned by
URLFields (both forms and models). This could handle unicode URLs and
be responsible for encoding/decoding depending where they were used.
What do people think?
Fraser
P.S. I noticed that #12988 has just been opened, which also relates to
IDN validation.
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: Issues surrounding IDN validation and URLs in general
Am 27.02.2010 um 21:02 schrieb Fraser Nevett: Validation of IDN (Internationalized Domain Names) was added in [12474], but I noticed that the verify_exists option doesn't work when you use an IDN. This is caused by urllib2 not supporting IDN and the validation code using the original unicode version of the URL when testing for existence. This seems only to be true for python 2.4. In 2.5 and above urlopen will happily accept IDNs. [...] It is probably overkill and overcomplicating things, but I had also thought about suggesting a "URL" object that would be returned by URLFields (both forms and models). This could handle unicode URLs and be responsible for encoding/decoding depending where they were used. What do people think? Which ever solution is chosen I think it's important to note RFC 3490 section 6.1 which states that: """Applications MAY allow input and display of ACE labels, but are not encouraged to do so except as an interface for special purposes, [...] ACE encoding is opaque and ugly, and should thus only be exposed to users who absolutely need it.""" Ulrich -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: dbsettings, and user configurable app settings
We use something called livesettings in Satchmo. Bruce has published the code on bitbucket so it can be used outside of Satchmo. I believe at one point it was a fork of dbsettings but I may be wrong on that account. Check it out here - http://bitbucket.org/bkroeze/django-livesettings/overview/ Anyway, we have been using extensively in Satchmo and it works for us. I know there was talk on this list a while back about including something more robust in Django. I don't think there's consensus on the best approach yet. Hope this helps, -Chris On Feb 26, 12:53 pm, Jared Forsyth wrote: > If you look at dbsettings (which again, might be dead - i don't know) it > solves a lot of those problems; settings are tied into the "sites" module, > and are presented to the user in a very friendly way. The problem of db > migration is still there, but I can envision an import/export to XML (or > something) functionality where you can get/transfer all settings (or just > for one/a few apps) to another installation. > I've forked my own copy of dbsettings for the project I'm currently working > on (as I said, it was last touched 2007, and out-of-the-box incompatible), > and improving, so I might try to get it polished and more integrated w/ > contrib.admin, and see how it works. > > Jared > > On Fri, Feb 26, 2010 at 11:00 AM, Wes Winham wrote: > > I'd love to see a better way of managing settings in the core of > > django. It's a real pain point sometimes for writing and using > > pluggable applications and there's a wide range of ways that > > application developers try to tackle it. Some have basically no > > settings, some plan on users reading the documentation and defining > > all required settings, some have a way of using defaults that can be > > overridden and some have hacked together ways of using the database > > for some settings (I think ReviewBoard does something like this, or > > did). The problem I see is that the *best* way hasn't really bubbled > > to the surface as far as I can tell. Kind of like how database > > migrations are a hard problem, settings management seems kind of > > tricky. > > > Are there things stopping someone from making a really good app for > > managing settings and letting that bubble up to the de facto solution > > like with South? If there are, maybe removing those things would be a > > good target before trying to build the actual solution in contrib. Of > > course, this is probably not the best time for discussing this since > > everyone is trying to get 1.2 out the door, but I'd love to see this > > area improved down the road. > > > My specific pain points: > > * With a given app, it's a crapshoot to find where they define their > > internal settings (some use app/conf/settings.py, some app/conf/ > > default.py, some app/default.py etc) > > * It's hard to tweak settings on deployment for different > > environments. Using a settings_local.py that gets included is a > > workable solution, but it requires you to maintain a monolithic file. > > Something like the conf.d/*.conf debian style of configuration might > > > Pinax is having a similar discussion right now. Maybe something will > > come out of that project that can be applied/adapted to some later > > version of django. That might be a place to apply some development if > > anyone is interested in working towards better settings management. > > >http://groups.google.com/group/pinax-core-dev/browse_thread/thread/d9... > > > -Wes > > > On Feb 26, 2:11 am, Jared Forsyth wrote: > > > I have been looking around for a way of managing user-configurable > > > application settings, and the only solution I have found is dbsettings, > > > which looks like it hasn't been touched in 3 years. > > > So, I would like to know: is dbsettings dead? Or is there a different > > > generally accepted method for having user-friendly app settings? (e.g. > > don't > > > require code modification) > > > > I think that this idea is pretty essential to django's ease of use; there > > > are many applications which have (or should have) settings which only > > effect > > > UI or minor behavioral issues, and shouldn't require a server restart to > > > effect (and imo shouldn't require server write access). It seems that the > > > most viable solution would be to have a database managed settings system > > (in > > > the form of a .contrib module) which would manage this. It also seems > > that > > > having such an infrastructure in place would really encourage app > > > maintainers to have more settings, thereby making the apps even more > > > portable. > > > > Any thoughts? > > > > Thanks, > > > Jared > > > -- > > You received this message because you are subscribed to the Google Groups > > "Django developers" group. > > To post to this group, send email to django-develop...@googlegroups.com. > > To unsubscribe from this group, send email to > > django-developers+unsubscr...@googlegroups.com > > . > > For more options, visit this group at > >http://
Re: Issues surrounding IDN validation and URLs in general
On Feb 27, 8:02 pm, Fraser Nevett wrote: [...] > I have a patch worked up for this and will raise a ticket shortly. Ticket created and patch uploaded... http://code.djangoproject.com/ticket/12989 Fraser -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
