Re: Complains about FileField not deleting files in 1.3.
It took me a while but I have opensourced a tool to clean the media folder from all the leftovers. It's a cron job with a simple admin gui. Also handles multiple upload folders. We have been using something similar to this internally for more than a year, and we are pretty happy about it. https://github.com/PuzzleDev/django-uploadcleaner Just out of curiosity, how do you normally handle file removal? On Tuesday, March 29, 2011 3:43:13 PM UTC+2, Carl Meyer wrote: > > Hi Alex, > > On 03/29/2011 01:36 AM, Alex Kamedov wrote: > > I think, cron jobs is an overhead in many simple cases where old > > behaviour was useful and more simpler. > > Why you don't want include DeletingFileField[1] in django? > > > > [1] https://gist.github.com/889692 > > Because, as mentioned above, it is known to cause data loss in certain > situations (rolled-back transactions, overlapping upload-to > directories), and we are not very fond of including things in Django > that cause some Django users to lose their data. If you understand those > risks and want to use DeletingFileField in your projects, it's not hard > to do so. > > Carl > > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To view this discussion on the web visit https://groups.google.com/d/msg/django-developers/-/wpPZcb_BKr0J. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: django.contrib.admin and null=True
By default Admin saves an empty string in those TextFields and CharFields defined as null = True. Whatever the semantic reasons are, this default behavior creates problems in those fields which are both unique = True and null = True because by saving an empty string they do not respect that null != null. For instance suppose you want to create a TextField containing an alphanumeric code which if exists is unique. The default behavior of the Admin forms will save "" in all the empty fields violating the unicity. A work around of would be to override the form used by the admin interface replacing the empty strings with None. On Dec 8, 12:26 am, schinckel wrote: > I haven't been able to find any documentation about this, but would be > happy to be pointed in the right direction. > > When you use null=True in a field, and then use that model in the > admin, it will not save NULL to the database, but will instead save an > empty string (or attempt to). > > I think this is broken behaviour: NULL values are semantically > different to empty strings, and in certain cases (I think it was > IPAddressField, but I will have another look later), it would fail to > write to the database. > > Is there a reason that the admin interface saves what should be a NULL > value as an empty string? Do I report this as a bug? > > Matt. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Intermittent IOError exception raised and emailed to admins during file upload.
Have you considered the possibility that it would be a client/browser
side error?
Both windows7 and chrome have a known open bug with large file
uploads. Many of my customers have reported this bug, all of them
using windows7 and chrome.
Has anyone seen this happening from a Mac or from Linux?
On Feb 4, 2:28 am, Tai Lee wrote:
> Thanks for your detailed explanation, Graham. I'll try to approach
> this from another angle and see if I can determine what is actually
> causing the connections to be dropped, as I'm seeing this error a few
> times a day on a low traffic site using Apache at the front for static
> media, with proxy pass to Django under Apache/mod_wsgi.
>
> Cheers.
> Tai.
>
> On Feb 4, 6:17 pm, Graham Dumpleton
> wrote:
>
>
>
> > The problem is you can't just catch IOError, because the presence of WSGI
> > middleware may mean that IOError could be generated for other reasons when
> > reading wsgi.input. To base detection on the description in the IOError,
> > ie., 'request data read error', would mean having to know what that
> > description could be on all WSGI hosting mechanisms, and then just hope that
> > there aren't other reasons why it could arise from WSGI middleware.
>
> > So, no simple solution. This unfortunately is one of the short comings of
> > the WSGI specification. That is, there is no standardised exception type to
> > represent actual real problems with reading original wsgi.input. Part of the
> > reason is that there is no standard module associated with the WSGI
> > specification to hold the exception type, so nothing to compare against. The
> > only solution would be for WSGI specification to be enhanced to require a
> > unique exception type to be used for wsgi.input exceptions such as aborted
> > connection, with the type of that exception being passed through in the WSGI
> > environ dictionary so it could be compare against. For example something
> > like:
>
> > try:
> > data = environ['wsgi.input'].read()
> > except wsgi['wsgi.input_exception], e:
> > # ignore it
>
> > In summary, no real practical solution at this time for this.
>
> > Graham
>
> > On Friday, February 4, 2011 1:55:25 PM UTC+11, Tai Lee wrote:
>
> > > There are many questions about this on django-users. The usual answer
> > > is to ignore the errors, which are probably generated as a result of
> > > the client prematurely terminating a connection, usually during an
> > > upload.
>
> > > Nobody seems to be able to reliably reproduce the error, and it seems
> > > to happen with large and small uploads alike, but always
> > > intermittently.
>
> > > It has been suggested that a client dropping the connection should be
> > > within what one can expect from a networked client, and thus Django
> > > should not email the exception and traceback to the admins in this
> > > case. From my preliminary investigation with this issue, I tend to
> > > agree.
>
> > > Is it feasible to handle this exception in Django, or are there other
> > > issues that would prevent us from isolating this case from other
> > > legitimate exceptions that may be raised at this point?
>
> > > Here's a recent traceback:
>
> > > Traceback (most recent call last):
>
> > > File "/home/tailee/django/core/handlers/base.py", line 105, in
> > > get_response
> > > response = middleware_method(request, callback, callback_args,
> > > callback_kwargs)
>
> > > File "/home/tailee/django/middleware/csrf.py", line 224, in
> > > process_view
> > > request_csrf_token = request.POST.get('csrfmiddlewaretoken', '')
>
> > > File "/home/tailee/django/core/handlers/wsgi.py", line 209, in
> > > _get_post
> > > self._load_post_and_files()
>
> > > File "/home/tailee/django/http/__init__.py", line 207, in
> > > _load_post_and_files
> > > self._post, self._files = self.parse_file_upload(self.META, self)
>
> > > File "/home/tailee/django/http/__init__.py", line 169, in
> > > parse_file_upload
> > > return parser.parse()
>
> > > File "/home/tailee/django/http/multipartparser.py", line 192, in
> > > parse
> > > for chunk in field_stream:
>
> > > File "/home/tailee/django/http/multipartparser.py", line 314, in next
> > > output = self._producer.next()
>
> > > File "/home/tailee/django/http/multipartparser.py", line 468, in next
> > > for bytes in stream:
>
> > > File "/home/tailee/django/http/multipartparser.py", line 314, in next
> > > output = self._producer.next()
>
> > > File "/home/tailee/django/http/multipartparser.py", line 375, in next
> > > data = self.flo.read(self.chunk_size)
>
> > > File "/home/tailee/django/http/multipartparser.py", line 405, in read
> > > return self._file.read(num_bytes)
>
> > > File "/home/tailee/django/http/__init__.py", line 231, in read
> > > return self._stream.read(*args, **kwargs)
>
> > > IOError: request data read error
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-d
Complains about FileField not deleting files in 1.3.
I am referring to this: http://docs.djangoproject.com/en/dev/releases/1.3/#filefield-no-longer-deletes-files Instead of preventing the data loss from happening a very usefull feature has been removed. Why not simply letting the developer decide when to enable or disable it with a constructor boolean parameter? My company sells multimedia web applications normally handling over 1 files over various models. I am sorry to say that but to me the idea of running a cron job to remove orphaned files does not seam to be practical. Shall I make a query for each file? This will make it very hard to switch projects from 1.2 to 1.3. The roll back data loss problem could have been solved by copying the file into a temporary file and by restoring it if necessary. Having to file fields pointing to the same file does not sound correct. However this is probably the only case in which it would make sense to disable the auto deletion. Am I the only one who would like to see the previous behaviour restored? Can we at least re-enable this feature from the file-field constructor? -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Complains about FileField not deleting files in 1.3.
> One query for each model > containing one or more FileFields is enough to build a list of the files > that ought to exist, and any file not in that list can presumably be > removed. How can I sleep at night knowing that there is a maintenance cron job deleting files which can be "presumably be removed"? My point is that yes it great that you have removed a data loss from FileField, but you have moved all the complexity to the developers and they will end up having even bigger data loss. Assuming that the purpose of improving django is to make it safer and easier to be used this "improvement" is a double bladed knife. Correct me if I am wrong: - Subclassing FileField will restore the previous issue. - A post delete signal will restore the previous issue. Having a maintenance job deleting file not listed will require a serious maintenance. Suppose a developer adding a file field and forgetting to update the maintenance script will cause all the file of that field to be deleted. Files which for a bad design are in the same folder as the file pointed by the file field will be removed. Thumbnails and other files eventually not pointed by a file field will be removed. And there will be more serious failures depending from the implementation. What I am trying to say is that removing orphaned files, even if with a cron job, should be done by django automatically and not assuming that the developers will take care of that. Said so I will start implementing such a maintenance job, and I am willing to share it so maybe we could include it in a future release of django. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
