Re: Ticket #3297 and File uploads

2007-06-05 Thread Malcolm Tredinnick
sign discussions when sick: Lying in bed last night, I was thinking through this whole issue again. Turns out I need to change the above behaviour to support updating the QueryDict with FILES anyway -- because at the moment I've broken file uploads for oldforms. Not something that anybody

Re: Ticket #3297 and File uploads

2007-06-04 Thread Malcolm Tredinnick
On Mon, 2007-06-04 at 13:58 +0200, David Danier wrote: > > Perhaps request.POST.copy() could do this translation/decoding? > > Shouldn't request.{POST,GET}.copy() return an decoded version anyway? They return an *exact* copy of the same data structure. So it has an encoding attribute and the dec

Re: Ticket #3297 and File uploads

2007-06-04 Thread Malcolm Tredinnick
On Mon, 2007-06-04 at 18:57 +0800, Russell Keith-Magee wrote: > On 6/4/07, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote: > > > > On Mon, 2007-06-04 at 12:44 +0800, Russell Keith-Magee wrote: > > > Hi All, > > > > > > I've been looking at ticket #3297 - a request to implement FileField > > > and Im

Re: Ticket #3297 and File uploads

2007-06-04 Thread Malcolm Tredinnick
On Mon, 2007-06-04 at 13:37 +0200, David Danier wrote: > > So the Unicode branch does decoding when > > things in request.GET and request.POST are accessed, not on > > construction, and you can change the encoding to whatever you like via > > an attribute on the HttpRequest (or http.QueryDict) cla

Re: Ticket #3297 and File uploads

2007-06-04 Thread David Danier
> Perhaps request.POST.copy() could do this translation/decoding? Shouldn't request.{POST,GET}.copy() return an decoded version anyway? Otherwise the copy() might look different to the vars under request: post_copy = request.POST.copy() if post_copy['foo'] == request.POST['foo']: print '

Re: Ticket #3297 and File uploads

2007-06-04 Thread David Danier
> So the Unicode branch does decoding when > things in request.GET and request.POST are accessed, not on > construction, and you can change the encoding to whatever you like via > an attribute on the HttpRequest (or http.QueryDict) class. Perhaps request.POST.copy() could do this translation/deco

Re: Ticket #3297 and File uploads

2007-06-04 Thread Russell Keith-Magee
On 6/4/07, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote: > > On Mon, 2007-06-04 at 12:44 +0800, Russell Keith-Magee wrote: > > Hi All, > > > > I've been looking at ticket #3297 - a request to implement FileField > > and ImageField in newforms. ... > > If we then modify value_from_datadict to also

Re: Ticket #3297 and File uploads

2007-06-03 Thread Malcolm Tredinnick
On Mon, 2007-06-04 at 12:44 +0800, Russell Keith-Magee wrote: > Hi All, > > I've been looking at ticket #3297 - a request to implement FileField > and ImageField in newforms. > > Adrian's comments on this ticket, plus the code in newforms-admin seem > to suggest that the 'approved' technique for

Ticket #3297 and File uploads

2007-06-03 Thread Russell Keith-Magee
T; cleaned_data for a field using a FileInput will then contain the file data. Comments? While we are on the subject of File uploads - is there a reason for omitting the automatic file-saving behaviour from newforms? Adrian's comments on the ticket suggests to me that the approved technique

Re: Re: File Uploads Problem

2006-12-19 Thread Russell Keith-Magee
On 12/19/06, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > > after a bit of tweaking with the MEDIA_ROOT i am able to post files > through my custom manipulator. I still have to work on the validation > scheme on the manipulator but this works for abeta launch. I should point out at this poin

Re: File Uploads Problem

2006-12-19 Thread [EMAIL PROTECTED]
I got this to work by changinging up the line here from the Manipulator >>def complete(self, request, data): >>self.save() I put this in my Participant manipulator self.participant.save() self.participant.save_photo_path_file(data['photo_path']['filename'],data['photo_path']['content'])

Re: File Uploads Problem

2006-12-18 Thread [EMAIL PROTECTED]
Anyone got any ideas on this one --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group

File Uploads Problem

2006-12-15 Thread [EMAIL PROTECTED]
Hey all, I have been bangin my head on this one for a while, any help would be appreciated. I have form that is saving via a custom manipulator that I put together, in this form is an image upload field that I can not get to upload a file - I can get the file name into the db no problem, but the f

Re: Serious problems with the way Django handles file uploads

2006-09-20 Thread jp
Actually I did apply that patch and it worked. Problem is, even though it kept the dev server form simply stopping when I went to upload large files, it did not speed up the upload process at all. --~--~-~--~~~---~--~~ You received this message because you are sub

Re: Serious problems with the way Django handles file uploads

2006-09-20 Thread SmileyChris
I can't get 2070 to work either. I found http://code.djangoproject.com/ticket/2613 which fixed up my uploading problems (at least on the dev server) - have you tried that one? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google

Re: Serious problems with the way Django handles file uploads

2006-09-20 Thread jp
I found out a solution to the problem! Today I thought about possibly subverting Django's way of parsing file uploads and using something else to do so, while still using Django itself for everything else. After a question in IRC about it, someone mentioned that request.POST and request.

Re: Serious problems with the way Django handles file uploads

2006-09-15 Thread Cheng Zhang
I have to admit that reading such informative issue reporting and django experience sharing email is always a pleasure. It's also interested to know that your path from PHP -> Rails -> Pylons -> Django. Maybe in another email thread, would like to know the reason behind your going beyond Pyl

Re: Serious problems with the way Django handles file uploads

2006-09-15 Thread jp
Turns out when I was trying to apply ticket 2070, I was forgetting to actually enable the upload middleware. I've enabled it now, but I'm getting errors. This patch *should* fix the problem if I can get it working. --~--~-~--~~~---~--~~ You received this message

Re: Re: Serious problems with the way Django handles file uploads

2006-09-14 Thread James Bennett
On 9/14/06, jp <[EMAIL PROTECTED]> wrote: > It is worth noting that parse_file_upload in django.http uses those > email and mail libraries a lot. For what I don't know. Yeah, I just realized that; I didn't poke deeply enough. I'm pretty certain that's because fil

Re: Serious problems with the way Django handles file uploads

2006-09-14 Thread Don Arbow
On Sep 14, 2006, at 9:41 PM, jp wrote: > > It is worth noting that parse_file_upload in django.http uses those > email and mail libraries a lot. For what I don't know. Django uses the email libraries to parse the uploaded content as HTTP uses MIME encoding for file

Re: Serious problems with the way Django handles file uploads

2006-09-14 Thread jp
It isn't using FileField, infact it isn't touching the DB or using any kind of manipulator at all. The form consists of a simple box which allows the user to upload a file. The file is then submitted in a POST request to the second view. The second view then calls to an outside library which con

Re: Serious problems with the way Django handles file uploads

2006-09-14 Thread James Bennett
On 9/13/06, jp <[EMAIL PROTECTED]> wrote: > http://paste.e-scribe.com/1564/ > http://paste.e-scribe.com/1565/ It's hard to infer exactly what's going on without knowing more about the actual code you're using; for example, that first set of profiler output is spending over 40% of its time in djan

Serious problems with the way Django handles file uploads

2006-09-13 Thread jp
Hello everyone. I started using Django about a week ago. I have a particular app which more or less accepts CSV files, Excel files, DBase files, and whatever else, takes the uploaded file, converts it to a tab-delimited format, and then does statistical work over it. Originally this application

Re: Any way to halt big file uploads?

2006-08-05 Thread Todd O'Bryan
oposal in that ticket: > http://code.djangoproject.com/ticket/2070#change_34 You are, of course, correct. One of the problems with a nice, object- oriented API is that it encourages you to forget that requests come in as long streams of bytes. But I'm not completely getting your idea

Re: Any way to halt big file uploads?

2006-08-05 Thread Ivan Sagalaev
Todd O'Bryan wrote: > Would it be better to expose the file-like object that comes with a > file upload, rather than reading the file's whole content into memory > (or into the server's file system, if the patch gets checked in)? > It's easy to retain backward compatibility by just having a

Re: Any way to halt big file uploads?

2006-08-05 Thread Bryan Chow
Agreed, it is a blunt solution. However, you could use it to address the requirement of allowing different groups of users different upload limits by using a separate virtual server for each group of user, e.g. https://admin.yoursite.com for content providers and http://www.yoursite.com for public

Re: Any way to halt big file uploads?

2006-08-05 Thread Todd O'Bryan
OK. That answers one concern. But that's a very blunt tool. You can't for example, allow some users (say content providers) to upload multiple megabytes, but limit others (content consumers, perhaps) to much smaller sizes. Todd On Aug 5, 2006, at 3:57 PM, Bryan Chow wrote: > > You can use

Re: Any way to halt big file uploads?

2006-08-05 Thread Bryan Chow
You can use Apache's LimitRequestBody directive to restrict the size of accepted uploads. http://groups.google.com/group/django-users/browse_thread/thread/ca95963aaa33ce1e On 8/5/06, Todd O'Bryan <[EMAIL PROTECTED]> wrote: > The way Django ships, isn't it possible for a user to hijack the > se

Any way to halt big file uploads?

2006-08-05 Thread Todd O'Bryan
The way Django ships, isn't it possible for a user to hijack the server by uploading HUUUGE files? Because the files are stored in memory, this seems like it could be a very bad thing. There's a ticket, #2070, with a patch that buffers files in small chunks, so that no more than about 64k i

Re: File uploads

2006-06-09 Thread [EMAIL PROTECTED]
Just for information: Now has a setting to keep it backwards compatible --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups

Re: File uploads

2006-06-02 Thread James Bennett
On 6/2/06, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > Fixed, is now ticket http://code.djangoproject.com/ticket/2070 Should've read this before I read that ticket. Or woken my brain up. -- "May the forces of evil become confused on the way to your house." -- George Carlin --~--~

Re: File uploads

2006-06-02 Thread [EMAIL PROTECTED]
Fixed, is now ticket http://code.djangoproject.com/ticket/2070 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To un

Re: File uploads

2006-06-02 Thread Jeroen Ruigrok van der Werven
On 6/2/06, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > Streaming is storing the output directly without loading the entire > file into memory. Just small parts of it are in memory at any given > time before it is appended to a file on disk. Ah right. Should've thought of that. I guess this wi

Re: File uploads

2006-06-02 Thread [EMAIL PROTECTED]
Jeroen Ruigrok van der Werven wrote: > Hei Oyvind, > > On 6/1/06, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > > Been working on a patch for streaming/ large file uploads. > > Can you briefly explain how I should interpret the concept streaming > file uploads? &

Re: File uploads

2006-06-02 Thread Jeroen Ruigrok van der Werven
Hei Oyvind, On 6/1/06, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > Been working on a patch for streaming/ large file uploads. Can you briefly explain how I should interpret the concept streaming file uploads? -- Jeroen Ruigrok van d

Re: File uploads

2006-06-01 Thread [EMAIL PROTECTED]
still have a issue with large files because of lib/python2.4/rfc822.py using readline without reading chunks, but more effective than the current handling of fileuploads if you really need large files just make readline() readline(64000) in this file --~--~-~--~~~-

File uploads

2006-06-01 Thread [EMAIL PROTECTED]
Been working on a patch for streaming/ large file uploads. http://code.djangoproject.com/ticket/1484 Reading bouandries done using 64kb chunks in FieldStorage to avoid reading a large file without linebreaks. Validation on filefields done using size of temporary file. File being moved in