[Tutor] Structured files?

[Válas Péter]

Good morning,

I can create files and write strings/unicodes.
Is it possible to write a list, a dictionary or an object or anything into a
file? Or do I have to transform them to strings?

Péter
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Structured files?

[Simon Yan]

2011/6/2 Válas Péter :
> Good morning,
>
> I can create files and write strings/unicodes.
> Is it possible to write a list, a dictionary or an object or anything into a
> file? Or do I have to transform them to strings?
Yes you can.
I guess the question is how you want the information to be structured.
IMHO, everything in Python can be "string-lized".

>
> Péter
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
>



-- 
Regards,
Simon Yan

http://www.google.com/profiles/simonyanix
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Structured files?

[Válas Péter]

2011. június 2. 9:29 Simon Yan írta, :

> Yes you can.
> I guess the question is how you want the information to be structured.
> IMHO, everything in Python can be "string-lized".
>
> What is the syntax then?  I have Windows XP. The code is:
f=open("xxx.dat","w")
f.write("fff")
d={'one':1, 2:'two'}
f.write(d)
f.close()

Python 2.5 message:
TypeError: argument 1 must be string or read-only character buffer, not dict
Python 3.1 message:
TypeError: must be str, not dict

Modified code:
f=open("xxx.dat","wb") The others remain as above, just I wrote wb.
Python 2.5 message:
TypeError: argument 1 must be string or read-only buffer, not dict
Python 3.1 message:
f.write("fff")
TypeError: must be bytes or buffer, not str
This won't write even a string.

I read something about a pack function, is that the key?
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Structured files?

[Simon Yan]

2011/6/2 Válas Péter :
>
>
> 2011. június 2. 9:29 Simon Yan írta, :
>>
>> Yes you can.
>> I guess the question is how you want the information to be structured.
>> IMHO, everything in Python can be "string-lized".
>>
> What is the syntax then?  I have Windows XP. The code is:
> f=open("xxx.dat","w")
> f.write("fff")
> d={'one':1, 2:'two'}
> f.write(d)
> f.close()
Try this:
f=open("xxx.dat","w")
f.write("fff")
d={'one':1, 2:'two'}
f.write(str(d))
f.close()

>
> Python 2.5 message:
> TypeError: argument 1 must be string or read-only character buffer, not dict
> Python 3.1 message:
> TypeError: must be str, not dict
>
> Modified code:
> f=open("xxx.dat","wb") The others remain as above, just I wrote wb.
> Python 2.5 message:
> TypeError: argument 1 must be string or read-only buffer, not dict
> Python 3.1 message:
>     f.write("fff")
> TypeError: must be bytes or buffer, not str
> This won't write even a string.
>
> I read something about a pack function, is that the key?
>
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
>



-- 
Regards,
Simon Yan

http://www.google.com/profiles/simonyanix
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Structured files?

[Walter Prins]

>Is it possible to write a list, a dictionary or an object or anything into
a file? Or do I have to transform them to strings?

Have a look at the pickle module:
http://docs.python.org/library/pickle.html
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Structured files?

[Alan Gauld]

"Válas Péter"  wrote

I can create files and write strings/unicodes.
Is it possible to write a list, a dictionary or an object or 
anything into a

file? Or do I have to transform them to strings?


You can write anything to a file, but the file will need to
be opened in binary mode(see the docs). The tricky bit
is reading the data back, you will need to find a way
to convert the bytes read from the file into whatever
the original data structure was.

For that reason it is usually better to use a tool like
pickle or shelve to store Python objects in a file
(again see the docs)

HTH,

--
Alan Gauld
Author of the Learn to Program web site
http://www.alan-g.me.uk/


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Structured files?

[Alan Gauld]

"Válas Péter"  wrote

Modified code:
f=open("xxx.dat","wb") The others remain as above, just I wrote wb.


Yes, you need to use binary mode.


I read something about a pack function, is that the key?


See the file handling topic in my tutorial for a simple
example of using the struct module(which includes the
pack function) to write/read binary data.

But, pickle or shelve are probably more useful
for your needs if you want to save entire Python
objects.

HTH,

--
Alan Gauld
Author of the Learn to Program web site
http://www.alan-g.me.uk/


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Structured files?

[Válas Péter]

2011. június 2. 10:52 Simon Yan írta, :

> Try this:
> f.write(str(d))
>
> That's what I first asked, this is not a structured file then, this is a
structure transformed to a string.

Thanks for the idea of pickle, I will look at it.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Structured files?

[Dave Angel]

On 01/-10/-28163 02:59 PM, Válas Péter wrote:

2011. június 2. 9:29 Simon Yan írta,:


Yes you can.
I guess the question is how you want the information to be structured.
IMHO, everything in Python can be "string-lized".

What is the syntax then?  I have Windows XP. The code is:

f=open("xxx.dat","w")
f.write("fff")
d={'one':1, 2:'two'}
f.write(d)
f.close()

Python 2.5 message:
TypeError: argument 1 must be string or read-only character buffer, not dict
Python 3.1 message:
TypeError: must be str, not dict

Modified code:
f=open("xxx.dat","wb") The others remain as above, just I wrote wb.
Python 2.5 message:
TypeError: argument 1 must be string or read-only buffer, not dict
Python 3.1 message:
 f.write("fff")
TypeError: must be bytes or buffer, not str
This won't write even a string.

I read something about a pack function, is that the key?



Look up shelve and pickle (both in the stdlib).  One of them will be 
suitable for the purpose.


DaveA
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Python Extensions in C

[James Reynolds]

I fixed it. I find if I sleep on things I have a better chance at fixing
them. Maybe I will get some real work done today instead of going all OCD on
this side project?

Here's the dif: http://pastebin.com/KYBab3H9

I'll address your points in a second, but here's how I found the problem
(Python is so much easier to work with, it's scary).

1. I thought to myself last night, I knew there was a problem with the sums
not adding up properly in the var() function (the sumall variable).

2. I then thought to myself, well, let me trace out every last variable in
the first step. I then found that sumall was being raised to the third
power, not the second power.

3. I traced out "moment" and it indeed was sending through a "3", or at
least seemed to be.

4. Meanwhile, back at line 362, I am calling the same function, but with "3"
in the moment arg.

5. It occurs to me that I need to save the value of the address that the
pointer returned from "return_varp" to a variable. I actually have such a
variable already, but just below the "third moment". The skew function now
seems to work correctly. My guess is Kurt is still broken, but most likely
this is the same problem.

I guess what happens is, the static variable in var() func is saving the
value to the same exact address. Thinking back on it, this would be
consistent with Python itself if I was to have a class like Class A(): a = 1
def update_a: A.a +=1. After calling update_a, if I print A.a my answer
should be 2.

But, what I was thinking was that each new call to var() would be like a
instance of class A above, in other words, it would create a static variable
called "temp_r", but in thinking about it, this isn't the case. Declaring
something static means (please correct me if I'm wrong) that the address you
using for "temp_r" is now and forever will be the address assigned
when initially created. So, any pointer pointing to this address will return
its changing content, whether intended to do so or not.

The entire reason why I was using static variables in the those functions
was so they didn't lose their values when returned to their calling
functions.

So anyway, back to your points.

I had gone through must of it to bring it in line with your suggestions from
earlier, but I'm afraid you caught me getting lazy.

This section:


avg = *return_avgp;
samp = 0;
return_varp = var(seq, count, &avg, samp, 2);
third_moment = var(seq, count, &avg, 0, 3);

Was an artifact from troubleshooting. It should have been (and I think is
now) how you suggested.


So at any rate, thank you for the help and if you see something else, please
don't hesitate to point it out!





On Wed, Jun 1, 2011 at 8:40 PM, ALAN GAULD wrote:

> OK, It seems to have grown a bit! :-)
>
> Your use of static seems a little odd. In particular the definition of avg
> as a static double.
> You then never use avg except here:
>
>
>1. avg = *return_avgp;
>2. samp = 0;
>3. return_varp = var(seq, count, &avg, samp, 2);
>4. third_moment = var(seq, count, &avg, 0, 3);
>
> You assign the content of a pointer to avg then take use address
> of avg in var() - effectively making avg back into a pointer?
>
> Why not just use *return_avgp inside the var() function?
> And why is avg static? - do you understand what static is doing?
> And the side effects of that? Is there a reason to use static?
> If so its a bit too subtle for me...
>
> BTW, On style issues:
> While I prefer the braces style you have made the mistake of mixing styles,
>
> particularly confusing when done inside a single function as in the middle
> of
> stat_kurt()
>
> And I don't personally like variables being declared in the middle of code.
>
> (eg lines 365 and 430) I can just about accept at the start of a block,
> but I prefer at the start of the function. It's just easier to find if they
>
> are all together.
>
> Other than that I can't see the error, but its late and thats
> quite a lot of code for a Python programmer to wade through! :-)
>
>
> Alan Gauld
> Author of the Learn To Program website
>
> http://www.alan-g.me.uk/
>
>
> --
> *From:* James Reynolds 
> *To:* Alan Gauld 
> *Cc:* tutor@python.org
> *Sent:* Wednesday, 1 June, 2011 20:49:44
>
> *Subject:* Re: [Tutor] Python Extensions in C
>
> So I've been continuing with my experiment above, and I've made some good
> progress and learned some new things as I've been going. I've expanded it
> out a little bit, and you can see the code here:
>
> http://pastebin.com/gnW4xQNv
>
> I've tried to incorporate most of your
> suggestions (although, the code itself looks more condensed in my editor
> than what it appears there).
>
> Unfortunately, I am stuck and I have no clue how to free myself from the
> mud.
>
>
> On line 370, you will notice I have "*return_varp" as the return value and
> not "

Re: [Tutor] checking if a variable is an integer?

[Prasad, Ramit]

>If not, he can google the bits he doesn't understand, or ask. We won't bite!   

Unless he asks for it ;)

Ramit



Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
712 Main Street | Houston, TX 77002
work phone: 713 - 216 - 5423


This communication is for informational purposes only. It is not
intended as an offer or solicitation for the purchase or sale of
any financial instrument or as an official confirmation of any
transaction. All market prices, data and other information are not
warranted as to completeness or accuracy and are subject to change
without notice. Any comments or statements made herein do not
necessarily reflect those of JPMorgan Chase & Co., its subsidiaries
and affiliates.

This transmission may contain information that is privileged,
confidential, legally privileged, and/or exempt from disclosure
under applicable law. If you are not the intended recipient, you
are hereby notified that any disclosure, copying, distribution, or
use of the information contained herein (including any reliance
thereon) is STRICTLY PROHIBITED. Although this transmission and any
attachments are believed to be free of any virus or other defect
that might affect any computer system into which it is received and
opened, it is the responsibility of the recipient to ensure that it
is virus free and no responsibility is accepted by JPMorgan Chase &
Co., its subsidiaries and affiliates, as applicable, for any loss
or damage arising in any way from its use. If you received this
transmission in error, please immediately contact the sender and
destroy the material in its entirety, whether in electronic or hard
copy format. Thank you.

Please refer to http://www.jpmorgan.com/pages/disclosures for
disclosures relating to European legal entities.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

[Tutor] Introduction

[Jamie Griffin]

Hello everyone

I'm not sure if introductions are customary on this list but I thought
I would anyway as I'm going to be using it quite a bit from now.

I am just starting out with python and I will no doubt get stuck and
need some assistance from you all soon. I'll try my best to keep the
stupidity to a minimum.

Jamie.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Introduction

[David Abbott]

On Thu, Jun 2, 2011 at 10:12 AM, Jamie Griffin  wrote:
> Hello everyone
>
> I'm not sure if introductions are customary on this list but I thought
> I would anyway as I'm going to be using it quite a bit from now.
>
> I am just starting out with python and I will no doubt get stuck and
> need some assistance from you all soon. I'll try my best to keep the
> stupidity to a minimum.
>
> Jamie.
Hi Jamie,
Welcome to the list, if you have not read one here is a pretty good
guide [1] for list etiquette, it would have saved me some
embarrassment if I had read it before I started posting to mail lists.
Have Fun :)
David

[1] http://curl.haxx.se/mail/etiquette.html
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Structured files?

[Alexandre Conrad]

2011/6/2 Válas Péter :
> I can create files and write strings/unicodes.
> Is it possible to write a list, a dictionary or an object or anything into a
> file? Or do I have to transform them to strings?

As suggested by Walter, you should use the Pickle module to serialize
your Python objects so they can be written to files and read back to
Python (de-serialized). This way you can also share your pickled
objects with other Python apps and Pickle is great for this. Actually,
that's how the multiprocessing module works under the hood to pass
back and forth Python objects between processes.

If you are only going to serialize data structures such as lists,
dictionaries with data types such as strings, numbers, bools, None and
you want to share that data between non-Python application, I could
also suggest the use of the JSON module. JSON is a standard format
(see json.org) supported by many programming languages. JSON is very
popular for web service APIs. Your Python server could expose web APIs
and applications written in different programming languages (Java, C,
C#, Javascript ...) can call an API and interpret the JSON data your
server sends back.

This is some food for thought.

HTH,
-- 
Alex | twitter.com/alexconrad
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Structured files?

[Alan Gauld]

"Alexandre Conrad"  wrote


you want to share that data between non-Python application, I could
also suggest the use of the JSON module. JSON is a standard format
(see json.org) supported by many programming languages. 


But isn't it a string based format?
I thought JSON converted everything into XML strings?

That makes it hugely wasteful of space which is usually 
the reason for using a binary format in the first place.


But I've only used JSON once from within TurboGears so 
I am no expert!


--
Alan Gauld
Author of the Learn to Program web site
http://www.alan-g.me.uk/


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Structured files?

[Walter Prins]

On 2 June 2011 17:13, Alan Gauld  wrote:

> But isn't it a string based format?
> I thought JSON converted everything into XML strings?
>
>
It's text based yes, although not XML.  As for efficiency, I'd be of the
view that it's probably efficient enough for the vast majority of use cases,
especially when the transmission channel has compression on it (which is
more often than not the case these days.)

Cheers

Walter
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Structured files?

[Steven D'Aprano]

Alan Gauld wrote:


"Alexandre Conrad"  wrote


you want to share that data between non-Python application, I could
also suggest the use of the JSON module. JSON is a standard format
(see json.org) supported by many programming languages. 


But isn't it a string based format?
I thought JSON converted everything into XML strings?


JSON is a string format, but not XML.

>>> import json
>>> json.dumps({'a': 1, 'b': 2, 'c': 3})
'{"a": 1, "c": 3, "b": 2}'


YAML is another alternative, arguably more human readable than JSON, but 
it's not in the standard library.


Perhaps you're thinking of plist ("property list"). In Python 3.1:


>>> from io import BytesIO
>>> import plistlib
>>> s = BytesIO()
>>> plistlib.writePlist({'a': 1, 'b': 2, 'c': 3}, s)
>>> print( s.getvalue().decode('ascii') )

"http://www.apple.com/DTDs/PropertyList-1.0.dtd";>



a
1
b
2
c
3





That makes it hugely wasteful of space which is usually the reason for 
using a binary format in the first place.


Meh, who cares whether your 100K of data takes 300K on disk? :)

If you need/want binary storage, pickle has a binary mode. But 
generally, I am a big believer in sticking to text formats unless you 
absolutely have to use binary. The usefulness of being able to read, or 
even edit, your data files using a simple text editor cannot be 
under-estimated!



--
Steven

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Structured files?

[Prasad, Ramit]

>But I've only used JSON once from within TurboGears so 
>I am no expert!
If you use Firefox you have used JSON ;) that is how it stores bookmarks. I am 
sure it is more widely used than that. Usage seems to be on the rise but that 
could be that I see it more now that I know what it is. 


Ramit



Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
712 Main Street | Houston, TX 77002
work phone: 713 - 216 - 5423


This communication is for informational purposes only. It is not
intended as an offer or solicitation for the purchase or sale of
any financial instrument or as an official confirmation of any
transaction. All market prices, data and other information are not
warranted as to completeness or accuracy and are subject to change
without notice. Any comments or statements made herein do not
necessarily reflect those of JPMorgan Chase & Co., its subsidiaries
and affiliates.

This transmission may contain information that is privileged,
confidential, legally privileged, and/or exempt from disclosure
under applicable law. If you are not the intended recipient, you
are hereby notified that any disclosure, copying, distribution, or
use of the information contained herein (including any reliance
thereon) is STRICTLY PROHIBITED. Although this transmission and any
attachments are believed to be free of any virus or other defect
that might affect any computer system into which it is received and
opened, it is the responsibility of the recipient to ensure that it
is virus free and no responsibility is accepted by JPMorgan Chase &
Co., its subsidiaries and affiliates, as applicable, for any loss
or damage arising in any way from its use. If you received this
transmission in error, please immediately contact the sender and
destroy the material in its entirety, whether in electronic or hard
copy format. Thank you.

Please refer to http://www.jpmorgan.com/pages/disclosures for
disclosures relating to European legal entities.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] __init__.py question

[Marilyn Davis]

Thank you for your careful explanation, Steven.

I guess there's nothing more for me to know.

While I think it is a mistake to put directories in your package directory
structure that aren't part of the package, that seems to be all it's
about.

Well, I guess I don't feel quite so stupid.

Thank you folks.

Marilyn Davis


On Wed, June 1, 2011 5:11 pm, Steven D'Aprano wrote:

> Marilyn Davis wrote:
> [...]
>
>> There's something I'm missing because I think you simply can't call
>> something string.py unless you are willing to give up the library
>> string.py.
>
> No, you're right about that. Python's search path is "first match wins".
> Whether it's a package or a module, it will match before the library
> string.py. Adding __init__.py doesn't change that.
>
> (Unless you re-arrange sys.path so that library modules come first.)
>
>
> However, you can have a directory called "string", containing arbitrary
> .py files. So you could drop this in your path:
>
>
>
> string/ +-- a.py
> +-- b.py
>
>
> and add string to the path so that the directory is searched, and still
> get the standard string.py. (I haven't actually tried this, but it should
> work.)
>
> Only if you put a file __init__.py into the directory will it shadow the
> standard library string.py.
>
> Personally, I think the emphasis given in the docs about shadowing
> standard library files is misplaced. I think that is actually irrelevant. I
> think a more useful explanation for the need for __init__.py is simple:
> how else could Python distinguish a package from a mere directory
> containing Python files?
>
> A mere directory is not searched unless it is explicitly added to the
> path; a mere directory cannot be imported. A package can be. But since a
> package *is* a directory at the file system level, how should Python
> distinguish them?
>
> I suppose Python could have the rule "the directory must end in .py to
> be considered a package". But then, when you import the main package
> instead of a submodule, which file should Python read? There needs to be a
> standard naming convention for the file to be treated as the top-level
> module. But since you have such a file, there's no longer any need for
> adding a .py to the end of the directory name, since the standard file is
> sufficient to determine that the directory is a package.
>
> That file could have been called anything, __init__.py was chosen by
> analogy with __init__ methods in classes and because it's unlikely to clash
> with any name package authors might choose for their own use.
>
>
>
> --
> Steven
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] checking if a variable is an integer?

[Marilyn Davis]

True that!

This is the nicest, most patient, most knowledgeable, technical list I've
ever experienced.

That's why I keep coming back when I am hopelessly confused.

Thank you.

Marilyn Davis


On Thu, June 2, 2011 7:04 am, Prasad, Ramit wrote:

>> If not, he can google the bits he doesn't understand, or ask. We won't
>> bite!
>
> Unless he asks for it ;)
>
>
> Ramit
>
>
>
>
> Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
> 712 Main Street | Houston, TX 77002
> work phone: 713 - 216 - 5423
>
>
> This communication is for informational purposes only. It is not
> intended as an offer or solicitation for the purchase or sale of any
> financial instrument or as an official confirmation of any transaction.
> All market prices, data and other information are not
> warranted as to completeness or accuracy and are subject to change without
> notice. Any comments or statements made herein do not necessarily reflect
> those of JPMorgan Chase & Co., its subsidiaries and affiliates.
>
> This transmission may contain information that is privileged,
> confidential, legally privileged, and/or exempt from disclosure under
> applicable law. If you are not the intended recipient, you are hereby
> notified that any disclosure, copying, distribution, or use of the
> information contained herein (including any reliance thereon) is STRICTLY
> PROHIBITED. Although this transmission and any
> attachments are believed to be free of any virus or other defect that might
> affect any computer system into which it is received and opened, it is the
> responsibility of the recipient to ensure that it is virus free and no
> responsibility is accepted by JPMorgan Chase & Co., its subsidiaries and
> affiliates, as applicable, for any loss or damage arising in any way from
> its use. If you received this transmission in error, please immediately
> contact the sender and destroy the material in its entirety, whether in
> electronic or hard copy format. Thank you.
>
> Please refer to http://www.jpmorgan.com/pages/disclosures for
> disclosures relating to European legal entities.
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Strategy to read a redirecting html page

[Robert Sjoblom]

> When you hit the page and you get an HTTP redirect code back (say,
> 302), you will need to make another call to the URL specified in the
> "Location" parameter in the response headers. Then you retrieve that
> new page and you can check you got an acceptable HTTP response code
> (such as 200) and read the page's body (or whatever you want to do
> with it). Otherwise, keep looping until you get an expected HTTP
> response code.
>
> Note: you may get stuck in an infinite loop if two URLs redirect to each 
> other.
>
> You might want to take a look at the higher level httplib module:
> http://docs.python.org/library/httplib.html
>
> Although I don't think it can automatically follow redirects for you.
> You'll have to implement the loop yourself.
>
> If you can rely on 3rd party packages (not part of the standard Python
> library), take a look at httplib2:
> https://httplib2.googlecode.com/hg/doc/html/libhttplib2.html
>
> This one can follow redirects.
>
> HTH,

Sorry for bringing up an old topic like this, but writing longer
messages on a phone is just not something that I want to do.

Python already has the urllib/urllib2 package that automatically
follow redirects, so I don't see why you'd need a 3rd-party module to
deal with it? When it encounters a 301 status code from the server,
urllib2 will search through its handlers and call the http_error_301
method, which will look for the Location: header and follow that
address. The behaviour is defined in HTTPRedirectHandler, which can be
overridden if necessary:

>>> help(urllib.request.HTTPRedirectHandler)
Help on class HTTPRedirectHandler in module urllib.request:

class HTTPRedirectHandler(BaseHandler)
 |  Method resolution order:
 |  HTTPRedirectHandler
 |  BaseHandler
 |  builtins.object
 |
 |  Methods defined here:
 |
 |  http_error_301 = http_error_302(self, req, fp, code, msg, headers)
 |
 |  http_error_302(self, req, fp, code, msg, headers)
 |  # Implementation note: To avoid the server sending us into an
 |  # infinite loop, the request object needs to track what URLs we
 |  # have already seen.  Do this by adding a handler-specific
 |  # attribute to the Request object.
 |
 |  http_error_303 = http_error_302(self, req, fp, code, msg, headers)
 |
 |  http_error_307 = http_error_302(self, req, fp, code, msg, headers)
 |
 |  redirect_request(self, req, fp, code, msg, headers, newurl)
 |  Return a Request or None in response to a redirect.
 |
 |  This is called by the http_error_30x methods when a
 |  redirection response is received.  If a redirection should
 |  take place, return a new Request to allow http_error_30x to
 |  perform the redirect.  Otherwise, raise HTTPError if no-one
 |  else should try to handle this url.  Return None if you can't
 |  but another Handler might.
 |
 |  --
 |  Data and other attributes defined here:
 |
 |  inf_msg = 'The HTTP server returned a redirect error that w...n infini...
 |
 |  max_redirections = 10
 |
 |  max_repeats = 4
 |
 |  --
 |  Methods inherited from BaseHandler:
 |
 |  __lt__(self, other)
 |
 |  add_parent(self, parent)
 |
 |  close(self)
 |
 |  --
 |  Data descriptors inherited from BaseHandler:
 |
 |  __dict__
 |  dictionary for instance variables (if defined)
 |
 |  __weakref__
 |  list of weak references to the object (if defined)
 |
 |  --
 |  Data and other attributes inherited from BaseHandler:
 |
 |  handler_order = 500

best regards,
Robert S.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Strategy to read a redirecting html page

[Brett Ritter]

On Thu, Jun 2, 2011 at 4:06 PM, Robert Sjoblom  wrote:
> Python already has the urllib/urllib2 package that automatically
> follow redirects, so I don't see why you'd need a 3rd-party module to
> deal with it? When it encounters a 301 status code from the server,

Ah, but I believe the issue is that the server isn't giving him a 301,
it's giving him a 200 with a page explaining that "this page has
moved, please update your bookmarks" etc, with a META HTTP-EQUIV tag
to redirect the page after N seconds.

(Based on the part where the OP said "I get the first warning/error
message page and not the redirection one.")

-- 
Brett Ritter / SwiftOne
swift...@swiftone.org
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Strategy to read a redirecting html page

[Robert Sjoblom]

On 2 June 2011 22:50, Brett Ritter  wrote:
> On Thu, Jun 2, 2011 at 4:06 PM, Robert Sjoblom  
> wrote:
>> Python already has the urllib/urllib2 package that automatically
>> follow redirects, so I don't see why you'd need a 3rd-party module to
>> deal with it? When it encounters a 301 status code from the server,
>
> Ah, but I believe the issue is that the server isn't giving him a 301,
> it's giving him a 200 with a page explaining that "this page has
> moved, please update your bookmarks" etc, with a META HTTP-EQUIV tag
> to redirect the page after N seconds.
>
> (Based on the part where the OP said "I get the first warning/error
> message page and not the redirection one.")
>
> --
> Brett Ritter / SwiftOne
> swift...@swiftone.org
>
Ah, I see -- but people were talking about 301 and 302's, so I assumed
the topic had moved to general redirections. My bad.


-- 
best regards,
Robert S.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Structured files?

[Alan Gauld]

"Prasad, Ramit"  wrote 

>But I've only used JSON once from within TurboGears so 
If you use Firefox you have used JSON ;) 


Yeah, OK, I've used JSON on loads of apps in that sense. 
I meant I've only programmed with JSON once... 
But I suspect you knew that! :-)



--
Alan Gauld
Author of the Learn to Program web site
http://www.alan-g.me.uk/


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Structured files?

[Alan Gauld]

"Steven D'Aprano"  wrote

That makes it hugely wasteful of space which is usually the reason 
for using a binary format in the first place.


Meh, who cares whether your 100K of data takes 300K on disk? :)


It depends on your volumes and your platform. About 10 years
ago I bought 10G of storage for a mainframe (remember the
millenium bug?!). At the time a 10G hard drive for a PC was
about $100. For the big iron I paid around $10,000! It was
fast and it had mirroring etc as standard but it was still
eye-wateringly expensive... Storage was not cheap and binary
files were common. If you are working in that kind of environment
binary may still be a worthwhile option.

If you need/want binary storage, pickle has a binary mode. But 
generally, I am a big believer in sticking to text formats unless 
you absolutely have to use binary.


In general though I agree with you. The mainframe example above
is the exception not the rule and disk storage is one place I don't
mind XML. For network traffic I hate it and avoid it if possible!
But for most file storage plain txt or XML is far easier to work
with - and can also be compressed if storage is an issue.
And if I need to store a lot of data I use database and let the
vendor (or Opensource programmers) worry about the binary
formatting!!

--
Alan Gauld
Author of the Learn to Program web site
http://www.alan-g.me.uk/


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor