[Tutor] CGI Script Fails on import cgi
Hello, As I've mentioned on the list before, I'm very new to Python and programming in general. Now I'm trying my hand at writing very very simple CGI scripts. I basically wanted to try play with cgi scripting as I feel my way around Python. I'm uploading to my account on my university's server. Unfortunately, I error out whenever I do 'import cgi'. I'm hoping to receive some advice! Thanks in advance for reading all this. Facts: 1. From what I can tell, the server is Red Hat Enterprise Linux with Apache 1.3.33. 2. I have successfully executed .cgi scripts that only included printing text or html. 3. I've tried adding form validation with cgi.FieldStorage() and it would run in the server command line okay as .py, without syntax errors. 4. However, these same scripts including cgi.FieldStorage() would fail to execute when uploaded as a cgi script. I get this error: "CGI Script Failure The web page you attempted to access seems to have a problem: the script failed before sending any headers. " 4. I tested by adding only 'import cgi' to one of my html printing files that worked, to confirm that it was the sole problem. That script failed - so I knew the module import was it. 5. I then used try: import cgi, except: traceback.format_exc(None) I received this: Traceback (most recent call last): File "cgi_errors.cgi", line 9, in import cgi File "/usr/local/depot/python.1250025008/lib/python2.6/cgi.py", line 40, in import urllib File "/usr/local/depot/python.1250025008/lib/python2.6/urllib.py", line 26, in import socket File "/usr/local/depot/python.1250025008/lib/python2.6/socket.py", line 46, in import _socket ImportError: ld.so.1: python: fatal: relocation error: file /usr/local/depot/python.1250025008/lib/python2.6/lib-dynload/_socket.so: symbol inet_aton: referenced symbol not found 6. From the command line I typed python2.6, and then asked for sys.path, sys.modules["cgi"], sys.modules["socket"], and sys.modules["urllib"], and sys.modules["_socket"] and this is what it gave me: '/usr/local/depot/python.1250025008/lib/python26.zip', '/usr/local/depot/python.1250025008/lib/python2.6', '/usr/local/depot/python.1250025008/lib/python2.6/plat-linux2', '/usr/local/depot/python.1250025008/lib/python2.6/lib-tk', '/usr/local/depot/python.1250025008/lib/python2.6/lib-old', '/usr/local/depot/python.1250025008/lib/python2.6/lib-dynload', '/usr/local/depot/python.1250025008/lib/python2.6/site-packages', '/afs/umbc.edu/common/python26/i386_linux26/lib/python2.6/site-packages' Any advice would be much appreciated! - Kris ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Logfile Manipulation
"Stephen Nelson-Smith" wrote * How does Python compare in performance to shell, awk etc in a big pipeline? The shell script kills the CPU Python should be significantly faster than the typical shell script and it should consume less resources, although it will probably still use a fair bit of CPU unless you nice it. * What's the best way to extract the data for a given time, eg - 2359 yesterday? I'm not familiar with Apache log files so I'll let somebody else answer, but I suspect you can either use string.split() or a re.findall(). You might even be able to use csv. Or if they are in XML you could use ElementTree. It all depends on the data! -- Alan Gauld Author of the Learn to Program web site http://www.alan-g.me.uk/ ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Logfile Manipulation
On Mon, Nov 9, 2009 at 8:47 AM, Alan Gauld wrote: > I'm not familiar with Apache log files so I'll let somebody else answer, > but I suspect you can either use string.split() or a re.findall(). You might > even be able to use csv. Or if they are in XML you could use ElementTree. > It all depends on the data! An apache logfile entry looks like this: 89.151.119.196 - - [04/Nov/2009:04:02:10 +] "GET /service.php?s=nav&arg[]=&arg[]=home&q=ubercrumb/node%2F20812 HTTP/1.1" 200 50 "-" "-" I want to extract 24 hrs of data based timestamps like this: [04/Nov/2009:04:02:10 +] I also need to do some filtering (eg I actually don't want anything with service.php), and I also have to do some substitutions - that's trivial other than not knowing the optimum place to do it? IE should I do multiple passes? Or should I try to do all the work at once, only viewing each line once? Also what about reading from compressed files? The data comes in as 6 gzipped logfiles which expand to 6G in total. S. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Logfile Manipulation
> An apache logfile entry looks like this: > >89.151.119.196 - - [04/Nov/2009:04:02:10 +] "GET > /service.php?s=nav&arg[]=&arg[]=home&q=ubercrumb/node%2F20812 > HTTP/1.1" 200 50 "-" "-" > >I want to extract 24 hrs of data based timestamps like this: > > [04/Nov/2009:04:02:10 +] OK It looks like you could use a regex to extract the first thing you find between square brackets. Then convert that to a time. > I also need to do some filtering (eg I actually don't want anything > with service.php), That's easy enough to detect. > and I also have to do some substitutions - that's > trivial other than not knowing the optimum place to do it? Assuming they are trivial then... > I do multiple passes? Or should I try to do all the work at once, I'd opt for doing it all in one pass. With such large files you really want to minimise the amount of time spent reading the file. Plus with such large files you will need/want to process them line by line anyway rather than reading the whole thing into memory. > Also what about reading from compressed files? > The data comes in as 6 gzipped logfiles Python has a module for that but I've never used it. BTW A quick google reveals that there are several packages for handling Apache log files. That is probably worth investigating before you start writing lots of code... Examples: Scratchy - The Apache Log Parser and HTML Report Generator for Python Scratchy is an Apache Web Server log parser and HTML report generator written in Python. Scratchy was created by Phil Schwartz ... scratchy.sourceforge.net/ - Cached - Similar - Loghetti: an apache log file filter in Python - O'Reilly ONLamp Blog 18 Mar 2008 ... Loghetti: an apache log file filter in Python ... This causes loghetti to parsethe query string, and return lines where the query parameter ... www.oreillynet.com/.../blog/.../loghetti_an_apache_log_file_fi.html - HTH Alan G.___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Logfile Manipulation
Sorry - forgot to include the list. On Mon, Nov 9, 2009 at 9:33 AM, Stephen Nelson-Smith wrote: > On Mon, Nov 9, 2009 at 9:10 AM, ALAN GAULD wrote: >> >>> An apache logfile entry looks like this: >>> >>>89.151.119.196 - - [04/Nov/2009:04:02:10 +] "GET >>> /service.php?s=nav&arg[]=&arg[]=home&q=ubercrumb/node%2F20812 >>> HTTP/1.1" 200 50 "-" "-" >>> >>>I want to extract 24 hrs of data based timestamps like this: >>> >>> [04/Nov/2009:04:02:10 +] >> >> OK It looks like you could use a regex to extract the first >> thing you find between square brackets. Then convert that to a time. > > I'm currently thinking I can just use a string comparison after the > first entry for the day - that saves date arithmetic. > >> I'd opt for doing it all in one pass. With such large files you really >> want to minimise the amount of time spent reading the file. >> Plus with such large files you will need/want to process them >> line by line anyway rather than reading the whole thing into memory. > > How do I handle concurrency? I have 6 log files which I need to turn > into one time-sequenced log. > > I guess I need to switch between each log depending on whether the > next entry is the next chronological entry between all six. Then on a > per line basis I can also reject it if it matches the stuff I want to > throw out, and substitute it if I need to, then write out to the new > file. > > S. > -- Stephen Nelson-Smith Technical Director Atalanta Systems Ltd www.atalanta-systems.com ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Logfile Manipulation
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Hello,
: An apache logfile entry looks like this:
:
: 89.151.119.196 - - [04/Nov/2009:04:02:10 +] "GET
: /service.php?s=nav&arg[]=&arg[]=home&q=ubercrumb/node%2F20812
: HTTP/1.1" 200 50 "-" "-"
:
: I want to extract 24 hrs of data based timestamps like this:
:
: [04/Nov/2009:04:02:10 +]
:
: I also need to do some filtering (eg I actually don't want
: anything with service.php), and I also have to do some
: substitutions - that's trivial other than not knowing the optimum
: place to do it? IE should I do multiple passes?
I wouldn't. Then, you spend decompression CPU, line matching CPU
and I/O several times. I'd do it all at once.
: Or should I try to do all the work at once, only viewing each
: line once? Also what about reading from compressed files? The
: data comes in as 6 gzipped logfiles which expand to 6G in total.
There are standard modules for handling compressed data (gzip and
bz2). I'd imagine that the other pythonistas on this list will give
you more detailed (and probably better) advice, but here's a sample
of how to use the gzip module and how to skip the lines containing
the '/service.php' string, and to extract an epoch timestamp from
the datestamp field(s). You would pass the filenames to operate on
as arguments to this script.
See optparse if you want fancier capabilities for option handling.
See re if you want to match multiple patterns to ignore.
See time (and datetime) for mangling time and date strings. Be
forewarned, time zone issues will probably be a massive headache.
Many others have been here before [0].
Look up itertools (and be prepared for some study) if you want the
output from the log files from your different servers sorted in
the output.
Note that the below snippet is a toy and makes no attempt to trap
(try/except) any error conditions.
If you are looking for a weblog analytics package once you have
reassambled the files into a whole, perhaps you could just start
there (e.g. webalizer, analog are two old-school packages that come
to mind for processing logging that has been produced in a Common
Log Format).
I will echo Alan Gauld's sentiments of a few minutes ago and note
that there are a probably many different Apache log parsers out
there which can accomplish what you hope to accomplish. On the
other hand, you may be using this as an excuse to learn a bit of
python.
Good luck,
- -Martin
[0] http://seehuhn.de/blog/52
Sample:
import sys, time, gzip
files = sys.argv[1:]
for file in files:
print >>sys.stderr, "About to open %s" % ( file )
f = gzip.open( file )
for line in f:
if line.find('/service.php') > 0:
continue
fields = line.split()
# -- ignoring time zone; you are logging in UTC, right?
#tz = fields[4]
d = int( time.mktime( time.strptime(fields[3], "[%d/%b/%Y:%H:%M:%S") ) )
print d, line,
- --
Martin A. Brown
http://linux-ip.net/
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.9 (GNU/Linux)
Comment: pgf-0.72 (http://linux-ip.net/sw/pine-gpg-filter/)
iD8DBQFK9+MGHEoZD1iZ+YcRAhITAKCLGF6GnEMYr50bgk4vAw3YMRZjuACg2VUg
I7/Vrw6KKjwqfxG0qfr10lo=
=oi6X
-END PGP SIGNATURE-
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Logfile Manipulation
On Mon, Nov 9, 2009 at 4:36 AM, Stephen Nelson-Smith wrote: I want to extract 24 hrs of data based timestamps like this: [04/Nov/2009:04:02:10 +] >>> >>> OK It looks like you could use a regex to extract the first >>> thing you find between square brackets. Then convert that to a time. >> >> I'm currently thinking I can just use a string comparison after the >> first entry for the day - that saves date arithmetic. As long as the times are all in the same time zone. >> How do I handle concurrency? I have 6 log files which I need to turn >> into one time-sequenced log. >> >> I guess I need to switch between each log depending on whether the >> next entry is the next chronological entry between all six. Then on a >> per line basis I can also reject it if it matches the stuff I want to >> throw out, and substitute it if I need to, then write out to the new >> file. If you create iterators from the files that yield (timestamp, entry) pairs, you can merge the iterators using one of these recipes: http://code.activestate.com/recipes/491285/ http://code.activestate.com/recipes/535160/ Kent ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Logfile Manipulation
On Sun, Nov 8, 2009 at 11:41 PM, Stephen Nelson-Smith wrote: > I've got a large amount of data in the form of 3 apache and 3 varnish > logfiles from 3 different machines. They are rotated at 0400. The > logfiles are pretty big - maybe 6G per server, uncompressed. > > I've got to produce a combined logfile for -2359 for a given day, > with a bit of filtering (removing lines based on text match, bit of > substitution). > > I've inherited a nasty shell script that does this but it is very slow > and not clean to read or understand. > > I'd like to reimplement this in python. > > Initial questions: > > * How does Python compare in performance to shell, awk etc in a big > pipeline? The shell script kills the CPU > * What's the best way to extract the data for a given time, eg - > 2359 yesterday? > > Any advice or experiences? > > go here and download the pdf! http://www.dabeaz.com/generators-uk/ Someone posted this the other day, and I went and read through it and played around a bit and it's exactly what you're looking for - plus it has one vs. slide of python vs. awk. I think you'll find the pdf highly useful and right on. HTH, Wayne ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
[Tutor] class initialization with a lot of parameters
Hello All, I'm making a class and the parameters I'm feeding the class is getting quite large. I'm up to 8 now. Is there any rules of thumb for classes with a lot of parameters? I was thinking to put the parameters into a tuple and then in the __init__ of the class, iterate over the tuple and assign attributes. Right now my class basically looks like this: class Foo(object): def __init__(self, arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8): ... Cheers T ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Logfile Manipulation
Hi, >> Any advice or experiences? >> > > go here and download the pdf! > http://www.dabeaz.com/generators-uk/ > Someone posted this the other day, and I went and read through it and played > around a bit and it's exactly what you're looking for - plus it has one vs. > slide of python vs. awk. > I think you'll find the pdf highly useful and right on. Looks like generators are a really good fit. My biggest question really is how to multiplex. I have 6 logs per day, so I don't know how which one will have the next consecutive entry. I love teh idea of making a big dictionary, but with 6G of data, that's going to run me out of memory, isn't it S. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Logfile Manipulation
Stephen Nelson-Smith wrote: Hi, Any advice or experiences? go here and download the pdf! http://www.dabeaz.com/generators-uk/ Someone posted this the other day, and I went and read through it and played around a bit and it's exactly what you're looking for - plus it has one vs. slide of python vs. awk. I think you'll find the pdf highly useful and right on. Looks like generators are a really good fit. My biggest question really is how to multiplex. I have 6 logs per day, so I don't know how which one will have the next consecutive entry. I love teh idea of making a big dictionary, but with 6G of data, that's going to run me out of memory, isn't it Perhaps "lookahead" generators could help? Though that would be getting into advanced territory: http://stackoverflow.com/questions/1517862/using-lookahead-with-generators ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Logfile Manipulation
Hi,
> If you create iterators from the files that yield (timestamp, entry)
> pairs, you can merge the iterators using one of these recipes:
> http://code.activestate.com/recipes/491285/
> http://code.activestate.com/recipes/535160/
Could you show me how I might do that?
So far I'm at the stage of being able to produce loglines:
#! /usr/bin/env python
import gzip
class LogFile:
def __init__(self, filename, date):
self.f=gzip.open(filename,"r")
for logline in self.f:
self.line=logline
self.stamp=" ".join(self.line.split()[3:5])
if self.stamp.startswith(date):
break
def getline(self):
ret=self.line
self.line=self.f.readline()
self.stamp=" ".join(self.line.split()[3:5])
return ret
logs=[LogFile("a/access_log-20091105.gz","[05/Nov/2009"),LogFile("b/access_log-20091105.gz","[05/Nov/2009"),LogFile("c/access_log-20091105.gz","[05/Nov/2009")]
while True:
print [x.stamp for x in logs]
nextline=min((x.stamp,x) for x in logs)
print nextline[1].getline()
--
Stephen Nelson-Smith
Technical Director
Atalanta Systems Ltd
www.atalanta-systems.com
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Logfile Manipulation
And the problem I have with the below is that I've discovered that the
input logfiles aren't strictly ordered - ie there is variance by a
second or so in some of the entries.
I can sort the biggest logfile (800M) using unix sort in about 1.5
mins on my workstation. That's not really fast enough, with
potentially 12 other files
Hrm...
S.
On Mon, Nov 9, 2009 at 1:35 PM, Stephen Nelson-Smith wrote:
> Hi,
>
>> If you create iterators from the files that yield (timestamp, entry)
>> pairs, you can merge the iterators using one of these recipes:
>> http://code.activestate.com/recipes/491285/
>> http://code.activestate.com/recipes/535160/
>
> Could you show me how I might do that?
>
> So far I'm at the stage of being able to produce loglines:
>
> #! /usr/bin/env python
> import gzip
> class LogFile:
> def __init__(self, filename, date):
> self.f=gzip.open(filename,"r")
> for logline in self.f:
> self.line=logline
> self.stamp=" ".join(self.line.split()[3:5])
> if self.stamp.startswith(date):
> break
>
> def getline(self):
> ret=self.line
> self.line=self.f.readline()
> self.stamp=" ".join(self.line.split()[3:5])
> return ret
>
> logs=[LogFile("a/access_log-20091105.gz","[05/Nov/2009"),LogFile("b/access_log-20091105.gz","[05/Nov/2009"),LogFile("c/access_log-20091105.gz","[05/Nov/2009")]
> while True:
> print [x.stamp for x in logs]
> nextline=min((x.stamp,x) for x in logs)
> print nextline[1].getline()
>
>
> --
> Stephen Nelson-Smith
> Technical Director
> Atalanta Systems Ltd
> www.atalanta-systems.com
>
--
Stephen Nelson-Smith
Technical Director
Atalanta Systems Ltd
www.atalanta-systems.com
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Logfile Manipulation
On Mon, Nov 9, 2009 at 7:46 AM, Stephen Nelson-Smith wrote: > And the problem I have with the below is that I've discovered that the > input logfiles aren't strictly ordered - ie there is variance by a > second or so in some of the entries. > Within a given set of 10 lines, is the first line and last line "in order" - i.e. 1 2 4 3 5 8 7 6 9 10 > I can sort the biggest logfile (800M) using unix sort in about 1.5 > mins on my workstation. That's not really fast enough, with > potentially 12 other files > If that's the case, then I'm pretty sure you can create sort of a queue system, and it should probably cut down on the sorting time. I don't know what the default python sorting algorithm is on a list, but AFAIK you'd be looking at a constant O(log 10) time on each insertion by doing something such as this: log_generator = (d for d in logdata) mylist = # first ten values while True: try: mylist.sort() nextdata = mylist.pop(0) mylist.append(log_generator.next()) except StopIteration: print 'done' #Do something with nextdata Or now that I look, python has a priority queue ( http://docs.python.org/library/heapq.html ) that you could use instead. Just push the next value into the queue and pop one out - you give it some initial qty - 10 or so, and then it will always give you the smallest value. HTH, Wayne ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Logfile Manipulation
> I can sort the biggest logfile (800M) using unix sort in about 1.5 > mins on my workstation. That's not really fast enough, with > potentially 12 other files You won't beat sort with Python. You have to be realistic, these are very big files! Python should be faster overall but for specific tasks the Unix tools written in C will be faster. But if you are merging multiple files into one then sorting them before processing will probably help. However if you expect to be pruning out more lines than you keep it might be easier just to throw all the data you want into a single file and then sort that at the end. It all depends on the data. HTH, Alan G___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
[Tutor] Cramer's Rule
Hello everyone I have been tryin to convert the
following C- codes in python but always receiving error messages. Please Help.
The program is about Cramer's rule
#include
#double finddet(double a1,double a2, double
a3,double b1, double b2,double b3, double c1, double c2, double c3);
void main():
{
double a1, a2, a3, b1,
b2, b3, c1, c2, c3, d1, d2, d3,det, detx, dety, detz;
printf("\n a1?"); /*Input Coefficients*/
scanf("%lf",&a1);
printf("\n b1?");
scanf("%lf",&b1);
printf("\n
c1?");
scanf("%lf",&c1);
printf("\n d1?");
scanf("%lf",&d1);
printf("\n a2?");
scanf("%lf",&a2);
printf("\n b2?");
scanf("%lf",&b2);
printf("\n c2?");
scanf("%lf",&c2);
printf("\n d2?");
scanf("%lf",&d2);
printf("\n a3?");
scanf("%lf",&a3);
printf("\n b3?");
scanf("%lf",&b3);
printf("\n c3?");
scanf("%lf",&c3);
printf("\n d3?");
scanf("%lf",&d3);
det=finddet(a1,a2,a3,b1,b2,b3,c1,c2,c3);
/*Find determinants*/
detx=finddet(d1,d2,d3,b1,b2,b3,c1,c2,c3);
dety=finddet(a1,a2,a3,d1,d2,d3,c1,c2,c3);
detz=finddet(a1,a2,a3,b1,b2,b3,d1,d2,d3);
if(d1==0 &&
d2==0 && d3==0 && det==0)
printf("\n Infinite
Solutions\n ");
else if(d1==0 && d2==0 && d3==0
&& det!=0) /*Print Answers depending on various
conditions*/
printf("\n x=0\n y=0, \n z=0\n ");
else
if(det!=0)
printf("\n x=%lf\n y=%lf\n z=%lf\n", (detx/det),
(dety/det), (detz/det));
else if(det==0 && detx==0 &&
dety==0 && detz==0)
printf("\n Infinite Solutions\n
");
else
printf("No Solution\n ");
}
double finddet(double a1,double a2, double
a3,double b1, double b2,double b3, double c1, double c2, double c3)
{
return ((a1*b2*c3)-(a1*b3*c2)-(a2*b1*c3)+(a3*b1*c2)+(a2*b3*c1)-(a3*b2*c1));
/*expansion of a 3x3 determinant*/
}
k...@v INDERJEET
_
Keep your friends updated—even when you’re not signed in.
http://www.microsoft.com/middleeast/windows/windowslive/see-it-in-action/social-network-basics.aspx?ocid=PID23461::T:WLMTAGL:ON:WL:en-xm:SI_SB_5:092010___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Cramer's Rule
"Keshav Inderjeet" wrote Hello everyone I have been tryin to convert the following C- codes in python but always receiving error messages. Please Help. So give us a clue. What does your code look like? What errors are you getting? What do you think the problem might be? -- Alan Gauld Author of the Learn to Program web site http://www.alan-g.me.uk/ ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] CGI Script Fails on import cgi
"Kristin Wilcox" wrote Unfortunately, I error out whenever I do 'import cgi'. I'm hoping to receive some advice! Thanks in advance for reading all this. My guess is that your web server user id is not set up to find the Python modules. You probably need the admin to do some web config But that is just a guess. Alan G. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] class initialization with a lot of parameters
"C.T. Matsumoto" wrote I'm making a class and the parameters I'm feeding the class is getting quite large. I'm up to 8 now. Is there any rules of thumb for classes with a lot of parameters? There are no rules as such. Some of the Tkinter classes for excample take a lot of arguments. But you can usually simplify it by using default values and named parameters. Also can any of the input values be grouped together into an object? OOP means programming with objects, and that includes passing whole objects aropund as arguments. class Foo(object): def __init__(self, arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8): But with only a generic example to guide us we can only offer limited help. -- Alan Gauld Author of the Learn to Program web site http://www.alan-g.me.uk/ ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Logfile Manipulation
On Mon, Nov 9, 2009 at 3:15 PM, Wayne Werner wrote: > On Mon, Nov 9, 2009 at 7:46 AM, Stephen Nelson-Smith > wrote: >> >> And the problem I have with the below is that I've discovered that the >> input logfiles aren't strictly ordered - ie there is variance by a >> second or so in some of the entries. > > Within a given set of 10 lines, is the first line and last line "in order" - On average, in a sequence of 10 log lines, one will be out by one or two seconds. Here's a random slice: 05/Nov/2009:01:41:37 05/Nov/2009:01:41:37 05/Nov/2009:01:41:37 05/Nov/2009:01:41:37 05/Nov/2009:01:41:36 05/Nov/2009:01:41:37 05/Nov/2009:01:41:37 05/Nov/2009:01:41:37 05/Nov/2009:01:41:37 05/Nov/2009:01:41:37 05/Nov/2009:01:41:37 05/Nov/2009:01:41:37 05/Nov/2009:01:41:36 05/Nov/2009:01:41:37 05/Nov/2009:01:41:37 05/Nov/2009:01:41:38 05/Nov/2009:01:41:38 05/Nov/2009:01:41:37 05/Nov/2009:01:41:38 05/Nov/2009:01:41:38 05/Nov/2009:01:41:38 05/Nov/2009:01:41:38 05/Nov/2009:01:41:37 05/Nov/2009:01:41:38 05/Nov/2009:01:41:36 05/Nov/2009:01:41:38 05/Nov/2009:01:41:38 05/Nov/2009:01:41:38 05/Nov/2009:01:41:38 05/Nov/2009:01:41:39 05/Nov/2009:01:41:38 05/Nov/2009:01:41:39 05/Nov/2009:01:41:39 05/Nov/2009:01:41:39 05/Nov/2009:01:41:39 05/Nov/2009:01:41:40 05/Nov/2009:01:41:40 05/Nov/2009:01:41:41 > I don't know > what the default python sorting algorithm is on a list, but AFAIK you'd be > looking at a constant O(log 10) I'm not a mathematician - what does this mean, in layperson's terms? > log_generator = (d for d in logdata) > mylist = # first ten values OK > while True: > try: > mylist.sort() OK - sort the first 10 values. > nextdata = mylist.pop(0) So the first value... > mylist.append(log_generator.next()) Right, this will add another one value? > except StopIteration: > print 'done' > Or now that I look, python has a priority queue ( > http://docs.python.org/library/heapq.html ) that you could use instead. Just > push the next value into the queue and pop one out - you give it some > initial qty - 10 or so, and then it will always give you the smallest value. That sounds very cool - and I see that one of the activestate recipes Kent suggested uses heapq too. I'll have a play. S. -- Stephen Nelson-Smith Technical Director Atalanta Systems Ltd www.atalanta-systems.com ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] CGI Script Fails on import cgi
Please use ReplyAll when replying to the list. And please do not hijack other threads, it confuses threaded mail tools/newsreaders. From: Keshav Inderjeet To: alan.ga...@btinternet.com Sent: Monday, 9 November, 2009 16:10:30 Subject: RE: [Tutor] CGI Script Fails on import cgi > In fact these codes will work just on C but I need them to work on python. > Is there any way to convert em?? Not automatically, you need to translate them into Python yourself. The good news is that its probably going to be shorter than the C code. But we won't do it for you, we will help you to do it for yourself. How much Python do you know? Did you try to translate it? How far did you get? What happened? What is preventing you from fixing it? Alan G___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Logfile Manipulation
> > what the default python sorting algorithm is on a list, but AFAIK you'd be > > looking at a constant O(log 10) > > I'm not a mathematician - what does this mean, in layperson's terms? O(log10) is a way of expressing the efficiency of an algorithm. Its execution time is proportional (in the Order of) log10. That is the time to process 100 items will be about twice the time to process 10 (rather than 100 times), since log(100) is 2 and log(10) is 1 These measures are indicative only, but the bottom line is that it will scale to high volumes better than a linear algorithm would. HTH, Alan G.___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] SQLite database not update correctly
> It's working fine now, but actually I didn't write exactly what you > suggested. > The "commit" method belongs to the connection, not to the cursor. Therefore, > in my script it should be conn.commit(). Whoops, you're quite right. Went a little too fast there. :D Che _ Windows 7: Unclutter your desktop. http://go.microsoft.com/?linkid=9690331&ocid=PID24727::T:WLMTAGL:ON:WL:en-US:WWL_WIN_evergreen:112009___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] class initialization with a lot of parameters
C.T. Matsumoto wrote: Hello All, I'm making a class and the parameters I'm feeding the class is getting quite large. I'm up to 8 now. Is there any rules of thumb for classes with a lot of parameters? I was thinking to put the parameters into a tuple and then in the __init__ of the class, iterate over the tuple and assign attributes. Don't do it. Putting the parameters into a tuple only makes sense if they're somehow related to each other. And if all the parameters are in the tuple, all you've done is to add another parenthesis pair around the argument list. Now, if the parameters are related to each other (like the coordinates of an n-dimensional point), then it makes sense to group them. As Alan said, a class can be good for that. So can tuples, but only if there's some connection, or if they already were being treated as a tuple. Note that if you want to declare your parameters as a tuple, you can use the * notation in the parameter list. And if you want to pass the arguments as a tuple, you can use the * notation in the argument list. Or both. But you need to have a reason, other than "too many parameters." DaveA ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Tutor Digest, Vol 69, Issue 21 - Change a text string from a list and change it into an integer(WinXP/py2.6.2/Beginner)
First I want to thank the following people for your help: Wayne W., Shashwat
A., and Alan G. I appreciate the input/criticism as I continually learn
more and more.
It seems that I am not sure if I am posting correctly to the right thread
and am not sure if it has to do with the fact that I am getting the posts in
a Digest manner. Let me know if you have any suggestions.
Now back to message at hand:
Message: 3
Date: Thu, 5 Nov 2009 21:34:31 -0600
From: Wayne Werner
To: Katt
Cc: tutor
Subject: Re: [Tutor] Change a text string from a list and change it
into an integer number.(WinXP/py2.6.2/Beginner)
Currently the above code does not work unless I change the "if" statement
to say:
"if check_year == "c".
It works correctly for me. Try modifying your code:
date = "cyear_11_05"
date2 = date.split("_")
check_year = date2[0]
print check_year
what does that do for you?
-Wayne
For me it prints to the screen the following : cyear
Message: 4
Date: Fri, 6 Nov 2009 09:06:02 +0530
From: Shashwat Anand
To: Katt
Cc: tutor
Subject: Re: [Tutor] Change a text string from a list and change it
into an integer number.(WinXP/py2.6.2/Beginner)
What do you want to say exactly ?
is 'cyear' an integer ?
let's say date1 = "1984_11_05"
Here cyear is just part of a text string which would be in the following:
"cyear_11_05"
My goal(which recent I accomplished with the help of Shashwat A.'s function)
was:
1. Split the string into a list. [cyear,11,5]
2. If date0 == "cyear" then it would replace cyear with the current year
based on the computer's date.
Then it would change this information into an integer and return it.
3. If date0 != "cyear" it would then change the number into an integer and
return the information.
find code that I used below at end of this message.
Then of course you can change it to an integer using following
list-comprehension,
date1 = "1984_11_05"
date1_list = [int(i) for i in date1.split("_")]
date1_list
[1984, 11, 5]
or alternatively,
date1_list_alternate=map(int,date1.split("_"))
date1_list_alternate
[1984, 11, 5]
I am still learning how to work with list comprehensions so I appreciate the
multiple examples using the "for" and "map"
choices.
also your code seems to work on my system.
date = "cyear_11_05"
date2 = date.split("_")
check_year = date2[0]
if check_year == "cyear":
year = localtime().tm_year
else:
year = int(date2[0])
print year
Found that I had not imported localtime from time. It worked once I
discovered this little oversight.
Message: 5
Date: Fri, 6 Nov 2009 09:27:46 +0530
From: Shashwat Anand
To: Katt
Cc: tutor
Subject: Re: [Tutor] Change a text string from a list and change it
into an integer number.(WinXP/py2.6.2/Beginner)
import time
def katt(d):
date0 = d.split("_")[0]
if date0 == "cyear":
return int(time.strftime("%Y"))
else:
return int(date0)
print katt("cyear_11_05")
print katt("1984_11_05")
http://codepad.org/RBjKmNcA
Hope this helps !
Thanks this helps. I actually changed it a little so that I could include
it into another function rather than its own seperate function.
My code is as follows:
if year_check == "cyear":
year = int(strftime("%Y"))
else:
year = int(year_check)
if month_check == "cmonth":
month = int(strftime("%m"))
else:
month = int(month_check)
I of course made sure to include the strftime in my import calls. I may
change the int(strftime("%Y")) for localtime().tm_year because I think I
heard it returns the value as an integer, but will have to experiment.
Thanks again for the inspiration on this section of code.
Thanks again to all.
Katt
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
[Tutor] parsing XML
Hi! I need to parse several XML documents into a Python dictionary. Is there a module that would be particularly good for this? I heard beginners should start with ElementTree. However, SAX seems to make a little more sense to me. Any suggestions? ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Tutor Digest, Vol 69, Issue 21 - Change a text string from a list and change it into an integer(WinXP/py2.6.2/Beginner)
On Tue, Nov 10, 2009 at 9:12 AM, Katt wrote:
> First I want to thank the following people for your help: Wayne W.,
> Shashwat A., and Alan G. I appreciate the input/criticism as I continually
> learn more and more.
>
> It seems that I am not sure if I am posting correctly to the right thread
> and am not sure if it has to do with the fact that I am getting the posts in
> a Digest manner. Let me know if you have any suggestions.
>
On http://mail.python.org/mailman/listinfo/tutor there is an option which
says - Would you like to receive list mail batched in a daily digest? May be
you checked it. Anyways you can try changing your settings to help remove
the Digest manner
> I of course made sure to include the strftime in my import calls. I may
> change the int(strftime("%Y")) for localtime().tm_year because I think I
> heard it returns the value as an integer, but will have to experiment.
>
Yes, it does. You can always check the type though.
>>> type(time.localtime().tm_year)
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] parsing XML
On Mon, Nov 9, 2009 at 7:48 PM, Christopher Spears wrote: > Hi! > > I need to parse several XML documents into a Python dictionary. Is there a > module that would be particularly good for this? I heard beginners should > start with ElementTree. However, SAX seems to make a little more sense to > me. Any suggestions? > I'd recommend ElementTree. I started out with minidom and wanted to rip my face off. Ok, exaggerating, but ElementTree made a lot more sense. 2c ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] parsing XML
"Christopher Spears" wrote I need to parse several XML documents into a Python dictionary. Is there a module that would be particularly good for this? I heard beginners should start with ElementTree. However, SAX seems to make a little more sense to me. XML parsers fall into 2 groups. Those that parse the whole structure and create a tree of objects - usually accessed like a dictionary, and those that parse line by line looking for patterns. ElementTree is of the former, sax of the latter. The former approach is usually slightly slower and more resource hungry but is much more flexible. SAX is fast but generally best if you only want to read something specific out of the XML. If SAX makes sense for you and meets your needs go with it. But ElementTree is worth persevering with if you need to do more complex editing of the XML. Its certainly easier than minidom. (The other standard tree parser in Python) Alan G. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Tutor Digest, Vol 69, Issue 21 - Change a text string from a list and change it into an integer(WinXP/py2.6.2/Beginner)
Katt wrote:
First I want to thank the following people for your help: Wayne W.,
Shashwat A., and Alan G. I appreciate the input/criticism as I
continually learn more and more.
It seems that I am not sure if I am posting correctly to the right
thread and am not sure if it has to do with the fact that I am getting
the posts in a Digest manner. Let me know if you have any suggestions.
If you're using the digest version, you should copy the subject line and
simply add RE: in front of it (don't add "Tutor Digest, Vol. 69, Issue
21 - ") to preserve threading in a threaded newsreader. If you want to
change the subject line but keep the threading (often discussion drifts
away from the original topic), you could copy the Message ID header
(better idea, don't change the subject line unless you're using a
threaded newsreaded)
Message: 5
Date: Fri, 6 Nov 2009 09:27:46 +0530
From: Shashwat Anand
To: Katt
Cc: tutor
Subject: Re: [Tutor] Change a text string from a list and change it
into an integer number.(WinXP/py2.6.2/Beginner)
import time
def katt(d):
date0 = d.split("_")[0]
if date0 == "cyear":
return int(time.strftime("%Y"))
else:
return int(date0)
print katt("cyear_11_05")
print katt("1984_11_05")
http://codepad.org/RBjKmNcA
Hope this helps !
Thanks this helps. I actually changed it a little so that I could
include it into another function rather than its own seperate function.
My code is as follows:
if year_check == "cyear":
year = int(strftime("%Y"))
else:
year = int(year_check)
if month_check == "cmonth":
month = int(strftime("%m"))
else:
month = int(month_check)
an alternative could be to us str.replace:
from time import strftime
current_year = strftime("%Y")
current_month = strftime("%m")
current_day = strftme("%d")
def katt(date):
date = date.replace("cyear", current_year)
date = date.replace("cmonth", current_month)
date = date.replace("cday", current_day)
return map(int, date.split("_"))
I of course made sure to include the strftime in my import calls. I may
change the int(strftime("%Y")) for localtime().tm_year because I think I
heard it returns the value as an integer, but will have to experiment.
Thanks again for the inspiration on this section of code.
Thanks again to all.
Katt
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
