Need 4 Python Developer // Times Sq - New York // 15+ Months Contract.
Hi Friends, Hope you are doing great. This is Rajesh from NYTP. I wanted to let you know about New Job opening in Times Sq - New York. It is a 15+ months Contract. Role : Python Developer Location : Times Sq - New York Duration : 15+ Months Contract Positions : 4 Project currently in the Development Phase Qualifications: 4-6 years of development experience with object-oriented languages 3+ years of Python development experience Knowledge of HTML5 and Javascript is a plus Experience developing applications for AWS Agile development experience a plus Demonstrated ability to work in a team environment Good unit testing practices Good communication and documentation skills Willingness to interact and work with different teams across organizations in different time zones Willingness to work overtime and weekends if required Bachelor’s degree in Computer Science Required Skills: Strong in object-oriented concepts and Python language Experience developing web applications with Tornado Working knowledge of software design patterns Familiar with N-Tier caching strategies Familiar with REST and JSON Knowledgeable about MongoDB and REDIS If you are available and interested in this positions. Please send me an updated resume. Please feel free to contact me for any further information. ___ New York Technology Partners – Rochester Rajesh Kaluri 332 Jefferson Rd. Rochester, NY 14623 Phone: (201) 680 - 0200 x7023 Fax: (201) 474 - 8533 [email protected] www.nytp.com Profile : http://in.linkedin.com/pub/k-rajeshwar/8/51a/13 -- http://mail.python.org/mailman/listinfo/python-list
How to write Smart Python programs?
Hi, We just executed a project with Python using TG. The feedback was to use more python like programming rather than C style code executed in Python. The feedback is from a Python purist and for some reasons we cannot solicity his help. So we'd like to do is to scrub through the codebase and identify places where the codebase needs improvement, both from styling as well as design. Is there any website that can provide me with advanced tips rather than just tutorials coz thats not of much help. The project involves: 1. A server module that receives messages from a datalogger to populate a database 2. A DB access module that interfaces with the UI The technologies we use are TurboGears, SQLAlchemy, CherryPy, Kid templates. Any help will be greatly appreciated. If you would like to get involved in it for $$ (experienced in the tools mentioned above) as well do let me know. I need help asap as the deadlines are very very short. Thanks, Raj. -- http://mail.python.org/mailman/listinfo/python-list
Post data using the libcurl
Hi,
I am newbie to programming with libcurl. the problem is i want to send
my gmail username and password outside the browser and get access to my
mail. this i am doing with LibCurl. Though i dont get any errors, i am
nt able to get the o/p Can anyone kindly help. i will attach the
program here
#include
#include
#include
int main(int argc, char *argv[])
{
CURL *curl;
CURLcode res;
struct curl_httppost *formpost=NULL;
struct curl_httppost *lastptr=NULL;
struct curl_slist *headerlist=NULL;
char buf[] = "Expect:";
curl_global_init(CURL_GLOBAL_ALL);
/* Fill in the file upload field */
curl_formadd(&formpost,
&lastptr,
CURLFORM_COPYNAME, "rajani.vemula",
CURLFORM_END);
/* Fill in the filename field */
curl_formadd(&formpost,
&lastptr,
CURLFORM_COPYNAME, "x",
CURLFORM_END);
curl_formadd(&formpost,
&lastptr,
CURLFORM_COPYNAME, "null",
CURLFORM_COPYCONTENTS, "Sign in",
CURLFORM_END);
curl = curl_easy_init();
/* initalize custom header list (stating that Expect: 100-continue
is
not
wanted */
headerlist = curl_slist_append(headerlist, buf);
if(curl) {
/* what URL that receives this POST */
curl_easy_setopt(curl, CURLOPT_URL,"
http://mail.google.com/mail";);
curl_easy_setopt(curl, CURLOPT_PROXY, "proxyname");
curl_easy_setopt(curl, CURLOPT_PROXYUSERPWD, "User:Password)";
/* only disable 100-continue header if explicitly requested */
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headerlist);
curl_easy_setopt(curl, CURLOPT_HTTPPOST, formpost);
res = curl_easy_perform(curl);
/* always cleanup */
curl_easy_cleanup(curl);
/* then cleanup the formpost chain */
curl_formfree(formpost);
curl_slist_free_all(headerlist
}
return 0;
}
Thanks
--
http://mail.python.org/mailman/listinfo/python-list
Post data using the libcurl
Hi,
I am newbie to programming with libcurl. the problem is i want to send
my gmail username and password outside the browser and get access to my
mail. this i am doing with LibCurl. Though i dont get any errors, i am
nt able to get the o/p Can anyone kindly help. i will attach the
program here
#include
#include
#include
int main(int argc, char *argv[])
{
CURL *curl;
CURLcode res;
struct curl_httppost *formpost=NULL;
struct curl_httppost *lastptr=NULL;
struct curl_slist *headerlist=NULL;
char buf[] = "Expect:";
curl_global_init(CURL_GLOBAL_ALL);
/* Fill in the file upload field */
curl_formadd(&formpost,
&lastptr,
CURLFORM_COPYNAME, "rajani.vemula",
CURLFORM_END);
/* Fill in the filename field */
curl_formadd(&formpost,
&lastptr,
CURLFORM_COPYNAME, "x",
CURLFORM_END);
curl_formadd(&formpost,
&lastptr,
CURLFORM_COPYNAME, "null",
CURLFORM_COPYCONTENTS, "Sign in",
CURLFORM_END);
curl = curl_easy_init();
/* initalize custom header list (stating that Expect: 100-continue
is
not
wanted */
headerlist = curl_slist_append(headerlist, buf);
if(curl) {
/* what URL that receives this POST */
curl_easy_setopt(curl, CURLOPT_URL,"
http://mail.google.com/mail";);
curl_easy_setopt(curl, CURLOPT_PROXY, "proxyname");
curl_easy_setopt(curl, CURLOPT_PROXYUSERPWD, "User:Password)";
/* only disable 100-continue header if explicitly requested */
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headerlist);
curl_easy_setopt(curl, CURLOPT_HTTPPOST, formpost);
res = curl_easy_perform(curl);
/* always cleanup */
curl_easy_cleanup(curl);
/* then cleanup the formpost chain */
curl_formfree(formpost);
curl_slist_free_all(headerlist
}
return 0;
}
Thanks
--
http://mail.python.org/mailman/listinfo/python-list
Post data to http server using libcurl
Hi,
I am newbie to programming with libcurl. the problem is i want to send
my gmail username and password outside the browser and get access to my
mail. this i am doing with LibCurl. Though i dont get any errors, i am
nt able to get the o/p Can anyone kindly help. i will attach the
program here
#include
#include
#include
int main(int argc, char *argv[])
{
CURL *curl;
CURLcode res;
struct curl_httppost *formpost=NULL;
struct curl_httppost *lastptr=NULL;
struct curl_slist *headerlist=NULL;
char buf[] = "Expect:";
curl_global_init(CURL_GLOBAL_ALL);
/* Fill in the file upload field */
curl_formadd(&formpost,
&lastptr,
CURLFORM_COPYNAME, "rajani.vemula",
CURLFORM_END);
/* Fill in the filename field */
curl_formadd(&formpost,
&lastptr,
CURLFORM_COPYNAME, "x",
CURLFORM_END);
curl_formadd(&formpost,
&lastptr,
CURLFORM_COPYNAME, "null",
CURLFORM_COPYCONTENTS, "Sign in",
CURLFORM_END);
curl = curl_easy_init();
/* initalize custom header list (stating that Expect: 100-continue
is
not
wanted */
headerlist = curl_slist_append(headerlist, buf);
if(curl) {
/* what URL that receives this POST */
curl_easy_setopt(curl, CURLOPT_URL,"
http://mail.google.com/mail";);
curl_easy_setopt(curl, CURLOPT_PROXY, "proxyname");
curl_easy_setopt(curl, CURLOPT_PROXYUSERPWD, "User:Password)";
/* only disable 100-continue header if explicitly requested */
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headerlist);
curl_easy_setopt(curl, CURLOPT_HTTPPOST, formpost);
res = curl_easy_perform(curl);
/* always cleanup */
curl_easy_cleanup(curl);
/* then cleanup the formpost chain */
curl_formfree(formpost);
curl_slist_free_all(headerlist
}
return 0;
}
Thanks
--
http://mail.python.org/mailman/listinfo/python-list
seperating directories from files
i'm surely a newbie in python and on the go, I just tried to write a small app to change the gnome desktop background daily. It's working now making use of listdir() to find the contents of each dir I encounter. But I'm sure that my code won't go that smooth forever. How can I find if the content is a Directory or not and if it's a file, whether it is an image file or not? Is there any method to read the metadata of files? Thanking all of you in advance. -- http://mail.python.org/mailman/listinfo/python-list
Re: seperating directories from files
Thanx adam 'n' fuzzyman! got that 'isdir'. But how the other thing? Can't find anything suitable. I'll be happy on getting atleast something like the file command in Unix -- http://mail.python.org/mailman/listinfo/python-list
Re: what is the 'host' for SMTP?
If you've a gmail account, you can set it as smtp.gmail.com. -- http://mail.python.org/mailman/listinfo/python-list
Re: problem with sorting
To ankit: Well, sort() doesn't return the sorted list. It returns None. Why not this straightforward way? dvals = dict.values() dvals.sort() print dvals -- http://mail.python.org/mailman/listinfo/python-list
cresitrepair
hi guys you can download the softwares and can get the latest tips and tricks about computers in this blog so just visit hte site and enjoy. If you don't gat what you want try the ads given there and son't forger to search in google search box what you want. the site is www.creditrepairs-rs.blogspot.com -- http://mail.python.org/mailman/listinfo/python-list
Reportlab - Splitting table by column
My Zope based application for a client has to generate annual audit reports with too wide tables of account-wise payment details. I turned to reportlab when HTML formatting became a headache for me. Reportlab doc claims it can split by row as well as column but doesn't explain how to. Then I came to learn that reportlab does not implement byCol splitting. But is there a way out for me to split my wide data table into pages? What if I try to implement splitByCol in the line of splitByRow? (Only if it won't take much time and effort!) Why don't they do that themselves? Any serious obstacles? -- http://mail.python.org/mailman/listinfo/python-list
Some interactive Python tutorials on basic stats, possibly useful for teaching
Dear Python folks, Those of you whose duties include teaching basic stats might be interested in these interactive tutorial files, designed to illustrate basic concepts. Running the code opens up an interactive figure window. When you click on a figure to add new points, the statistical tests shown in the figure change accordingly. http://www.dartmouth.edu/~raj/intro-stats.html The code has lots of comments in it, which attempt to explain the concepts as explicitly as possible. No prior knowledge of Python or statistics is assumed. These programs require the SciPy and Matplotlib modules. The tutorials are: - interactive_mean_std_normal_distribution.py - interactive_one_sample_t_test.py - interactive_two_sample_t_test.py - interactive_correlation_plot.py The same webpage also contains Matlab versions of the scripts. Please feel more than free to use any of the code for teaching, if you find it useful. Yours, Rajeev Raizada Research Assistant Professor Neukom Institute for Computational Science Dartmouth College HB 6255 Hanover NH 03755 Tel: 603 646 0175 E.mail: [email protected] WWW: http://www.dartmouth.edu/~raj -- http://mail.python.org/mailman/listinfo/python-list
Python 3.1.2 and marshal
Hi,
I am using 64 bit Python on an x86_64 platform (Fedora 13). I have
some code that uses the python marshal module to serialize some
objects to files. However, in moving the code to python 3 I have come
across a situation where, if more than one object has been serialized
to a file, then while trying to de-serialize only the first object is
de-serialized. Trying to de-serialize the second object raises an
EOFError. De-serialization of multiple objects works fine in Python
2.x. I tried going through the Python 3 documentation to see if
marshal functionality has been changed, but haven't found anything to
that effect. Does anyone else see this problem? Here is some
example code:
bash-4.1$ cat marshaltest.py
import marshal
numlines = 1
numwords = 25
stream = open('fails.mar','wb')
marshal.dump(numlines, stream)
marshal.dump(numwords, stream)
stream.close()
tmpstream = open('fails.mar', 'rb')
value1 = marshal.load(tmpstream)
value2 = marshal.load(tmpstream)
print(value1 == numlines)
print(value2 == numwords)
Here are the results of running this code
bash-4.1$ python2.7 marshaltest.py
True
True
bash-4.1$ python3.1 marshaltest.py
Traceback (most recent call last):
File "marshaltest.py", line 13, in
value2 = marshal.load(tmpstream)
EOFError: EOF read where object expected
Interestingly the file created by using Python 3.1 is readable by both
Python 2.7 as well as Python 2.6 and both objects are successfully
read.
Cheers,
raj
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python 3.1.2 and marshal
On Jul 17, 10:11 pm, Thomas Jollans wrote:
[Snip]
> So, the contents of the file is identical, but Python 3 reads the whole
> file, Python 2 reads only the data it uses.
>
> This looks like a simple optimisation: read the whole file at once,
> instead of byte-by-byte, to improve performance when reading large
> objects. (such as Python modules...)
>
Good analysis and a nice catch. Thanks. It is likely that the intent
is to optimize performance.
> The question is: was storing multiple objects in sequence an intended
> use of the marshal module?
The documentation (http://docs.python.org/py3k/library/marshal.html)
for marshal itself states (emphasis added by me),
marshal.load(file)¶
Read *one value* from the open file and return it. If no valid
value is read (e.g. because the data has a different Python version’s
incompatible marshal format), raise EOFError, ValueError or TypeError.
The file must be an open file object opened in binary mode ('rb' or 'r
+b').
This suggests that support for reading multiple values is intended.
> I doubt it. You can always wrap your data in
> tuples or use pickle.
>
The code that I am moving to 3.x dates back to the python 1.5 days,
when marshal was significantly faster than pickle and Zope was
evolutionarily at the Bobo stage :-). I have switched the current code
to pickle - makes more sense. The pickle files are a bit larger and
loading it is a tad bit slower, but nothing that makes even a
noticeable difference for my use case. Thanks.
raj
--
http://mail.python.org/mailman/listinfo/python-list
Append to python List
Checkout the following code: sample2 = [x+y for x in range(1,10) for y in range(1,10) if x!=y] output=[] output=[x for x in sample2 if x not in output] the output I get is 3 4 5 6 7 8 9 10 3 5 6 7 8 9 10 11 4 5 7 8 9 10 11 12 5 6 7 9 10 11 12 13 6 7 8 9 11 12 13 14 7 8 9 10 11 13 14 15 8 9 10 11 12 13 15 16 9 10 11 12 13 14 15 17 10 11 12 13 14 15 16 17 which contains duplicate values. But if I do like this: sample2 = [x+y for x in range(1,10) for y in range(1,10) if x!=y] output=[] for x in sample2: if x not in output: output.append(x) the value of 'output' I get like this: 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 I know that both the programs have the same functionality, but why do I have different outputs? Please help! -- http://mail.python.org/mailman/listinfo/python-list
Re: Append to python List
Then what about this code part? [(x, y) for x in [1,2,3] for y in [3,1,4] if x != y] and the following code part: for x in [1,2,3]: for y in [3,1,4]: if x != y: combs.append((x, y)) On Thursday, May 9, 2013 12:24:24 PM UTC+5:30, Gary Herron wrote: > On 05/08/2013 11:36 PM, RAHUL RAJ wrote: > > > Checkout the following code: > > > > > > sample2 = [x+y for x in range(1,10) for y in range(1,10) if x!=y] > > > output=[] > > > > > output=[x for x in sample2 if x not in output] > > This statement is not doing what you expect. It is not building a list > > in the variable named output, it is building a list (anonymously) then > > binding it to the variable output once it's built. Therefore output is > > [] for the whole list building operation. > > > > The later operation works, because your *are* building the list in place > > as you go. > > > > > > > > the output I get is > > > 3 4 5 6 7 8 9 10 3 5 6 7 8 9 10 11 4 5 7 8 9 10 11 12 5 6 7 9 10 11 12 13 6 > > 7 8 9 11 12 13 14 7 8 9 10 11 13 14 15 8 9 10 11 12 13 15 16 9 10 11 12 13 > > 14 15 17 10 11 12 13 14 15 16 17 > > > > > > which contains duplicate values. > > > > > > > > > > > > > > > But if I do like this: > > > > > > sample2 = [x+y for x in range(1,10) for y in range(1,10) if x!=y] > > > output=[] > > > for x in sample2: > > >if x not in output: > > > output.append(x) > > > > > > > > > the value of 'output' I get like this: > > > 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 > > > > > > I know that both the programs have the same functionality, but why do I > > have different outputs? > > > > > > Please help! -- http://mail.python.org/mailman/listinfo/python-list
Re: Append to python List
I'm getting same output for both code parts, why not for th code parts in question? On Thursday, May 9, 2013 1:48:51 PM UTC+5:30, RAHUL RAJ wrote: > Then what about this code part? > > > > [(x, y) for x in [1,2,3] for y in [3,1,4] if x != y] > > > > and the following code part: > > > > for x in [1,2,3]: > > for y in [3,1,4]: > > if x != y: > > combs.append((x, y)) > > > > > > On Thursday, May 9, 2013 12:24:24 PM UTC+5:30, Gary Herron wrote: > > > On 05/08/2013 11:36 PM, RAHUL RAJ wrote: > > > > > > > Checkout the following code: > > > > > > > > > > > > > > sample2 = [x+y for x in range(1,10) for y in range(1,10) if x!=y] > > > > > > > output=[] > > > > > > > > > > > > > > > > > output=[x for x in sample2 if x not in output] > > > > > > This statement is not doing what you expect. It is not building a list > > > > > > in the variable named output, it is building a list (anonymously) then > > > > > > binding it to the variable output once it's built. Therefore output is > > > > > > [] for the whole list building operation. > > > > > > > > > > > > The later operation works, because your *are* building the list in place > > > > > > as you go. > > > > > > > > > > > > > > > > > > > > the output I get is > > > > > > > 3 4 5 6 7 8 9 10 3 5 6 7 8 9 10 11 4 5 7 8 9 10 11 12 5 6 7 9 10 11 12 13 > > > 6 7 8 9 11 12 13 14 7 8 9 10 11 13 14 15 8 9 10 11 12 13 15 16 9 10 11 12 > > > 13 14 15 17 10 11 12 13 14 15 16 17 > > > > > > > > > > > > > > which contains duplicate values. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > But if I do like this: > > > > > > > > > > > > > > sample2 = [x+y for x in range(1,10) for y in range(1,10) if x!=y] > > > > > > > output=[] > > > > > > > for x in sample2: > > > > > > >if x not in output: > > > > > > > output.append(x) > > > > > > > > > > > > > > > > > > > > > the value of 'output' I get like this: > > > > > > > 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 > > > > > > > > > > > > > > I know that both the programs have the same functionality, but why do I > > > have different outputs? > > > > > > > > > > > > > > Please help! -- http://mail.python.org/mailman/listinfo/python-list
MySQL dymanic query in Python
Can anyone tell me the proper way in which I can execute dynamic MySQL queries in Python? I want to do dynamic queries for both CREATE and INSERT statement. Here is my attempted code: sql="create table %s (%%s, %%s, %%s ... )" % (tablename,''.join(fields)+' '.join(types)) cur.execute(sql) where 'field' is the fieldnames stored in a list and 'types' are the fieldtypes stored in a list. -- http://mail.python.org/mailman/listinfo/python-list
Re: MySQL dymanic query in Python
On Wednesday, May 29, 2013 3:32:51 PM UTC+5:30, Fábio Santos wrote: > On 29 May 2013 10:13, "RAHUL RAJ" wrote: > > > > > > Can anyone tell me the proper way in which I can execute dynamic MySQL > > queries in Python? > > > > > > I want to do dynamic queries for both CREATE and INSERT statement. > > > > > > Here is my attempted code: > > > > > > > > > sql="create table %s (%%s, %%s, %%s ... )" % (tablename,''.join(fields)+' > > '.join(types)) > > > cur.execute(sql) > > > > > > > > > where 'field' is the fieldnames stored in a list and 'types' are the > > fieldtypes stored in a list. > > You need to join the fields and the field types. Use zip(). > > Then join with commas. > > fields_and_types = ['%s %s' % (field, type) for field, type in zip(fields, > types)] > > what_goes_between_the_parens = ', '.join(fields_and_types) > > sql = 'create table %s (%s)' % (tablename, what_goes_between_the_parens) > > See where that gets you. Fantastic! It worked, Thanks :) -- http://mail.python.org/mailman/listinfo/python-list
join this group
http://123maza.com/65/Cape201/ -- http://mail.python.org/mailman/listinfo/python-list
join this group
http://123maza.com/65/Cape201/ -- http://mail.python.org/mailman/listinfo/python-list
Problem using pip
dear team I have encountered a problem while using pip , that it did not allowed to install webbrowsing and OS module it show the error that it does not found any matching distributor and does not fount the version which can fulfill the requirement. .So , I want to ask you what I can do to solve this problem because I downloaded every module except for these two modules . please give the solution as soon as possible. regards -- https://mail.python.org/mailman/listinfo/python-list
Re: Problem using pip
Yes, that's it. Thanks for your help On Tue, Jan 14, 2020, 7:47 PM Jerry Hill wrote: > On Tue, Jan 14, 2020 at 2:18 AM proplayer raj > wrote: > > > > dear team > > I have encountered a problem while using pip , that it did not allowed to > > install webbrowsing and OS module it show the error that it does not > found > > any matching distributor and does not fount the version which can fulfill > > the requirement. > > Can you be specific about which modules you are trying to install? I > don't see any sign of a python package called 'webbrowsing' anywhere > on the internet, for instance. Are you possibly looking for the > 'webbrowser' module? If so, it's not installed via pip, it's part of > the python standard library (see docs here: > https://docs.python.org/3/library/webbrowser.html). There's also an > 'os' module in the standard library > (https://docs.python.org/3/library/os.html). Are those the modules you > were looking for? > > -- > Jerry > -- https://mail.python.org/mailman/listinfo/python-list
About some problem
Hello, I am beginner to python and i am writing following code from pytesser import * and i am getting an error as follow Traceback (most recent call last): File "", line 1, in File "C:\Python33\lib\site-packages\pytesser.py", line 61 print text ^ SyntaxError: invalid syntax How to resolve it ? Give me all steps to resolve it. -- https://mail.python.org/mailman/listinfo/python-list
Problem with using ConfigParser.py
Hi Folks,
Having a very strange problem in using python with apache/mod_python.
[conf.py attached]
>>> from ast.conf import conf
>>> c = conf()
>>> a = c.readPath('sandbox')
>>> len (a)
>>> 30
The same code when run thru the a web script gives following error -
[Errno 13] Permission denied: '/home/ast/ast-linux.conf'
Mod_python error: "PythonHandler mod_python.publisher"
Traceback (most recent call last):
File "/usr/lib/python2.4/site-packages/mod_python/apache.py", line 299, in
HandlerDispatch
result = object(req)
File "/usr/lib/python2.4/site-packages/mod_python/publisher.py", line 213,
in handler
published = publish_object(req, object)
File "/usr/lib/python2.4/site-packages/mod_python/publisher.py", line 412,
in publish_object
return publish_object(req,util.apply_fs_data(object, req.form, req=req))
File "/usr/lib/python2.4/site-packages/mod_python/util.py", line 439, in
apply_fs_data
return object(**args)
File "/var/www/html/ast-sandbox/testConf.py", line 9, in index
path = c.readPath('sandbox')
File "/usr/lib/python2.4/ast/conf.py", line 40, in readPath
self.vars = self.conf.options(target)
File "/usr/lib/python2.4/ConfigParser.py", line 241, in options
raise NoSectionError(section)
NoSectionError: No section: 'sandbox'
>
WebScript -
import sys
from ast.conf import conf
def index(req, **kw):
sys.stdout = req
req.content_type = 'text/html'
c = conf()
path = c.readPath('sandbox')
print path
The configuration file is the standard format used by ConfigParser module.
The permission for the ast-linux.conf file is the same as the user/group
defined in httpd.conf.
[EMAIL PROTECTED] ast-sandbox]# ls -l /home/ast/ast-linux.conf
-rw-rw-rw- 1 777 ast 7936 Jun 21 11:11 /home/ast/ast-linux.conf
Any pointers would help.
TIA
~Anthony
DISCLAIMER
==
This e-mail may contain privileged and confidential information which is the
property of Persistent Systems Pvt. Ltd. It is intended only for the use of the
individual or entity to which it is addressed. If you are not the intended
recipient, you are not authorized to read, retain, copy, print, distribute or
use this message. If you have received this communication in error, please
notify the sender and delete all copies of this message. Persistent Systems
Pvt. Ltd. does not accept any liability for virus infected mails.
""" common.py
A common set of functions used across production and sandbox
"""
import os, time
import ConfigParser
class conf:
def __init__(self):
self.dir = dict()
self.vars = []
self.astConfig = self.getAST()
#self.vars =
['rootDir','codeDir','dataDir','searchDir','snippetsDir','snippetStemsDir']
self.readConf()
def getAST(self):
#astConfig = os.path.expandvars('$ASTCONF')
astConfig = ''
if astConfig == '':
astConfig = "/home/ast/ast-linux.conf"
os.environ['ASTCONF'] = astConfig
return astConfig
def readConf(self):
self.conf = ConfigParser.ConfigParser()
#self.astConfig = os.path.expandvars('$ASTCONF')
#astConfig = os.path.expandvars('$ASTCONF')
#self.astconfig = '/home/ast/ast-linux.conf1'
a = self.conf.read(self.astConfig)
#print 'self.conf.read=',a
def readPath(self,target):
target = target.strip('')
target = target.strip('\r')
#print 'CONFIG=',self.astConfig
#print 'SECTIONS=',self.conf.sections()
self.vars = self.conf.options(target)
#print '\n+++vars=',self.vars
if self.conf.has_section(target):
for var in self.vars:
self.dir[var] = self.conf.get(target,var)
#print '+++dir=',self.dir
return self.dir
def _test(self):
import doctest
doctest.testmod()
def getTargets(self):
pass
#self.conf._sections()
if __name__ == "__main__":
time.clock()
c = conf()
dir = c.readPath('production')
print c.dir
print 'Elapsed time(s): ', time.clock()
--
http://mail.python.org/mailman/listinfo/python-list
New-style classes and special methods
Hi My question is about how special methods are stored internally in Python objects. Consider a new-style class which implements special methods such as __call__ and __new__ class C(type): def __call__(...): class B: __metaclass__ = C b= B() The type of C is 'type', that of B is 'C'. When B is instantiated, the __call__ method of C is first invoked, since C is the metaclass for B. Internally, when a Python callable object 'obj' is called, the actual function called seems to be 'obj->ob_type->tp_call'. Does this that somehow the '__call__' method defined in C above is assigned to the 'tp_call' slot in the object representing the class C, instead of it just being stored in the dictionary like a normal attribute? Where and how does this magic happen exactly? I'd appreciate any level of detail. Thanks! Raj -- http://mail.python.org/mailman/listinfo/python-list
Re: New-style classes and special methods
> Yes, special methods populate the slots in the structures which Python > uses to represent types. Objects/typeobject.c in the Python source > distribution does the hard work, particularly in function type_new Thanks for that quick response. I am quite comfortable with C code and am trying to understand exactly what happens when a new-style class is created, and then instantiated. I have been reading typeobject.c and type_new() inside it in detail, and there are a few issues I am trying to figure out. I can see a lot of *SLOT() macros in the file that seem to set the slots to appropriate values. What I am having trouble figuring out is the connection i.e. at what point during construction of the class object in type_new() are those slots allotted? Is it the tp_alloc() function which does this? Is there some kind of descriptor or other mechanism connecting special method names with their slots in the object representation? (e.g. "__call__" with type->tp_call) Also, what happens when a class inherits from multiple classes with their own __call__ methods? Where and how is it decided which __call__ goes into the tp_call slot? I'm sure I'll eventually figure it out if I stare at the code hard enough, but would totally appreciate any help I can get :) Thanks again! Raj -- http://mail.python.org/mailman/listinfo/python-list
Representation of new-style instance
Consider a new-style class
class rabbit(object):
def __init__(self,c):
self.color = c
r1=rabbit("blue")
r2=rabbit("purple")
Which C struct in the Python implementation is used to represent the
instances c1 and c2 of the
new-style class? I understand that when the class 'rabbit' is
created, the type_new function
in typeobject.c creates a copy of a 'struct typeobject' with
dictionary tp_dict appropriately modified.
However, I can't figure out which structure is used for new-style
instances and where the instance dictionary is stored. Could anyone
please clarify?
Thanks
Raj
--
http://mail.python.org/mailman/listinfo/python-list
How to know the starting point
Hi, I need help regarding the starting point in python project, As we can find main() function in java class to know the starting class in java, what is the starting point in python project? How to find the starting point. Thank you - Get the freedom to save as many mails as you wish. Click here to know how.-- http://mail.python.org/mailman/listinfo/python-list
Microdom xml file
Hi every one, Can any one send me the sample code/links for creating a XML file and adding a tag by using microdom in twisted python. Thank you in advanced. - Now you can chat without downloading messenger. Click here to know how.-- http://mail.python.org/mailman/listinfo/python-list
Regarding Python.h
Hi to all, I'm new to Python. I am using python2.4. I have one application written in c language in which it includes python like #include "Python.h" But when i try to install it, i'm getting an error like Python.h: No such file or directory I feel gcc is unable to get the path for Python.h What should i do to get rid of this problem. If i have to add any path pls give me details of how to do that. Thanks in advance. Chat on a cool, new interface. No download required. Go to http://in.messenger.yahoo.com/webmessengerpromo.php-- http://mail.python.org/mailman/listinfo/python-list
check a directory
Hi all, I'm using following code... for x in listdir(path) some code. but how i can check whether x is a directory or not? Because listdir() is giving all the files present in that path Thanks in advance. Did you know? You can CHAT without downloading messenger. Go to http://in.messenger.yahoo.com/webmessengerpromo.php/ -- http://mail.python.org/mailman/listinfo/python-list
library
hi, how to open python api documetation in linux machine? i think there should be a way to open installed library as a HTML pages. Thanks in advance. Did you know? You can CHAT without downloading messenger. Go to http://in.messenger.yahoo.com/webmessengerpromo.php/ -- http://mail.python.org/mailman/listinfo/python-list
Inconsistency in retrieving exceptions via sys module
Hi
I am writing some C code which sets and retrieves a Python exception. I
set the exception using PyErr_SetString(), and retrieve it in 2 ways: 1)
using PyErr_Occurred() and 2) Using sys.exc_type. However, I get two
different results and am very puzzled by this apparent inconsistency.
Could anyone please clarify what I'm missing? I need both to work
consistently for my application.
Here is the C code and the output
#include "Python.h"
#define OPRINT(o) PyObject_Print((o),stdout,0);printf("\n")
int main() {
PyObject *sysmod,*etype1,*etype2;
Py_Initialize();
sysmod = PyImport_ImportModule("sys");
PyErr_SetString(PyExc_Exception,"HI"); //set exception
etype1 = PyObject_GetAttrString(sysmod,"exc_type"); //sys.exc_type
etype2 = PyErr_Occurred();
printf("sys.exc_type : "); OPRINT(etype1);
printf("PyErr_Occurred() : "); OPRINT(etype2);
}
The output
sys.exc_type : None
PyErr_Occurred() :
Thank you
Raj
--
http://mail.python.org/mailman/listinfo/python-list
regarding xml elements
Hi,
I have an xml file in my application,
I have created an element using
Example goes like this.
document.createElement("abc")
and i appeneded it by using append() method.
But how i can reflect this change to my xml file?
and one more thing is i want to create element with some other parameters
like.
and i have m and n values as strings with me.
can anybody help me to create this element and write it to the existing xml
file
as a child of an existing element?
Thanks in advance..
Raj..
Forgot the famous last words? Access your message archive online at
http://in.messenger.yahoo.com/webmessengerpromo.php--
http://mail.python.org/mailman/listinfo/python-list
Question about overloading of binary operators
Hi Here's a simple class example I've defined # class myInt(int): def __add__(self,other): return 0 print 5 + myInt(4) #prints 9 print myInt(4) + 5 #prints 0 # The Python binary operation function (binary_op1() in Objects/abstract.c) states the rules for binary operations as follows: v w Action --- new new w.op(v,w)[*], v.op(v,w), w.op(v,w) new old v.op(v,w), coerce(v,w), v.op(v,w) old new w.op(v,w), coerce(v,w), v.op(v,w) old old coerce(v,w), v.op(v,w) [*] only when v->ob_type != w->ob_type && w->ob_type is a subclass of v->ob_type It seems that my example should fall in case 1, and in both cases, the __add__ function of the subclass should be used, returning 0, regardless of operand order. However, in one case the subclass's function is used and not in the other case. What am I missing here? Thanks Raj -- http://mail.python.org/mailman/listinfo/python-list
Free chat for u.....hot.hot.....hot..............
http://chattingfree.blogspot.com/ -- http://mail.python.org/mailman/listinfo/python-list
MAKE UPTO $5000 PER MONTH! $2000 IN FIRST 30 DAYS!
MAKE UPTO $5000 PER MONTH! $2000 IN FIRST 30 DAYS! Generate $50 to $100 whenever you have a couple of hours free time tospare. You could make $50 or more in the next 2 hours. Starting right Now! Today! Awesome earnings get paid for your honest work Join as a free member and get paid to your bank account To join the Network follow the link http://www.go-easy-money.com/home.html?a_aid=4c0be5bc81f9c Get paid for your real work and earn awesome -- http://mail.python.org/mailman/listinfo/python-list
option argument length
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, I'm using optparse module to parse all options and arguments. My program uses mostly "option arguments" hence my len(args) value is always zero. I need to check if the user has passed the correct number of "option arguments". Something like: (options,args) = parser.parse_args() len(options) != 1 or len(options) > 2: print "Incorrect number of arguments passed." How do I accomplish it ? Regards, rrs - -- Ritesh Raj Sarraf RESEARCHUT -- http://www.researchut.com "Stealing logics from one person is plagiarism, stealing from many is research." "Necessity is the mother of invention." Note: Please CC me. I'm not subscribed to the list -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2 (GNU/Linux) iD8DBQFDk1Nh4Rhi6gTxMLwRApx0AJ9XHlWFU1J0NdN02gtvimogUSgDkACgmkOO 2pX8ocoC7pot1a8R4u2BWrY= =piNo -END PGP SIGNATURE- -- http://mail.python.org/mailman/listinfo/python-list
Re: option argument length
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Marc 'BlackJack' Rintsch on Monday December 5 2005 03:24 wrote: > In <[EMAIL PROTECTED]>, Ritesh Raj > Sarraf wrote: > >> My program uses mostly "option arguments" hence my len(args) value is >> always zero. I need to check if the user has passed the correct number of >> "option arguments". Something like: >> >> (options,args) = parser.parse_args() >> >> len(options) != 1 or len(options) > 2: >> print "Incorrect number of arguments passed." >> >> How do I accomplish it ? > > Just insert an ``if`` in front of the condition and end the program with > ``sys.exit()`` after the message. > > Ciao, > Marc 'BlackJack' Rintsch This won't help because "options" is an instance. Regards, rrs - -- Ritesh Raj Sarraf RESEARCHUT -- http://www.researchut.com "Stealing logics from one person is plagiarism, stealing from many is research." "Necessity is the mother of invention." Note: Please CC me. I'm not subscribed to the list -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2 (GNU/Linux) iD8DBQFDlIbH4Rhi6gTxMLwRAlgSAJ0Y3TT9eCBgrck5N2Y9YjOTZMxUgwCcDeO5 qqgzY6rz2E4YKvurnlHL0nQ= =hlZO -END PGP SIGNATURE- -- http://mail.python.org/mailman/listinfo/python-list
Re: option argument length
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Peter Otten on Monday December 5 2005 03:34 wrote:
> options, args = parser.parse_args(values=MyValues())
>
> but you should do your users a favour and give them meaningful error
> messages. I can't conceive how you could achieve this by checking the
> number of options. Explicit constraint checks like
>
> options, args = parser.parse_args()
> if options.eat_your_cake and options.have_it:
> parser.error("Sorry, you cannot eat your cake and have it")
>
> will increase your script's usability and make it easier to maintain for
> only a tiny amount of work.
I'm using this for "option arguments" which are mutually inclusive.
But I want the user to pass atleast one "option argument" for the program to
function properly.
For example, I have an option "--fetch-update" which requires a file "foo"
to check what it has to fetch. If the file is provided as an argument, it
uses it, else I add a parser.set_defaults("foo") which makes the program to
look for it in the current working directory.
WHen the program see the "--fetch-update" option, it should execute the
required code. Now how do I check if at least one option has been passed at
the command-line ?
I have multiple options but I have parser.set_defaults() for each of them.
Regards,
rrs
- --
Ritesh Raj Sarraf
RESEARCHUT -- http://www.researchut.com
"Stealing logics from one person is plagiarism, stealing from many is
research."
"Necessity is the mother of invention."
Note: Please CC me. I'm not subscribed to the list
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.2 (GNU/Linux)
iD8DBQFDlIhF4Rhi6gTxMLwRAsLTAJ9ydjppGrFVbH2kL00vL00HgtrxqQCghZBq
pC0/1HftWE+9Eipx6vF+3Bo=
=Ay8m
-END PGP SIGNATURE-
--
http://mail.python.org/mailman/listinfo/python-list
Re: option argument length
On Tue, 6 Dec 2005, Peter Otten wrote:
> Ritesh Raj Sarraf wrote:
>
>> I'm using this for "option arguments" which are mutually inclusive.
>> But I want the user to pass atleast one "option argument" for the program
>> to function properly.
>>
>> For example, I have an option "--fetch-update" which requires a file "foo"
>> to check what it has to fetch. If the file is provided as an argument, it
>> uses it, else I add a parser.set_defaults("foo") which makes the program
>> to look for it in the current working directory.
>>
>> WHen the program see the "--fetch-update" option, it should execute the
>> required code. Now how do I check if at least one option has been passed
>> at the command-line ?
>> I have multiple options but I have parser.set_defaults() for each of them.
>
> I'm sorry I don't understand your example. Wouldn't you need at least two
> options to demonstrate "mutually inclusive" options? The set_default()
> method seems to accept only keyword arguments -- but even it were used
> correctly I'm still unclear why you would need it at all.
>
> Perhaps you can post a few sample invocations (both correct and illegal) of
> your script together with a description (in English, not code) of how the
> script should react?
>
> Peter, puzzled
>
>
try:
version = "0.6b"
reldate = "03/10/2005"
copyright = "(C) 2005 Ritesh Raj Sarraf - RESEARCHUT
(http://www.researchut.com/)"
#FIXME: Option Parsing
# There's a flaw with either optparse or I'm not well understood with
it
# Presently you need to provide all the arguments to it to work.
# No less, no more. This needs to be converted to getopt sometime.
#parser = OptionParser()
#parser = optparse.OptionParser()
parser = optparse.OptionParser(usage="%prog [OPTION1, OPTION2, ...]",
version="%prog " + version)
parser.add_option("-d","--download-dir", dest="download_dir",
help="Root directory path to save the downloaded files", action="store",
type="string")
parser.set_defaults(download_dir="foo")
parser.add_option("-s","--cache-dir", dest="cache_dir", help="Root
directory path where the pre-downloaded files will be searched. If not, give a
period '.'",action="store", type="string", metavar=".")
parser.set_defaults(cache_dir=".")
#parser.set_defaults(cache_dir=".")
#parser.add_option("-u","--uris", dest="uris_file", help="Full path of
the uris file which contains the main database of files to be
downloaded",action="store", type="string")
# We'll have additional options
# --set-update - This will extract the list of uris which need to be
fetched
# --fetch-update - This will fetch the list of uris which need for
update.
# --install-update - This will install the fetched database files
# The same will happen for upgradation.
# --set-upgrade - This will extract the list of uris which need to be
fetched
# --fetch-upgrade - This will fetch the list of uris which need for
upgrade
# --install-upgrade - This will install the fetched database files
parser.add_option("","--set-update", dest="set_update", help="Extract
the list of uris which need to be fetched for _updation_", action="store",
type="string", metavar="foo")
parser.set_defaults(set_update="foo")
parser.add_option("","--fetch-update", dest="fetch_update",
help="Fetch the list of uris which are needed for _updation_.", action="store",
type="string", metavar="foo")
parser.set_defaults(fetch_update="foo")
parser.add_option("","--install-update", dest="install_update",
help="Install the fetched database files ", action="store", type="string",
metavar="foo.zip")
parser.set_defaults(install_update="foo.zip")
parser.add_option("","--set-upgrade", dest="set_upgrade",
help="Extract the list of uris which need to be fetched ", action="store",
type="string", metavar="foo.dat")
parser.set_defaults(set_upgrade="foo.dat")
parser.add_option("","--fetch-upg
Re: option argument length
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Hi Peter,
Peter Otten on Wednesday December 7 2005 21:25 wrote:
> This can be simplified to
>
> parser.add_option("-d", "--download-dir", default="foo",
> help="Root directory path to save the downloaded files")
>
> which seems to be the reason why I've never seen the set_defaults() call
> before.
As per python docs, it's mentioned that "default=" has been deprecated.
That's why I changed to parser.set_defaults()
>
>> if len(options) != 1 or len(options) > 2:
>
> It doesn't matter much as it won't work anyway, but
>
> len(options) > 2 implies len(options) != 1, so
>
> if len(options) != 1:
> #...
>
Yes, you're right. Sorry.
> would suffice here.
>
But will len(options) give a meaningful output. "options" is an instance.
It gives a Value Error.
> Now to the actual problem: I think you didn't understand my previous
> question. I cannot infer from your non-working code what it actually
> should do. I asked for examples of how your script would be used. E. g,
> assuming the code above is in a file called sarraf.py, what should the
> following invocations
>
> ./sarraf.py --fetch-update bar
> ./sarraf.py --fetch-update bar --the-mutually-inclusive-option baz
>
> do? Would the first terminate with an error message that another option
> must also be given? Would it use the default? Would the second be
> accepted? Try to describe it as simple and clear as possible. Imagine you
> were talking to someone who has never written a line of code.
>
./sarraf.py --fetch-update /bar
If the user gives the /bar argument, the program should save the downloaded
files to /bar. But I'm assuming that the user could be dumb or too lazy, in
which case --fetch-udpate should use the parser.set_defaults value
i.e. /foo
./sarraf.py --set-upgrade foo.dat --upgrade-type minimal
set-upgrade will again write data to foo.dat. If the user doesn't pass it as
an arguemnt it should take the defaults (again whatever is there in
parser.set_defaults). This will be inclusive with the --upgrade-type option
because the user will have a choice of selecting what kind of upgrade he'd
like to do, minimal or full or blah.
For this I think this should be enough:
if not options.set_upgrade and options.upgrade_type:
parser.error("They are mutually inclusive options")
But my main concern is what if the user doesn't pass any arguemtns. Every
option I have has a default value. So I want to check what the user has
passed.
But unfortunately the args variable has "0" as its value always.
Is my way (up till now) of using optparse logically incorrect or improper ?
Regards,
rrs
- --
Ritesh Raj Sarraf
RESEARCHUT -- http://www.researchut.com
"Stealing logics from one person is plagiarism, stealing from many is
research."
"Necessity is the mother of invention."
Note: Please CC me. I'm not subscribed to the list
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.2 (GNU/Linux)
iD8DBQFDlzC84Rhi6gTxMLwRAtCBAJ9z5zDQ8oyx8Jy/rLe9JrwLII3xtACfTaEV
VhMmj8OD+/p+yN/8wF6xe+8=
=atC6
-END PGP SIGNATURE-
--
http://mail.python.org/mailman/listinfo/python-list
urllib.urlretireve problem
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Hello Everybody,
I've got a small problem with urlretrieve.
Even passing a bad url to urlretrieve doesn't raise an exception. Or does
it?
If Yes, What exception is it ? And how do I use it in my program ? I've
searched a lot but haven't found anything helping.
Example:
try:
urllib.urlretrieve("http://security.debian.org/pool/updates/main/p/perl/libparl5.6_5.6.1-8.9_i386.deb";)
except IOError, X:
DoSomething(X)
except OSError, X:
DoSomething(X)
urllib.urlretrieve doesn't raise an exception even though there is no
package named libparl5.6
Please Help!
rrs
- --
Ritesh Raj Sarraf
RESEARCHUT -- http://www.researchut.com
Gnupg Key ID: 04F130BC
"Stealing logic from one person is plagiarism, stealing from many is
research".
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
iD8DBQFCRcCk4Rhi6gTxMLwRAlb2AJ0fB3V5ZpwdAiCxfl/rGBWU92YBEACdFYIJ
8bGZMJ5nuKAqvjO0KEAylUg=
=eaHC
-END PGP SIGNATURE-
--
http://mail.python.org/mailman/listinfo/python-list
Re: urllib.urlretireve problem
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Larry Bates wrote: > I noticed you hadn't gotten a reply. When I execute this it put's the > following in the retrieved file: > > > > 404 Not Found > > Not Found > The requested URL /pool/updates/main/p/perl/libparl5.6_5.6.1-8.9_i386.deb > was no t found on this server. > > > You will probably need to use something else to first determine if the URL > actually exists. I'm happy that at least someone responded as this was my first post to the python mailing list. I'm coding a program for offline package management. The link that I provided could be obsolete by newer packages. That is where my problem is. I wanted to know how to raise an exception here so that depending on the type of exception I could make my program function. For example, for Temporary Name Resolution Failure, python raises an exception which I've handled well. The problem lies with obsolete urls where no exception is raised and I end up having a 404 error page as my data. Can we have an exception for that ? Or can we have the exit status of urllib.urlretrieve to know if it downloaded the desired file. I think my problem is fixable in urllib.urlopen, I just find urllib.urlretrieve more convenient and want to know if it can be done with it. Thanks for responding. rrs - -- Ritesh Raj Sarraf RESEARCHUT -- http://www.researchut.com Gnupg Key ID: 04F130BC "Stealing logic from one person is plagiarism, stealing from many is research". -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.5 (GNU/Linux) iD8DBQFCSuYS4Rhi6gTxMLwRAu0FAJ9R0s4TyB7zHcvDFTflOp2joVkErQCfU4vG 8U0Ah5WTdTQHKRkmPsZsHdE= =OMub -END PGP SIGNATURE- -- http://mail.python.org/mailman/listinfo/python-list
Re: urllib.urlretireve problem
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Diez B. Roggisch wrote: > You could for instance try and see what kind of result you got using the > unix file command - it will tell you that you received a html file, not a > deb. > > Or check the mimetype returned - its text/html in the error case of yours, > and most probably something like application/octet-stream otherwise. > Using the unix file command is not possible at all. The whole goal of the program is to help people get their packages downloaded from some other (high speed) machine which could be running Windows/Mac OSX/Linux et cetera. That is why I'm sticking strictly to python libraries. The second suggestion sounds good. I'll look into that. Thanks, rrs - -- Ritesh Raj Sarraf RESEARCHUT -- http://www.researchut.com Gnupg Key ID: 04F130BC "Stealing logic from one person is plagiarism, stealing from many is research". -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.5 (GNU/Linux) iD8DBQFCTDhV4Rhi6gTxMLwRAi2BAJ4zp7IsQNMZ1zqpF/hGUAjUyYwKigCeKaqO FbGuuFOIHawZ8y/ICf87wOI= =btA5 -END PGP SIGNATURE- -- http://mail.python.org/mailman/listinfo/python-list
A simple single line, triple-quoted comment is giving syntax error. Why?
I always do single line comments with # but just for the sake of it I tried it with ''' ''' and it gives me a syntax error. In both the interpreter, and the source code text file, doing - a = 5 '''a comment''' results in a syntax error, with the very last quote at the end of the line highlighted in red. Of course, if I do - a = 5 #'''a comment''' it works. I searched for the problem, and arrived at http://stackoverflow.com/questions/397148/why-doesnt-python-have-multiline-comments which says that there are no 'true' multiline comments in python and that all those 'block' comments are actually triple-quoted strings. Then I looked in the documentation and found https://docs.python.org/3/reference/lexical_analysis.html#string-and-bytes-literals but it is a little bit too complex for my understanding (I'm just starting python). So can someone tell me why a triple-quoted string gives a syntax error if only in one line? Actually, there are other confusions I have too, regarding using backslashes inside triple-quoted strings to form multi-line comments, and a general uncertainty about triple-quoted strings. Can someone also provide a sort of a 'guide' to triple-quoted comments in general? Something like how I can just sum up index slices by saying in [a:b], the 'counting' for a always starts with 0, a is included, everything up to b but not b is included (assuming this is in fact the correct explanation ;-)) -- https://mail.python.org/mailman/listinfo/python-list
Re: A simple single line, triple-quoted comment is giving syntax error. Why?
On Wednesday, March 18, 2015 at 1:04:39 PM UTC-5, Laurent Pointal wrote: > > Can someone also provide a sort of a 'guide' to triple-quoted comments > > in general? > > A triple ' or " string is a Python string, allowing line-return in string. What do you mean by line-return in string? Is it newline? Does it mean I can write - '''first part second part''' ? > If it is in an expression (like a = 5 '''a comment'''), then it must be a > valid expression (and here it is not). What is not a valid expression here? > > A+ > Laurent. Thank you for the fast and detailed response, Aditya -- https://mail.python.org/mailman/listinfo/python-list
urllib2.urlopen Progress bar
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Hi,
In urllib.urlretrieve I can use reporthook to implement a progress bar.
But in urllib2.urlopen I can't.
I have to use urllib2.urlopen because it throws exceptions which are
important to be handled where as urlretrieve is dumb.
Is there a way to implement urllib2.urlopen to enable reporthook ?
progress.py code:
#!/usr/bin/env python
import time, urllib2, urllib
class progressBar:
def __init__(self, minValue = 0, maxValue = 10, totalWidth=12):
self.progBar = "[]" # This holds the progress bar string
self.min = minValue
self.max = maxValue
self.span = maxValue - minValue
self.width = totalWidth
self.amount = 0 # When amount == max, we are 100% done
self.updateAmount(0) # Build progress bar string
def updateAmount(self, newAmount = 0):
if newAmount < self.min: newAmount = self.min
if newAmount > self.max: newAmount = self.max
self.amount = newAmount
# Figure out the new percent done, round to an integer
diffFromMin = float(self.amount - self.min)
percentDone = (diffFromMin / float(self.span)) * 100.0
percentDone = round(percentDone)
percentDone = int(percentDone)
# Figure out how many hash bars the percentage should be
allFull = self.width - 2
numHashes = (percentDone / 100.0) * allFull
numHashes = int(round(numHashes))
# build a progress bar with hashes and spaces
self.progBar = "[" + '#'*numHashes + ' '*(allFull-numHashes) + "]"
# figure out where to put the percentage, roughly centered
percentPlace = (len(self.progBar) / 2) - len(str(percentDone))
percentString = str(percentDone) + "%"
# slice the percentage into the bar
#self.progBar = self.progBar[0:percentPlace] + percentString +
self.progBar[percentPlace+len(percentString):]
self.progBar = self.progBar[0:percentPlace] +
self.progBar[percentPlace+len(percentString):] + percentString
def __str__(self):
return str(self.progBar)
def myReportHook(count, blockSize, totalSize):
import sys
global prog
if prog == "":
prog = progressBar(0,totalSize,50)
#print count, blockSize, totalSize
#prog = progressBar(0, totalSize, 77)
prog.updateAmount(count*blockSize)
sys.stdout.write (str(prog))
sys.stdout.write ("\r")
#print count * (blockSize/1024) , "kb of " , (totalSize/1024) , "kb
downloaded.\n"
prog = ""
sFile = "new.rpm"
sUrl="http://localhost/new.rpm";
urllib.urlretrieve(sUrl, sFile, reporthook=myReportHook)
print "\n\n"
#temp = urllib2.urlopen(sUrl)
#lastval = int(temp.headers['Content-Length'])
#prog = progressBar(0, lastval, 77)
#
#for x in range(101):
#prog.updateAmount(x)
#print prog, "\r", time.sleep(0.5)
#
#data = open(sFile,'wb')
#data.write(temp.read())
#data.close()
#temp.close()
Regards,
Ritesh
- --
Ritesh Raj Sarraf
RESEARCHUT -- http://www.researchut.com
Gnupg Key ID: 04F130BC
"Stealing logic from one person is plagiarism, stealing from many is
research."
"Necessity is the mother of invention."
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.2 (GNU/Linux)
iD8DBQFDyfkM4Rhi6gTxMLwRAlmMAJ9yePv6DvXCcnUb0DYhWSuIonciSACgrxDj
aKSGkfg8I1T/q3r6zWLTXmw=
=uomB
-END PGP SIGNATURE-
--
http://mail.python.org/mailman/listinfo/python-list
[FIXED] Re: urllib2.urlopen Progress bar
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Calling urllib2.urlopen the below mentioned way fixed my problem.
try:
os.chdir (sSourceDir)
block_size = 4096
i = 0
count = 0
temp = urllib2.urlopen(sUrl)
headers = temp.info()
size = int(headers['Content-Length'])
data = open (sFile, 'wb')
while i < size:
data.write (temp.read(block_size))
i += block_size
counter += 1
progressbar.myReportHook(counter, block_size, size)
print "\n"
data.close()
temp.close()
Thanks,
Ritesh
Ritesh Raj Sarraf on Sunday 15 Jan 2006 12:55 wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Hi,
>
> In urllib.urlretrieve I can use reporthook to implement a progress bar.
>
> But in urllib2.urlopen I can't.
> I have to use urllib2.urlopen because it throws exceptions which are
> important to be handled where as urlretrieve is dumb.
>
> Is there a way to implement urllib2.urlopen to enable reporthook ?
>
> progress.py code:
>
> #!/usr/bin/env python
>
> import time, urllib2, urllib
>
> class progressBar:
> def __init__(self, minValue = 0, maxValue = 10, totalWidth=12):
> self.progBar = "[]" # This holds the progress bar string
> self.min = minValue
> self.max = maxValue
> self.span = maxValue - minValue
> self.width = totalWidth
> self.amount = 0 # When amount == max, we are 100% done
> self.updateAmount(0) # Build progress bar string
>
> def updateAmount(self, newAmount = 0):
> if newAmount < self.min: newAmount = self.min
> if newAmount > self.max: newAmount = self.max
> self.amount = newAmount
>
> # Figure out the new percent done, round to an integer
> diffFromMin = float(self.amount - self.min)
> percentDone = (diffFromMin / float(self.span)) * 100.0
> percentDone = round(percentDone)
> percentDone = int(percentDone)
>
> # Figure out how many hash bars the percentage should be
> allFull = self.width - 2
> numHashes = (percentDone / 100.0) * allFull
> numHashes = int(round(numHashes))
>
> # build a progress bar with hashes and spaces
> self.progBar = "[" + '#'*numHashes + ' '*(allFull-numHashes) + "]"
>
> # figure out where to put the percentage, roughly centered
> percentPlace = (len(self.progBar) / 2) - len(str(percentDone))
> percentString = str(percentDone) + "%"
>
> # slice the percentage into the bar
> #self.progBar = self.progBar[0:percentPlace] + percentString +
> self.progBar[percentPlace+len(percentString):]
> self.progBar = self.progBar[0:percentPlace] +
> self.progBar[percentPlace+len(percentString):] + percentString
>
> def __str__(self):
> return str(self.progBar)
>
> def myReportHook(count, blockSize, totalSize):
> import sys
> global prog
>
> if prog == "":
> prog = progressBar(0,totalSize,50)
> #print count, blockSize, totalSize
> #prog = progressBar(0, totalSize, 77)
> prog.updateAmount(count*blockSize)
> sys.stdout.write (str(prog))
> sys.stdout.write ("\r")
> #print count * (blockSize/1024) , "kb of " , (totalSize/1024) , "kb
> downloaded.\n"
> prog = ""
> sFile = "new.rpm"
> sUrl="http://localhost/new.rpm";
> urllib.urlretrieve(sUrl, sFile, reporthook=myReportHook)
> print "\n\n"
> #temp = urllib2.urlopen(sUrl)
> #lastval = int(temp.headers['Content-Length'])
> #prog = progressBar(0, lastval, 77)
> #
> #for x in range(101):
> #prog.updateAmount(x)
> #print prog, "\r", time.sleep(0.5)
> #
> #data = open(sFile,'wb')
> #data.write(temp.read())
> #data.close()
> #temp.close()
>
>
> Regards,
> Ritesh
> - --
> Ritesh Raj Sarraf
> RESEARCHUT -- http://www.researchut.com
> Gnupg Key ID: 04F130BC
> "Stealing logic from one person is plagiarism, stealing from many is
> research."
> "Necessity is the mother of invention."
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v1.4.2 (GNU/Linux)
>
> iD8DBQFDyfkM4Rhi6gTxMLwRAlmMAJ9yePv6DvXCcnUb0DYhWSuIonciSACgrxDj
> aKSGkfg8I1T/q3r6zWLTXmw=
> =uomB
> -END PGP SIGNATURE-
>
- --
Ritesh Raj Sarraf
RESEARCHUT -- http://www.researchut.com
"Stealing logic from one person is plagiarism, stealing from many is
research."
"Necessity is the mother of invention."
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.2 (GNU/Linux)
iD8DBQFDyicF4Rhi6gTxMLwRAldhAJ9cdnma+FerMVxe/EAG+eokL1POdQCgr5A3
ZyMQFN96Yrkp5frgHPYUnfU=
=Y9wS
-END PGP SIGNATURE-
--
http://mail.python.org/mailman/listinfo/python-list
python logging module problem
import os, sys, logging
logger = logging.getLogger("my_app")
conerr = logging.StreamHandler(sys.stderr)
conerr.setLevel(logging.DEBUG)
conerr_formatter = logging.Formatter('%(levelname)s %(message)s')
conerr.setFormatter(conerr_formatter)
console = logging.StreamHandler(sys.stdout)
console.setLevel(logging.INFO)
console_formatter = logging.Formatter('%(message)s')
console.setFormatter(console_formatter)
logger.addHandler(conerr)
logger.addHandler(console)
logger.info("Ritesh Raj Sarraf.\n")
logger.warning("Ricky Raj Sarraf.\n")
Hi,
When I execute the above code, logger.info()'s messages don't get
displayed. And logger.warning()'s messages get displayed twice.
C:\Eclipse\Workspace\Python Fun>python log.py
WARNING Ricky Raj Sarraf.
Ricky Raj Sarraf.
Is there something I am doing wrong ?
I basically want to use Python's logging module for my entire program.
I want is something like logger.message() which would contain normal
program messages which shouldbe passed to stdout.
I also want to implement a logger.verbose() handler which would execute
when we enable verbose mode.
Am I doing it the correct way ? Or am I using the wrong tool ? Should
logging be used for it ?
TIA,
Ritesh
--
http://mail.python.org/mailman/listinfo/python-list
Re: python logging module problem
Ritesh Raj Sarraf wrote:
> import os, sys, logging
>
> logger = logging.getLogger("my_app")
>
I tried this code:
import logging, sys
# set up logging to file - see previous section for more details
logging.basicConfig(level=logging.DEBUG,
format='%(asctime)s %(name)-12s %(levelname)-8s
%(message)s',
stream=sys.stderr)
# define a Handler which writes INFO messages or higher to the
sys.stderr
console = logging.StreamHandler(sys.stdout)
console.setLevel(logging.INFO)
# set a format which is simpler for console use
formatter = logging.Formatter('%(message)s')
# tell the handler to use this format
console.setFormatter(formatter)
# add the handler to the root logger
#logging.getLogger('').addHandler(console)
logging.RootLogger(console)#.addHandler(console)
# Now, we can log to the root logger, or any other logger. First the
root...
logging.info('Jackdaws love my big sphinx of quartz.')
logging.debug('Ritesh raj Sarraf.\n')
With this it seems to be working halfway.
logging.debug() works perfect. But logging.info() is inheriting the
settings of logging.debug(). For example it is using logging.debug()'s
formatter while displaying. :-(
Ritesh
--
http://mail.python.org/mailman/listinfo/python-list
Re: python logging module problem
Vinay Sajip wrote: > > It's usual to rely on logger levels and to set handler levels for > additional refinement of what goes to a particular handler's > destination. > The problem is that for StreamHandler, logging module logs to sys.stderr. I want to use the logging feature for most of the messages my program displays. So some messages would be going to sys.stdout and some to sys.stderr. So, I'm ended up creating two handlers, one for sys.stdout and the other for sys.stderr. I'd then make INFO level messages go to sys.stdout and DEBUG level messages go to sys.stderr. What do you suggest ?? Is it good doing this way ? Ritesh -- http://mail.python.org/mailman/listinfo/python-list
Re: python logging module problem
Peter Otten wrote:
> You can achieve the desired behaviour by adding a custom Filter:
>
> import sys
> import logging
>
> logger = logging.getLogger("my_app")
> logger.setLevel(logging.DEBUG)
>
> class LevelFilter(logging.Filter):
> def __init__(self, level):
> self.level = level
> def filter(self, record):
> return self.level == record.levelno
>
> def make_handler(outstream, format, level):
> handler = logging.StreamHandler(outstream)
> formatter = logging.Formatter(format)
> handler.setFormatter(formatter)
> handler.addFilter(LevelFilter(level))
> return handler
>
> logger.addHandler(make_handler(sys.stderr,
> 'STDERR %(levelname)s %(message)s', logging.WARN))
> logger.addHandler(make_handler(sys.stdout,
> 'STDOUT %(levelname)s %(message)s', logging.INFO))
>
> logger.info("the world is flat")
> logger.warning("take care not to fall off its rim")
>
> Not sure whether this is a good idea. Another way might be to use distinct
> loggers.
>
> Peter
Thanks. This looks similar to what I wanted. I'll try customizing it to
my requirements and see if this helps.
Thanks,
Ritesh
--
http://mail.python.org/mailman/listinfo/python-list
class instance scope
Hi, I have a class defined in a file called foo.py In bar.py I've imported foo.py In bar.py's main function, I instantiate the class as follows: log = foo.log(x, y, z) Now in main I'm able to use log.view(), log.error() et cetera. But when I call the same method from some functions which are in bar.py, it fails giving me the following error: NameError: global name 'log' is not defined 1) I tried lookng into the docs but couldn't find anything on instance scope. 2) How is such situation tackled ? Will I have to instantiate in every function ? Ritesh -- http://mail.python.org/mailman/listinfo/python-list
Re: class instance scope
Steve Holden wrote: > Ritesh Raj Sarraf wrote: > > But when I call the same method from some functions which are in > > bar.py, it fails giving me the following error: > > > > NameError: global name 'log' is not defined > > > Well, that's preumbaly because your > >log = foo.log(x, y, z) > > statement was inside a function, and so the name "foo" was created in > that function's local namespace rather than in the module's global > namespace. > So if I do the instantiation before calling main(), will it work. Something like: if __name__ == "__main__": log = foo.log(x, y, z) main() In this case, will log be global ? But still I get the same error. > > 1) I tried lookng into the docs but couldn't find anything on instance > > scope. > > 2) How is such situation tackled ? Will I have to instantiate in every > > function ? > > > The best thing to do would be to pass the instance in as an argument to > the functions that need to manipulate it. > But then how do os, sys, and other modules which are imported, become accessible to all the functions ? I'm a newbie, so please bear with me. Ritesh -- http://mail.python.org/mailman/listinfo/python-list
Re: class instance scope
> > log = foo.log(x, y, z) > > Resulting line is: log = foo.log(x, y, z) global log Making the instance "log" global makes it accessible to all the functions. Now I have only one question, Is this a correct way to do it ? Or are there better way ? Ritesh -- http://mail.python.org/mailman/listinfo/python-list
Thread Question
Hi, I have some basic doubts about thread. I have a list which has items in it which need to be downloaded from the internet. Let's say list is: list_items[] which has 100 items in it. I have a function download_from_web() which does the work of downloading the items from the web. It does error handling in case it gets errors like 404 et cetera. Currently, this is how I'm using it. for item in list_items: download_from_web(item) This way, one items is downloaded at a time. I'm planning to implement threads in my application so that multiple items can be downloaded concurrently. I want the thread option to be user-defined. Looking at the documentation of threads (Core Python Programming), I've noticed that all threads are executed a once. Depending upon what they are doing, some finish early and some later. But I want to implement something like: for item in list_items: for num in thread_args: thread[num].start() thread[num].start() Is this the correct way of threading applications ? This is the first time I'd be doing threading. So was looking for comments and suggestions. Thanks, Ritesh -- http://mail.python.org/mailman/listinfo/python-list
Re: Thread Question
Duncan, I couldn't make out much from the code. Instead this is what I did. threads = [] nloops = range(len(lRawData)) for i in nloops: (sUrl, sFile, download_size, checksum) = stripper(lRawData[i]) t = threading.Thread(target=download_from_web, args=(sUrl, sFile, sSourceDir, None)) # = pypt_thread(download_from_web, i, stripper(lRawData[i])) threads.append(t) i = 0 join_i = 0 while i < nloops: counter = 0 while counter < 3: threads[i].start() counter += 1 i += 1 counter = 0 join_i = i - 3 while counter < 3: threads[join_i].join() counter += 1 join_i += 1 Is this correct ? Comments!! Ritesh Duncan Booth wrote: > Ritesh Raj Sarraf wrote: > > > I'm planning to implement threads in my application so that multiple > > items can be downloaded concurrently. I want the thread option to be > > user-defined. > > > > Looking at the documentation of threads (Core Python Programming), I've > > noticed that all threads are executed a once. Depending upon what they > > are doing, some finish early and some later. > > > > But I want to implement something like: > > > > for item in list_items: > > for num in thread_args: > >thread[num].start() > >thread[num].start() > > > > Is this the correct way of threading applications ? > > This is the first time I'd be doing threading. So was looking for > > comments and suggestions. > > > > What you want is to use a pool of threads so that you can configure how > many requests are issued at a time (you don't want to try to issue 100 > requests all in parallel). You can communicate with the threads through a > Queue. > > So if the code for a thread looks like: > >def run(request, response): >while 1: >item = request.get() >if item is None: > break >response.put(download_from_web(item)) > > # your main loop can be something like: > > requestQueue = Queue() > responseQueue = Queue() > thread_pool = [ > Thread(target=run, args=(requestQueue, responseQueue) > for i in range(numthreads)] > for t in thread_pool: t.start() > > for item in list_items: > requestQueue.put(item) > > for i in range(len(list_items)): > response = responseQueue.get() > handle_response(response) > > # and then to shut down the threads when you've finished: > for t in thread_pool: > requestQueue.put(None) > for t in thread_pool: > t.join() -- http://mail.python.org/mailman/listinfo/python-list
Re: Thread Question
Duncan Booth on Thursday 27 Jul 2006 17:17 wrote: > What you want is to use a pool of threads so that you can configure how > many requests are issued at a time (you don't want to try to issue 100 > requests all in parallel). You can communicate with the threads through a > Queue. > Thanks to both Duncan and Simon for your motivating replies. I had forgotten the truth that the lists are a medium to learn and educate. I hope you won't be very frustrated by my dummy questions. > So if the code for a thread looks like: > > def run(request, response): > while 1: > item = request.get() > if item is None: > break > response.put(download_from_web(item)) > This is where I'm most confused. I have a list, lRawData which has around, say, 100 items which are read from a file. Currently (without threads), I iterate upon each item in the list, split the item and then manipulate and pass it to download_from_web() to do the real work. I'm mainly confused in your run(). The line response.put(download_from_web(item)) is there. But if I use it, how am I going to iterate on my list lRawData. I'm not able to see where and how, run() or the line item = request.get() goes into my list lRawData to get the items. Can you please explain the functioning of the above code block. > # your main loop can be something like: > > requestQueue = Queue() > responseQueue = Queue() > thread_pool = [ > Thread(target=run, args=(requestQueue, responseQueue) > for i in range(numthreads)] > for t in thread_pool: t.start() This part seems understandable. > > for item in list_items: > requestQueue.put(item) > > for i in range(len(list_items)): > response = responseQueue.get() > handle_response(response) > I guess these would be clear once I'm clear on the first problem. > # and then to shut down the threads when you've finished: > for t in thread_pool: > requestQueue.put(None) > for t in thread_pool: > t.join() Thanks a lot for your replies. Ritesh -- Ritesh Raj Sarraf RESEARCHUT - http://www.researchut.com "Necessity is the mother of invention." "Stealing logic from one person is plagiarism, stealing from many is research." "The great are those who achieve the impossible, the petty are those who cannot - rrs" -- http://mail.python.org/mailman/listinfo/python-list
Re: Thread Question
And people, Is there any documentation on Python Threads or Threads in general. It'd be of great help to really understand. Ritesh Ritesh Raj Sarraf on Thursday 27 Jul 2006 16:37 wrote: > Is this the correct way of threading applications ? > This is the first time I'd be doing threading. So was looking for > comments and suggestions. -- Ritesh Raj Sarraf RESEARCHUT - http://www.researchut.com "Necessity is the mother of invention." "Stealing logic from one person is plagiarism, stealing from many is research." "The great are those who achieve the impossible, the petty are those who cannot - rrs" -- http://mail.python.org/mailman/listinfo/python-list
Re: Thread Question
Simon Forman on Thursday 27 Jul 2006 22:47 wrote: > def run(request, response, func=dummy_func): > ''' > Get items from the request Queue, process them > with func(), put the results along with the > Thread's name into the response Queue. > > Stop running once an item is None. > ''' > name = currentThread().getName() > while 1: > item = request.get() > if item is None: > break > response.put((name, func(item))) > Meanwhile, instead of sitting idle and waiting for a reply, I thought of trying to understand the code (the example by Simon). Good part is that I was able to use it. :-) Here's the changed code: from Queue import Queue from threading import Thread, currentThread # Imports for the dummy testing func. from time import sleep from random import random NUMTHREADS = 3 def run(request, response, func=download_from_web): ''' Get items from the request Queue, process them with func(), put the results along with the Thread's name into the response Queue. Stop running once an item is None. ''' name = currentThread().getName() while 1: item = request.get() (sUrl, sFile, download_size, checksum) = stripper(item) if item is None: break response.put((name, func(sUrl, sFile, sSourceDir, None))) # Create two Queues for the requests and responses requestQueue = Queue() responseQueue = Queue() # Pool of NUMTHREADS Threads that run run(). thread_pool = [ Thread( target=run, args=(requestQueue, responseQueue) ) for i in range(NUMTHREADS) ] # Start the threads. for t in thread_pool: t.start() # Queue up the requests. for item in lRawData: requestQueue.put(item) # Shut down the threads after all requests end. # (Put one None "sentinel" for each thread.) for t in thread_pool: requestQueue.put(None) # Don't end the program prematurely. # # (Note that because Queue.get() is blocking by # default this isn't strictly necessary. But if # you were, say, handling responses in another # thread, you'd want something like this in your # main thread.) for t in thread_pool: t.join() I'd like to put my understanding over here and would be happy if people can correct me at places. So here it goes: Firstly the code initializes the number of threads. Then it moves on to initializing requestQueue() and responseQueue(). Then it moves on to thread_pool, where it realizes that it has to execute the function run(). From NUMTHREADS in the for loop, it knows how many threads it is supposed to execute parallelly. So once the thread_pool is populated, it starts the threads. Actually, it doesn't start the threads. Instead, it puts the threads into the queue. Then the real iteration, about which I was talking in my earlier post, is done. The iteration happens in one go. And requestQueue.put(item) puts all the items from lRawData into the queue of the run(). But there, the run() already known its limitation on the number of threads. No, I think the above statement is wrong. The actual pool about the number of threads is stored by thread_pool. Once its pool (at a time 3 as per this example) is empty, it again requests for more threads using the requestQueue() And in function run(), when the item of lRawData is None, the thread stops. The the cleanup and checks of any remaining threads is done. Is this all correct ? I also do have a couple of questions more which would be related to locks. But I'd post them once I get done with this part. Thanks, Ritesh -- Ritesh Raj Sarraf RESEARCHUT - http://www.researchut.com "Necessity is the mother of invention." "Stealing logic from one person is plagiarism, stealing from many is research." "The great are those who achieve the impossible, the petty are those who cannot - rrs" -- http://mail.python.org/mailman/listinfo/python-list
Re: Thread Question
Hi, I have this following situation: #INFO: Thread Support # Will require more design thoughts from Queue import Queue from threading import Thread, currentThread NUMTHREADS = variables.options.num_of_threads def run(request, response, func=download_from_web): '''Get items from the request Queue, process them with func(), put the results along with the Thread's name into the response Queue. Stop running once an item is None.''' name = currentThread().getName() while 1: item = request.get() (sUrl, sFile, download_size, checksum) = stripper(item) if item is None: break response.put((name, func(sUrl, sFile, sSourceDir, None))) My download_from_web() returns True or False depending upon whether the download was successful or failed. How can I check that in the above code ? One other question I had, If my user passes the --zip option, download_from_web() internally (when the download is successful) zips the downloaded data to a zip file. Since in case of threading there'll be multiple threads, and say if one of the thread completes 2 seconds before others and is doing the zipping work: What will the other thread, at that moment do, if it completes while the previous thread is doing the zipping work ? Thanks, Ritesh Justin Azoff on Thursday 27 Jul 2006 22:33 wrote: > Ritesh Raj Sarraf wrote: > [snip] >> for item in list_items: >> download_from_web(item) >> >> This way, one items is downloaded at a time. >> >> I'm planning to implement threads in my application so that multiple >> items can be downloaded concurrently. I want the thread option to be >> user-defined. > [snip] > > See my post about the iterthreader module I wrote... > http://groups.google.com/group/comp.lang.python/browse_frm/thread/2ef29fae28cf44c1/ > > for url, result in Threader(download_from_web, list_items): > print url, result > #... > -- Ritesh Raj Sarraf RESEARCHUT - http://www.researchut.com "Necessity is the mother of invention." "Stealing logic from one person is plagiarism, stealing from many is research." "The great are those who achieve the impossible, the petty are those who cannot - rrs" -- http://mail.python.org/mailman/listinfo/python-list
Re: Thread Question
Simon Forman wrote: > > One other question I had, > > If my user passes the --zip option, download_from_web() internally (when the > > download is successful) zips the downloaded data to a zip file. Since in > > case > > of threading there'll be multiple threads, and say if one of the thread > > completes 2 seconds before others and is doing the zipping work: > > What will the other thread, at that moment do, if it completes while the > > previous thread is doing the zipping work ? > > The other threads will just take the next request from the Queue and > process it. They won't "care" what the one thread is doing, > downloading, zipping, whatever. > > The thread will be marked as complete only when the function that it executed exits. Right ? download_from_web() internally calls a funtion to zip the file. So it doesn't return before zipping. Till then this thread is not complete and therefore is busy working (i.e. zipping). during the same time if another thread (which is also calling download_from_web) completes the download, the download function will again call the zip code. At that particular situtation, will it wait for the previous thread to complete the zipping and release the file so that it can zip more data to it or will it just panic and quit ? I think this shouldn't be much concern. The d_f_w() calls zip(). And since zip opens the file.zip file in append mode, no matter how many threads access it, it should be okay. Ritesh -- http://mail.python.org/mailman/listinfo/python-list
Re: Thread Question
Simon Forman wrote: > > The other threads will just take the next request from the Queue and > process it. They won't "care" what the one thread is doing, > downloading, zipping, whatever. > As I mentioned in my previous post, the other threads will also have to go through the same "zip the file if the download was successful" method. I implemented it but am seeing some issues. If I use a single thread, all files are zipped to the archive. Obviously this has to work. If threads are 2 or 3 or 4 in numbers, some of the files don't show up in the archive. What would a thread do (which wants to add files to an archive) if it finds that another thread has opened the archive in append mode ? Will it wait, overwrite or just die ? Looking at the way my program is behaving, it looks like at that condition the threads are skipping/dying or something like that. Ritesh -- http://mail.python.org/mailman/listinfo/python-list
Re: Thread Question
Gerhard Fiedler wrote: > Rather than downloading and zipping in the same thread, you could run > multiple threads like you're doing that only download files, and one > zip-it-all-up thread. After downloading a file, the download threads place > a message in a queue that indicates the file they have downloaded, and the > zip-it-all-up thread takes messages out of that queue, one at a time, and > zips the files. > I was using this approach earlier. The problem with this approach is too much temporary disk usage. Say I'm downloading 2 GB of data which is a combination of, say 600 files. Now following this approach, I'll have to make sure that I have 4 GB of disk space available on my hard drive. Where as downloading in pieces, adding them to the archive, and then unlinking the downloaded file helps proper utilization of disk resource. Thanks, Ritesh -- http://mail.python.org/mailman/listinfo/python-list
Re: Thread Question
Carl Banks wrote: > If you have multiple threads trying to access the same ZIP file at the > same time, whether or not they use the same ZipFile object, you'll have > trouble. You'd have to change download_from_web to protect against > simultaneous use. A simple lock should suffice. Create the lock in > the main thread, like so: > > ziplock = threading.Lock() > Thanks. This looks to be the correct way to go. I do have access to all the source code as it is under GPL. > Then change the zipping part of download_from_web to acquire and > release this lock; do zipfile operations only between them. > > ziplock.acquire() > try: > do_all_zipfile_stuff_here() > finally: > ziplock.release() > I hope while one thread has acquired the lock, the other threads (which have done the downloading work and are ready to zip) would wait. > If you can't change download_from_web, you might have no choice but to > download sequentially. > > OTOH, if each thread uses a different ZIP file (and a different ZipFile > object), you wouldn't have to use a lock. It doesn't sound like you're > doing that, though. > > It shouldn't be a problem if one thread is zipping at the same time > another is downloading, unless there's some common data between them > for some reason. > > Thanks, Ritesh -- http://mail.python.org/mailman/listinfo/python-list
Re: Thread Question
Carl Banks wrote: > > Exactly. Only one thread can hold a lock at a time. If a thread tries > to acquire a lock that some other thread has, it'll wait until the > other thread releases it. You need locks to do this stuff because most > things (such as zipfile objects) don't wait for other threads to > finish. > I would heartly like to thank you for the suggestion you made. My program now works exactly as I wanted. Thanks. :-) Ritesh -- http://mail.python.org/mailman/listinfo/python-list
Re: Thread Question
Bryan Olson on Saturday 05 Aug 2006 13:31 wrote:
>> Exactly. Only one thread can hold a lock at a time.
>
> In the code above, a form called a "critical section", we might
> think of a thread as holding the lock when it is between the
> acquire() and release(). But that's not really how Python's
> locks work. A lock, even in the locked state, is not held by
> any particular thread.
>
>> If a thread tries
>> to acquire a lock that some other thread has, it'll wait until the
>> other thread releases it.
>
> More accurate: If a thread tries to acquire a lock that is in
> the locked state, it will wait until some thread releases it.
> (Unless it set the blocking flag false.) If more that one thread
> is waiting to acquire the lock, it may be blocked longer.
>
> I think the doc for threading.Lock is good:
>
> http://docs.python.org/lib/lock-objects.html
>
You're correct.
I noticed that even though while one thread acquires the lock, the other threads
don't respect the lock. In fact they just go ahead and execute the statements
within the lock acquire statement. With this behavior, I'm ending up having a
partially corrupted zip archive file.
def run(request, response, func=copy_first_match):
'''Get items from the request Queue, process them
with func(), put the results along with the
Thread's name into the response Queue.
Stop running once an item is None.'''
name = threading.currentThread().getName()
ziplock = threading.Lock()
while 1:
item = request.get()
if item is None:
break
(sUrl, sFile, download_size, checksum) = stripper(item)
response.put((name, sUrl, sFile, func(cache, sFile, sSourceDir,
checksum)))
# This will take care of making sure that if downloaded, they
are zipped
(thread_name, Url, File, exit_status) = responseQueue.get()
if exit_status == False:
log.verbose("%s not available in local cache %s\n" % (File,
cache))
if download_from_web(sUrl, sFile, sSourceDir, checksum) !=
True:
log.verbose("%s not downloaded from %s and NA in local
cache %s\n\n" % (sFile, sUrl, sRepository))
else:
# We need this because we can't do join or exists
operation on None
if cache is None or os.path.exists(os.path.join(cache,
sFile)):
#INFO: The file is already there.
pass
else:
shutil.copy(sFile, cache)
if zip_bool:
ziplock.acquire()
try:
compress_the_file(zip_type_file, sFile,
sSourceDir)
os.remove(sFile) # Remove it because we
don't need the file once it is zipped.
finally:
ziplock.release()
elif exit_status == True:
if zip_bool:
ziplock.acquire()
try:
compress_the_file(zip_type_file, sFile, sSourceDir)
os.unlink(sFile)
finally:
ziplock.release()
--
Ritesh Raj Sarraf
RESEARCHUT - http://www.researchut.com
"Necessity is the mother of invention."
"Stealing logic from one person is plagiarism, stealing from many is research."
"The great are those who achieve the impossible, the petty are those who
cannot - rrs"
--
http://mail.python.org/mailman/listinfo/python-list
Re: Thread Question
Bryan Olson on Saturday 05 Aug 2006 23:56 wrote: > You don't want "ziplock = threading.Lock()" in the body of > the function. It creates a new and different lock on every > execution. Your threads are all acquiring different locks. > To coordinate your threads, they need to be using the same > lock. > > Try moving "ziplock = threading.Lock()" out of the function, so > your code might read, in part: > > > ziplock = threading.Lock() > > def run(request, response, func=copy_first_match): > # And so on... Thanks. That did it. :-) Ritesh -- Ritesh Raj Sarraf RESEARCHUT - http://www.researchut.com "Necessity is the mother of invention." "Stealing logic from one person is plagiarism, stealing from many is research." "The great are those who achieve the impossible, the petty are those who cannot - rrs" -- http://mail.python.org/mailman/listinfo/python-list
thread lock question
Hi, I have a question regarding locks in threads. Following is the code example: if download_file(foo, bar) == True: if some condition: do_something() if zip_bool: ziplock.acquire() try: compress_the_file(zip_type_file, foo, bar) os.unlink(foo) # Unlink it once it has been zipped finally: ziplock.release() if zip_bool: ziplock.acquire() try: compress_the_file(zip_type_file, foo, bar) os.unlink(foo) finally: ziplock.release() Basically, I'm testing for some condition, if that is True, do something and then do the zipping. I'm also doing the zipping even if that's not true. My question is, in a code example like this which is threaded, does the locking mechanism work correctly ? Or are two different locks being acquired ? Thanks, Ritesh -- Ritesh Raj Sarraf RESEARCHUT - http://www.researchut.com "Necessity is the mother of invention." "Stealing logic from one person is plagiarism, stealing from many is research." "The great are those who achieve the impossible, the petty are those who cannot - rrs" -- http://mail.python.org/mailman/listinfo/python-list
Re: thread lock question
I'm sorry. My example code wasn't clear enough. Please see the following: exit_status = copy_first_match(foo, bar) if exit_status == False: if download_file(foo, bar) == True: if zip_bool: ziplock.acquire() try: compress_the_file(zip_type_file, foo, bar) os.unlink(foo) # Unlink it once it has been zipped finally: ziplock.release() else: if zip_bool: ziplock.acquire() try: compress_the_file(zip_type_file, foo, bar) os.unlink(foo) finally: ziplock.release() I think this code should be clear enough. The program, before trying to download from the web, checks into its cache repository to see if the file is available or not. If available, it copies it from the local cache repository and adds to the archive, else it downloads from the web and archives it. Now in this scenario, with threads, say a file of 100 mb got downloaded and then being zipped. It might take a couple of seconds for the zip to complete. During those couple of seconds, if another thread's file (a couple kb) gets downloaded/copied, will it wait for the lock to be released or will it create another lock ? Thanks, Ritesh Dennis Lee Bieber wrote: > On Wed, 11 Oct 2006 18:39:32 +0530, Ritesh Raj Sarraf > <[EMAIL PROTECTED]> declaimed the following in comp.lang.python: > > > > > if download_file(foo, bar) == True: > > if some condition: > > do_something() > > if zip_bool: > > > if zip_bool: > > > > Basically, I'm testing for some condition, if that is True, do something and > > then do the zipping. I'm also doing the zipping even if that's not true. > > > Actually, given the code you show, if "some condition" is true, you > are /attempting/ to zip the file twice; the second one likely fails due > to the first one deleting the file -- a condition masked by the finally: > clause. > > All you really need there is > > if download_file(foo, bar): > if some condition: > do_something() > if zip_bool: > > > > > My question is, in a code example like this which is threaded, does the > > locking > > mechanism work correctly ? > > Or are two different locks being acquired ? > > Is the lock object itself defined globally (shared by the threads)? > -- > WulfraedDennis Lee Bieber KD6MOG > [EMAIL PROTECTED] [EMAIL PROTECTED] > HTTP://wlfraed.home.netcom.com/ > (Bestiaria Support Staff: [EMAIL PROTECTED]) > HTTP://www.bestiaria.com/ -- http://mail.python.org/mailman/listinfo/python-list
determining file type
Hi, I have a funtion named unzipper() which does the work of unzipping the files. Is there a way I can identify what is the type of the file which'll be passed to unzipper(). If yes, I'll be able to make out if it's a zip file or a tar or a bz2 file. Thanks, Ritesh -- http://mail.python.org/mailman/listinfo/python-list
Re: determining file type
But isn't there any library function ?
Something like
XX.filetype(name)
Directory
File-Tar
File-Zip
File-MPEG
Ritesh
Maric Michaud wrote:
> Le Mercredi 14 Juin 2006 11:22, Ritesh Raj Sarraf a écrit :
> > Hi,
> >
> > I have a funtion named unzipper() which does the work of unzipping the
> > files.
> >
> > Is there a way I can identify what is the type of the file which'll be
> > passed to unzipper().
> > If yes, I'll be able to make out if it's a zip file or a tar or a bz2
> > file.
> >
> Under Linux you can use the file command in a pipe.
> In all cases, you use the "magic" repository of the file command directly like
> this (I found the '\xff\xd8' for jpeg in /usr/share/file/magic.mime on my
> debian)
>
> In [69]: f = file ('samurai_tux.jpg')
>
> In [70]: s = f.read(2)
>
> In [71]: s == '\xff\xd8'
> Out[71]: True
>
>
>
>
> > Thanks,
> > Ritesh
>
> --
> _
>
> Maric Michaud
> _
>
> Aristote - www.aristote.info
> 3 place des tapis
> 69004 Lyon
> Tel: +33 426 880 097
--
http://mail.python.org/mailman/listinfo/python-list
Re: determining file type
Also,
f = file ('some_file.jpg')
throws an error.
"str object is not callable"
Ritesh
Maric Michaud wrote:
> Le Mercredi 14 Juin 2006 11:22, Ritesh Raj Sarraf a écrit :
> > Hi,
> >
> > I have a funtion named unzipper() which does the work of unzipping the
> > files.
> >
> > Is there a way I can identify what is the type of the file which'll be
> > passed to unzipper().
> > If yes, I'll be able to make out if it's a zip file or a tar or a bz2
> > file.
> >
> Under Linux you can use the file command in a pipe.
> In all cases, you use the "magic" repository of the file command directly like
> this (I found the '\xff\xd8' for jpeg in /usr/share/file/magic.mime on my
> debian)
>
> In [69]: f = file ('samurai_tux.jpg')
>
> In [70]: s = f.read(2)
>
> In [71]: s == '\xff\xd8'
> Out[71]: True
>
>
>
>
> > Thanks,
> > Ritesh
>
> --
> _
>
> Maric Michaud
> _
>
> Aristote - www.aristote.info
> 3 place des tapis
> 69004 Lyon
> Tel: +33 426 880 097
--
http://mail.python.org/mailman/listinfo/python-list
zipfile module doesn't allow append
Hi,
I've got a problem here.
def compress_the_file(zip_file_name, files_to_compress, sSourceDir):
"""
Condenses all the files into one single file for easy transfer
"""
try:
import zipfile
except ImportError:
sys.stderr.write("Ai! module not found.\n")
try:
os.chdir(sSourceDir)
except:
#TODO: Handle this exception
pass
filename = zipfile.ZipFile(zip_file_name, "a")
#try:
#filename = zipfile.ZipFile(zip_file_name, "a")
#except:
##TODO Handle the exception
#sys.stderr.write("\nAieee! Some error exception in creating
zip file %s\n" % (zip_file_name))
#sys.exit(1)
filename.write(files_to_compress, files_to_compress,
zipfile.ZIP_DEFLATED)
filename.close()
The line
filename = zipfile.ZipFile(zip_file_name, "a")
throws an exception if the given filename is not present already.
Shouldn't it create a file (in case one is not there) since it is
"append" mode ??
Ritesh
--
http://mail.python.org/mailman/listinfo/python-list
optparse multiple arguments
Hi,
I'm having some minor problems with optparse. I'm just worried that
someone shouldn't say that multiple argument feature isn't implemented
in optpartse.
How to tackle multiple arguments to an option ?
As far as I dug, I've found that >1 arguments are being ignored.
parser.add_option("", "--my-option", dest="my_option", action="store",
type="string")
Now if someone uses it as:
./foo --my-option a b c
I want somehow to store all the three arguments but my_option stores
only "a" while ignoring b and c.
Any help?
Ritesh
--
http://mail.python.org/mailman/listinfo/python-list
Re: optparse multiple arguments
Ritesh Raj Sarraf wrote:
> Hi,
>
> I'm having some minor problems with optparse. I'm just worried that
> someone shouldn't say that multiple argument feature isn't implemented
> in optpartse.
>
> How to tackle multiple arguments to an option ?
> As far as I dug, I've found that >1 arguments are being ignored.
>
> parser.add_option("", "--my-option", dest="my_option", action="store",
> type="string")
>
> Now if someone uses it as:
> ./foo --my-option a b c
>
> I want somehow to store all the three arguments but my_option stores
> only "a" while ignoring b and c.
>
I just noticed that the args variable is holding values b and c.
the args variables comes from:
(options, args) = parser.parse_args()
I guess I only need to figure out now is why args isn't storing
argument "a" also...
Ritesh
--
http://mail.python.org/mailman/listinfo/python-list
FIXED: Re: optparse multiple arguments
Ritesh Raj Sarraf wrote:
> I just noticed that the args variable is holding values b and c.
> the args variables comes from:
> (options, args) = parser.parse_args()
>
> I guess I only need to figure out now is why args isn't storing
> argument "a" also...
>
> Ritesh
I fixed it, I guess.
parser.add_option("", "--my-option", dest="my_option",
action="store_true")
sets my_option to True and the arguments are all stored in the list
"args". :-)
Ritesh
--
http://mail.python.org/mailman/listinfo/python-list
Re: FIXED: Re: optparse multiple arguments
Simon Percivall wrote:
>
> It might do you good to read the documentation instead of blindly
> experimenting.
>
> Anyway,
>
> parser.add_option("", "--my-option", nargs=3)
>
> http://docs.python.org/lib/optparse-standard-option-actions.html
That won't help because by design of my program, I can't limit the
number of arguments a user can pass to it.
Ritesh
--
http://mail.python.org/mailman/listinfo/python-list
Re: FIXED: Re: optparse multiple arguments
Fredrik Lundh wrote: > Ritesh Raj Sarraf wrote: > > >> http://docs.python.org/lib/optparse-standard-option-actions.html > > > > That won't help because by design of my program, I can't limit the > > number of arguments a user can pass to it. > > do you want options, arguments, or are you just somewhat confused ? > > I'm not sure about that. My program will accept user input. The user can input multiple arguments. maybe 1 or maybe 100, that's not definite. I do need option because this particular (multiple) argument criteria is for one single option. Thanks, Ritesh -- http://mail.python.org/mailman/listinfo/python-list
compressed file ended before the logical end-of-stream was detected
Hi,
The program downloads the files from the internet and compresses them
to a single zip archive using compress_the_file().
Upon running syncer() which calls the decompress_the_file(), the first
iteration succeeds. But upon second iteration, I get an IOError
exception with the message:
"compressed file ended before the logical end-of-stream was detected"
Any ideas why this happens ? I don't think it has anything to do with
bad packets from the internet.
Here's the code.
There are two functions:
1) compress_the_file() - This function takes files as an argument to it
and put all of them into a zip archive
def compress_the_file(zip_file_name, files_to_compress, sSourceDir):
'''Condenses all the files into one single file for easy
transfer'''
try:
import zipfile
except ImportError:
sys.stderr.write("Ai! module not found.\n")
try:
os.chdir(sSourceDir)
except:
#TODO: Handle this exception
pass
try:
filename = zipfile.ZipFile(zip_file_name, "a")
except IOError:
#INFO By design zipfile throws an IOError exception when you
open
# in "append" mode and the file is not present.
filename = zipfile.ZipFile(zip_file_name, "w")
except:
#TODO Handle the exception
sys.stderr.write("\nAieee! Some error exception in creating zip
file %s\n" % (zip_file_name))
sys.exit(1)
filename.write(files_to_compress, files_to_compress,
zipfile.ZIP_DEFLATED)
filename.close()
My actual program iterates over a loop and sends files to this funtion
as it downloads from the internet.
2) decompress_the_file() - This function takes archive file as an
argument and decompresses them.
The above function decompress_the_file() is called by function syncer()
syncer(): This function tells decompress_the_file() what type file it
is and at which path to decompress.
def syncer(install_file_path, target_path, type=None):
'''Syncer does the work of syncing the downloaded files.
It syncs "install_file_path" which could be a valid file path
or a zip archive to "target_path'''
if type == 1:
try:
import zipfile
except ImportError:
sys.stderr.write("Ai! Module zipfile not found.\n")
sys.exit(1)
file = zipfile.ZipFile(install_file_path, "r")
for filename in file.namelist():
try:
import file_magic
except ImportError:
sys.stderr.write("Ai! Module file_magic not
found.\n")
sys.exit(1)
data = open(filename, "wb")
data.write(file.read(filename))
data.close()
#data = file.read(filename)
if file_magic.file(filename) == "application/x-bzip2":
decompress_the_file(os.path.abspath(filename),
target_path, filename, 1)
elif file_magic.file(filename) == "PGP armored data":
try:
shutil.copy(filename, target_path)
sys.stdout.write("%s file synced.\n" % (filename))
except shutil.Error:
sys.stderr.write("%s is already present.\n" %
(filename))
--
http://mail.python.org/mailman/listinfo/python-list
Threads and Progress Bar
Hi, I have a small application, written in Python, that uses threads. The application uses function foo() to download files from the web. As it reads data from the web server, it runs a progress bar by calling an install of a progress bar class. When using threads, I get the problem that the progress bar gets over-written by the download progress of files from other threads. I believe my change has to go into the progress bar class to make it thread aware. Are they any docs/suggestions on how to implement progress bars along with threads ? Thanks, Ritesh -- Ritesh Raj Sarraf RESEARCHUT - http://www.researchut.com "Necessity is the mother of invention." "Stealing logic from one person is plagiarism, stealing from many is research." "The great are those who achieve the impossible, the petty are those who cannot - rrs" -- http://mail.python.org/mailman/listinfo/python-list
Re: Threads and Progress Bar
single thread at the moment.\n")
log.warn("Threads is still in alpha stage. It's better to use
just a single thread at the moment.\n")
NUMTHREADS = variables.options.num_of_threads
name = threading.currentThread().getName()
ziplock = threading.Lock()
def run(request, response, func=download_from_web):
'''Get items from the request Queue, process them
with func(), put the results along with the
Thread's name into the response Queue.
Stop running once an item is None.'''
while 1:
item = request.get()
if item is None:
break
(sUrl, sFile, download_size, checksum) = stripper(item)
response.put((name, sUrl, sFile, func(sUrl, sFile,
sSourceDir, None)))
# This will take care of making sure that if downloaded,
they are zipped
(thread_name, Url, File, exit_status) = responseQueue.get()
if exit_status == True:
if zip_bool:
ziplock.acquire()
try:
compress_the_file(zip_type_file, File,
sSourceDir)
os.remove(File) # Remove it because we don't
need the file once it is zipped.
finally:
ziplock.release()
else:
variables.errlist.append(File)
pass
# Create two Queues for the requests and responses
requestQueue = Queue.Queue()
responseQueue = Queue.Queue()
# Pool of NUMTHREADS Threads that run run().
thread_pool = [
threading.Thread(
target=run,
args=(requestQueue, responseQueue)
)
for i in range(NUMTHREADS)
]
# Start the threads.
for t in thread_pool: t.start()
# Queue up the requests.
for item in lRawData: requestQueue.put(item)
# Shut down the threads after all requests end.
# (Put one None "sentinel" for each thread.)
for t in thread_pool: requestQueue.put(None)
# Don't end the program prematurely.
#
# (Note that because Queue.get() is blocking by
# defualt this isn't strictly necessary. But if
# you were, say, handling responses in another
# thread, you'd want something like this in your
# main thread.)
for t in thread_pool: t.join()
> Second... It sounds like you only created one progress bar, and each
> thread is referencing that single bar. I'd suspect you need to create a
> bar for EACH thread you create, and tell the thread which bar to update.
Yes, you're correct. That's what I'm also suspecting. I tried to do some minor
changes but couldn't succeed.
Request you to, if you reply with code, give a little explanation so that I can
understand and learn from it.
Thanks,
Ritesh
--
Ritesh Raj Sarraf
RESEARCHUT - http://www.researchut.com
"Necessity is the mother of invention."
"Stealing logic from one person is plagiarism, stealing from many is research."
"The great are those who achieve the impossible, the petty are those who
cannot - rrs"
--
http://mail.python.org/mailman/listinfo/python-list
screen output problem
Hi, I have, for very long, been trying to find a consistent solution (which could work across major python platforms - Linux, Windows, Mac OS X) for the following problem. I have a function which downloads files from the web. I've made the function threaded. I'm trying to implement a progress bar for it which could work across all the platforms. The problem is that when the progress bar is displayed, all the threads overwrite the progress bar. I've thought of using curses but AFAIK curses is not available for Windows. Is there a generic way to accomplish the following: progress = "[===]\n[===]" for x in range(5): sys.stdout.write(progress + "\r") Basically, I want a way through which I want to display data on two lines and make sure that "\r" overwrites the two lines without putting a newline making sure that whatever library it uses is available across all major Python platforms. Thanks, Ritesh -- http://mail.python.org/mailman/listinfo/python-list
Re: screen output problem
On Sunday 26 November 2006 13:07, Calvin Spealman wrote:
> Take a look at this:
> http://cheeseshop.python.org/pypi/progressbar
Hi,
Sorry for being a little late in replying.
The progressbar implementation is excellent but it has the same problems that
the current progressbar implementation I use, has.
In the above mentioned progressbar implementation also, the progressbar status
is overwritten if you're using multiple threads.
If your application is threaded, and multiple threads are downloading multiple
files, you get a progressbar which is overwritten by all the 3 threads.
I had brought this issue some time back too.
Dennis Lee Bieber had a solution which looked better but not very much. His
implementation had os.system(clear/cls) being called (to update the progress)
which made the screen flicker if your network latency was low (especially in
cases where the data is indirectly being downloaded from a Proxy or a product
like NetApp's Netcache).
If there's a way to print multiple lines of text withouth the newline ("\n")
character, we can then use carriage return ("\r") and implement a proper
progressbar with a bar for each separate progress action.
Thanks,
Ritesh
--
Ritesh Raj Sarraf
RESEARCHUT - http://www.researchut.com
"Necessity is the mother of invention."
"Stealing logic from one person is plagiarism, stealing from many is
research."
"The great are those who achieve the impossible, the petty are those who
cannot - rrs"
pgp4GnEOe0frx.pgp
Description: PGP signature
--
http://mail.python.org/mailman/listinfo/python-list
Re: screen output problem
I'm not sure if there is a definite solution to this problem.
I've noticed that one of the applications, which I use on a daily basis
(apt from Debian) does address the progress bar issue in another way.
When apt tries to download multiple files, it displays the progress of
all the downloads on a single line. Probably the apt developers also
might have run into the same issue and hence settled down with this
workaround.
Thanks,
Ritesh
Dennis Lee Bieber wrote:
> On Tue, 28 Nov 2006 15:15:28 +0530, Ritesh Raj Sarraf
> <[EMAIL PROTECTED]> declaimed the following in comp.lang.python:
>
>
> > If there's a way to print multiple lines of text withouth the newline ("\n")
> > character, we can then use carriage return ("\r") and implement a proper
> > progressbar with a bar for each separate progress action.
> >
> If you were running on an Amiga, or via a serial port connection to
> an old VT-100 terminal, it would be child's play... Since both
> understood the same terminal control codes for moving the cursor around
> a text display.
>
> The old MS-DOS "ANSI" console driver may have understood that set.
> But M$, in its great and all-powerful wisdom, seems to feel that the
> only need for a command line interface is for batch configuration
> scripts that run with no human interaction, and hence with no need for
> status. One is expected to code a GUI instead if such things as
> progress-bars are needed.
>
> A google for "windows console cursor control" brings up (a bit of a
> surprise since I my search terms didn't mention Python):
>
> http://newcenturycomputers.net/projects/wconio.html
>
> Of course, this isn't portable to LINUX or other operating systems...
>
> Also, looking at win32api, I find a SetCursorPos() (and matching
> get...).
>
> --
> WulfraedDennis Lee Bieber KD6MOG
> [EMAIL PROTECTED] [EMAIL PROTECTED]
> HTTP://wlfraed.home.netcom.com/
> (Bestiaria Support Staff: [EMAIL PROTECTED])
> HTTP://www.bestiaria.com/
--
http://mail.python.org/mailman/listinfo/python-list
breaking from loop
Hi,
Following is the code:
def walk_tree_copy(sRepository, sFile, sSourceDir, bFound = None):
try:
if sRepository is not None:
for name in os.listdir(sRepository):
path = os.path.join(sRepository, name)
if os.path.isdir(path):
walk_tree_copy(path, sFile, sSourceDir, bFound)
elif name.endswith('.foo') or name.endswith('.bar'):
if name == sFile:
try:
shutil.copy(path, sSourceDir)
except IOError, (errno, errstring):
errfunc(errno, errstring)
except shutil.Error:
print name + " is available in " +
sSourceDir + "Skipping Copy!"
bFound = True
break
return bFound
except OSError, (errno, strerror):
print errno, strerror
This function allows me to walk into a directory based tree and search
for files ending with .foo and .bar. My requirement is to copy
.foo/.bar.
Say in directory temp=>test=>test.bar is found. So shutil.copy will
copy it. I want that once the copy is done, it should make bFound =
True and get out.
But since test directory is under temp, work_tree_copy makes two calls
of the same function _but_ break only is able to get out from the inner
call.
Where am I wrong in this code ? Is there a better way to implement it ?
Regards,
rrs
--
http://mail.python.org/mailman/listinfo/python-list
Re: breaking from loop
Thanks to everyone. It is really the best place and the best people to
learn from.
Here's what I followed from the discussion:
def files(root):
for path, folders, files in os.walk(root):
for file in files:
yield path, file
def copy_first_match(repository, filename, dest_dir): # aka
walk_tree_copy()
for path, file in files(repository):
if file == filename:
try:
shutil.copy(os.path.join(path, file), dest_dir)
sys.stdout.write("%s copied from local cache %s." %
(file, repository))
except shutil.Error:
sys.stdout.write("%s is available in %s. Skipping
Copy!" % (file, dest_dir))
return True
return False
All I've added is the exception check because in case "file" is
available in "dest_dir", I want to display a message.
Since I'm still new and learning will be great if you let me know if
this is the proper way or not.
Thanks to everyone once again. It's been a very good experience
understanding everyone's comments.
Regards,
rrs
--
http://mail.python.org/mailman/listinfo/python-list
Namespace issue
Hi, I need a little help in understanding how Namespaces and scoping works with Classes/Functions in Python. Here's my code: class FetchData: def __init__(self, dataTypes=["foo", "bar", "spam"], archive=False): self.List = [] self.Types = dataTypes if archive: self.Archiver = Archiver(True) def FetchData(self, PackageName, Filename=None): try: import my_module except ImportError: return False if Filename != None: try: file_handle = open(Filename, 'a') except IOError: sys.exit(1) (amnt, header, self.List) = my_module.get_data(PackageName) This is the only way this code will work. As per my understanding, the bad part is that on every call of the method FetchData(), an import would be done. To not let that happen, I can put the import into __init__(). But when I put in there, I get a NameError saying that my_module is not available even though it got imported. All I noticed is that the import has to be part of the method else I end up getting a NameError. But always importing my_module is also not good. What is the correct way of doing this ? IMO, ideally it should be part of __init__() and be imported only when the class is instantiated. Thanks, Ritesh -- If possible, Please CC me when replying. I'm not subscribed to the list. -- http://mail.python.org/mailman/listinfo/python-list
Module imports during object instantiation
Hi, I've been very confused about why this doesn't work. I mean I don't see any reason why this has been made not to work. class Log: def __init__(self, verbose, lock = None): if verbose is True: self.VERBOSE = True else: self.VERBOSE = False if lock is None or lock != 1: self.DispLock = False else: self.DispLock = threading.Lock() self.lock = True if os.name == 'posix': self.platform = 'posix' self.color = get_colors() elif os.name in ['nt', 'dos']: self.platform = 'microsoft' try: import SomeModule except ImportError: self.Set_Flag = None if self.Set_Flag is not None: self.color = SomeModule.get_colors_windows() else: self.platform = None self.color = None When I create an object the "import" part never gets executed. Is there a reason behind it ? I mean I'd like to keep my class as independent as I want. So that when later I need to use it somewhere else, I don't need to know if it depends on any modules. Currently, the way I'm left is to globally go and import the module and set a flag there. Ritesh -- If possible, Please CC me when replying. I'm not subscribed to the list. -- http://mail.python.org/mailman/listinfo/python-list
Re: Module imports during object instantiation
On Aug 11, 3:17 am, James Stroud <[EMAIL PROTECTED]> wrote: > You do realize your import statement will only be called for nt and dos > systems don't you? > Yes. I would like to load a Windows Python Module (which is, say a specific implementation for Windows only) in such a condition where I find that the platform is Dos/NT. Loading it globally doesn't make sense because when from the same class an object is created on a non-windows platfom, it would be waste. It would either ImportError or else just load the module and never use it. As per my understanding __init__ is executed only once during object creation. For such situations (that I've explained above), I think __init__() is the correct place to put imports. Ritesh -- http://mail.python.org/mailman/listinfo/python-list
Re: Module imports during object instantiation
Bruno Desthuilliers wrote: > Ritesh Raj Sarraf a écrit : >> >> if lock is None or lock != 1: >> self.DispLock = False >> else: >> self.DispLock = threading.Lock() >> self.lock = True >> >> if os.name == 'posix': >>self.platform = 'posix' >>self.color = get_colors() >> >> elif os.name in ['nt', 'dos']: >> self.platform = 'microsoft' >> >> try: >> import SomeModule >> except ImportError: >> self.Set_Flag = None >> >> if self.Set_Flag is not None: >> self.color = SomeModule.get_colors_windows() >> >> else: >> self.platform = None >> self.color = None >> >> When I create an object the "import" part never gets executed. Is there a >> reason behind it ? > > what does "print os.name" yields ? On Windows: nt On Linux: posix > >> I mean I'd like to keep my class as independent as I want. So that when >> later I need to use it somewhere else, I don't need to know if it depends >> on any modules. > > > > Then pass the module to the initializer. Python's modules are objects... > Yes, that's an option. But is that the only one? To me it looks like an ugly way of doing. Ritesh -- If possible, Please CC me when replying. I'm not subscribed to the list. -- http://mail.python.org/mailman/listinfo/python-list
Re: Module imports during object instantiation
Bruno Desthuilliers wrote: > Ritesh Raj Sarraf a écrit : > > The initializer will be called *each time* you instanciate the class. > And nothing prevents client code from calling it explicitelly as many > times as it wants - ok, this would be rather strange, but this is still > technically possible. What I mean that you have no assurance about the > number of times an initializer will be called. > Yes, it should be called _each time_ I do an instantiation. But the point is, I'm doing it only once. And I don't see people instantiating multiple times. And even if you do, it should do separate imports. log1 = Log() log2 = Log() Both the objects are separate from each other in every manner and they share nothing. > > wrt/ your problem, remember that top-level code is executed when the > module is loaded (either as a main program or as an imported module). > The canonical solution to os-specific imports is to handle them at the > top-level: > > if os.name == 'posix: >from some_posix_module import get_colors > elif os.name in ['nt', 'dos']: >from some_nt_module import get_windows_color as get_colors > else: >get_colors = lambda: None # or any other sensible default value... > > class SomeClass(object): >def __init__(self, *args, **kw): > self.colors = get_colors() This is what I'm left with to do currently. But I doubt if that makes by classes independent and to "Just Work". If someone was to cut/paste just the class, it won't work. He'll have to go through the imports and figure out which one relates to the class he want's to import and add similar code to his. Ritesh -- If possible, Please CC me when replying. I'm not subscribed to the list. -- http://mail.python.org/mailman/listinfo/python-list
Re: Module imports during object instantiation
Steve Holden wrote: > Ritesh Raj Sarraf wrote: >> class Log: >> >> def __init__(self, verbose, lock = None): >> >> if verbose is True: >> self.VERBOSE = True >> else: self.VERBOSE = False >> > Better: > > self.VERBOSE = verbose > > or, if you suspect verbose might pass in a mutable value, > > self.VERBOSE bool(verbose) > Thanks for pointing this out. > What's leading you to conclude the import isn't being executed? You > realise, I trust, that the module's code will only be executed on the > first call to __init__()? > Well. Putting it in a "try" inside __init__() doesn't do anything. The import never happens. And thus if I make a call in any of the methods, it fails with an error message. A NameError IIRC. > You are right in assuming that __init__() is called once per instance > created, and it's legitimate to make an import conditional in the way > you have because of the "execute code only once" behavior - if the > module is already in sys.modules then it won't be re-imported, the > existing one will be used. > This is what even my understanding is. But am afraid to say that this understanding is not implemented in Python. Have you tried doing this? Does it work for you? > Having said all that, I still don't see why you can't just put the > try/except at the top level of your code and have color be a global. Why > repeat the attempted import and the computation for each object you > create? Alternatively, do it at the class level, so it's only executed > once when the class is declared? > Hmmm!! This is where I might not have done my homework. I just tried to import the same class (Log) the following way and it worked. from module import Log It worked. Now I have some questions. Going with your point of try/except imports at the top level, I am having multiple imports at the top level for the multiple classes that I have in the module. Not everything from the top level imports is required for the class Log (Say, just one module is what is required). So when I do a `from module import Log`, do all the modules at the top level get imported? My understanding says Yes, and if that's true, that's bad IMO. (Sorry! I'd really not thought about this when reading about Python module imports) My ultimate goal is to make all my classes as independent as possible while keeping them as lightweight as possible. If all top level module import statements are getting executed during a `from module import class`, this is _not_ lightweight IMO. That's what led me to try imports in __init__() but unfortunately I haven't seen that working for me even once. Ritesh -- If possible, Please CC me when replying. I'm not subscribed to the list. -- http://mail.python.org/mailman/listinfo/python-list
Re: Module imports during object instantiation
Bruno Desthuilliers wrote: >>> What's leading you to conclude the import isn't being executed? You >>> realise, I trust, that the module's code will only be executed on the >>> first call to __init__()? >>> >> >> Well. Putting it in a "try" inside __init__() doesn't do anything. > > This would be highly suprising. > Yes, it is surprising. Unfortunately. :-( Or maybe I'm doing something really dumb. >> The >> import never happens. > > As soon as your running this code on a platform where os.name yields > either 'nt' or 'dos', the import statement is executed. You can bet your > ass on this. Now this import can fail for a lot of reasons (imported > module not in sys.path, error in the imported module, etc), and since > your catching *and dismissing* the ImportError, you don't know what > effectively happens. It's like someone send you mails, you set your > filter to trash mails from this person, and then complain this person > never send you mails !-) > >> And thus if I make a call in any of the methods, it >> fails with an error message. A NameError IIRC. > > "IIRC" ? Inspecting the traceback may help, you know. There's no traceback because the import never happens. Here's what I've captured for you exactly I see on my terminal [EMAIL PROTECTED]:/tmp$ cat rick.py class Log: def __init__(self, verbose, lock = None): self.VERBOSE = bool(verbose) self.lock = bool(lock) if self.lock: self.dispLock = threading.Lock() else: self.dispLock = None if os.name == 'posix': try: import foobar except ImportError, e: print >> sys.stderr, e self.platform = 'posix' self.color = get_colors() elif os.name in ['nt', 'dos']: self.platform = 'microsoft' try: import SomeModule except ImportError, e: # comment out next line before going to prod... print >> sys.stderr, e self.color = None else: self.color = SomeModule.get_colors_windows() else: self.platform = None self.color = None [EMAIL PROTECTED]:/tmp$ python Python 2.4.4 (#2, Jul 21 2007, 11:00:24) [GCC 4.1.3 20070718 (prerelease) (Debian 4.1.2-14)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import os, sys >>> os.path.abspath(os.curdir) '/tmp' >>> from rick import Log >>> import rick >>> os.name 'posix' >>> import foobar Traceback (most recent call last): File "", line 1, in ? ImportError: No module named foobar >>> For the 'posix' OS, I'm doing an erratic import which should fail and print the error message. See, it prints nothing. This _is_ what has made me conclude that imports aren't executed in __init__(). But hey, they will get executed if you put them in any of the methods (which is again illogical because methods can/are called multiple times). Ritesh -- If possible, Please CC me when replying. I'm not subscribed to the list. -- http://mail.python.org/mailman/listinfo/python-list
Re: Module imports during object instantiation
Neil Cerutti wrote: > If you want an import inside an __init__ to run, you must call > the __init__ function that contains it. Doesn't __init__ get called automatically ? I thought __init__ was required to be called explicitly only when you were doing inheritance and wanted to pass separate values to the base class. Ritesh -- If possible, Please CC me when replying. I'm not subscribed to the list. -- http://mail.python.org/mailman/listinfo/python-list
Re: Module imports during object instantiation
Neil Cerutti wrote: >> >> Doesn't __init__ get called automatically ? > > It gets called automatically when you construct an instance of > the class in which it's defined. I am a little confused by your statements now. In my earlier posts in the same thread, I gave some code example which was something like this: class Log: def __init__(): if os.name == 'posix': try: import foobar except ImportError, e: print >> sys.stderr, e (Sorry for the bad indentation) Now, what do you mean here by "construct an instance of the class in which it is defined" ? __init__() _is_ defined in the Log class and I try to create an instance of the class Log as: log = Log() Assuming it is run on a box which is POSIX compliant, the try/import should fail and I should see an error message because there is no module named foobar. But I don't see that. And this is what exactly I wanted to get clarified in this whole thread. (But still haven't got a clear answer to a Yes/No). Or am I terribly missing something that you are trying to tell ? Ritesh -- If possible, Please CC me when replying. I'm not subscribed to the list. -- http://mail.python.org/mailman/listinfo/python-list
Re: Module imports during object instantiation
On Aug 15, 11:42 pm, Neil Cerutti <[EMAIL PROTECTED]> wrote: > On 2007-08-15, Ritesh Raj Sarraf <[EMAIL PROTECTED]> wrote: > > Or am I terribly missing something that you are trying to tell ? > > I didn't see log = Log() in your example. Sorry for the > excursion. > > Are you sure os.name is 'posix' on your system? > Here again for you: In [6]: cat rick.py IPython system call: cat rick.py import os, sys class Log: def __init__(self, verbose, lock = None): self.VERBOSE = bool(verbose) self.lock = bool(lock) if self.lock: self.dispLock = threading.Lock() else: self.dispLock = None if os.name == 'posix': try: import foobar except ImportError, e: print >> sys.stderr, e self.platform = 'posix' self.color = get_colors() elif os.name in ['nt', 'dos']: self.platform = 'microsoft' try: import SomeModule except ImportError, e: # comment out next line before going to prod... print >> sys.stderr, e self.color = None else: self.color = SomeModule.get_colors_windows() else: self.platform = None self.color = None In [7]: from rick import Log In [8]: log = Log(verbose = True) No module named foobar --- exceptions.NameError Traceback (most recent call last) /tmp/ /tmp/rick.py in __init__(self, verbose, lock) 18 19 self.platform = 'posix' ---> 20 self.color = get_colors() 21 22 elif os.name in ['nt', 'dos']: NameError: global name 'get_colors' is not defined In [9]: os.name Out[9]: 'posix' In [10]: import foobar --- exceptions.ImportError Traceback (most recent call last) /tmp/ ImportError: No module named foobar In [11]: Okay!! Ignore line 20 for now because that is not what this whole thread has been about. In the code, just 2 lines above "self.color = get_colors()" you see the try/except import statements. Since there is no "foobar" named module, an ImportError should have been thrown and a relevant error message should have been printed. With these example I'm just trying to prove that imports put into a class' __init__() never get executed. They get executed if you put them in any of the methods. (And I don't know why it is done this way). Ritesh -- http://mail.python.org/mailman/listinfo/python-list
Re: Module imports during object instantiation
On Aug 16, 12:16 am, Ritesh Raj Sarraf <[EMAIL PROTECTED]> wrote: > On Aug 15, 11:42 pm, Neil Cerutti <[EMAIL PROTECTED]> wrote: > > > On 2007-08-15, Ritesh Raj Sarraf <[EMAIL PROTECTED]> wrote: > > > Or am I terribly missing something that you are trying to tell ? > > > I didn't see log = Log() in your example. Sorry for the > > excursion. > > > Are you sure os.name is 'posix' on your system? > > Here again for you: > > In [6]: cat rick.py > IPython system call: cat rick.py > import os, sys > > class Log: > def __init__(self, verbose, lock = None): > self.VERBOSE = bool(verbose) > self.lock = bool(lock) > > if self.lock: > self.dispLock = threading.Lock() > else: > self.dispLock = None > > if os.name == 'posix': > try: > import foobar > except ImportError, e: > print >> sys.stderr, e > > self.platform = 'posix' > self.color = get_colors() > > elif os.name in ['nt', 'dos']: > self.platform = 'microsoft' > try: > import SomeModule > except ImportError, e: > # comment out next line before going to prod... > print >> sys.stderr, e > > self.color = None > else: > self.color = SomeModule.get_colors_windows() > else: > self.platform = None > self.color = None > > In [7]: from rick import Log > > In [8]: log = Log(verbose = True) > No module named foobar > --- > exceptions.NameError Traceback (most > recent call last) > > /tmp/ > > /tmp/rick.py in __init__(self, verbose, lock) > 18 > 19 self.platform = 'posix' > ---> 20 self.color = get_colors() > 21 > 22 elif os.name in ['nt', 'dos']: > > NameError: global name 'get_colors' is not defined > Oops!!! Looks like I completely missed this. It _did_ print the error message. Apologies to all for not keeping a close eye on the error message. -- http://mail.python.org/mailman/listinfo/python-list
