[Tutor] python module to search a website

[vineeth]

Hello all,

I am looking forward for a python module to search a website and extract 
the url.


For example I found a module for Amazon with the name "amazonproduct", 
the api does the job of extracting the data based on the query it even 
parses the url data. I am looking some more similar query search python 
module for other websites like Amazon.


Any help is appreciated.

Thank You
Vin
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] python module to search a website

[vineeth]

Hi Bill,

Thanks for the reply, I know how the urllib module works I am not 
looking for scraping. I am looking to obtain the html page that my query 
is going to return. Just like when you type in a site like Amazon you 
get a bunch of product listing the module has to search the website and 
return the html link. I can ofcourse scrap the information from that link.


Thanks
Vin

On 02/27/2011 12:04 AM, Bill Allen wrote:
n Sat, Feb 26, 2011 at 21:11, vineeth <mailto:vineethrak...@gmail.com>> wrote:


Hello all,

I am looking forward for a python module to search a website and
extract the url.

For example I found a module for Amazon with the name
"amazonproduct", the api does the job of extracting the data based
on the query it even parses the url data. I am looking some more
similar query search python module for other websites like Amazon.

Any help is appreciated.

Thank You
Vin

I am not sure what url you are trying to extract, or from where, but I 
can give you an example of basic web scraping if that is your aim.


The following works for Python 2.x.

#This one module that gives you the needed methods to read the html 
from a webpage

import urllib

#set a variable to the needed website
mypath = "http://some_website.com";

#read all the html data from the page into a variable and then parse 
through it looking for urls

mylines = urllib.urlopen(mypath).readlines()
for item in mylines:
if "http://"; in item:
 ...do something with the url that was found in the page html...
 ...etc...


--Bill
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

[Tutor] help with re module and parsing data

[vineeth]

Hello all I am doing some analysis on my trace file. I am finding the 
lines Recvd-Content and Published-Content. I am able to find those lines 
but the re module as predicted just gives the word that is being 
searched. But I require the entire  line similar to a grep in unix. Can 
some one tell me how to do this. I am doing the following way.


import re
file = open('file.txt','r')
file2 = open('newfile.txt','w')

LineFile = ' '

for line in file:
LineFile += line

StripRcvdCnt = re.compile('(P\w+\S\Content|Re\w+\S\Content)')

FindRcvdCnt = re.findall(StripRcvdCnt, LineFile)

for SrcStr in FindRcvdCnt:
file2.write(SrcStr)


Thanks
Vin




___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

[Tutor] Help return a pattern from list

[Vineeth Rakesh]

Hello all,

Can some one help me to return a special pattern from a list.

say list =
["something1.mp3","something2.mp3","something4.pdf","something5.odt"]

now say I just need to return the files with .mp3 extension. How to go about
doing this?

Thanks
Vin
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

[Tutor] Help execution time calculation:

[Vineeth Rakesh]

Hello all,

I want to calculate the execution time of a program. Say I have a function
like below:

def RepAdd(i):
  j = 0
  while(j___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

[Tutor] position of an element in list:

[Vineeth Rakesh]

Hello all,

How to return the position of a character in a string. Say I have str1 =
"welcome to the world" if i want to return the position of the first
occurrence of "o" how to do it?

Thanks
Vin
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

[Tutor] passing by reference

[Vineeth Mohan]

Hi,

How are variables in python passed. By value or by referrence?  For  
example if we consider a simple function below wherein the value of a 
does not get modified in the main function.


def addition(a,b):
a = a+1
return a+b

if __name__ == '__main__':

a = 10
b = 15
addition(a,b)

Is the following the only way to pass by reference? or is there any 
other way


def addition(x):
x.a = x.a+1
return x.a+x.b

class variables(object):
a = 10
b = 15

if __name__ == '__main__':

obj = variables()
addition(obj)
print obj.a


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor