Re: [Tutor] weather scraping with Beautiful Soup

2009-07-18 Thread Michiel Overtoom
Sander Sweers wrote: 2009/7/18 Che M : table = soup.find("table","dataTable tm10") #find the table tbody = table.find("tbody") #find the table's body You can do this in one step. tbody = soup.find('tbody') Yeah, but what if the document contains multiple tables, and you w

Re: [Tutor] weather scraping with Beautiful Soup

2009-07-18 Thread Sander Sweers
2009/7/18 Che M : > table = soup.find("table","dataTable tm10")  #find the table > tbody = table.find("tbody")  #find the table's body You can do this in one step. tbody = soup.find('tbody') Greets Sander ___ Tutor maillist - Tutor@py

Re: [Tutor] weather scraping with Beautiful Soup

2009-07-17 Thread Che M
> Date: Sat, 18 Jul 2009 01:09:32 +0200 > From: sander.swe...@gmail.com > To: tutor@python.org > Subject: Re: [Tutor] weather scraping with Beautiful Soup > > 2009/7/17 Che M : > > table = soup.find("td",id="dataTable tm10") > > Almos

Re: [Tutor] weather scraping with Beautiful Soup

2009-07-17 Thread Sander Sweers
2009/7/17 Che M : > table = soup.find("td",id="dataTable tm10") Almost right. attrs should normall be a dict so {'class':'dataTable tm10'} but you can use a shortcut, read on. > > > When I look at the page source for that page, there is this section, which > contains the

Re: [Tutor] weather scraping with Beautiful Soup

2009-07-17 Thread Che M
OK, got the very basic case to work when using Beautiful Soup 3.0.7a and scraping the Weather Underground Lite page. That gives me the current temperature, etc. Good. But now I would like the average temperature from, say, yesterday. I am again having trouble finding elements in the soup.

Re: [Tutor] weather scraping with Beautiful Soup

2009-07-17 Thread Che M
> 2009/7/17 Michiel Overtoom : > > This is actually the first time I see that BeautifulSoup is NOT able to > > parse a webpage... > > Depends on which version is used. If 3.1 then it is much worse with > malformed html than prior releases. See [1] for more info. > > Greets > Sander > > [1] htt

Re: [Tutor] weather scraping with Beautiful Soup

2009-07-17 Thread Che M
> Date: Fri, 17 Jul 2009 20:02:22 +0200 > From: mot...@xs4all.nl > To: tutor@python.org > CC: pine...@hotmail.com > Subject: Re: [Tutor] weather scraping with Beautiful Soup > > Che M wrote: > > > "http://www.wund.com/cgi-bin/findweather/getForecast?query=Wo

Re: [Tutor] weather scraping with Beautiful Soup

2009-07-17 Thread Sander Sweers
2009/7/17 Michiel Overtoom : > This is actually the first time I see that BeautifulSoup is NOT able to > parse a webpage... Depends on which version is used. If 3.1 then it is much worse with malformed html than prior releases. See [1] for more info. Greets Sander [1] http://www.crummy.com/softw

Re: [Tutor] weather scraping with Beautiful Soup

2009-07-17 Thread Michiel Overtoom
Che M wrote: "http://www.wund.com/cgi-bin/findweather/getForecast?query=Worthington%2C+OH"; > Any help is appreciated. That would be: daytemp = soup.find("div",id="main").findNext("span").renderContents() -- "The ability of the OSS process to collect and harness the collective IQ of tho

Re: [Tutor] weather scraping with Beautiful Soup

2009-07-17 Thread Che M
> Date: Fri, 17 Jul 2009 08:09:10 -0400 > Subject: Re: [Tutor] weather scraping with Beautiful Soup > From: ken...@tds.net > To: pine...@hotmail.com > CC: tutor@python.org > > On Thu, Jul 16, 2009 at 11:21 PM, Che M wrote: > > Hi, > > > > I am interested

Re: [Tutor] weather scraping with Beautiful Soup

2009-07-17 Thread Michiel Overtoom
Che M wrote: Thanks, but that isn't working for me. That's because BeautifulSoup isn't able to parse that webpage, not because the statement I posted doesn't work. I had BeautifulSoup parse the HTML fragment you posted earlier instead of the live webpage. This is actually the first time I se

Re: [Tutor] weather scraping with Beautiful Soup

2009-07-17 Thread Che M
> Date: Fri, 17 Jul 2009 12:27:36 +0200 > From: mot...@xs4all.nl > To: tutor@python.org > CC: pine...@hotmail.com > Subject: Re: [Tutor] weather scraping with Beautiful Soup > > Che M wrote: > > > The 60.3 is the value I want to extract. > > soup.find(&

Re: [Tutor] weather scraping with Beautiful Soup

2009-07-17 Thread Stefan Behnel
Che M wrote: > > > West of Town, Jamestown, Pennsylvania > (PWS) > Updated: pwsid="KPAJAMES1" pwsunit="english" pwsvariable="lu" value="1247814018">3:00 > AM EDT on July 17, 2009 > > > > >

Re: [Tutor] weather scraping with Beautiful Soup

2009-07-17 Thread Kent Johnson
On Thu, Jul 16, 2009 at 11:21 PM, Che M wrote: > Hi, > > I am interested in gathering simple weather data using Beautiful Soup, but > am having trouble understanding what I'm doing.  I have searched the > archives and so far haven't found enough to get me moving forward. > > Basically I am trying t

Re: [Tutor] weather scraping with Beautiful Soup

2009-07-17 Thread Michiel Overtoom
Che M wrote: The 60.3 is the value I want to extract. soup.find("div",id="curcondbox").findNext("span","b").renderContents() -- "The ability of the OSS process to collect and harness the collective IQ of thousands of individuals across the Internet is simply amazing." - Vinod Valloppillil ht

Re: [Tutor] weather scraping with Beautiful Soup

2009-07-17 Thread Che M
> The posts basocally say go and look at the HTML and find the > right tags for the data you need. This is fubndamental to any kind of web > scraping, you need to understand the HTML tree well enough to identify > where yourt data exists. > > How familiar are you with HTML and its structures? R

Re: [Tutor] weather scraping with Beautiful Soup

2009-07-16 Thread Alan Gauld
"Che M" wrote Grabbing Weather Underground Data with BeautifulSoup http://flowingdata.com/2007/07/09/grabbing-weather-underground-data-with-beautifulsoup/ But I get to the exact same problem that this other person got to in this post: http://groups.google.com/group/beautifulsoup/browse_threa

[Tutor] weather scraping with Beautiful Soup

2009-07-16 Thread Che M
Hi, I am interested in gathering simple weather data using Beautiful Soup, but am having trouble understanding what I'm doing. I have searched the archives and so far haven't found enough to get me moving forward. Basically I am trying to start off this example: Grabbing Weather Underground