[Tutor] General question rgrd. usage of libraries
Hi there,
I just recently learned how to build a basic web scraper with Python
3.5 (I am learning Python for data analytics purposes). Being new to
coding, I have a question:
How do I know which libraries I need to perform a certain task?
For example, in case of this web scraper (which I built with help of a
tutorial on YouTube) I need to have urrlib and Beautiful Soup
import urllib
import urllib.request
from bs4 import BeautifulSoup
theurl = "https://twitter.com/rafaelknuth";
thepage = urllib.request.urlopen(theurl)
soup = BeautifulSoup(thepage, "html.parser")
print(soup.title.text)
i = 1
for tweets in soup.findAll("div",{"class":"content"}):
print(i)
print(tweets.find("p").text)
i = i + 1
Is there a way I can figure out which libraries I need when drafting my code?
Can you share your experiences? Right now, if I wanted for example to
populate a Google Sheet with my scraped web content - how would I know
which libraries I would need to actually make this happen? I am trying
wondering if there is a process to figure out what I exactly need
library-wise.
Thank you,
Rafael
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] General question rgrd. usage of libraries
On 05/05/17 14:45, Rafael Knuth wrote: > How do I know which libraries I need to perform a certain task? Mostly youn learn by experience, but otherwise google (or other search engine) is your friend. It doesn't do any harm to read the Library Reference document on the python.org web site too, it gives a good overview of the most common ones. But for stuff outside the standard library its pretty much down to google and asking on fora like the tutor list or a more topic related one. > populate a Google Sheet with my scraped web content One thing that will definitely help when asking on forums is to identify the correct (or at least the common) terminology. For example I've no idea what a "Google Sheet" is - that may be the correct term - but but I recognize the term I might be able to offer help. If I don't recognize the term I'll assume I don't know the answer. So some research before asking definitely helps. And don't forget PyPI which has libraries categorised to make browsing/searching easier. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] General question rgrd. usage of libraries
On 05/05/2017 08:45 AM, Rafael Knuth wrote:
Hi there,
I just recently learned how to build a basic web scraper with Python
3.5 (I am learning Python for data analytics purposes). Being new to
coding, I have a question:
How do I know which libraries I need to perform a certain task?
For example, in case of this web scraper (which I built with help of a
tutorial on YouTube) I need to have urrlib and Beautiful Soup
import urllib
import urllib.request
from bs4 import BeautifulSoup
theurl = "https://twitter.com/rafaelknuth";
thepage = urllib.request.urlopen(theurl)
soup = BeautifulSoup(thepage, "html.parser")
print(soup.title.text)
i = 1
for tweets in soup.findAll("div",{"class":"content"}):
print(i)
print(tweets.find("p").text)
i = i + 1
Is there a way I can figure out which libraries I need when drafting my code?
Can you share your experiences? Right now, if I wanted for example to
populate a Google Sheet with my scraped web content - how would I know
which libraries I would need to actually make this happen? I am trying
wondering if there is a process to figure out what I exactly need
library-wise.
There is a Python API to google sheets but when I had a look, it seemed
fairly complex. I haven't tried it yet but depending on what you need to
do this library may be what you need:
https://pypi.python.org/pypi/gspread.
Regards, Jim
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
