[Tutor] Python3: looping through web address

[Umed Saidov]

Hello Tutor(s),

I want to create a loop that loops through a web address, changing only 
a part of that address with each loop. I can't seem to be able to get 
the code to do this. Below is the code with some explanation of what I 
am trying to do. The code seems to loop through the ticker symbol file 
ignoring the web address. Is there another way of doing the same thing?


import urllib.request, json
import csv
import pandas as pd
import itertools

ticker = []
ticker = pd.DataFrame(ticker)

#open a cvs file with 100+ stock tickers from S&P500. Save in a 
dataframe 'ticker'.

ticker =pd.read_csv('tickers.csv')

#Loop through ticker, changing the designated part of the web address. 
Download the data into 'response'


for t in ticker.itertuples():
response += 
urllib.request.urlopen('https:websiteaddress/{ct}/financials'.format(ct=ticker))


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Python3: looping through web address

[Umed Saidov]

On 07/18/2016 06:35 AM, Alan Gauld via Tutor wrote:
Caveat: I'm not a Pandas expert but... This looks odd. You first 
assign an empty list to ticker and pass that into pandas.DataFrame 
then assign the result back to ticker. Is that what you intended? Why 
not just ticker = pd.DataFrame([]) Or is there something clever going on? 
No. nothing clever I am afraid. I wanted to create a variable to store 
data in pandas format. This seemed like a good way of doing it... but 
perhaps not.

#open a cvs file with 100+ stock tickers from S&P500. Save in a
dataframe 'ticker'.
ticker =pd.read_csv('tickers.csv')

And now you overwrite the dataframe with the content of a csv file?
I'm not sure what read_csv returns but it seems to me you are
creating and throwing away a lot of stuff here?

the csv is just a column of tickers. I wanted to store them in pandas 
format. this returns a pandas dataframe with contents of the csv file 
stored in two columns (index and ticker values)

for t in ticker.itertuples():
  response +=
urllib.request.urlopen('https:websiteaddress/{ct}/financials'.format(ct=ticker))

ticker is the iterable, it does not change during the loop.
Yet in the url it is ticker you assign to the ct marker.
Don't you want 't'? That's the thing that changes.
I want the content of the column one in ticker. With each iteration 
of t, which I thought was just a simple numeric progression representing 
a row number, I wanted to get the content of the column one.

It might help to create and print the address before calling urlopen:

for t in ticker.itertuples():
 address = 'https:websiteaddress/{ct}/financials'.format(ct=ticker)
 print address   # remove when done
 response += urllib.request.urlopen(address)



Thanks for the suggestion. I tried this and received the badstatusline 
error message:


Traceback (most recent call last):
  File "something.py", line 23, in 
response += urllib.request.urlopen(address)
  File "...request.py", line 162, in urlopen
return opener.open(url, data, timeout)
  File "..request.py", line 465, in open
response = self._open(req, data)
  File "..request.py", line 483, in _open
'_open', req)
  File "python3.5/urllib/request.py", line 443, in _call_chain
result = func(*args)
  File "python3.5/urllib/request.py", line 1283, in https_open
context=self._context, check_hostname=self._check_hostname)
  File "/python3.5/urllib/request.py", line 1243, in do_open
r = h.getresponse()
  File "python3.5/http/client.py", line 1174, in getresponse
response.begin()
  File "python3.5/http/client.py", line 282, in begin
version, status, reason = self._read_status()
  File "python3.5/http/client.py", line 264, in _read_status
raise BadStatusLine(line)
http.client.BadStatusLine: 

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor