Re: [Tutor] Code to download credit card statement and add to DB

2018-04-20 Thread Kuan Lu
Hello Mr. Tutor:

I constructed a pieced of code utilizing requests and pyodbc to download 
creditcard statement transactions and insert them into a DB table based on the 
code published by Louis Millette here: 
https://github.com/louismillette/Banking/tree/master.

Now I am a total newbie to Python even though I had some experience writing in 
other languages. The code works like a charm until where it comes to inserting 
the transactions into the table. As you can see in my code, in order to avoid 
duplicates being inserted into the table I’m trying to compared the downloaded 
statement line to each existing line in the table through nested loop and 
insert only when there’s no match. I kind of intuitively expect that there is 
another much simpler way to compare each downloaded line to all the inserted 
lines and find out if it is already in there. Obviously the approach I’m using 
doesn’t really work as I want it to. Can you please give me some hints as to 
how to get the loops to work as I explained or, if there is another better way 
to do it, what would that look like?

Thanks a lot for your guidance.

Kaun



   KUAN LU, CPA, CGA  |  FINANCE TEAM LEAD


  Email: k...@talize.com |
  Tel.:416-757-7008,212 | Fax.:416-757-9656
67 Alexdon Rd. Unit 1, North York, M3J 2B5 
[cid:image8b62b4.PNG@56bf8aa6.478ff844] <http://www.talize.com>

   [cid:image60c6be.PNG@4e534b25.42997eac] 
<http://www.linkedin.com/company/talize/>   
[cid:imagecf6195.PNG@85b08044.4f853faf] 
<http://www.instagram.com/talizethrift/>   
[cid:image5aee5c.PNG@df0af238.4cb43c7f] <http://www.facebook.com/TalizeThrift>  
 [cid:image53396f.PNG@356ad215.46ac24a4] 
<https://plus.google.com/+TalizeThriftStoreTalizeThriftStore>   
[cid:imagede763d.PNG@0b16b53d.47948094] <https://twitter.com/TalizeThrift>

The information transmitted, including attachments, is intended only for the 
person(s) or entity to which it is addressed and may contain confidential 
and/or privileged material. Any review, retransmission, dissemination or other 
use of, or taking of any action in reliance upon this information by persons or 
entities other than the intended recipient is prohibited. If you received this 
in error please contact the sender and destroy any copies of this information.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor


Re: [Tutor] Code to download credit card statement and add to DB

2018-04-20 Thread Kuan Lu
dex.html";,
"Content-Type": "application/vnd.api+json",
"Client-Type": "default_web",
"brand": "cibc",
"X-Auth-Token": X_Auth_Token,
"X-Requested-With": "XMLHttpRequest",
"Connection": "keep-alive",
},
cookies=cookies
)

#after logging in as yourself, go ahead and pull your default account id from 
the response
login_request_response = login_request.json()

#The Visa Account ID
defaultAccountId = "XXX"

#the dateFrom and dateUntil arguments are python datetimes representing from 
when until when you want to pull credit and debit entries
account_requests = requests.get(

url="https://www.cibconline.cibc.com/ebm-ai/api/v1/json/transactions?accountId={}&filterBy=range&fromDate={}&lastFilterBy=range&limit=150&lowerLimitAmount=&offset=0&sortAsc=true&sortByField=date&toDate={}&transactionLocation=&transactionType=&upperLimitAmount=".format(
defaultAccountId,
dateFrom.strftime("%Y-%m-%d"),
dateUntil.strftime("%Y-%m-%d")
),
headers={
"Host": "www.cibconline.cibc.com",
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:53.0) 
Gecko/20100101 Firefox/53.0",
"Accept": "application/vnd.api+json",
"Accept-Language": "en",
"Accept-Encoding": "gzip, deflate, br",

"Referer":"https://www.cibconline.cibc.com/ebm-resources/public/banking/cibc/client/web/index.html";,
"Content-Type": "application/vnd.api+json",
"Client-Type": "default_web",
"brand": "cibc",
"X-Auth-Token": X_Auth_Token,
"X-Requested-With": "XMLHttpRequest",
"Connection": "keep-alive",
},
cookies=cookies
   )
transactions = account_requests.json()['transactions']

#for T in transactions:
#ListOfRows=[r for r in T]
#print(ListOfRows[0],ListOfRows[1],ListOfRows[2],ListOfRows[3])

#The function to load the statement dataset
def Generate_List():

#There are a couple of ways to do this one, I set it up as an iterator.  This 
is just to format the output as a nice dictionary.
for transaction in transactions:
transaction_type = 'Debit' if transaction['debit'] else 'Credit'
date_datetime = 
datetime.datetime.strptime(transaction['date'].split('T')[0],"%Y-%m-%d")
amount = transaction['debit'] if transaction_type == 'Debit' else 
-transaction['credit']
yield {
'transaction': transaction_type,  # 'Debit' or 'Credit'
'TransDate': date_datetime,
'TransDescription': transaction['transactionDescription'],
'CreditCardNumber': transaction['creditCardNumber'],
'Amount': amount,
'balance': transaction['runningBalance']
}

#call the function to populate the generator
Statement=Generate_List()

#read the credit card list from the DB
Card_Dict = dict(cursor.execute("SELECT * FROM dbo.CardNameLookup"))

#read the transactions already loaded from the latest date existing in the 
statement table
Loaded_Trans = cursor.execute("SELECT * FROM Visa.CreditCardStatement WHERE 
TransDate=(SELECT MAX(TransDate) FROM Visa.CreditCardStatement)")
#gather the column name info for the dictionary
columns = [column[0] for column in Loaded_Trans.description]
#create the empty dictionary to hold the loaded transactions
Card_Trans = []
#iterate through the cursor and load the dictionary with transactions
for line in Loaded_Trans.fetchall():
Card_Trans.append(dict(zip(columns,line)))

Trans_Count = 0
#loop through the generator and insert the new visa statement lines
for Statement_Line in Statement:
for trans in Card_Trans:
Trans_Count += 1

print(Statement_Line['TransDate'],Statement_Line['TransDescription'],Card_Dict.get(Statement_Line['CreditCardNumber']),\
Statement_Line['Amount'],Statement_Line['balance'])
if trans['TransDate'] == Statement_Line['TransDate'].date() and 
trans['TransDescription'] == Statement_Line['TransDescription'] and \
trans['CreditCardID'] == 
Card_Dict.get(Statement_Line['CreditCardNumber']) and float(trans.Amount) == 
float(Statement_Line['Amount']):
pass
elif Trans_Count==le