[Tutor] How to write tests for main() function that does not return anything
Hi guys, I'm trying to test the code in the main() but I'm a bit unsure how to
go about it since I'm not passing any arguments or even returning anything
other then logging. For the purposes of the example I've shortened the tree
statements in the function..
Can anyone point me in the right direction on how to test the logic below? Or
perhaps give me an example on how to test the code below (that would be really
appreciated).
I've posted this question on Stackoverflow but the guys over there told me to
mock it but no one provided an example (I'm fairly new on mocking)..and how
could I mock the if statements or even the for loop?
script.py
from . import settings
def main():
if settings.PATHS: # list containing full paths to a directory of files
paths = settings.PATHS
for path in paths:
data = read_file(path)
modified_data = do_something_with_the_data_collected(data)
write_to_new_file(modified_data)
else:
logger.warning("There are no files in
{}".format(settings.FILES_DIRECTORY))
if __name__ == '__main__':
main()
tests/file_tests.py
import unittest
from module.script import main
class FileManagerTests(unittest.TestCase):
def test_main_func(self):
main() # ?? this is where I am stuck, should I just test
# that it logs correctly if certain data exists
# in settings file?
if __name__ == '__main__':
unittest.main()
RegardsPedro+44(0)7549646235pedro.mig...@live.co.uk
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Help with error
First, Please try to be more creative with your message subject line.
Help, isn't -- helpfull
Second. Don't paraphrase the error, copy it from your actual terminal
screen and paste it here.
See below
On Tue, Feb 2, 2016 at 10:37 AM, Chelsea G wrote:
> Hi,
> When I run my code I am getting an error it says "Attribute Error:
> Dictionary instance has no attribute 'search'. So the whole idea for my
> code is to input a csv file and output results that we want. This specific
> piece of code def search is to search the csv file for any keyword like
> 'issues' and output those results into a text file. I just need some help
> with figuring out how to fix the error I am receiving.
> This is my functions:
>
> import csvimport jsonimport sysfrom collections import defaultdictfrom
> collections import Counter
> data = defaultdict(list)
>
> upper_limit = 5
> lower_limit = 4
> class dictionary():
> def __init__(self):
> self.dict = defaultdict(list)
> self.counted_dict = defaultdict(list)
> self.grouped_dict = defaultdict(list)
> self.other_dict = defaultdict(list)
> self.final_dict = defaultdict(list)
> self.total_dict = defaultdict(list)
> self.search_dict = defaultdict(list)
>
> def populate_dict(self, filename):
> with open(filename, 'rb') as f:
> reader = csv.reader(f)
> next(reader, None)
> for row in reader:
> self.dict[row[2]].append(row[3])
>
> def all_counts(self):
> data_count = Counter()
> for key in self.dict.keys():
> self.counted_dict.update({key: Counter(self.dict[key])})
>
> def total_counts(self):
> for key in self.dict.keys():
> total = 0
> b = Counter(self.dict[key])
> for value in b:
> total += b[value]
> new_list = str(total)
> #self.total_dict.update({key: 'Total count for this
> application: ' + str(total)})
> #self.total_dict.update({key: str(total)})
> self.total_dict[key].append(new_list)
>
> def grouped_counts(self):
> for key in self.dict.keys():
> total = 0
> c = Counter(self.dict[key])
> for value in c:
> if c[value] >= upper_limit:
> new_list = value, str(c[value])
> self.grouped_dict[key].append(new_list)
> elif c[value] <= lower_limit:
> total += c[value]
> self.other_dict.update({key: 'other: ' + str(total)})
>
> for d in (self.grouped_dict, self.other_dict, self.total_dict):
> for key, value in d.iteritems():
> self.final_dict[key].append(value)
>
> def json_output(self):
> with open('test.txt', 'w') as text_file:
> json.dump(self.final_dict, text_file, sort_keys = True, indent
> = 4)
>
> Your search method below is indented so as to be a function within
json_output. Remove the indentation so that it lines up with your other
class methods.
> def search(self, filename):
> with open(filename, 'r') as searchfile,
> open('weekly_test.txt', 'w')
> as search_results_file:
> for line in searchfile:
> if 'PBI 43125' in line:
> print >>search_results_file, line
>
>
>
> And then I have another python file where I import my functions and
> run the results. Here is the code for that:
>
> import functions2
>
> week_choice = raw_input("Previous Week or Current Week?")if
> week_choice == "current week":
> data = functions2.dictionary()
> filename = raw_input("What file do you want to use?")
> data.populate_dict(filename)
> data.total_counts()
> data.grouped_counts()
> data.json_output()
> data.search(filename)
> elif week_choice == "previous week":
> previous_data = functions2.dictionary()
> filename = raw_input("What file do you want to use?")
> previous_data.populate_dict(filename)
> previous_data.total_counts()
> previous_data.grouped_counts()
> previous_data.json_output()
> previous_data.search(filename)
> else:
> print "That choice is not correct"
>
>
> It says the error is with data.search(filename).. Not sure how to get
> this working.
> ___
> Tutor maillist - Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
--
Joel Goldstick
http://joelgoldstick.com/stats/birthdays
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] 2016-02-01 Filter STRINGS in Log File and Pass as VARAIBLE within PYTHON script
hello Cameron,
thank you for the positive input. this is my new code.
NEW CODE
$ more fail2ban-banned-ipAddress.py
#VERSION CONTROL:
#2016-01-31 - Initial build by Kuenn Leow
#- fail2ban package has to be installed
#- fail2ban leverage on linux iptables to work
#2016-02-02 - modified with recommendation from Cameron Simpson
#
#FIXED MODULE IMPORT and FIXED ARGV IMPORT
import sys
import os
import subprocess
import time
import traceback
myArray = sys.argv
def checkInputs():
if('-date' not in myArray):
#print(__doc__)
print('''
USAGE:python fail2ban-banned-ipAddress.py -date
EXAMPLE: python fail2ban-banned-ipAddress.py -date 2016-01-31
''')
sys.exit(1)
def main():
#START MAIN PROGRAM HERE!!!
try:
checkInputs()
myDate = myArray[myArray.index('-date') + 1]
timestamp01 = time.strftime("%Y-%m-%d")
timestamp02 = time.strftime("%Y-%m-%d-%H%M%S")
wd01 = ("/var/tmp/myKNN/1_mySAMPLEpython-ver-001/" +
timestamp01)
wd02 = ("/var/tmp/myKNN/1_mySAMPLEpython-ver-001/" +
timestamp02)
#print(" ")
#print(40 * "-")
#print("START DEBUG Log of MAIN Defined VARIABLE")
#print(40 * "-")
#print("myDate: " + myDate)
#print(" ")
#print("timestamp01: " + timestamp01)
#print("timestamp01: " + timestamp02)
#print(" ")
#print("wd01: " + wd01)
#print("wd02: " + wd02)
#print(38 * "-")
#print("END DEBUG Log of MAIN Defined VARIABLE")
#print(38 * "-")
#print(" ")
print(" ")
with open("/var/log/fail2ban.log") as fail_log:
for line in fail_log:
if("ssh" in line and "Ban" in line and
myDate in line):
words = line.split()
banIP = words[6]
print("banIP:" , banIP)
whoisFile = os.popen("whois -H
" + banIP + " |egrep -i \"name|country|mail\" |sort -u").read()
print("whoisFile:", whoisFile)
except KeyboardInterrupt:
print('Shutdown requested...exiting')
except Exception:
traceback.print_exc(file=sys.stdout)
sys.exit(0)
#END MAIN PROGRAM HERE!!!
#START RUN main program/functions HERE!!!
if __name__ == "__main__":
main()
#END RUN main program/functions HERE!!!
TEST RESULT:
---
$ python ./fail2ban-banned-ipAddress.py -date 2016-01-31
banIP: 183.3.202.109
whoisFile: abuse-mailbox: anti-s...@ns.chinanet.cn.net
abuse-mailbox: antispam_gd...@189.cn
country:CN
e-mail: anti-s...@ns.chinanet.cn.net
e-mail: gdnoc_h...@189.cn
netname:CHINANET-GD
banIP: 183.3.202.109
whoisFile: abuse-mailbox: anti-s...@ns.chinanet.cn.net
abuse-mailbox: antispam_gd...@189.cn
country:CN
e-mail: anti-s...@ns.chinanet.cn.net
e-mail: gdnoc_h...@189.cn
netname:CHINANET-GD
banIP: 27.75.97.233
whoisFile: abuse-mailbox: hm-chan...@vnnic.net.vn
country:VN
e-mail: hm-chan...@vnnic.net.vn
e-mail: tie...@viettel.com.vn
e-mail: truon...@viettel.com.vn
netname:Newass2011xDSLHN-NET
remarks:For spamming matters, mail to tie...@viettel.com.vn
banIP: 183.3.202.109
whoisFile: abuse-mailbox: anti-s...@ns.chinanet.cn.net
abuse-mailbox: antispam_gd...@189.cn
country:CN
e-mail: anti-s...@ns.chinanet.cn.net
e-mail: gdnoc_h...@189.cn
netname:CHINANET-GD
Cameron Simpson wrote:
On 01Feb2016 15:53, knnleow GOOGLE wrote:
trying out on how to port my unix shell script to python.
get more complicated than i expected.: (
i am not familiar with the modules available in python.
anyone care to share how to better the clumsy approach below.
regards,
kuenn
timestamp02 = time.strftime("%Y-%m-%d-%H%M%S")
banIPaddressesFile = os.popen("cat
/var/log/fail2ban.log| egrep ssh| egrep Ban| egrep " + myDate + "|
awk \'{print $7}\'| sort -n| uniq >/tmp/banIPaddressesFile." +
timestamp02).read()
First up, this is still essentially a shell script. You're
constructing a shell pipeline like this (paraphrased):
cat >/var/log/fail2ban.log
| egrep ssh
| egrep Ban
| egrep myDate
| awk '{print $7}'
| sort -n
| uniq >/tmp/banIPaddressesFile-timestamp
So really, you're doing almost nothing in Python. You're also writing
intermediate results to a temporary filename, then reading from it.
Unless you really need to keep that file around, you won't need that
either.
Before I g
[Tutor] Help with error
Hi,
When I run my code I am getting an error it says "Attribute Error:
Dictionary instance has no attribute 'search'. So the whole idea for my
code is to input a csv file and output results that we want. This specific
piece of code def search is to search the csv file for any keyword like
'issues' and output those results into a text file. I just need some help
with figuring out how to fix the error I am receiving.
This is my functions:
import csvimport jsonimport sysfrom collections import defaultdictfrom
collections import Counter
data = defaultdict(list)
upper_limit = 5
lower_limit = 4
class dictionary():
def __init__(self):
self.dict = defaultdict(list)
self.counted_dict = defaultdict(list)
self.grouped_dict = defaultdict(list)
self.other_dict = defaultdict(list)
self.final_dict = defaultdict(list)
self.total_dict = defaultdict(list)
self.search_dict = defaultdict(list)
def populate_dict(self, filename):
with open(filename, 'rb') as f:
reader = csv.reader(f)
next(reader, None)
for row in reader:
self.dict[row[2]].append(row[3])
def all_counts(self):
data_count = Counter()
for key in self.dict.keys():
self.counted_dict.update({key: Counter(self.dict[key])})
def total_counts(self):
for key in self.dict.keys():
total = 0
b = Counter(self.dict[key])
for value in b:
total += b[value]
new_list = str(total)
#self.total_dict.update({key: 'Total count for this
application: ' + str(total)})
#self.total_dict.update({key: str(total)})
self.total_dict[key].append(new_list)
def grouped_counts(self):
for key in self.dict.keys():
total = 0
c = Counter(self.dict[key])
for value in c:
if c[value] >= upper_limit:
new_list = value, str(c[value])
self.grouped_dict[key].append(new_list)
elif c[value] <= lower_limit:
total += c[value]
self.other_dict.update({key: 'other: ' + str(total)})
for d in (self.grouped_dict, self.other_dict, self.total_dict):
for key, value in d.iteritems():
self.final_dict[key].append(value)
def json_output(self):
with open('test.txt', 'w') as text_file:
json.dump(self.final_dict, text_file, sort_keys = True, indent = 4)
def search(self, filename):
with open(filename, 'r') as searchfile, open('weekly_test.txt',
'w')
as search_results_file:
for line in searchfile:
if 'PBI 43125' in line:
print >>search_results_file, line
And then I have another python file where I import my functions and
run the results. Here is the code for that:
import functions2
week_choice = raw_input("Previous Week or Current Week?")if
week_choice == "current week":
data = functions2.dictionary()
filename = raw_input("What file do you want to use?")
data.populate_dict(filename)
data.total_counts()
data.grouped_counts()
data.json_output()
data.search(filename)
elif week_choice == "previous week":
previous_data = functions2.dictionary()
filename = raw_input("What file do you want to use?")
previous_data.populate_dict(filename)
previous_data.total_counts()
previous_data.grouped_counts()
previous_data.json_output()
previous_data.search(filename)
else:
print "That choice is not correct"
It says the error is with data.search(filename).. Not sure how to get
this working.
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] 2016-02-01 Filter STRINGS in Log File and Pass as VARAIBLE within PYTHON script
Sorry, forget to make use of SET() ... this is the new update.
appreciate your advice if we can still optimized further...
$ more fail2ban-banned-ipAddress.py
#VERSION CONTROL:
#2016-01-31 - Initial build by Kuenn Leow
# - fail2ban package has to be installed
# - fail2ban leverage on linux iptables to work
#2016-0-02- modified with recommandation from Carmeron Simpson
#FIXED MODULE IMPORT and FIXED ARGV IMPORT
import sys
import os
import subprocess
import time
import traceback
myArray = sys.argv
def checkInputs():
if('-date' not in myArray):
#print(__doc__)
print('''
USAGE:python fail2ban-banned-ipAddress.py -date
EXAMPLE: python fail2ban-banned-ipAddress.py -date 2016-01-31
''')
sys.exit(1)
def main():
#START MAIN PROGRAM HERE!!!
try:
checkInputs()
myDate = myArray[myArray.index('-date') + 1]
timestamp01 = time.strftime("%Y-%m-%d")
timestamp02 = time.strftime("%Y-%m-%d-%H%M%S")
wd01 = ("/var/tmp/myKNN/1_mySAMPLEpython-ver-001/" +
timestamp01)
wd02 = ("/var/tmp/myKNN/1_mySAMPLEpython-ver-001/" +
timestamp02)
#print(" ")
#print(40 * "-")
#print("START DEBUG Log of MAIN Defined VARIABLE")
#print(40 * "-")
#print("myDate: " + myDate)
#print(" ")
#print("timestamp01: " + timestamp01)
#print("timestamp02: " + timestamp02)
#print(" ")
#print("wd01: " + wd01)
#print("wd02: " + wd02)
#print(38 * "-")
#print("END DEBUG Log of MAIN Defined VARIABLE")
#print(38 * "-")
#print(" ")
# store all the BANNED IP in a SET
print(" ")
banIP_addrs = set()
with open("/var/log/fail2ban.log") as fail_log:
for line in fail_log:
if("ssh" in line and "Ban" in line and
myDate in line):
words = line.split()
word6 = words[6]
print("word6:" , word6)
banIP_addrs.add(word6)
print("banIP_addrs:" , banIP_addrs)
# LOOP through the SET and WHOIS
for i in banIP_addrs:
print("i:", i)
whoisVAR = os.popen("whois -H " + i + " |egrep
-i \"name|country|mail\" |sort -u").read()
print("whoisVAR:", whoisVAR)
except KeyboardInterrupt:
print('Shutdown requested...exiting')
except Exception:
traceback.print_exc(file=sys.stdout)
sys.exit(0)
#END MAIN PROGRAM HERE!!!
#START RUN main program/functions HERE!!!
if __name__ == "__main__":
main()
#END RUN main program/functions HERE!!!
TEST RESULT
---
$ python ./fail2ban-banned-ipAddress.py -date 2016-01-31
word6: 183.3.202.109
word6: 183.3.202.109
word6: 27.75.97.233
word6: 183.3.202.109
word6: 222.187.222.220
word6: 183.3.202.109
word6: 77.73.91.28
word6: 59.47.0.149
word6: 183.3.202.109
word6: 77.73.91.28
word6: 77.73.91.28
word6: 185.130.5.184
word6: 183.3.202.109
word6: 221.203.142.71
banIP_addrs: {'183.3.202.109', '59.47.0.149', '222.187.222.220',
'77.73.91.28', '27.75.97.233', '221.203.142.71', '185.130.5.184'}
i: 183.3.202.109
whoisVAR: abuse-mailbox: anti-s...@ns.chinanet.cn.net
abuse-mailbox: antispam_gd...@189.cn
country:CN
e-mail: anti-s...@ns.chinanet.cn.net
e-mail: gdnoc_h...@189.cn
netname:CHINANET-GD
i: 59.47.0.149
whoisVAR: abuse-mailbox: anti-s...@ns.chinanet.cn.net
country:CN
e-mail: anti-s...@ns.chinanet.cn.net
e-mail: lnab...@lntele.com
netname:CHINANET-LN
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Help with error
On 02/02/16 15:37, Chelsea G wrote:
> 'issues' and output those results into a text file. I just need some help
> with figuring out how to fix the error I am receiving.
Joel has already pointed out the indentation error in your class.
However there is another problem... but possibly not with your code.
> And then I have another python file where I import my functions and
> run the results. Here is the code for that:
>
> import functions2
>
> week_choice = raw_input("Previous Week or Current Week?")if
> week_choice == "current week":
> data = functions2.dictionary()
> filename = raw_input("What file do you want to use?")
> data.populate_dict(filename)
> data.total_counts()
> data.grouped_counts()
> data.json_output()
> data.search(filename)
> elif week_choice == "previous week":
> previous_data = functions2.dictionary()
> filename = raw_input("What file do you want to use?")
> previous_data.populate_dict(filename)
> previous_data.total_counts()
> previous_data.grouped_counts()
> previous_data.json_output()
> previous_data.search(filename)
> else:
> print "That choice is not correct"
The if, elif and else indentation is way off
Now that is probably due to emails being messed up,
but it makes it hard to give sensible feedback since
there could be bugs but we can't see them.sensibly...
Please be sure to send the complete error message
and not just summarize it and also use plain text,
especially when posting code.
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
[Tutor] Fwd: RE: Finding the max value from a dictionary that does not exceed a variable's value.
-- Forwarded message --
From: "Danny Yoo"
Date: Feb 2, 2016 1:22 PM
Subject: RE: [Tutor] Finding the max value from a dictionary that does not
exceed a variable's value.
To: "Lawrence Lorenzo"
Cc:
On Feb 2, 2016 10:34 AM, "Lawrence Lorenzo" wrote:
>
>
> I can do it with a list, however I need to be able to print the change
amount in coin names. Thanks for your reply sorry I didn't respond earlier
I didn't see it until just now.
> Just to expand on what I'm trying with this is to print an arbitrary
value as 1p, 2p, 5p, 10p, 20p, 50p, £1 in the least amount required so if
the change needed is £2.37 then the program prints that much as ("£1, £1,
20p, 10p, 5p, 2p")
Consider separating the process of computing the answer from presentation
of that answer.
That is: just because you have to print or the coin names at the end
doesn't mean you have to *compute* the change in terms of those string
names.
You can compute in terms of the numerical coin values first, and as a
separate step, translate the answer back in terms of coin names when
presenting the answer.
Think of a problem where someone asks you to add two Roman numerals
together. It would be madness to try to do that without first changing the
representation into something more computer-friendly. Same line of
reasoning. Choose the data representation to make the problem easy to
solve. As long as we have a way to go back from that representation to
something that's human readable, were good to go.
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] 2016-02-01 Filter STRINGS in Log File and Pass as VARAIBLE within PYTHON script
On 02Feb2016 21:14, knnleow GOOGLE wrote:
Sorry, forget to make use of SET() ... this is the new update.
appreciate your advice if we can still optimized further...
A few remarks, interleaved below.
myArray = sys.argv
I would not make this a global. Instead, pass sys.argv to main at the bottom
where you call main:
main(sys.argv)
and set up main as:
def main(argv):
def checkInputs():
if('-date' not in myArray):
#print(__doc__)
print('''
Also pass argv to checkInputs, so:
def checkInputs(argv):
if '-date' not in argv:
BTW, this is not C or Perl, you don't need brackets around your "if"
conditions.
Also, have a read of PEP 8:
https://www.python.org/dev/peps/pep-0008/
Although it is the style guide for the standard library, most of its
recommendations are followed by most Python code. I have in mind the section on
function names; in Python it is common to name functions and variables using
lowercase_with_underscores, so one would normally call "checkInputs" then name
"check_inputs".
USAGE:python fail2ban-banned-ipAddress.py -date
EXAMPLE: python fail2ban-banned-ipAddress.py -date 2016-01-31
''')
sys.exit(1)
def main():
As remarked, go for:
def main(argv):
#START MAIN PROGRAM HERE!!!
try:
checkInputs()
and then:
check_inputs(argv)
myDate = myArray[myArray.index('-date') + 1]
It is better to properly examine the argument list.
For example (for something this simple), I often go:
cmd = argv.pop(0)
badopts = False
if not argv:
print("%s: missing options" % (cmd,), file=sys.stderr)
badopts = True
else:
option = argv.pop(0)
if option != '-date':
print("%s: unrecognised option: %s" % (cmd, option)", file=sys.stderr)
badopts = True
elif not argv:
print("%s: %s: missing date" % (cmd, option), file=sys.stderr)
badopts = True
else:
my_date = argv.pop(0)
... check date for sanity here ...
if argv:
print("%s: extra arguments: %r" % (cmd, argv), file=sys.stderr)
badopts = True
if badopts:
print(USAGE, file=sys.stderr)
return 2
... proceed with main program here ...
See how it checks for options and their arguments, and has the scope to make
many complaints before quitting?
If you have several options you will want to reach for a module like argparse,
but your program has only one.
timestamp01 = time.strftime("%Y-%m-%d")
timestamp02 = time.strftime("%Y-%m-%d-%H%M%S")
wd01 = ("/var/tmp/myKNN/1_mySAMPLEpython-ver-001/" +
timestamp01)
wd02 = ("/var/tmp/myKNN/1_mySAMPLEpython-ver-001/" +
timestamp02)
You never use these. Also, these pathnames are very special looking; I would
put them up the top as tunable constants (Python doesn't have constants, but it
has an idiom of globals with UPPERCASE names for the same purpose).
[...]
# LOOP through the SET and WHOIS
for i in banIP_addrs:
You might do better to use a better name than "i", for example "ipaddr". More
readable to others, and also to yourself later.
Also, you _may_ want to sort the addresses for reporting purposes (entirely
your call), you could go:
for ipaddr in sorted(banIP_addrs):
print("i:", i)
whoisVAR = os.popen("whois -H " + i + " |egrep
-i \"name|country|mail\" |sort -u").read()
Again, here you are running a shell pipeline which could be done in Python.
Look at the subprocess module, and invoke:
whois = subprocess.Popen(['whois', '-H', ipaddr], stdout=subprocess.PIPE)
for line in whois:
... gather up name/country/mail as you did with the log file ...
whois.wait()
... now print report ...
Cheers,
Cameron Simpson
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
[Tutor] File extension change
Hi,
So what I am working on is taking a csv file and only taking 2 columns from
the spreadsheet and out putting that to a text file. Then taking those two
columns and organize them by product(key) and outputting the
description(values) that are associated. Some products have a lot of
duplicate descriptions and I am trying to get the counts of those. I have a
piece of code that takes anything greater then 5 and prints that and also
anything 4 or less goes into an 'other' category with the counts.So what I
am trying to do now is import a csv and change it to a text file with the
same naming convention as the csv. But I want the text file to have the
results from the json function. I want the text file that I am trying to
output to have the same name as the filename I am importing. The file that
I am inputting in the def populate_dict (usually the files are named
weekly_and the date...ex: weekly_20160102.csv) and then i want to run all
my code i have, but then in the def json_output instead of having the
filename which i have now is 'test.txt' I want to have weekly_20160102.txt
same naming convention as the csv file i am inputting. In my code I am soft
coding the filename so that the user is prompted to enter what file they
want to use.
import csvimport jsonimport sysimport osfrom collections import
defaultdictfrom collections import Counter
upper_limit = 5
lower_limit = 4
class dictionary():
def __init__(self):
self.dict = defaultdict(list)
self.counted_dict = defaultdict(list)
self.grouped_dict = defaultdict(list)
self.other_dict = defaultdict(list)
self.final_dict = defaultdict(list)
self.total_dict = defaultdict(list)
def populate_dict(self, filename):
with open(filename, 'rb') as f:
reader = csv.reader(f)
next(reader, None)
for row in reader:
self.dict[row[2]].append(row[3])
with open(filename, 'r') as searchfile, open('weekly_test.txt',
'w')
as search_results_file:
for line in searchfile:
if 'PBI 43125' in line:
print >>search_results_file, line
def all_counts(self):
data_count = Counter()
for key in self.dict.keys():
self.counted_dict.update({key: Counter(self.dict[key])})
def total_counts(self):
for key in self.dict.keys():
total = 0
b = Counter(self.dict[key])
for value in b:
total += b[value]
new_list = str(total)
#self.total_dict.update({key: 'Total count for this
application: ' + str(total)})
#self.total_dict.update({key: str(total)})
self.total_dict[key].append(new_list)
def grouped_counts(self):
for key in self.dict.keys():
total = 0
c = Counter(self.dict[key])
for value in c:
if c[value] >= upper_limit:
new_list = value, str(c[value])
self.grouped_dict[key].append(new_list)
elif c[value] <= lower_limit:
total += c[value]
self.other_dict.update({key: 'other: ' + str(total)})
for d in (self.grouped_dict, self.other_dict, self.total_dict):
for key, value in d.iteritems():
self.final_dict[key].append(value)
def json_output(self):
with open('test.txt', 'w') as text_file:
json.dump(self.final_dict, text_file, sort_keys = True,
indent = 4
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] File extension change
Chelsea G writes: > import csvimport jsonimport sysimport osfrom collections import > defaultdictfrom collections import Counter Your email client has, probably without your consent, mangled the content of your message. Please ensure your email client does not attempt to apply pretty formatting. Set it explicitly to send messages as “plain text” only, so your deliberate code formatting will not be mangled. -- \ “Capitalism is the astounding belief that the most wickedest of | `\men will do the most wickedest of things for the greatest good | _o__) of everyone.” —John Maynard Keynes | Ben Finney ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] date range
Hi, So I am taking in a csv file with several rows and one of those rows in a date row. I am trying to see if I can read in a csv file and search for a certain date range like 1/2/2016 to 1/5/2016. Something like that or a bigger date range. And then I want to ouput the results to either another csv or to a textfile. I am not sure how to even beginning this and was looking for some guidance on this. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] date range
Chelsea G writes: > So I am taking in a csv file with several rows and one of those rows > in a date row. That doesn't make much sense, as a sepcification. CSV is by design a data format in which every data row has the same structure: the fields always have the same meaning, in the same order. > I am trying to see if I can read in a csv file and search for a > certain date range like 1/2/2016 to 1/5/2016. You can, by breaking the problem into simpler problems: * Get the rows into some collection of structured records. (The Python standard library's ‘csv.DictReader’ is your friend here.) * Identify which field of the record is the data on which you want to filter. (For a collection of records you retrieved from reading a ‘csv.DictReader’, you need to know which field name has the dates you're interested in.) * Construct an expression that takes an arbitrary date value as found in that field, and interrogates it by some comparison that evaluates to True when the value is in your specified range and False otherwise. * Construct a new collection of records, by iterating the full collection and only accumulating the ones which match your criteria. Each of those is much simpler that the original problem. Solve each of them separately, and it should then be much easier to put the solutions together to solve the larger problem. -- \ “Friendship is born at that moment when one person says to | `\another, ‘What! You too? I thought I was the only one!’” —C.S. | _o__)Lewis | Ben Finney ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
