Re: [Tutor] write dictionary to file
This is driving me nuts.
I have tried many different things, but I just do not understand this csv
library.
I have tried passing various parameters to the writerow method and I am really
getting nowhere fast.
I just want to read from a file, join text to column and write to file.
The writerow section I really do not understand, I copied an example as it
seems to make zero sense when looking at examples or reading documentation.
I thought the csv library would simplify the task but it seems to complicate
it. I do not remember having this problem with sed/awk or perl.
#so far this should read a file
#using dictreader and take a column and join some text onto it
import csv
csvfile= open('StudentListToSort.csv', newline='')
spamreader = csv.DictReader(csvfile,delimiter=',',quotechar='|')
#open a file to write to later
fields = ['user','first','last','password','year']
csvoutput = open('output.csv', 'wb+')
spamwriter = csv.DictWriter(csvoutput,fieldnames=fields, delimiter=' ')
for row in spamreader:
if row['year'] == '40':
username = row['user']
email = "".join([username,'@email.com])
output = row['user'],
row['first'],row['last'],row['password'],row['year']
spamwriter.writerow([spamreader[fields] for fieldnames in fields])
print(output)
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] write dictionary to file
Hi,
On 20 June 2014 09:38, Ian D wrote:
> #so far this should read a file
> #using dictreader and take a column and join some text onto it
>
> import csv
>
> csvfile= open('StudentListToSort.csv', newline='')
> spamreader = csv.DictReader(csvfile,delimiter=',',quotechar='|')
>
> #open a file to write to later
> fields = ['user','first','last','password','year']
> csvoutput = open('output.csv', 'wb+')
> spamwriter = csv.DictWriter(csvoutput,fieldnames=fields, delimiter=' ')
>
>
>
> for row in spamreader:
> if row['year'] == '40':
> username = row['user']
> email = "".join([username,'@email.com])
>
> output = row['user'],
> row['first'],row['last'],row['password'],row['year']
> spamwriter.writerow([spamreader[fields] for fieldnames in fields])
> print(output)
Using DictReader and DictWriter means you retrieve and provide Python
dict's when interacting with the CSV module. Maybe this is adding some
confusion? Anyhow, here's a quick&dirty example modified from the
source you posted which adds a column to an existing CSV file.
(Initially I create the CSV just using a plain CSV writer. Then that
file is read in and a column added to selected records and written out
again.)
# -*- coding: utf-8 -*-
import csv
def create_demo_file(csv_demo_filename):
csvfile=open(csv_demo_filename, 'wb')
csvw = csv.writer(csvfile, quoting=csv.QUOTE_MINIMAL)
csvw.writerow(['user','first','last','password','year'])
csvw.writerows([
(1, 'john', 'smith', 'LKJ£$_£(*$£', 35),
(2, 'joe', 'bloggs','5^££J"HLLDD', 40),
(3, 'alice','jones', '^%!*&^%1681', 43),
(4, 'bob', 'white', '!&££JHLKJ*F', 28),
])
csvfile.close()
def add_email_to_csv(csv_input_filename, csv_output_filename):
csvfile= open(csv_input_filename)
spamreader = csv.DictReader(csvfile)
fields = ['user','first','last','password','year', 'email']
csvoutput = open(csv_output_filename, 'wb+')
spamwriter = csv.DictWriter(csvoutput,fieldnames=fields)
spamwriter.writeheader()
for row in spamreader:
if row['year'] in ('43', '40'):
username = row['user']
row['email'] = username+'@email.com'
spamwriter.writerow(row)
csvoutput.close()
###
create_demo_file('StudentListToSort.csv')
print 'Demo input file created contents:'
print open('StudentListToSort.csv', 'r').read()
add_email_to_csv('StudentListToSort.csv', 'output.csv')
print 'Demo output file created contents:'
print open('output.csv', 'r').read()
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] write dictionary to file
Thanks for your help I am not much closer in understanding this so I am going to try and start with a simpler example for myself. I will try and write some values to a file as I am struggling even doing this. TypeError: 'str' does not support the buffer interface TypeError: 'tuple' does not support the buffer interface these are the errors I tend to hit. So until I can write a list or a tuple to a file I will not try and get to adventurous. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] write dictionary to file
Hi,
Firstly an apology -- I only just noticed your original code was
Python 3 -- my example was Python 2, so there would be some changes
required to make the example work on Python 3...
On 20 June 2014 11:19, Ian D wrote:
> Thanks for your help
>
>
> I am not much closer in understanding this so I am going to try and start
> with a simpler example for myself.
Yes, that's usually a good idea.
> I will try and write some values to a file as I am struggling even doing this.
>
>
> TypeError: 'str' does not support the buffer interface
>
> TypeError: 'tuple' does not support the buffer interface
Could you post the lines of code that generates this exception and the
full stack trace that goes with it?
Nonetheless, having re-read your question and having googled a bit, it
seems that your problem might be related to Python 2 vs. Python 3, see
here:
http://stackoverflow.com/questions/24294457/python-typeerror-str-does-not-support-the-buffer-interface
In short: In Python 2 you are expected to open the CSV file in binary
mode ('wb'). In Python 3 this should be text mode as per the above
question, else you'll only be able to write "bytes" streams, hence the
"buffer" interface errors. If you've perhaps been cribbing/using
Python 2.x examples and documentation while in fact using using Python
3, then that would help explain the confusion...?
Walter
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] write dictionary to file
Thanks
>
> Nonetheless, having re-read your question and having googled a bit, it
> seems that your problem might be related to Python 2 vs. Python 3, see
> here:
> http://stackoverflow.com/questions/24294457/python-typeerror-str-does-not-support-the-buffer-interface
>
> In short: In Python 2 you are expected to open the CSV file in binary
> mode ('wb'). In Python 3 this should be text mode as per the above
> question, else you'll only be able to write "bytes" streams, hence the
> "buffer" interface errors. If you've perhaps been cribbing/using
> Python 2.x examples and documentation while in fact using using Python
> 3, then that would help explain the confusion...?
Ok I see this error and the example shows a different type of syntax.
Rather than a file open for writing:
outfile = open('output.csv', 'wb')
it uses
with open('data.csv', 'w', newline='') as out:
now is this written differently in order to implement this text mode thing or
is it just the omission of the 'b' on the 'wb' that causes text mode?
and if so could it be written:
outfile = open('output.csv', 'w')
?
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] write dictionary to file
Ok making some progress by changing the 'wb' to 'w'
>
> Ok I see this error and the example shows a different type of syntax.
>
>
> Rather than a file open for writing:
>
> outfile = open('output.csv', 'wb')
>
>
> it uses
>
> with open('data.csv', 'w', newline='') as out:
>
>
>
> now is this written differently in order to implement this text mode thing or
> is it just the omission of the 'b' on the 'wb' that causes text mode?
>
>
>
> and if so could it be written:
>
> outfile = open('output.csv', 'w')
>
>
>
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] write dictionary to file
>
> Ok making some progress by changing the 'wb' to 'w'
>
err no.
unstuck again.
import csv
csvfile= open('StudentListToSort.csv', newline='')
spamreader = csv.reader(csvfile,delimiter=',',quotechar='|')
outfile = open('outfile.csv','w')
for row in spamreader:
if row[4] == '6':
print("".join([row[0],'@email.com']),row[1])
email = "".join([row[0],'@email.com'])
output = email,row[1]
outfile.write(output)
outfile.close()
when I start to concatenate the results, it ends up as a Tuple and the write to
file stuff doesn't like Tuples
TypeError: must be str, not tuple
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] write dictionary to file
On 20/06/2014 15:11, Ian D wrote:
Ok making some progress by changing the 'wb' to 'w'
err no.
unstuck again.
import csv
csvfile= open('StudentListToSort.csv', newline='')
spamreader = csv.reader(csvfile,delimiter=',',quotechar='|')
outfile = open('outfile.csv','w')
for row in spamreader:
if row[4] == '6':
print("".join([row[0],'@email.com']),row[1])
email = "".join([row[0],'@email.com'])
output = email,row[1]
outfile.write(output)
outfile.close()
when I start to concatenate the results, it ends up as a Tuple and the write to
file stuff doesn't like Tuples
TypeError: must be str, not tuple
Please give the full traceback, not just the last line, as it gives us a
lot more information. As it happens you're creating a tuple when you
assign output, as it's the comma that makes a tuple, so unless I've
missed something there's your problem.
Further advice is to slow down a bit, remember more haste, less speed.
Walking away from the problem for a few minutes to clear your head often
works miracles :)
--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.
Mark Lawrence
---
This email is free from viruses and malware because avast! Antivirus protection
is active.
http://www.avast.com
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] write dictionary to file
Hi,
You've had a very good reply from Mark already however I want to add
to it and further clarify what he pointed out (why exactly *are* you
getting the tuple error after all?), also I've updated the prior
example to help explain, see below:
On 20 June 2014 15:11, Ian D wrote:
>
> import csv
>
> csvfile= open('StudentListToSort.csv', newline='')
> spamreader = csv.reader(csvfile,delimiter=',',quotechar='|')
> outfile = open('outfile.csv','w')
>
> for row in spamreader:
>
> if row[4] == '6':
> print("".join([row[0],'@email.com']),row[1])
> email = "".join([row[0],'@email.com'])
> output = email,row[1]
> outfile.write(output)
... Note that you're writing using the ///plain file object itself///,
which indeed would expect a simple string to write. Obviously a plain
file object, such as outfile, doesn't know by itself how to write a
tuple of objects, that's more the CSV object's job, hence you get the
error you're getting.
Instead you want to be writing using a CSV object instead, as you
originally indeed were doing. I suspect you simply forgot to use the
CSV writer and accidentally tried to write the output tuple directly
with the file object? So I'm with Mark -- less haste, more speed. :)
Finally I've updated the previous example to work with Python 3 and
added some comments, so you should be able to run it without problems.
Hopefully this should be enough to get you going. :)
# -*- coding: utf-8 -*-
import csv
def create_demo_file(csv_demo_filename):
csvfile=open(csv_demo_filename, 'w', newline='')
#Here we instantiate a CSV writer that accepts plain list like objects for
#writing, and then use it to write some demo data by passing it a list of
#tuples.
csvwriter = csv.writer(csvfile, quoting=csv.QUOTE_MINIMAL)
csvwriter.writerow(['user','first','last','password','year'])
csvwriter.writerows([
(1, 'john', 'smith', 'LKJ£$_£(*$£', 35),
(2, 'joe', 'bloggs','5^££J"HLLDD', 40),
(3, 'alice','jones', '^%!*&^%1681', 43),
(4, 'bob', 'white', '!&££JHLKJ*F', 28),
])
csvfile.close()
def add_email_to_csv(csv_input_filename, csv_output_filename):
csvfile= open(csv_input_filename)
csvoutput = open(csv_output_filename, 'w+', newline='')
#Here we instantiate a CSV reader that gives us each row as a dict object
#(as opposed to one which simply gives the values as a plain list)
csvreader = csv.DictReader(csvfile)
fields = ['user','first','last','password','year', 'email']
#Here we instantiate a csv writer that accepts dicts for writing.
#We pass it the formal field list as part of the constructor params
#and ask it to write the header line as a first step.
csvwriter = csv.DictWriter(csvoutput,fieldnames=fields)
csvwriter.writeheader()
#Step through each row dict object...
for row in csvreader:
if row['year'] in ('43', '40'):
#Again: The row is a dict, containing an entry for every
column in the table.
#We cheat, and simply add a new column to this existing dict,
# by simply assigning to the dict as normal. Then we ask
the CSV output
# writer to write this (now modified) dict as the row to
the output file:
row['email'] = row['user']+'@email.com'
csvwriter.writerow(row)
csvoutput.close()
###
create_demo_file('StudentListToSort.csv')
print('Demo input file created contents:')
print(open('StudentListToSort.csv', 'r').read() )
add_email_to_csv('StudentListToSort.csv', 'output.csv')
print('Demo output file created contents:')
print(open('output.csv', 'r').read())
HTH,
Walter
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
[Tutor] Fwd: Re: Tips
On 2014-06-19 15:52, Steven D'Aprano wrote: On Thu, Jun 19, 2014 at 11:10:26AM -0700, Alex Kleider wrote: The idea of a singleton class is new to me as is this comparison of class vs module. Can anyone suggest a place to turn for more discussion of the topic? thks, alexK "Singleton" is one of the classic "design patterns", although these days people are equally divided on whether it's a design pattern or anti-pattern. The idea of design patterns is that they are a standard way of solving a certain type of problem. For example, in the real world, there are various problems which have a certain factor in common: Example problems: - Painting a mural on the ceiling. Building a house. Replacing a broken window on the 2nd story. Making a tree house. Class of problem: - There is work needed at a height well above what you can reach from the ground. General solution: - Use scaffolding to raise the height at which you can comfortably work. So "scaffolding" is the design pattern. Actual scaffolds may be made from many different materials (steel, timber, bamboo) and in many different shapes and sizes, but they're all scaffolds. Rather than there being a "one size fits all" solution for all problems, instead there is a general solution that you customize for the specific problem. The size and shape of the scaffolding needed to replace a broken window will be different than that needed to build a house. Design patterns for software are like scaffolds: a design pattern is not a language feature or function you can call, but a general technique to be used to solve a class of problems. https://en.wikipedia.org/wiki/Design_pattern_%28computer_science%29 Explicit use of design patterns is very, very big in the Java and Dot Net worlds, less so in other programming languages. To learn more, Duck Duck Go is your friend: https://duckduckgo.com/html/?q=singleton+design+pattern https://duckduckgo.com/html/?q=singleton+anti-pattern If you still prefer Google: https://www.google.com/search?q=singleton+design+pattern or feel free to ask questions here. You've got me thinking but I'm still at a loss as to what questions need asked! The only applicability that fits in with anything I've experienced has to do with the necessity of globals to represent command line parameters which must be read at the outset and then be available to direct program execution. Thanks for the "tip." alex ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] write dictionary to file
On Fri, Jun 20, 2014 at 08:38:52AM +, Ian D wrote:
> This is driving me nuts.
>
> I have tried many different things, but I just do not understand this
> csv library.
Have you tried reading the documentation? It sounds like you're just
throwing random bits of code at it and hoping something works.
A better approach is to slow down and try to understand what the csv is
doing, what it expects from you, and how you can best use it. Ask
*focused* questions, rather than just blast us with blobs of code over
and over again.
> #so far this should read a file
> #using dictreader and take a column and join some text onto it
>
> import csv
>
> csvfile= open('StudentListToSort.csv', newline='')
> spamreader = csv.DictReader(csvfile,delimiter=',',quotechar='|')
Are you sure that your input file uses | as a quote character and , as
the field delimiter?
> #open a file to write to later
> fields = ['user','first','last','password','year']
> csvoutput = open('output.csv', 'wb+')
I'm pretty sure you don't want to use "wb+" mode. Since you're using
Python 3, I think you should just use "w" mode.
The "b" turns on binary mode, and in Python 3 you don't want that. The
"+" turns on either "read/write" mode or "append" mode, I don't remember
which, but either way I don't think it's necessary for what you are
doing.
> spamwriter = csv.DictWriter(csvoutput,fieldnames=fields, delimiter=' ')
Now you're turning every , delimiter into a space. Are you sure you want
that?
> for row in spamreader:
> if row['year'] == '40':
> username = row['user']
> email = "".join([username,'@email.com])
Syntax error: you left out the closing single quote. You need:
email = "".join([username,'@email.com'])
Other than that, so far so good. You're going through each row of the
input file, extracting the "year" field, checking if it equals "40", and
if it is, creating an email address from the user field. Perhaps a
shorter way to do this would be:
email = row['user'] + '@email.com'
rather than mucking about with "".join for only two pieces of text.
> output = row['user'],
> row['first'],row['last'],row['password'],row['year']
I'd move the print(output) line directly under this line, so it will
print the output even if the following line fails.
> spamwriter.writerow([spamreader[fields] for fieldnames in fields])
> print(output)
I'm sure the writerow line is completely wrong :-(
It expects a dict {fieldname: fieldvalue} not a list [fieldvalue]. And
you extract the field values through row, not through the spamreader
object directly. Try this instead:
spamwriter.writerow({name: row[name] for name in fields})
Let me put all those changes together, plus a few more fixes. Take note
of my comments.
import csv
# Open the file we're reading from.
csvfile= open('StudentListToSort.csv', newline='')
# Open a file to write to.
csvoutput = open('output.csv', 'w', newline='')
fields = ['user', 'first', 'last', 'password', 'year']
# Are you sure you want | as the quote character?
spamreader = csv.DictReader(csvfile, delimiter=',', quotechar='|')
# Still using , as a delimiter, not space.
spamwriter = csv.DictWriter(csvoutput, fieldnames=fields, delimiter=',')
for row in spamreader:
if row['year'] == '40':
email = row['user'] + '@email.com'
output = [ row[fieldname] for fieldname in fields ]
print(output)
# DictWriter needs a dict, not a list.
spamwriter.writerow({name: row[name] for name in fields})
print("Warning: email calculated but never used:", email)
# Good practice is to close the files when done.
csvfile.close()
csvoutput.close()
Try that. If you get any errors, please COPY AND PASTE the ***entire***
traceback, not just the last line.
--
Steven
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
