Re: [Tutor] Coming from R, what's a good IDE editor? I've tried PyCharm and Spyder
On 06/04/2017 04:44 AM, Abdur-Rahmaan Janhangeer wrote: Wing. Wing IDE personal works awesome for me Abdur-Rahmaan Janhangeer, Mauritius abdurrahmaanjanhangeer.wordpress.com On 3 Jun 2017 02:59, "C W" wrote: Dear Python list, I am an R user learning Python. What is a good editor? 1) Pycharm PyCharm evaluates the entire script, I just want to change a few lines in the script. For example, import matplotlib.pyplot as plt import numpy as np x = np.arange(0, 1,0.1) y = np.sin(2 * np.pi * x) plt.figure(1) plt.clf() plt.plot(x, y) plt.show() Now, I want to do a scatter plot, but don't want to generate the data again. I just want the "line by line" evaluation like in R and Matlab. Basically, you can type in console and add to the existing variables. 2) Spyder Spyder looks a lot like RStudio, I like it! But, it does not have an app icon in applications. I am baffled. I do ~/anaconda/bin/spyder every time. Am I missing something or is this the way it is? Thank you very much! ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor Here's how I start Spyder, in Xfce: /home/comp/Apps/anaconda3/bin/spyder -- Stephen P. Molnar, Ph.D.Life is a fuzzy set www.molecular-modeling.net Stochastic and multivariate (614)312-7528 (c) Skype: smolnar1 ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] Python - help with something most essential
I need someone's help. I am not proficient in Python and I wish to understand something. I was in a job pre-screening process where I was asked to solve a simple problem. The problem was supposed to be solved in Python and it was supposed to take two arguments: filename and word. The program reads the file which is a .txt file containing a bunch of words and counts how many of those words in the file are anagrams of the argument. First I concocted this solution: import sys from collections import Counter def main(args): filename = args[1] word = args[2] print countAnagrams(word, filename) def countAnagrams(word, filename): fileContent = readFile(filename) counter = Counter(word) num_of_anagrams = 0 for i in range(0, len(fileContent)): if counter == Counter(fileContent[i]): num_of_anagrams += 1 return num_of_anagrams def readFile(filename): with open(filename) as f: content = f.readlines() content = [x.strip() for x in content] return content if __name__ == '__main__': main(sys.argv) Very quickly I received this comment: "Can you adjust your solution a bit so you less loops (as little as possible) and also reduce the memory usage footprint of you program?" I tried to rework the methods into this: def countAnagrams(word, filename): fileContent = readFile(filename) return sum(1 for _ in filter(lambda x: Counter(word) == Counter(x.strip()), fileContent)) def readFile(filename): with open(filename) as f: content = f.readlines() return content And I was rejected. I just wish to understand what I could have done for this to be better? I am a Python beginner, so I'm sure there are things I don't know, but I was a bit surprised at the abruptness of the rejection and I'm worried I'm doing something profoundly wrong. Thank you in advance ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Coming from R, what's a good IDE editor? I've tried PyCharm and Spyder
+1 for Wing IDE. I have been using it for about 6-7 months now and absolutely love it. Ryan On Sun, Jun 4, 2017 at 11:37 AM Ryan Smith wrote: > > On Sun, Jun 4, 2017 at 8:13 AM wolfrage8...@gmail.com < > wolfrage8...@gmail.com> wrote: > >> Atom.io Editor is my current favorite after having swapped around a lot. >> >> http://www.marinamele.com/install-and-configure-atom-editor-for-python >> ___ >> Tutor maillist - Tutor@python.org >> To unsubscribe or change subscription options: >> https://mail.python.org/mailman/listinfo/tutor >> > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] Huge list comprehension
hi,
I would appreciate if you can help me suggesting a quick and efficient strategy
for comparing multiple lists with one principal list
I have about 125 lists containing about 100,000 numerical entries in each
my principal list contains about 6 million entries.
I want to compare each small list with main list and append yes/no or 0/1 in
each new list corresponding to each of 125 lists
The program is working but it takes ages to process huge files,
Can someone pleases tell me how can I make this process fast. Right now it
takes arounf 2 weeks to complete this task
the code I have written and is working is as under:
sample_name = []
main_op_list,principal_list = [],[]
dictionary = {}
with open("C:/Users/INVINCIBLE/Desktop/T2D_ALL_blastout_batch.txt", 'r') as f:
reader = csv.reader(f, dialect = 'excel', delimiter='\t')
list2 = filter(None, reader)
for i in range(len(list2)):
col1 = list2[i][0]
operon = list2[i][1]
main_op_list.append(operon)
col1 = col1.strip().split("_")
sample_name = col1[0]
if dictionary.get(sample_name):
dictionary[sample_name].append(operon)
else:
dictionary[sample_name] = []
dictionary[sample_name].append(operon)
locals().update(dictionary) ## converts dictionary keys to variables
##print DLF004
dict_values = dictionary.values()
dict_keys = dictionary.keys()
print dict_keys
print len(dict_keys)
main_op_list_np = np.array(main_op_list)
DLF002_1,DLF004_1,DLF005_1,DLF006_1,DLF007_1,DLF008_1,DLF009_1,DLF010_1,DLF012_1,DLF013_1,DLF014_1,DLM001_1,DLM002_1,DLM003_1,DLM004_1,DLM005_1,DLM006_1,DLM009_1,DLM011_1,DLM012_1,DLM018_1,DOF002_1,DOF003_1
=[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]
DOF004_1,DOF006_1,DOF007_1,DOF008_1,DOF009_1,DOF010_1,DOF011_1,DOF012_1,DOF013_1,DOF014_1,DOM001_1,DOM003_1,DOM005_1,DOM008_1,DOM010_1,DOM012_1,DOM013_1,DOM014_1,DOM015_1,DOM016_1,DOM017_1,DOM018_1,DOM019_1
=[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]
DOM020_1,DOM021_1,DOM022_1,DOM023_1,DOM024_1,DOM025_1,DOM026_1 =
[],[],[],[],[],[],[]
NLF001_1,NLF002_1,NLF005_1,NLF006_1,NLF007_1,NLF008_1,NLF009_1,NLF010_1,NLF011_1,NLF012_1,NLF013_1,NLF014_1,NLF015_1,NLM001_1,NLM002_1,NLM003_1,NLM004_1,NLM005_1,NLM006_1,NLM007_1,NLM008_1,NLM009_1,NLM010_1
=[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]
NLM015_1,NLM016_1,NLM017_1,NLM021_1,NLM022_1,NLM023_1,NLM024_1,NLM025_1,NLM026_1,NLM027_1,NLM028_1,NLM029_1,NLM031_1,NLM032_1,NOF001_1,NOF002_1,NOF004_1,NOF005_1,NOF006_1,NOF007_1,NOF008_1,NOF009_1,NOF010_1
=[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]
NOF011_1,NOF012_1,NOF013_1,NOF014_1,NOM001_1,NOM002_1,NOM004_1,NOM005_1,NOM007_1,NOM008_1,NOM009_1,NOM010_1,NOM012_1,NOM013_1,NOM015_1,NOM016_1,NOM017_1,NOM018_1,NOM019_1,NOM020_1,NOM022_1,NOM023_1,NOM025_1
=[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]
NOM026_1,NOM027_1,NOM028_1,NOM029_1 = [],[],[],[]
for i in main_op_list_np:
if i in DLF002: DLF002_1.append('1')
else:DLF002_1.append('0')
if i in DLF004: DLF004_1.append('1')
else:DLF004_1.append('0')
if i in DLF005: DLF005_1.append('1')
else:DLF005_1.append('0')
if i in DLF006: DLF006_1.append('1')
else:DLF006_1.append('0')
if i in DLF007: DLF007_1.append('1')
else:DLF007_1.append('0')
if i in DLF008: DLF008_1.append('1')
else:DLF008_1.append('0')
## if main_op_list[i] in DLF009: DLF009_1.append('1')
## else:DLF009_1.append('0')
if i in DLF010: DLF010_1.append('1')
else:DLF010_1.append('0')
if i in DLF012: DLF012_1.append('1')
else:DLF012_1.append('0')
if i in DLF013: DLF013_1.append('1')
else:DLF013_1.append('0')
if i in DLF014: DLF014_1.append('1')
else:DLF014_1.append('0')
if i in DLM001: DLM001_1.append('1')
else:DLM001_1.append('0')
if i in DLM002: DLM002_1.append('1')
else:DLM002_1.append('0')
if i in DLM003: DLM003_1.append('1')
else:DLM003_1.append('0')
if i in DLM004: DLM004_1.append('1')
else:DLM004_1.append('0')
if i in DLM005: DLM005_1.append('1')
else:DLM005_1.append('0')
if i in DLM006: DLM006_1.append('1')
else:DLM006_1.append('0')
if i in DLM009: DLM009_1.append('1')
else:DLM009_1.append('0')
if i in DLM011: DLM011_1.append('1')
else:DLM011_1.append('0')
if i in DLM012: DLM012_1.append('1')
else:DLM012_1.append('0')
if i in DLM018: DLM018_1.append('1')
else:DLM018_1.append('0')
if i in DOF002: DOF002_1.append('1')
else:DOF002_1.append('0')
if i in DOF003: DOF003_1.append('1')
else:DOF003_1.append('0')
if i in DOF004: DOF004_1.append('1')
else:DOF004_1.append('0')
if i in DOF006: DOF006_1.append('1')
else:DOF006_1.append('0')
if i in DOF007: DOF007_1.append('1')
else:DOF007_1.append('0')
if i in DOF008: DOF008_1.append('1')
else:DOF008_1.append('0')
if i in
[Tutor] f.readlines(size)
Hi,
I'm trying to understand the optional size argument in file.readlines method.
The help(file) shows:
| readlines(...) | readlines([size]) -> list of strings, each a line
from the file. | | Call readline() repeatedly and return a list of
the lines so read. | The optional size argument, if given, is an
approximate bound on the | total number of bytes in the lines returned.
From the documentation:f.readlines() returns a list containing all the lines of
data in the file.
If given an optional parameter sizehint, it reads that many bytes from the file
and enough more to complete a line, and returns the lines from that.
This is often used to allow efficient reading of a large file by lines,
but without having to load the entire file in memory. Only complete lines
will be returned.
I wrote the function below to try it, thinking that it would print multiple
times, 3 lines at a time, but it printed all in one shot, just like when I
din't specify the optional argument. Could someone explain what I've missed?
See input file and output below.
Thanks,Nancy
def readLinesWithSize(): # bufsize = 65536
bufsize = 45 with open('input.txt') as f: while True:
# print len(f.readlines(bufsize)) # this will print 33 print
lines = f.readlines(bufsize) print lines if
not lines: break for line in lines:
pass readLinesWithSize()
Output:
['1CSCO,100,18.04\n', '2ANTM,200,45.03\n', '3CSCO,150,19.05\n',
'4MSFT,250,80.56\n', '5IBM,500,22.01\n', '6ANTM,250,44.23\n',
'7GOOG,200,501.45\n', '8CSCO,175,19.56\n', '9MSFT,75,80.81\n',
'10GOOG,300,502.65\n', '11IBM,150,25.01\n', '12CSCO1,100,18.04\n',
'13ANTM1,200,45.03\n', '14CSCO1,150,19.05\n', '15MSFT1,250,80.56\n',
'16IBM1,500,22.01\n', '17ANTM1,250,44.23\n', '18GOOG1,200,501.45\n',
'19CSCO1,175,19.56\n', '20MSFT1,75,80.81\n', '21GOOG1,300,502.65\n',
'22IBM1,150,25.01\n', '23CSCO2,100,18.04\n', '24ANTM2,200,45.03\n',
'25CSCO2,150,19.05\n', '26MSFT2,250,80.56\n', '27IBM2,500,22.01\n',
'28ANTM2,250,44.23\n', '29GOOG2,200,501.45\n', '30CSCO2,175,19.56\n',
'31MSFT2,75,80.81\n', '32GOOG2,300,502.65\n', '33IBM2,150,25.01\n']
[]
The input file contains 33 lines of text, 15 or 16 letter each (15 - 16
bytes):1CSCO,100,18.042ANTM,200,45.033CSCO,150,19.054MSFT,250,80.565IBM,500,22.016ANTM,250,44.237GOOG,200,501.458CSCO,175,19.569MSFT,75,80.8110GOOG,300,502.6511IBM,150,25.0112CSCO1,100,18.0413ANTM1,200,45.0314CSCO1,150,19.0515MSFT1,250,80.5616IBM1,500,22.0117ANTM1,250,44.2318GOOG1,200,501.4519CSCO1,175,19.5620MSFT1,75,80.8121GOOG1,300,502.6522IBM1,150,25.0123CSCO2,100,18.0424ANTM2,200,45.0325CSCO2,150,19.0526MSFT2,250,80.5627IBM2,500,22.0128ANTM2,250,44.2329GOOG2,200,501.4530CSCO2,175,19.5631MSFT2,75,80.8132GOOG2,300,502.6533IBM2,150,25.0
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Python - help with something most essential
> On Jun 5, 2017, at 09:36, Schtvveer Schvrveve wrote: > > > And I was rejected. I just wish to understand what I could have done for > this to be better? > > I am a Python beginner, so I'm sure there are things I don't know, but I > was a bit surprised at the abruptness of the rejection and I'm worried I'm > doing something profoundly wrong. The main thing that jumps out to me is the memory issue they asked you to address was not addressed. In your readFile, you have with open(filename) as f: content = f.readlines() Which reads the entire file into memory. They specifically did not want you to do that. The implication is they were looking for a solution where you read the file maybe one line (or even one word) at a time and look for anagrams in smaller groups. — David ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Python - help with something most essential
Schtvveer Schvrveve wrote: > I need someone's help. I am not proficient in Python and I wish to > understand something. I was in a job pre-screening process where I was > asked to solve a simple problem. > > The problem was supposed to be solved in Python and it was supposed to > take two arguments: filename and word. The program reads the file which is > a .txt file containing a bunch of words and counts how many of those words > in the file are anagrams of the argument. > > First I concocted this solution: > > import sys > from collections import Counter > > def main(args): > filename = args[1] > word = args[2] > print countAnagrams(word, filename) > > def countAnagrams(word, filename): > > fileContent = readFile(filename) > > counter = Counter(word) > num_of_anagrams = 0 > > for i in range(0, len(fileContent)): > if counter == Counter(fileContent[i]): > num_of_anagrams += 1 > > return num_of_anagrams > > def readFile(filename): > > with open(filename) as f: > content = f.readlines() > > content = [x.strip() for x in content] > > return content > > if __name__ == '__main__': > main(sys.argv) > > Very quickly I received this comment: > > "Can you adjust your solution a bit so you less loops (as little as > possible) and also reduce the memory usage footprint of you program?" > > I tried to rework the methods into this: > > def countAnagrams(word, filename): > > fileContent = readFile(filename) > > return sum(1 for _ in filter(lambda x: Counter(word) == > Counter(x.strip()), fileContent)) > > def readFile(filename): > > with open(filename) as f: > content = f.readlines() > > return content > > And I was rejected. I just wish to understand what I could have done for > this to be better? > > I am a Python beginner, so I'm sure there are things I don't know, but I > was a bit surprised at the abruptness of the rejection and I'm worried I'm > doing something profoundly wrong. for i in range(0, len(stuff)): ... instead of for item in stuff: ... and content = file.readlines() # read the whole file into memory process(content) are pretty much the most obvious indicators that you are a total newbie in Python. Looks like they weren't willing to give you the time to iron that out on the job even though you knew about lambda, Counter, list comprehensions and generator expressions which are not newbie stuff. When upon their hint you did not address the root cause of the unbounded memory consumption they might have come to the conclusion that you were reproducing snippets you picked up somewhere and thus were cheating. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
