[Tutor] global interpreter lock

[anish singh]

Can someone explain global interpreter lock with
some source code examples?

I didn't understand explanation offered here:
https://docs.python.org/3/glossary.html#term-global-interpreter-lock
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] receiving regular expression from command line

[anish singh]

I am trying to implement grep to just increase my knowledge
about regular expression.

Below is the program usage:
python test.py -i Documents/linux/linux/ -s '\w+_readalarm*'

However, due to my lack of knowledge about string handling
in python, I am getting wrong results.

def read_file(file, pattern):
   with open(file, 'r') as outfile:
 for line in outfile:
   match = re.compile(str(pattern)).match(line)
   if match:
 print(file + " " + match.group())

Can someone let me know how can I pass regular expression
from command line?

Whole code: http://ideone.com/KxLJP2

Any other comments about the code are most welcome.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Tutor Digest, Vol 152, Issue 3

[anish singh]

> On 01/10/16 09:16, anish singh wrote:
> > I am trying to implement grep to just increase my knowledge
> > about regular expression.
> >
> > Below is the program usage:
> > python test.py -i Documents/linux/linux/ -s '\w+_readalarm*'
> >
> > However, due to my lack of knowledge about string handling
> > in python, I am getting wrong results.
>
> Possibly, but we can't tell because
> a) You don't show us the code that parses your input
>


import os, sys, getopt

import re

import glob


def get_full_path(path, pattern):

  for (dirpath, dirnames, filenames) in os.walk(path):

match = re.search(pattern, dirpath)

for filename in filenames:

  if filename.endswith(('.c', '.h')):

yield os.path.join(dirpath, filename)


def read_file(file, pattern):

  with open(file, 'r') as infile:

for line in infile:

  match = re.compile(str(pattern)).match(line)

  if match:

print(file + " " + match.group())



def main(argv):

  path, f_pattern, s_pattern = '', '', ''

  try:

opts, args =
getopt.getopt(argv,"hi:p:f:s:",["ifile=","file_pattern=","string_pattern="])

  except getopt.GetoptError:

print 'test.py -i  -p '

sys.exit(2)

  for opt, arg in opts:

if opt == '-h':

   print 'test.py -i '

   sys.exit()

elif opt in ("-i", "--ifile"):

  path = arg

elif opt in ("-f", "--file_pattern"):

  f_pattern = arg

elif opt in ("-s", "--string_pattern"):

  s_pattern = arg.encode().decode('unicode_escape')

  print(s_pattern)


files = get_full_path(path, f_pattern)

for file in files:

  read_file(file, s_pattern)


if __name__ == "__main__":

  main(sys.argv[1:])


> b) You don't show us your output/error message
>

output is only file names. I don't see any other output.
I am running it like this:
python test.py -i ~/Documents/linux-next/ -s '\w*_read_register\w*'


>
> How are you parsing the input? Are you using
> the argparse module? Or one of the older ones?
> Or are you trying to just test the values in sys.argv?
>

You can see the code now.


>
> How do you determine the input filename and the pattern?
>

Yes. Those are correct that is why i am getting all the file names.
You can run this code on any directory and see it just provides
the output as file names.

> Have you proved that those values are correct before you
> call your read_file() function?
>

Yes.

>
>
> > def read_file(file, pattern):
> >with open(file, 'r') as outfile:
>
> Since you are reading it it probably should be
> called infile?
>

Done.

>
> >  for line in outfile:
> >match = re.compile(str(pattern)).match(line)
> >if match:
> >  print(file + " " + match.group())
>
>
> > Can someone let me know how can I pass regular expression
> > from command line?
>
> The real issue here is probably how you parse the
> input line but we can't tell. The recommended module
> for doing that is argparse. Try reading the module
> documentation which includes many examples.
>
> If you are still having problems show us your full
> program plus any error messages
>
> > Whole code: http://ideone.com/KxLJP2
>
> Unless its very long (>100lines?) just post it
> in your email.
>
>
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] python grep implementation

[anish singh]

I am trying to implement grep functionality in python.

import os, sys, getopt
import multiprocessing as mp
import re
import itertools

def get_files(path, pattern):
  for (dirpath, dirnames, filenames) in os.walk(path):
for filename in filenames:
  if filename.endswith(pattern):
yield os.path.join(dirpath, filename)

def worker_search_fn(arg):
  fname, regex = arg
  with open(fname, 'rt') as f:
match = regex.search(f.read())
if match:
  print(fname + " " +":"+ match.group())
  return

def main(argv):
  path, f_pattern, s_pattern = '', '', ''
  try:
 opts, args =
getopt.getopt(argv,"hi:p:f:s:S:",["ifile=","file_pattern=","string_pattern=","string_flags="])
  except getopt.GetoptError:
 print 'test.py -i  -p  -f 
-S '
 print 'example usage python a.py -i . -s \'.*_i2c_register.*\' -f
.c,.h,.cpp -S "S"'
 sys.exit(2)
  for opt, arg in opts:
 if opt == '-h':
print 'test.py -i  -p , -f '
sys.exit()
 elif opt in ("-i", "--ifile"):
path = arg
 elif opt in ("-f", "--file_pattern"):
f_pattern = arg.split(",")
 elif opt in ("-s", "--string_pattern"):
s_pattern = arg
 elif opt in ("-S", "--string_flags"):
s_pattern_flags = arg

  regex = re.compile(s_pattern, getattr(re, s_pattern_flags))
  files = get_files(path, tuple(f_pattern))
  mp.Pool().map(worker_search_fn, itertools.izip(files,
itertools.repeat(regex)))

if __name__ == "__main__":
  main(sys.argv[1:])


I want to see if I can further speedup in any way possible and also some
code review.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Help with Python Queue

[anish singh]

I was just writing to read a file using popen and wanted to use queue along
with it.
I have below code but it is giving this error:
AttributeError: Queue instance has no attribute 'taskdone'

import threading
from Queue import Queue

def worker(q):
while True:
  dataset = q.get()
  print("q is taken out")
  q.taskdone()

def create_data(q):
print("Inside create data")
output = [1, 2, 3]
for i in output:
  print("Inside", i)
  print("queue is put")
  q.put(i)

if __name__ == '__main__':
q = Queue()
t = threading.Thread(target=worker, args=(q,))
t.daemon = True
t.start()
create_data(q)
q.join()
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] decorators in a class

[anish singh]

Trying to use decorators in my class. I am calling
build_tree from the main function and i want to increment
the arguments by a constant factor by using decorators.
However as build_tree is a recursive function, I don't want
to call it recursively with increased constant factor always.

Probably decorators are not the ideal way for this but still
how to go about it using decorators.

Simple solution would be to just pass the parameters after
incrementing with constant values in the main function.
However, I want the caller of build_tree to not know that
internally we increment the indexes in the class and work
on that.
I can also call a intermediate function and then call build_tree
but then would that be the right way?

def pow_of_2(n):
n -= 1
n |= n >> 1
n |= n >> 2
n |= n >> 4
n |= n >> 8
n |= n >> 16
n += 1
return n

def p_decorate(func):
def func_wrapper(self, left, right, root):
return func(self, left+self.n, right+self.n, root)
return func_wrapper

class segment_tree(object):
def __init__(self, data):
self.n = pow_of_2(len(data))
self.tree = [0]*self.n + data + [0]*(self.n - len(data))

@p_decorate
def build_tree(self, left, right, root):
if left == right:
return self.tree[left]
#below build_tree should not use decorated function,
#how to achieve that?
s = self.build_tree(left, (left+right)/2, 2*root) +
  self.build_tree(1+(left+right)/2, right, 2*root+1)
self.tree[root] = s

def __repr__(self):
return " ".join(str(i) for i in self.tree)

data = [1, 2, 3, 4]
sg = segment_tree(data)
sg.build_tree(0, 7, 1)
print(sg)
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] custom comparator with ordered list

[anish singh]

I need a custom comparator.

dictionary = {a:[b,c], d:[e,f]}

If both 'b' and 'e' belong to the same bin
then it should be compared based on 'c' and 'f'.

However, I want to also represent the result of the
sorted operation in a ordered dictionary as order is
important.

My custom comparator is something like this:


''' x and y is a list of two elements each'''
def cmpr(x, y):
r = 3
if x[0]//r != y[0]//r:
return x[0]//r < y[0]//r
return x[1] < y[1]

Please note it is not exactly comparing the first elements
of the value but checking if they belong to the same bin
and they do then it checks the second element as as shown
above.

Example:
{0:[0, 8], 1:[2, 5], 2:[2, 11], 3:[16, 17], 4:[13, 14], 5:[1, 17], 6:[17,
17] }
output should be:
{1:[2, 5], 0:[0, 8], 2:[2, 11], 5:[1, 17], 4:[13, 14], 3:[16, 17], 6:[17,
17] }

http://ideone.com/lXBdr2
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Tutor Digest, Vol 160, Issue 34

[anish singh]

> >> anish singh wrote:
> >>
> >>> I need a custom comparator.
> >>>
> >>> dictionary = {a:[b,c], d:[e,f]}
> >>>
> >>> If both 'b' and 'e' belong to the same bin
> >>> then it should be compared based on 'c' and 'f'.
> >>>
> >>> However, I want to also represent the result of the
> >>> sorted operation in a ordered dictionary as order is
> >>> important.
> >>>
> >>> My custom comparator is something like this:
> >>>
> >>>
> >>> ''' x and y is a list of two elements each'''
> >>> def cmpr(x, y):
> >>>r = 3
> >>>if x[0]//r != y[0]//r:
> >>>return x[0]//r < y[0]//r
> >>>return x[1] < y[1]
> >>
> >> This looks like it should be called less() rather than compare() as it
> >> doesn't differentiate between the x < y and x == y case.
> >>
> >>> Please note it is not exactly comparing the first elements
> >>> of the value but checking if they belong to the same bin
> >>> and they do then it checks the second element as as shown
> >>> above.
> >>
> >> The effect should be the same.
> >
> > Well no, take the case of [1,100] and [2,0]
> > Both belong to same bin suppose then it should
> > be sorted based on second index and I would
> > expect [2,0] [1,100] as output.
> > This is not happening currently with the original
> > code I have sent.
>
> I think that is because you do not consider all three cases.
> Let's start with a function cmp() modeled after the Python 2 built-in
>
> def cmp(a, b):
> if a < b:
> return -1
> elif a > b:
> return 1
> return 0
>
> Then your comparator could be fixed (I think) as follows
>
> def compare(x, y):
> def bin(a): return a[0] // 3
>
> result = cmp(bin(x), bin(y))
> if result:
> return result
> return cmp(x[1], y[1])
>
> and that "fixed" version would be equivalent (I think) to
>
> def compare(x, y)
> def key(a): return (a[0] // 3, a[1])
>
> return cmp((key(x), key(y))
>
> That said, even if you use Python 2 you should use sorted() with a key
> function rather than a comparison -- as shown below. Did that work for you?
>

Yes it did. Thanks.

>
> >>> Example:
> >>> {0:[0, 8], 1:[2, 5], 2:[2, 11], 3:[16, 17], 4:[13, 14], 5:[1, 17],
> >>> {6:[17,
> >>> 17] }
> >>> output should be:
> >>> {1:[2, 5], 0:[0, 8], 2:[2, 11], 5:[1, 17], 4:[13, 14], 3:[16, 17],
> >>> {6:[17,
> >>> 17] }
> >>
> >>>>> input = {0:[0, 8], 1:[2, 5], 2:[2, 11], 3:[16, 17], 4:[13, 14], 5:[1,
> >> 17], 6:[17,
> >> ... 17] }
> >>>>> wanted = {1:[2, 5], 0:[0, 8], 2:[2, 11], 5:[1, 17], 4:[13, 14],
> 3:[16,
> >> 17], 6:[17,
> >> ... 17] }
> >>>>> output = {k: v for k, v in sorted(input.items(), key=lambda x: (x[1]
> >> [0]//3, x[1][1]))}
> >>>>> assert list(output.items()) == list(wanted.items())
> >>
> >> As written it will work with CPython 3.6. However, for compatibility
> with
> >> other versions of Python I recommend that you replace the plain dicts
> >> above with collections.OrderedDict instances. Quoting
> >>
> >> https://docs.python.org/dev/whatsnew/3.6.html#whatsnew36-pep520
> >>
> >> """
> >> The order-preserving aspect of this new [dict] implementation is
> >> considered an implementation detail and should not be relied upon [...]
> >> """
>
>
>
>
> --
>
> Message: 3
> Date: Mon, 26 Jun 2017 11:22:21 -0600
> From: Mats Wichmann 
> To: tutor@python.org
> Subject: Re: [Tutor] custom comparator with ordered list
> Message-ID: <9ecb3bcb-ceaa-8e2c-d6a2-edf78689a...@wichmann.us>
> Content-Type: text/plain; charset=utf-8
>
> On 06/26/2017 10:38 AM, Anish Kumar wrote:
> >
> >> anish singh wrote:
> >>
> >>> I need a custom comparator.
> >>>
> >>> dictionary = {a:[b,c], d:[e,f]}
> >>>
> >>> If both 'b' and 'e' belong to the same bin
>
> if would help alot if your problem statement included a description of
> what "same bin" is.
>
> > Well no, take the case of [1,100] and [2,0]
> &

[Tutor] Python creating trie

[anish singh]

Can someone explain me this code to create a trie
from words?

import collections
words = ["bad", "sad", "abyss"]

Trie = lambda: collections.defaultdict(Trie)
trie = Trie()
END = True

for i, word in enumerate(words):
reduce(dict.__getitem__, word, trie)[END] = i
print(trie.values())


I am not able to understand lambda usage and reduce
function here.

Thanks,
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] sort by value and then by key

[anish singh]

document = "Practice makes perfect. you'll only get
 Perfect by practice. just practice!"

output: [ ["practice", "3"], ["perfect", "2"], ["by", "1"],
  ["get", "1"], ["just", "1"], ["makes", "1"],
  ["only", "1"], ["youll", "1"] ]

I am supposed to return a list of all unique words
in it and their number of occurrences, sorted by
the number of occurrences in a descending order.
If two or more words have the same count, they
should be sorted alphabetically (in an ascending order).

However, I am stuck. I have below code which is not working.



import collections

x = collections.OrderedDict()
import collections
import operator


def make_comparator(x, y):
if x[1] > y[1]:
return 1
elif x[1] < y[1]:
return -1
elif x[1] == y[1]:
if x > y:
return 1
elif x < y:
return -1
return 0


document = "Practice makes perfect. you'll only get Perfect by
practice. just practice!"
words = document.split()
d = collections.defaultdict(int)
for word in words:
word = word.lower()
word = [c if c >= 'a' and c <= 'z' else "" for c in word]
word = "".join(word)
d[word] += 1
output = []
for key, value in sorted(d, cmp = make_comparator(x)):
output.append([key, value])
print(output)


Thanks,
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Tutor Digest, Vol 166, Issue 21

[anish singh]

>> However, I am stuck. I have below code which is not working.

I don't know how to achieve this programmatically: sorted by the
number of occurrences in a descending order. If two or more words
have the same count, they should be sorted
alphabetically (in an ascending order).

>
> Define "not working"
> Do you get an error message? (show us)

I am not getting any error message but i don't know
how to get the expected result.

#getting: [('just', 4), ('practice', 3), ('perfect', 2), ('youll', 1),
('makes', 1), ('get', 1), ('by', 1)]
#expected: 
[["just","4"],["practice","3"],["perfect","2"],["makes","1"],["youll","1"],["get","1"],["by","1"]]


> If the output different to what you expect (show us)

explained above.
>
> Have you tried printing the intermediate results?

I did but I don't know how to get what i am looking for.

> For example the dictionary before you sort it?
> Or what about using the default sort, how close is that?
>
> Don't expect us to run your code and check the output.

https://paste.pound-python.org/show/NappXV9daDMFz64yA805/

With the above code, i don't know what we can modify this to get
expected result as below:

#getting: [('just', 4), ('practice', 3), ('perfect', 2), ('youll', 1),
('makes', 1), ('get', 1), ('by', 1)]
#expected: 
[["just","4"],["practice","3"],["perfect","2"],["makes","1"],["youll","1"],["get","1"],["by","1"]]
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor