Re: HTTP Header Capitalization in urllib.request.AbstractHTTPHandler (Python 3.3)
Chris, That is genius. Thank you! -- Logan -- https://mail.python.org/mailman/listinfo/python-list
Re: Re: HTTP Header Capitalization in urllib.request.AbstractHTTPHandler (Python 3.3)
On 11/20/2013 02:05 AM, Chris Angelico wrote: On Wed, Nov 20, 2013 at 1:14 PM, Logan wrote: Chris, That is genius. Thank you! Then it works? Awesome!! (Permit me an evil laugh. Muahahah!) This is why I love working with open source languages. Even if you don't end up actually changing anything, you can go and snoop the code and see what happens - sometimes you can tweak your code based on that knowledge. And hey. This is duck typing at its best! ChrisA Not exactly as written, but close enough to get me working. At one point the following code is executed, turning the value into a string to be "title"d next time it is called: name = name.title() So, I worked around it with the following class, adapted from yours: class CaseSensitiveHeader(object): def __init__(self, name): self.name = name def capitalize(self): return self def title(self): return self def lower(self): return self.name def encode(self, encoding): return self.name.encode(encoding) With that, I am now able to post a case sensitive HTTP header. -- Logan -- https://mail.python.org/mailman/listinfo/python-list
Override the interpreter used by multiprocessing subprocesses?
Hello, I am using a product that has a built-in Python interpreter (ESRI ArcGIS Desktop 10.0 SP3) and have implemented multiprocessing in script that can be run by a tool within the application using the built-in interpreter. The way the built-in interpreter works is incompatible with multiprocessing as it appears to want to start multiple instances of the host application when the subprocesses are created. I am using multiprocessing.Pool with the apply_async function. It works great in a standalone script and I'd rather not rewrite what I've done so far if I can help it, but if there is a way to simply force the standard interpreter to be used for the subprocesses that would be ideal. Thinking about it I realize that this might not be possible due to whatever messaging, pickling, and queueing that the main processes has to handle for the subprocesses, but thought I would ask anyways if only for my own enlightenment. TIA! -- http://mail.python.org/mailman/listinfo/python-list
Python not Running
I downloaded Python but when I try to run Idle it says it is not found and I should try to redownload it. When I try to do that all that comes up is a page that says modify, repair or uninstall. I have repaired several times but it still gives me the same message. Can anyone help me with this? -- https://mail.python.org/mailman/listinfo/python-list
Deletion of Environmental Variables
Hello- I think I subscribed now. I planned on using python to simulate different cipher to challenge myself, but I kinda deleted my environmental variables to python. I run Windows 10 and wanted to get NumPy and MatPlotLib modules imported to python. Basically, I downloaded python 3.7.1 first, but when I realized it was not compatible with TensorFlow (I wanted to make a machine learning algorithm at a point in time), I uninstalled it to get python 3.6.1. Somehow, maybe I misread a stack overflow post, but I deleted all my path files to python36 for whatever reason. I still have no idea why I did this. Then, I thought, if I could uninstall python 3.6.1 and reinstall it pip would come back and I could use python from the cmd. I uninstalled it and tried reinstalling (with chocolatey this time) it for no avail. I tried searching for all the correct environment variables to put them back in manually, but I cannot seem to find most of them since python is still “not recognized as an internal or external command, operable program or batch file.” In my cmd. I really want pip to work so I can download modules. I may follow: https://stackoverflow.com/questions/4750806/how-do-i-install-pip-on-windows to try and help reinstall pip and I may redownload it through: https://pypi.org/project/pip/#files with help from: https://pypi.org/project/setuptools/#files As an aside, I have so far taught myself everything I know about python, and I am willing to learn even more. Thank you for any help, Logan Vogelsong -- https://mail.python.org/mailman/listinfo/python-list
HTTP Header Capitalization in urllib.request.AbstractHTTPHandler (Python 3.3)
Hello everyone, I was hoping for some advice in dealing with an edge case related to Python's HTTP Header handling. Python is correctly assuming that the HTTP header field name (eg Content-Type) is case insensitive, but I have a webservice I am interacting with that does not follow the standards. I am passing in the header "SOAPAction" and do_request of AbstractHTTPHandler is applying title() to the string, leading to the field name being "Soapaction", which the server does not recognize. So my question is, what does everyone suggest is the best solution for being able to pass a case sensitive header to the server? Thanks! Logan -- https://mail.python.org/mailman/listinfo/python-list
Possible PEP Submission
Just checking whether 1) a PEP is the proper place for this and 2) what
y'all think about it.
I would like to propose a change to the the 're' standard library to
support iterables.
So, something like the following would work:
import re
text = """hello user
hello user
hello user"""
users = ["Bob", "Alice", "Jeffery"]
newtext = re.sub("user", users, text)
# newtext = "hello Bob\nhello Alice\nhello Jeffery"
There are a few corner cases I'm not sure what would be the best behavior.
Nor am I entirely certain this should be modified functionality or just...
a new function. What do y'all think?
--
---Logan Collins
GPG Public Key 2AAE8775
0xfc1185c82aae8775
http://keyserver.ubuntu.com:11371/pks/lookup?op=get&search=0xFC1185C82AAE8775
--
https://mail.python.org/mailman/listinfo/python-list
Re: how can this iterator be optimized?
On Feb 11, 8:22 pm, Basilisk96 wrote:
> Hello all,
>
> I have the following function that uses an intermediate iterator
> "rawPairs":
>
> def MakePairs(path):
> import os
> import operator
> join = os.path.join
> rawPairs = (
> (join(path, s), func(s))
> for s in os.listdir(path)
> if func(s) is not None and func(s).endswith("some criterion")
> )
> #Use the second item in the pair as the sort criterion
> result = sorted(rawPairs, key=operator.itemgetter(1))
> return result
>
> where "func" is a single-argument function that returns either a
> string or None, but is an expensive call.
> I am pretty sure that the sorted() construct cannot be improved much
> further, but...
> ...does anyone have ideas on improving the "rawPairs" iterator so that
> it calls "func(s)" only once per iteration? Perhaps a lambda
> construct, but I am not sure how to go about it...?
>
> Cheers,
> Basilisk96
Hi,
Try something like this:
import os
from os.path import join
from itertools import ifilter #assuming 2.5 and earlier, for 3.0 just
use the filter builtin
from operator import itemgetter
def isvalid(pair):
return (s[1] is not None) and s[1].endswith('some criteria')
def makepairs(path):
pair_iter = ((join(path,s), func(s)) for s in os.listdir(path))
pair_iter = ifilter(isvalid, pair_iter)
result = sorted(pair_iter, key=itemgetter(1))
return result
--
http://mail.python.org/mailman/listinfo/python-list
Re: Read Only attributes, auto properties and getters and setters
On Feb 12, 10:58 am, TechieInsights wrote: > Oh... one other thing that would be really cool is to do this with AOP/ > descriptors! I just haven't been able to get that to work either. > Basics... > > @readonly > class MyClass(object): > def __init__(self, x): > self.set_x(x) > > def get_x(self): > return self.__x > > def set_x(self, x): > print 'Hello the setter was called!' > self.__x = x > > and it's done! I don't think this will work because (maybe I'm wrong) > but I don't think you can set the metaclass in the descriptor??? If > this is the case, it is more work to do the AOP and set a metaclass... > better to just inherit. If it can be done... that would be awesome! > > Greg Are your needs not already satisfied by the Python built-in property? http://docs.python.org/dev/3.0/library/functions.html#property -- http://mail.python.org/mailman/listinfo/python-list
Re: Read Only attributes, auto properties and getters and setters
On Feb 12, 12:27 pm, TechieInsights wrote:
> Ok... for some closure I have written a class to automate the
> process. It takes getters and setters and deleters and then sets the
> property automatically. Sweet!
>
> class AutoProperty(type):
> def __new__(cls, name, bases, methoddict):
> processed = []
> getter = 'get_'
> setter = 'set_'
> deleter = 'del_'
>
> starters = {getter:PropertyAttr(getter, PropertyAttr.FGET),
> setter:PropertyAttr(setter,
> PropertyAttr.FSET),
> deleter:PropertyAttr(deleter,
> PropertyAttr.FDEL)
> }
> for key, value in methoddict.items():
> var = None
> for start in starters.keys():
> if key.startswith(start):
> var = key[len(start):]
> break
> if var is None or var in processed:
> continue
> property_values = []
>
> for start in starters.keys():
> if '%s%s' %(start, var) in methoddict.keys():
>
> property_values.append(starters[start].tostring(var))
> else:
>
> property_values.append(starters[start].tostring(None))
> property_map = 'methoddict["%s"] = property(%s)'
> %(var, ','.join
> (property_values))
> exec(property_map)
> return type.__new__(cls, name, bases, methoddict)
>
> class PropertyAttr(object):
> FGET = 'fget'
> FSET = 'fset'
> FDEL = 'fdel'
> def __init__(self, start, type = FGET):
> self.start = start
> self.type = type
>
> def tostring(self, var):
> if self.type == self.FSET:
> vars = ['v']
> else:
> vars = []
> fullvar = ['self'] + vars
> if var is None:
> return '%s=None' %(self.type)
> return '%s=lambda %s: self.%s%s(%s)' %(self.type,
> ','.join(fullvar),
> self.start, var, ','.join(vars))
>
> class ReadOnly(object):
> __metaclass__ = AutoProperty
>
> class MyClass(ReadOnly):
> def __init__(self, x, y):
> self.__x = x
> self.__y = y
>
> def get_x(self):
> return self.__x
>
> def set_x(self, x):
> self.__x = x
>
> def get_y(self):
> return self.__y
>
> mc = MyClass(10, 100)
> print mc.x, mc.y
> mc.x = 10
> print mc.x
> try:
> mc.y = 100
> except AttributeError:
> print 'Yea it worked!'
>
> try:
> del mc.y
> except AttributeError:
> print "It's read only!"
>
> And the output:
> 10 100
> 10
> Yea it worked!
> It's read only!
>
> Now to work on descriptors doing it for you.
I think I'll stick with the built-in, Thanks :)
--
http://mail.python.org/mailman/listinfo/python-list
Re: how can this iterator be optimized?
On Feb 13, 7:44 pm, Basilisk96 wrote: > On Feb 12, 1:15 am, Steven D'Aprano > > wrote: > > > I usually strive > > > for comprehensions if a for loop can be reduced to such. > > > Any particular reason? > > Only two. > 1.) I was impressed by their clarity and conciseness when I first > discovered them. > 2.) I also read now and then that simple list comprehensions are > faster when compared with their for-loop equivalents because of the > way comprehensions are implemented under the hood. My example is a far > cry from a "simple" comprehension, however. :) > > > If there's only one call to func(), and you ignore the (probably) fixed > > cost of jumping into a generator each time, then it shouldn't make any > > difference. > > > If you are comparing one call to func() in a for loop versus three calls > > to func() in a list comp or generator expression, then of course the for > > loop will be more efficient. > > I agree. I would rather call func() only once per iteration in any > case. I will revise it to a plain for loop with a single call. > > Thanks, > -Basilisk96 Just as long as you do realize that it is possible to do what you were looking with one call to func() using chained generators, as demonstrated in my post above. Whichever way is clearest for you would be the way I'd go. -- http://mail.python.org/mailman/listinfo/python-list
Re: Candidate for a new itertool
On Mar 7, 8:47 pm, Raymond Hettinger wrote:
> The existing groupby() itertool works great when every element in a
> group has the same key, but it is not so handy when groups are
> determined by boundary conditions.
>
> For edge-triggered events, we need to convert a boundary-event
> predicate to groupby-style key function. The code below encapsulates
> that process in a new itertool called split_on().
>
> Would love you guys to experiment with it for a bit and confirm that
> you find it useful. Suggestions are welcome.
>
> Raymond
>
> -
>
> from itertools import groupby
>
> def split_on(iterable, event, start=True):
> 'Split iterable on event boundaries (either start events or stop
> events).'
> # split_on('X1X23X456X', 'X'.__eq__, True) --> X1 X23 X456 X
> # split_on('X1X23X456X', 'X'.__eq__, False) --> X 1X 23X 456X
> def transition_counter(x, start=start, cnt=[0]):
> before = cnt[0]
> if event(x):
> cnt[0] += 1
> after = cnt[0]
> return after if start else before
> return (g for k, g in groupby(iterable, transition_counter))
>
> if __name__ == '__main__':
> for start in True, False:
> for g in split_on('X1X23X456X', 'X'.__eq__, start):
> print list(g)
> print
>
> from pprint import pprint
> boundary = '--===2615450625767277916==\n'
> email = open('email.txt')
> for mime_section in split_on(email, boundary.__eq__):
> pprint(list(mime_section, 1, None))
> print '= = ' * 30
I've found this type of splitting quite useful when grouping sections
of a text file. I used the groupby function directly in the file, when
i would have rather used something like this.
However, I wonder if it would be helpful to break that function into
two instead of having the "start" flag. The flag feels odd to me
(maybe it's the name?), and the documentation might have a better feel
to it, coming from a newcomer's perspective. Also, it would be cool if
the function took keywords; I wonder why most of the other functions
in the itertools module don't take keywords.
I wouldn't split out the keys separately from the groups. But the idea
of a flag to exclude the keys sounds interesting to me.
Thank you for giving me the opportunity to use the nonlocal keyword
for the first time since trying out Python 3.0. I hope this is an
appropriate usage:
def split_on(iterable, key=bool, start=True):
'Split iterable on boundaries (either start events or stop
events).'
# split_on('X1X23X456X', 'X'.__eq__, True) --> X1 X23 X456 X
# split_on('X1X23X456X', 'X'.__eq__, False) --> X 1X 23X 456X
flag = 0
def event_marker(x, start_flag=start):
nonlocal flag, key
before = flag
if key(x):
flag += 1
after = flag
return after if start_flag else before
return (g for k, g in it.groupby(iterable, key=event_marker))
--
http://mail.python.org/mailman/listinfo/python-list
Re: Stripping non-numbers from a file parse without nested lists?
On Apr 1, 2:35 am, [email protected] wrote: > On Mar 31, 6:47 pm, "Rhodri James" > wrote: > > > What you're doing (pace error checking) seems fine for the data > > structures that you're using. I'm not entirely clear what your usage > > pattern for "dip" and "dir" is once you've got them, so I can't say > > whether there's a more appropriate shape for them. I am a bit curious > > though as to why a nested list is non-ideal? > > > ... > > if "/" in word and "dip" not in word: > > dip_n_dir.append(word.split("/", 1)) > > > is marginally shorter, and has the virtue of making it harder to use > > unrelated dip and dir values together. > > > -- > > Rhodri James *-* Wildebeeste Herder to the Masses > > Rhodri, > > Thanks. That works better than what I had before and I learned a new > method of parsing what I was looking for. > > Now I'm on to jumping a set number of lines from a given positive > search match: > > ...(lines of garbage)... > 5656 (or some other value I want, but don't explicitly know) > ...(18 lines of garbage)... > search object > ...(lines of garbage)... > > I've tried: > > def read_poles(filename): > index = 0 > fh = None > try: > fh = open(filename, "r") > lines=fh.readlines() > while True: > > if "search object" in lines[index] > poles = int(lines[index-18]) > print(poles) > > index +=1 > > except(IndexError): pass > > finally: > if fh is not None: # close file > fh.close() > > -- > > Which half works. If it's not found, IndexError is caught and passed > (avoids quitting on lines[index out of range]. The print(poles) > properly displays the value I am looking for (_always_ 18 lines before > the search object). > > However, since it is assigned using the index variable, the value of > poles doesn't keep (poles is always zero when referenced outside of > the read_poles function). I'm assuming because I'm pointing to a > certain position of an object and once index moves on, it no longer > points to anything valid. My python book suggested using > copy.deepcopy, but that didn't get around the fact I am calling it on > (index-18). > > Any experience jumping back (or forward) a set number of lines once a > search object is found? This is the only way I can think of doing it > and it clearly has some problems. > > Reading the file line by line using for line in blah works for finding > the search object, but I can't see a way of going back the 18 lines to > grabbing what I need. > > Thanks for the help! I'm slowly getting this mangled mess of a file > into something automated (hand investigating the several thousand > files I need to do would be unpleasant). # You could try using a deque holding 18 lines and search using that deque # This is untested, but here's a try (>=Python 3.0) from collections import deque import itertools as it import sys def read_poles(filename): with open(filename) as f: line_iter = iter(f) d = deque(it.islice(line_iter,17), maxlen=18) for line in line_iter: d.append(line) if 'search object' in line: poles = int(d[0]) print(poles) return poles else: print('No poles found in', filename, file=sys.err) -- http://mail.python.org/mailman/listinfo/python-list
Re: Stripping non-numbers from a file parse without nested lists?
On Apr 1, 11:05 am, jay logan wrote: > On Apr 1, 2:35 am, [email protected] wrote: > > > > > On Mar 31, 6:47 pm, "Rhodri James" > > wrote: > > > > What you're doing (pace error checking) seems fine for the data > > > structures that you're using. I'm not entirely clear what your usage > > > pattern for "dip" and "dir" is once you've got them, so I can't say > > > whether there's a more appropriate shape for them. I am a bit curious > > > though as to why a nested list is non-ideal? > > > > ... > > > if "/" in word and "dip" not in word: > > > dip_n_dir.append(word.split("/", 1)) > > > > is marginally shorter, and has the virtue of making it harder to use > > > unrelated dip and dir values together. > > > > -- > > > Rhodri James *-* Wildebeeste Herder to the Masses > > > Rhodri, > > > Thanks. That works better than what I had before and I learned a new > > method of parsing what I was looking for. > > > Now I'm on to jumping a set number of lines from a given positive > > search match: > > > ...(lines of garbage)... > > 5656 (or some other value I want, but don't explicitly know) > > ...(18 lines of garbage)... > > search object > > ...(lines of garbage)... > > > I've tried: > > > def read_poles(filename): > > index = 0 > > fh = None > > try: > > fh = open(filename, "r") > > lines=fh.readlines() > > while True: > > > if "search object" in lines[index] > > poles = int(lines[index-18]) > > print(poles) > > > index +=1 > > > except(IndexError): pass > > > finally: > > if fh is not None: # close file > > fh.close() > > > -- > > > Which half works. If it's not found, IndexError is caught and passed > > (avoids quitting on lines[index out of range]. The print(poles) > > properly displays the value I am looking for (_always_ 18 lines before > > the search object). > > > However, since it is assigned using the index variable, the value of > > poles doesn't keep (poles is always zero when referenced outside of > > the read_poles function). I'm assuming because I'm pointing to a > > certain position of an object and once index moves on, it no longer > > points to anything valid. My python book suggested using > > copy.deepcopy, but that didn't get around the fact I am calling it on > > (index-18). > > > Any experience jumping back (or forward) a set number of lines once a > > search object is found? This is the only way I can think of doing it > > and it clearly has some problems. > > > Reading the file line by line using for line in blah works for finding > > the search object, but I can't see a way of going back the 18 lines to > > grabbing what I need. > > > Thanks for the help! I'm slowly getting this mangled mess of a file > > into something automated (hand investigating the several thousand > > files I need to do would be unpleasant). > > # You could try using a deque holding 18 lines and search using that > deque > # This is untested, but here's a try (>=Python 3.0) > from collections import deque > import itertools as it > import sys > > def read_poles(filename): > with open(filename) as f: > line_iter = iter(f) > d = deque(it.islice(line_iter,17), maxlen=18) > > for line in line_iter: > d.append(line) > > if 'search object' in line: > poles = int(d[0]) > print(poles) > return poles > else: > print('No poles found in', filename, file=sys.err) Notice that I returned the "pole" from the function so you could catch the return value as follows: pole = read_poles(filename) if pole is None: # no poles found else: print('Function returned this pole:', pole) If you need a list of poles, then return a list: def read_poles(filename): all_poles = [] with open(filename) as f: line_iter = iter(f) d = deque(it.islice(line_iter,17), maxlen=18) for line in line_iter: d.append(line) if 'search object' in line: all_poles.append(int(d[0])) return all_poles ... poles = read_poles(filename) if poles: print('Here are the poles:\n', '\n'.join(map(str,poles))) else: print('There were no poles found in', filename) -- http://mail.python.org/mailman/listinfo/python-list
Where is the correct round() method?
Hello, I need a round function that _always_ rounds to the higher integer if the argument is equidistant between two integers. In Python 3.0, this is not the advertised behavior of the built-in function round() as seen below: >>> round(0.5) 0 >>> round(1.5) 2 >>> round(2.5) 2 I would think this is a common need, but I cannot find a function in the Python library to do it. I wrote my own, but did I miss such a method in my search of the Python library? Thanks -- http://mail.python.org/mailman/listinfo/python-list
Re: Where is the correct round() method?
On Jul 27, 7:58 pm, Gary Herron <[EMAIL PROTECTED]> wrote: > josh logan wrote: > > Hello, > > > I need a round function that _always_ rounds to the higher integer if > > the argument is equidistant between two integers. In Python 3.0, this > > is not the advertised behavior of the built-in function round() as > > seen below: > > >>>> round(0.5) > > > 0 > > >>>> round(1.5) > > > 2 > > >>>> round(2.5) > > > 2 > > Huh? > > >>> round(2.5) > 3.0 > > Works for me on Python 2.5 on Linux running on "Intel(R) Core(TM)2 Duo > CPU". What system are you on? > > It could be that 2.5 is really 2.4... which would round down to 2, > but on any modern CPU (using IEEE floating point), 2.5 should be > representable exactly. > > However, as with any floating point calculations, if you expect exact > representation or calculations with any numbers, then you are misusing > floating points. > > Gary Herron > > > I would think this is a common need, but I cannot find a function in > > the Python library to do it. I wrote my own, but did I miss such a > > method in my search of the Python library? > > > Thanks > > -- > >http://mail.python.org/mailman/listinfo/python-list > > I should reiterate that I am using Python 3.0 and not Python 2.x. It looks like the behavior round() has changed between these two versions. Here is the documentation for round() in Python 3.0: http://docs.python.org/dev/3.0/library/functions.html#round Of interest in this discussion is the second paragraph, which explains the change: Does anyone know the reason behind this change, and what replacement method I can use to get the original behavior? -- http://mail.python.org/mailman/listinfo/python-list
Re: Where is the correct round() method?
On Jul 27, 8:45 pm, pigmartian <[EMAIL PROTECTED]> wrote: > it could be that 3.0 is using "banker's rounding" --- rounding to the > even digit. the idea behind it behind it being to reduce error > accumulation when working with large sets of values. > > > Works for me on Python 2.5 on Linux running on "Intel(R) Core(TM)2 Duo > > CPU". What system are you on? > > > It could be that 2.5 is really 2.4... which would round down to 2, > > but on any modern CPU (using IEEE floating point), 2.5 should be > > representable exactly. > > That's exactly what's happening, pigmartian. Thank you for explaining the reasoning behind this change. So am I relegated to building my own round() function that behaves like the original function? Or did they move the functionality to a new method somewhere for backwards-compatibility? -- http://mail.python.org/mailman/listinfo/python-list
Re: Concise way to format list/array to custom(hex) string
On Aug 2, 9:29 am, Grant Edwards <[EMAIL PROTECTED]> wrote:
> On 2008-08-02, Zoltán Nagy <[EMAIL PROTECTED]> wrote:
>
>
>
>
>
> > Kurien Mathew írta:
> >> Hello,
>
> >> What will be a concise & efficient way to convert a list/array.array of
> >> n elements into a hex string? For e.g. given the bytes
> >> [116, 111, 110, 103, 107, 97]
> >> I would like the formatted string
> >> 0x74 0x6f 0x6e 0x67 0x6b 0x61
>
> >> Is there an approach better than below:
> >> hex = ''
> >> for b in bytes:
> >> hex += ('0x%x '%b)
>
> >
>
> > You should avoid multiple string additions, as each one creates a new
> > string object (str objects are immutable). Try this:
>
> > bytes = [116, 111, 110, 103, 107, 97]
> > string = ''.join( ['0x%x '%b for b in bytes] )
>
> That results in an extra ' ' and the end of the string. Try
> this:
>
> string = ' '.join(['0x%02x' % b for b in bytes])
>
> --
> Grant Edwards grante Yow! ... Just enough
> at time to do my LIBERACE
> visi.com impression...
There is also a built-in hex() method in the Python Standard Library.
So, to make it cleaner:
' '.join(hex(x) for x in bytes)
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to ignore the first line of the text read from a file
On Aug 28, 3:47 am, Santiago Romero <[EMAIL PROTECTED]> wrote: > > I want to read text line-by-line from a text file, but want to ignore > > only the first line. I know how to do it in Java (Java has been my > > primary language for the last couple of years) and following is what I > > have in Python, but I don't like it and want to learn the better way > > of doing it. > > Why don't you read and discard the first line before processing the > rest of the file? > > file = open(filename, 'r') > file.readline() > for line in file: print line, > > (It works). # You could also do the following: from itertools import islice f = open(filename) for each_line in islice(f, 1, None): print line -- http://mail.python.org/mailman/listinfo/python-list
Re: How to check is something is a list or a dictionary or a string?
But this changes with Python 3, right? On Aug 30, 7:15 am, Ken Starks <[EMAIL PROTECTED]> wrote: > George Sakkis wrote: > > On Aug 29, 12:16 pm, [EMAIL PROTECTED] wrote: > >> Hi, > > >> How to check if something is a list or a dictionary or just a string? > >> Eg: > > >> for item in self.__libVerDict.itervalues(): > >> self.cbAnalysisLibVersion(END, item) > > >> where __libVerDict is a dictionary that holds values as strings or > >> lists. So now, when I iterate this dictionary I want to check whether > >> the item is a list or just a string? > > > if isinstance(item,basestring): > > # it's a string > > ... > > else: # it should be a list > > # typically you don't have to check it explicitly; > > # even if it's not a list, it will raise an exception later anyway > > if you call a list-specific method > > > HTH, > > George > > For a bit more explanation see, for example, > > http://evanjones.ca/python-utf8.html > > (Quote) > Working With Unicode Strings > > Thankfully, everything in Python is supposed to treat Unicode strings > identically to byte strings. However, you need to be careful in your own > code when testing to see if an object is a string. Do not do this: > > if isinstance( s, str ): # BAD: Not true for Unicode strings! > > Instead, use the generic string base class, basestring: > > if isinstance( s, basestring ): # True for both Unicode and byte strings -- http://mail.python.org/mailman/listinfo/python-list
SAXReaderNotAvailble: No parsers found
> Vincent Yau <[EMAIL PROTECTED]> writes:
> > I am trying to use Python SAX API to parse XML files. I do see expat.py
> > somewhere underneath my Python 2.1.1 installation (on Solaris).
> > But I got this error when invoking the xml.sax.make_parser() call. Any
> > tip/help much appreciated.
>
> You should install Expat before building Python. Best, you edit
> Modules/Setup to build pyexpat explicitly.
>
> Regards,
> Martin
Fast-forward to 2008
I installed Python 3.0b2 on a Windows Vista laptop (after having
previously installed Python 2.5), and I am getting this same error:
Traceback (most recent call last):
File "Programming\Python\monkeys.py", line 24, in
test_parse(sys.argv[1])
File "Programming\Python\monkeys.py", line 21, in test_parse
xml.sax.parse(f, handler)
File "C:\Python30\lib\xml\sax\__init__.py", line 30, in parse
parser = make_parser()
File "C:\Python30\lib\xml\sax\__init__.py", line 90, in make_parser
raise SAXReaderNotAvailable("No parsers found", None)
xml.sax._exceptions.SAXReaderNotAvailable: No parsers found
I see a pyexpat.lib in the C:\Python30\libs folder.
I also see a pyexpat.pyd in the C:\Python30\DLLs folder.
It works in Python 2.5. I installed Python 3.0b2 as admin.
Does anyone know what is wrong and how to fix it?
--
http://mail.python.org/mailman/listinfo/python-list
Python 3.0b2 cannot map '\u12b'
Hello, I am using Python 3.0b2. I have an XML file that has the unicode character '\u012b' in it, which, when parsed, causes a UnicodeEncodeError: 'charmap' codec can't encode character '\u012b' in position 26: character maps to This happens even when I assign this character to a reference in the interpreter: Python 3.0b2 (r30b2:65106, Jul 18 2008, 18:44:17) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> s = '\u012b' >>> s Traceback (most recent call last): File "", line 1, in File "C:\Python30\lib\io.py", line 1428, in write b = encoder.encode(s) File "C:\Python30\lib\encodings\cp437.py", line 19, in encode return codecs.charmap_encode(input,self.errors,encoding_map)[0] UnicodeEncodeError: 'charmap' codec can't encode character '\u012b' in position 1: character maps to Is this a known issue, or am I doing something wrong? Here is a link to the XML file. The character is on line 600, char 54 http://rubyquiz.com/SongLibrary.xml.gz -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 3.0b2 cannot map '\u12b'
On Sep 1, 8:19 am, Marc 'BlackJack' Rintsch <[EMAIL PROTECTED]> wrote: > On Mon, 01 Sep 2008 02:27:54 -0400, Terry Reedy wrote: > > I doubt the OP 'chose' cp437. Why does Python using cp437 even when the > > default encoding is utf-8? > > > On WinXP > > >>> sys.getdefaultencoding() > > 'utf-8' > > >>> s='\u012b' > > >>> s > > Traceback (most recent call last): > > File "", line 1, in > > File "C:\Program Files\Python30\lib\io.py", line 1428, in write > > b = encoder.encode(s) > > File "C:\Program Files\Python30\lib\encodings\cp437.py", line 19, in > > encode > > return codecs.charmap_encode(input,self.errors,encoding_map)[0] > > UnicodeEncodeError: 'charmap' codec can't encode character '\u012b' in > > position > > 1: character maps to > > Most likely because Python figured out that the terminal expects cp437. > What does `sys.stdout.encoding` say? > > > To put it another way, how can one 'choose' utf-8 for display to screen? > > If the terminal expects cp437 then displaying utf-8 might give some > problems. > > Ciao, > Marc 'BlackJack' Rintsch So, it is not a problem with the program, but a problem when I print it out. sys.stdout.encoding does say cp437. Now, when I don't print anything out, the program hangs. I will try this again and let the board know the results. Thanks for all of your insight. -- http://mail.python.org/mailman/listinfo/python-list
Re: Eleganz way to get rid of \n
On Sep 1, 9:25 am, Hans Müller <[EMAIL PROTECTED]> wrote:
> Hello,
>
> I'm quite often using this construct:
>
> for l in open("file", "r"):
> do something
>
> here, l contains the \n or \r\n on windows at the end.
> I get rid of it this way:
>
> for l in open("file", "r"):
> while l[-1] in "\r\n":
> l = l[:-1]
>
> I find this a little bit clumsy, but it works fine.
>
> Has someone a better solution ?
>
> Thanks
>
> Hans
Can you do this:
f = open(fname)
for x in f:
line = x.rstrip('\r\n')
--
http://mail.python.org/mailman/listinfo/python-list
Re: SAXReaderNotAvailble: No parsers found
On Aug 30, 8:59 pm, josh logan <[EMAIL PROTECTED]> wrote:
> > Vincent Yau <[EMAIL PROTECTED]> writes:
> > > I am trying to use Python SAX API to parse XML files. I do see expat.py
> > > somewhere underneath my Python 2.1.1 installation (on Solaris).
> > > But I got this error when invoking the xml.sax.make_parser() call. Any
> > > tip/help much appreciated.
>
> > You should install Expat before building Python. Best, you edit
> > Modules/Setup to build pyexpat explicitly.
>
> > Regards,
> > Martin
>
> Fast-forward to 2008
>
> I installed Python 3.0b2 on a Windows Vista laptop (after having
> previously installed Python 2.5), and I am getting this same error:
>
> Traceback (most recent call last):
> File "Programming\Python\monkeys.py", line 24, in
> test_parse(sys.argv[1])
> File "Programming\Python\monkeys.py", line 21, in test_parse
> xml.sax.parse(f, handler)
> File "C:\Python30\lib\xml\sax\__init__.py", line 30, in parse
> parser = make_parser()
> File "C:\Python30\lib\xml\sax\__init__.py", line 90, in make_parser
> raise SAXReaderNotAvailable("No parsers found", None)
> xml.sax._exceptions.SAXReaderNotAvailable: No parsers found
>
> I see a pyexpat.lib in the C:\Python30\libs folder.
> I also see a pyexpat.pyd in the C:\Python30\DLLs folder.
>
> It works in Python 2.5. I installed Python 3.0b2 as admin.
> Does anyone know what is wrong and how to fix it?
Does anyone have an answer for this?
I uninstalled both Python 2.5 and Python 3.0b2 and then re-installed
3.0b2, thinking that the installer was confusing 2.5 and 3.0b2 on
Windows Vista. Still have the same issue.
I had to use my XP machine, since the Vista installation seems broken
for Python 3.0b2. How do I fix this? How do I get Python to notice the
pyexpat.lib in the C:\Python30\DLLs folder in Vista?
Thanks
--
http://mail.python.org/mailman/listinfo/python-list
Re: Eleganz way to get rid of \n
On Sep 1, 9:41 am, Wojtek Walczak <[EMAIL PROTECTED]> wrote:
> On Mon, 01 Sep 2008 15:25:03 +0200, Hans Müller wrote:
> > I'm quite often using this construct:
>
> > for l in open("file", "r"):
> > do something
> > Has someone a better solution ?
>
> The most general would be to use rstrip() without
> arguments:
>
>
>
> >>> a="some string\r\n"
> >>> a.rstrip()
> 'some string'
>
> but be careful, because it will also cut whitespaces:
>
>
>
> >>> a="some string\t \r\n"
> >>> a.rstrip()
> 'some string'
>
> so maybe you could do this:
>
>
>
> >>> a.rstrip('\n').rstrip('\r')
> 'some string\t '
>
> HTH.
>
> --
> Regards,
> Wojtek Walczak,http://tosh.pl/gminick/
You can send both '\n' and '\r' in one rstrip call. No need for 2
separate calls.
--
http://mail.python.org/mailman/listinfo/python-list
Re: String/Number Conversion
On Sep 6, 5:04 pm, Andreas Hofmann <[EMAIL PROTECTED]>
wrote:
> Hello Folks!
>
> I've got a little problem here, which which really creeps me out at the
> moment.
> I've got some strings, which only contain numbers plus eventually one
> character as si-postfix (k for kilo, m for mega, g for giga). I'm trying
> to convert those strings to integers, with this function:
>
> def eliminate_postfix(value):
> if type(value) is str:
> value.upper()
> if value.endswith('K'):
> mult = 1000
> elif value.endswith('M'):
> mult = 100
> elif value.endswith('G'):
> mult = 10
> else:
> mult = 1
>
> if mult is 1:
> value = string.atoi(value)
> else:
> value = string.atoi(value[:-1]) * mult
> return value
>
> The problem is as follows: Everytime a string with a postfix should get
> converted, mult does not get set properly. It is always 1. Does anyone
> have an idea how to fix this? I just don't see it, maybe because I'm
> pretty new to python or because I'm just blind I would be really greatful.
>
> Kind regards,
> Andy
Hello,
1. You call value.upper(), but you do not capture the results of that
method. Strings are immutable in Python.
>> value = value.upper()
2. You should use == instead of "is" in the comparison of mult being
1.
>> if mult == 1:
--
http://mail.python.org/mailman/listinfo/python-list
Question about sorted in Python 3.0rc1
Hello,
I have 2 questions. Say I have this class:
class Player(object):
def __init__(self, fname, lname, score):
self.score = score
self.fname = fname
self.lname = lname
def __cmp__(self, other):
return (-cmp(self.score, other.score) or
cmp(self.lname, other.lname) or
cmp(self.fname, other.fname))
def __repr__(self):
return 'Player(fname={0.fname}, lname={0.lname},
score={0.score})'.format(self)
def __eq__(self, others):
if isinstance(other, Player):
return (self.score == other.score and
self.lname == other.lname and
self.fname == other.fname)
return False
def __ne__(self, others):
return not self.__eq__(others)
fnames = ['Julie', 'Ben', 'Jason', 'David']
lnames = ['Parks', 'Smith']
scores = [100, 95, 95, 130, 58, 74]
import itertools as it
score_iter = it.cycle(scores)
P = [Player(fn, ln, next(score_iter)) for fn in fnames for ln in
lnames]
cmp(P[0], P[1]) # returns -1
sorted(P) # throws TypeError: unorderable types Player() < Player()
The sorted function works when I define __lt__.
I must be misreading the documentation, because I read for the
documentation __cmp__ that it is called if none of the other rich
comparison functions are defined.
Is this a bug in Python 3.0rc1, or am I missing something?
Secondly, say that we suddenly need another sorting order, where we
want to sort by decreasing score and then by DECREASING last name
(instead of increasing last name, defined above). Now that the
comparison function argument is taken away from the sorted builtin,
how do we accomplish this with the "key" parameter?
Thank you
--
http://mail.python.org/mailman/listinfo/python-list
Re: Question about sorted in Python 3.0rc1
On Sep 22, 3:41 am, Arnaud Delobelle <[EMAIL PROTECTED]> wrote:
> On 22 Sep, 04:05, josh logan <[EMAIL PROTECTED]> wrote:
>
>
>
> > Hello,
>
> > I have 2 questions. Say I have this class:
>
> > class Player(object):
> > def __init__(self, fname, lname, score):
> > self.score = score
> > self.fname = fname
> > self.lname = lname
> > def __cmp__(self, other):
> > return (-cmp(self.score, other.score) or
> > cmp(self.lname, other.lname) or
> > cmp(self.fname, other.fname))
> > def __repr__(self):
> > return 'Player(fname={0.fname}, lname={0.lname},
> > score={0.score})'.format(self)
> > def __eq__(self, others):
> > if isinstance(other, Player):
> > return (self.score == other.score and
> > self.lname == other.lname and
> > self.fname == other.fname)
> > return False
> > def __ne__(self, others):
> > return not self.__eq__(others)
>
> > fnames = ['Julie', 'Ben', 'Jason', 'David']
> > lnames = ['Parks', 'Smith']
> > scores = [100, 95, 95, 130, 58, 74]
>
> > import itertools as it
>
> > score_iter = it.cycle(scores)
>
> > P = [Player(fn, ln, next(score_iter)) for fn in fnames for ln in
> > lnames]
>
> > cmp(P[0], P[1]) # returns -1
>
> > sorted(P) # throws TypeError: unorderable types Player() < Player()
>
> > The sorted function works when I define __lt__.
> > I must be misreading the documentation, because I read for the
> > documentation __cmp__ that it is called if none of the other rich
> > comparison functions are defined.
> > Is this a bug in Python 3.0rc1, or am I missing something?
>
> > Secondly, say that we suddenly need another sorting order, where we
> > want to sort by decreasing score and then by DECREASING last name
> > (instead of increasing last name, defined above). Now that the
> > comparison function argument is taken away from the sorted builtin,
> > how do we accomplish this with the "key" parameter?
>
> > Thank you
>
> I don't know about __cmp__ but for the second part of the question you
> can probably do:
>
> sorted(P, key=lambda p: (p.score, p.lname), reverse=True)
>
> HTH
>
> --
> Arnaud
Thank you for the prompt reply. I didn't think my second question
completely through.
A better example would be sorting by increasing last name and
decreasing first name. This would be easy with the sort function
comparator, but I can't see how to do the same with the key argument.
Is the only solution to decorate the Player objects in another class
that has the appropriate __cmp__ function (or whatever is needed) and
then retrieve the Player objects back?
--
http://mail.python.org/mailman/listinfo/python-list
Re: Question about sorted in Python 3.0rc1
On Sep 22, 7:32 am, Sion Arrowsmith <[EMAIL PROTECTED]> wrote: > josh logan <[EMAIL PROTECTED]> wrote: > > >sorted(P) # throws TypeError: unorderable types Player() < Player() > > >The sorted function works when I define __lt__. > >I must be misreading the documentation, because I read for the > >documentation __cmp__ that it is called if none of the other rich > >comparison functions are defined. > > You're either misreading or forgetting that __eq__ and __ne__, > which you define, are rich comparison functions. __cmp__ will only > be called for a comparison when *none* of the rich comparison > functions are defined, not just the one in question. > > -- > \S -- [EMAIL PROTECTED] --http://www.chaos.org.uk/~sion/ > "Frankly I have no feelings towards penguins one way or the other" > -- Arthur C. Clarke > her nu becomeþ se bera eadward ofdun hlæddre heafdes bæce bump bump bump Hello Sion, When I don't define the __eq__ and __ne__ comparison functions, the same unexpected behavior occurs. -- http://mail.python.org/mailman/listinfo/python-list
Re: Question about sorted in Python 3.0rc1
On Sep 22, 9:29 am, Hrvoje Niksic <[EMAIL PROTECTED]> wrote: > josh logan <[EMAIL PROTECTED]> writes: > > sorted(P) # throws TypeError: unorderable types Player() < Player() > > > The sorted function works when I define __lt__. > > I must be misreading the documentation, because I read for the > > documentation __cmp__ that it is called if none of the other rich > > comparison functions are defined. > > Is this a bug in Python 3.0rc1, or am I missing something? > > What documentation are you referring to, exactly? The whole __cmp__ > thing was supposed to be removed from Python 3, so mention of it > sounds like a documentation bug. > > > Secondly, say that we suddenly need another sorting order, where we > > want to sort by decreasing score and then by DECREASING last name > > (instead of increasing last name, defined above). Now that the > > comparison function argument is taken away from the sorted builtin, > > how do we accomplish this with the "key" parameter? > > By calling sort twice on the sequence: > > lst.sort(key=lambda x: x.score) > lst.sort(key=lambda x: x.last_name, reverse=True) > > As much as I like the key argument, I believe it was a mistake to > remove cmp, simply because it was the more general mechanism. > Emulating cmp with key is possible, but it requires creating a bunch > of objects for each sort. (I'm aware that list.sort caches the > calculated keys, but it still has to create as many of them as there > are list items.) It looks like __cmp__ is still in the documentation, and it seems to work somewhat in Python 3.0rc1. Here is the link to the documnetation http://docs.python.org/dev/3.0/reference/datamodel.html#object.__cmp__ -- http://mail.python.org/mailman/listinfo/python-list
Milenko Kindl bnvnvnvb
NEW YORK - Financial markets grew more upbeat Thursday as political leaders said they struck an agreement in principle on a massive spending plan to revive the crippled financial system. The Dow Jones industrial average jumped about 200 points on optimism about the bailout, and demand for safe-haven assets remained high but eased slightly as some investors placed bets that a deal would help unclog credit markets. ADVERTISEMENT Stock market investors got a lift when key lawmakers said they would present the $700 billion plan to the Bush administration and hoped for a vote by both houses of Congress within days. Still, some resistance remained from House Republicans as the closing bell on Wall Street rang ahead of a meeting of congressional leaders at the White House. And after the close of trading, it was clear that plan could still face some obstacles. Stock futures weakened, signaling a lower open Friday, after Sen. Richard Shelby, the top Republican on the Banking Committee, left the White House meeting and said the announced deal "is, obviously, no agreement." Trading that has been difficult for more than a week is likely to remain so in the coming days. "The market's going to experience volatility as the terms become known," said Doug Roberts, chief investment strategist at Channel Capital Research. Treasury Secretary Henry Paulson and Federal Reserve Chairman Ben Bernanke urged lawmakers Tuesday and Wednesday to quickly sign off on the plan, which they said would help prop up the economy by removing billions of dollars in risky mortgage-related assets from financial firms' balance sheets. Fear of heavy losses on these assets has made banks hesitant to extend credit, which in turn threatens the overall economy by making it harder and more expensive for businesses and consumers to borrow money. President Bush highlighted what he sees as the urgency in a national address Wednesday night. Major elements are still being worked out, including how to phase in the mammoth cost of the package and whether the government will get an ownership stake in troubled companies. Alan Lancz, director at investment research group LanczGlobal, said stock market investors were encouraged that the rescue looked more likely than it had earlier in the week. He said the move could help unclog credit markets by allowing banks and investors to place values on assets tied to mortgages. "How do you establish a floor? Well, this is the bazooka. This is how you establish a floor," he said of the plan's goal of buying up the toxic debt. Still, some investors had their doubts. Demand eased but remained high for the 3-month Treasury bill, considered the safest short-term investment. Its yield rose to 0.72 percent from 0.49 percent late Wednesday. That means investors are still willing to earn the slimmest of returns in exchange for a safe place to put their money. The yield on the benchmark 10-year Treasury note, which moves opposite its price, rose to 3.84 percent from 3.81 late Wednesday. The Dow rose 196.89, or 1.82 percent, to 11,022.06. The gain helped erase some of the losses from heavy selling earlier in the week, though the blue chips still remain down by more than 360 points, or 3.2 percent. Broader stock indicators also rose Thursday. The Standard & Poor's 500 index advanced 23.31, or 1.97 percent, to 1,209.18 and the Nasdaq composite index rose 30.89, or 1.43 percent, to 2,186.57. Advancing issues outnumbered decliners by nearly 3 to 1 on the New York Stock Exchange, where consolidated volume came to 5.73 billion shares, compared with 4.66 billion traded Wednesday. Roberts noted that the market's back-and-forth moves of late might be unnerving for investors but ultimately can leave stocks with little to show for all the volatility. "Most of this is just oscillating around a straight line," he said, noting that last week's huge daily moves, which also included triple- digit moves in the Dow, left stocks largely unchanged for the week. The dollar was mixed against other major currencies Thursday, while gold prices fell. Light, sweet crude for November delivery rose $2.29 to settle at $108.02 a barrel on the New York Mercantile Exchange. Meanwhile, disappointing readings on employment, housing and demand for big-ticket manufactured goods, as well as a sobering forecast from General Electric Co., underscored the difficulties facing the economy. The Labor Department said the number of people seeking unemployment benefits increased by 32,000 to a seasonally adjusted 493,000 last week — the highest level in seven years and well above analysts' expectations of 445,000. Hurricanes Ike and Gustav added about 50,000 new claims in Louisiana and Texas, the department said. The Commerce Department said sales of new homes fell sharply in August to the slowest pace in 17 years. The average sales price also fell by the largest amount on record. New homes sales dropped by 11.5 percent in August to a seasonally adjusted annua
I need your opinion...
It should be interesting to add new funcionality to "copytree" function into a "shutil.py" module?, I mean...I have developed a very silly function "copytree" with a different fourth argument. I have modified the "ignore" parameter to "target" parameter; this new one is used to copy the files wich matched with the given pattern. Basically, the opposite of the current "copytree". Maybe were a stupid idea but I throw it anyway. Thanks in advance. Logan!! PS: def copytree(src, dst, symlinks=False, target=None) def target_patterns(*patterns) -- http://mail.python.org/mailman/listinfo/python-list
Re: I need your opinion...
Thanks for the answer, it's perfect for my purposes. See you in other thread!!! On Mon, Dec 21, 2009 at 6:47 PM, Stephen Hansen wrote: > On Mon, Dec 21, 2009 at 8:23 AM, logan tag wrote: > >> It should be interesting to add new funcionality to "copytree" function >> into a "shutil.py" module?, I mean...I have developed a very silly function >> "copytree" with a different fourth argument. I have modified the "ignore" >> parameter to "target" parameter; this new one is used to copy the files wich >> matched with the given pattern. >> Basically, the opposite of the current "copytree". >> >> Maybe were a stupid idea but I throw it anyway. >> >> Thanks in advance. >> > > I don't understand what you're actually asking. Are you asking if its > possible to 'add' new functionality into shutil yourself/manually? If so, > yes, you can just 'shutil.mynewfunction = myfunction'. But doing so is > highly discouraged, it's best to simply put this function into your own > module and import that module when needed. > > Or are you asking if such a function would be useful as an addition to the > shutil in the standard library? If so, I'd say no: every "ignore" option can > trivially be turned into a "target" by simply inverting the test and > vice-versa. There's no reason to expand the stdlib's API and make people > choose between two functions which do the same thing in opposite directions. > > Meaning, an argument which defines "ignore=" can serve the purpose of > "target=" by simply choosing to ignore everything which doesn't match what > you want to target. > > Or you might be asking something else entirely, in which case I'm sorry, > I'm not sure what it is :) > > HTH, > > --S > -- http://mail.python.org/mailman/listinfo/python-list
question about a program
question about an assignment:
>>> places("home sweet home is here",' ')
[4, 10, 15, 18]
this is my code:
def places(x, y):
return [x.index(y) for v in x if (v == y)]
so far I'm only getting
[4, 4, 4, 4]
so the first value is correct, it is just not iterating on to the next
three items it needs to
--
http://mail.python.org/mailman/listinfo/python-list
HTMLParser not parsing whole html file
Hello,
I wanted to use python to scrub an html file for score data, but I'm
having trouble.
I'm using HTMLParser, and the parsing seems to fizzle out around line
192 or so. None of the event functions are being called anymore
(handle_starttag, handle_endtag, etc.) and I don't understand why,
because it is a html page over 1000 lines.
Could someone tell me if this is a bug or simply a misunderstanding on
how HTMLParser works? I'd really appreciate some help in
understanding.
I am using Python 3.1.2 on Windows 7 (hopefully shouldn't matter).
I put the HTML file on pastebin, because I couldn't think of anywhere
better to put it:
http://pastebin.com/wu6Pky2W
The source code has been pared down to the simplest form to exhibit
the problem. It is displayed below, and is also on pastebin for
download (http://pastebin.com/HxwRTqrr):
import sys
import re
import os.path
import itertools as it
import urllib.request
from html.parser import HTMLParser
import operator as op
base_url = 'http://www.dci.org'
class TestParser(HTMLParser):
def handle_starttag(self, tag, attrs):
print('position {}, staring tag {} with attrs
{}'.format(self.getpos(), tag, attrs))
def handle_endtag(self, tag):
print('ending tag {}'.format(tag))
def do_parsing_from_file_stream(fname):
parser = TestParser()
with open(fname) as f:
for num, line in enumerate(f, start=1):
# print('Sending line {} through parser'.format(num))
parser.feed(line)
if __name__ == '__main__':
do_parsing_from_file_stream(sys.argv[1])
--
http://mail.python.org/mailman/listinfo/python-list
Re: HTMLParser not parsing whole html file
On Oct 24, 4:36 pm, josh logan wrote:
> Hello,
>
> I wanted to use python to scrub an html file for score data, but I'm
> having trouble.
> I'm using HTMLParser, and the parsing seems to fizzle out around line
> 192 or so. None of the event functions are being called anymore
> (handle_starttag, handle_endtag, etc.) and I don't understand why,
> because it is a html page over 1000 lines.
>
> Could someone tell me if this is a bug or simply a misunderstanding on
> how HTMLParser works? I'd really appreciate some help in
> understanding.
>
> I am using Python 3.1.2 on Windows 7 (hopefully shouldn't matter).
>
> I put the HTML file on pastebin, because I couldn't think of anywhere
> better to put it:http://pastebin.com/wu6Pky2W
>
> The source code has been pared down to the simplest form to exhibit
> the problem. It is displayed below, and is also on pastebin for
> download (http://pastebin.com/HxwRTqrr):
>
> import sys
> import re
> import os.path
> import itertools as it
> import urllib.request
> from html.parser import HTMLParser
> import operator as op
>
> base_url = 'http://www.dci.org'
>
> class TestParser(HTMLParser):
>
> def handle_starttag(self, tag, attrs):
> print('position {}, staring tag {} with attrs
> {}'.format(self.getpos(), tag, attrs))
>
> def handle_endtag(self, tag):
> print('ending tag {}'.format(tag))
>
> def do_parsing_from_file_stream(fname):
> parser = TestParser()
>
> with open(fname) as f:
> for num, line in enumerate(f, start=1):
> # print('Sending line {} through parser'.format(num))
> parser.feed(line)
>
> if __name__ == '__main__':
> do_parsing_from_file_stream(sys.argv[1])
Sorry, the group doesn't like how i surrounded the Python code's
pastebin URL with parentheses:
http://pastebin.com/HxwRTqrr
--
http://mail.python.org/mailman/listinfo/python-list
Re: HTMLParser not parsing whole html file
On Oct 24, 4:38 pm, josh logan wrote:
> On Oct 24, 4:36 pm, josh logan wrote:
>
>
>
>
>
> > Hello,
>
> > I wanted to use python to scrub an html file for score data, but I'm
> > having trouble.
> > I'm using HTMLParser, and the parsing seems to fizzle out around line
> > 192 or so. None of the event functions are being called anymore
> > (handle_starttag, handle_endtag, etc.) and I don't understand why,
> > because it is a html page over 1000 lines.
>
> > Could someone tell me if this is a bug or simply a misunderstanding on
> > how HTMLParser works? I'd really appreciate some help in
> > understanding.
>
> > I am using Python 3.1.2 on Windows 7 (hopefully shouldn't matter).
>
> > I put the HTML file on pastebin, because I couldn't think of anywhere
> > better to put it:http://pastebin.com/wu6Pky2W
>
> > The source code has been pared down to the simplest form to exhibit
> > the problem. It is displayed below, and is also on pastebin for
> > download (http://pastebin.com/HxwRTqrr):
>
> > import sys
> > import re
> > import os.path
> > import itertools as it
> > import urllib.request
> > from html.parser import HTMLParser
> > import operator as op
>
> > base_url = 'http://www.dci.org'
>
> > class TestParser(HTMLParser):
>
> > def handle_starttag(self, tag, attrs):
> > print('position {}, staring tag {} with attrs
> > {}'.format(self.getpos(), tag, attrs))
>
> > def handle_endtag(self, tag):
> > print('ending tag {}'.format(tag))
>
> > def do_parsing_from_file_stream(fname):
> > parser = TestParser()
>
> > with open(fname) as f:
> > for num, line in enumerate(f, start=1):
> > # print('Sending line {} through parser'.format(num))
> > parser.feed(line)
>
> > if __name__ == '__main__':
> > do_parsing_from_file_stream(sys.argv[1])
>
> Sorry, the group doesn't like how i surrounded the Python code's
> pastebin URL with parentheses:
>
> http://pastebin.com/HxwRTqrr
I found the error. The HTML file I'm parsing has invalid HTML at line
193.
It has something like:
Note there is no space between the closing quote for the "href" tag
and the class attribute. I guess I'll go through each file and correct
these issues as I parse them.
Thanks for reading, anyways.
--
http://mail.python.org/mailman/listinfo/python-list
Total Beginner - Extracting Data from a Database Online (Screenshot)
Hey guys, I'm learning Python and I'm experimenting with different projects -- I like learning by doing. I'm wondering if you can help me here: http://i.imgur.com/KgvSKWk.jpg What this is is a publicly-accessible webpage that's a simple database of people who have used the website. Ideally what I'd like to end up with is an excel spreadsheet with data from the columns #fb, # vids, fb sent?, # email tm. I'd like to use Python to do it -- crawl the page and extract the data in a usable way. I'd love your input! I'm just a learner. -- http://mail.python.org/mailman/listinfo/python-list
Re: Total Beginner - Extracting Data from a Database Online (Screenshot)
Sorry to be unclear -- it's a screenshot of the webpage, which is publicly accessible, but it contains sensitive information. A bad combination, admittedly, and something that'll be soon fixed. -- http://mail.python.org/mailman/listinfo/python-list
Re: Total Beginner - Extracting Data from a Database Online (Screenshot)
On Saturday, May 25, 2013 6:33:25 PM UTC-7, John Ladasky wrote:
> On Friday, May 24, 2013 4:36:35 PM UTC-7, Carlos Nepomuceno wrote:
>
> > #to create the tables list
>
> > tables=[[re.findall('(.*?)',r,re.S) for r in
> > re.findall('(.*?)',t,re.S)] for t in
> > re.findall('(.*?)',page,re.S)]
>
> >
>
> >
>
> > Pretty simple.
>
>
>
> Two nested list comprehensions, with regex pattern matching?
>
>
>
> Logan did say he was a "total beginner." :^)
Oh goodness, yes, I have no clue.
--
http://mail.python.org/mailman/listinfo/python-list
