Re: Plotting in batch with no display
[email protected] writes: > It's possible to plot with matplotlib without a display. I'm not > surprised you didn't figure out how, though, it's not all that > obvious. http://matplotlib.sourceforge.net/faq/howto_faq.html#plotting-howto , | Generate images without having a window popup ` -- http://mail.python.org/mailman/listinfo/python-list
Re: Plotting in batch with no display
On Fri, 04 Jun 2010 22:27:33 -, [email protected] wrote: : It's possible to plot with matplotlib without a display. I'm not : surprised you didn't figure out how, though, it's not all that obvious. Thank you very much. That's a good start. Do you know of any good documentation on how to choose and use backends? What I find is very thin... : Check out the matplotlib.use function. For example: : : import matplotlib : matplotlib.use('agg') : import pylab : pylab.plot([1, 3, 5]) : fig = file('foo.png', 'wb') : pylab.savefig(fig, format='png') : fig.close() Raster graphics is not good enough, I will need a backend which does vector graphics and pdf output. AFAICS from the FAQ at sourceforge, agg only supports raster and png. Cairo supports vector graphics and PDF, but I cannot find any information about how/if it deals with X11... :-- George -- http://mail.python.org/mailman/listinfo/python-list
Re: Diff of Text
On 06/05/10 15:43, GZ wrote: > On Jun 4, 8:37 pm, Lie Ryan wrote: >> On06/05/10 07:51, GZ wrote: >>> No, rsync does not solve my problem. >> >>> I want a library that does unix 'diff' like function, i.e. compare two >>> strings line by line and output the difference. Python's difflib does >>> not work perfectly for me, because the resulting differences are >>> pretty big. I would like an algorithm that generates the smallest >>> differences. >> >> is n=0 not short enough? >> >> pprint.pprint(list(difflib.context_diff(s, t, n=0))) > > This still does not do what I want it to do. It only displays the diff > results in a different format. I want a different algorithm to > generate a smaller diff -- in other words less differences No, I meant I was confirming that you already have turned off context lines (i.e. the n=0 part), right? Also, what's the nature of the changes? You might be able to minimize difflib's output by using word-based or character-based diff-ing instead of traditional line-based diff-ing. diff output is fairly compressable, so you might want to look at zipping the output. -- http://mail.python.org/mailman/listinfo/python-list
Re: changing format of time duration.
On 4 jun, 06:14, "Günther Dietrich" wrote:
> GabrielGenellina wrote:
> >Try the strptime method with a suitable format, like this (untested):
> >delta = now2-now1
> >delta.strftime('%H:%M:%S.%f')
>
> Throws an exception:
>
> |Traceback (most recent call last):
> | File "", line 1, in
> |AttributeError: 'datetime.timedelta' object has no attribute 'strftime'
>
> What seems logical, since the documentation doesn't mention an strftime
> method for timedelta.
You're right. Second try (still untested):
def nice_timedelta_str(d):
result = str(d)
if result[1] == ':':
result = '0' + result
return result
delta = now2-now1
print nice_timedelta_str(delta)
--
Gabriel Genellina
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python Forum
On 06/05/10 12:34, John Bokma wrote: > Lie Ryan writes: > >> If you look at Stack Overflow, the highest voted questions are: >> >> - Hidden Features of C#? >> - What is the single most influential book every programmer should read? >> - What's your favorite "programmer" cartoon? >> - What is your best programmer joke? >> ... and so on >> >> many of them are nearly out-of-topic. > > What do you mean with out-of-topic? (off topic?) yeah, "off-topic", that's the word. > http://stackoverflow.com/questions/tagged/python > > But to be honest I mostly end up on Stack Overflow when I google for a > specific problem, and most of the time I find a nice concise answer > without much noise. Same here. But the point is, since Google bypasses the voting system, that's why I don't see much added value in having a voting system. -- http://mail.python.org/mailman/listinfo/python-list
Re: Diff of Text
On Fri, 04 Jun 2010 22:43:48 -0700, GZ wrote: > This still does not do what I want it to do. It only displays the diff > results in a different format. I want a different algorithm to generate > a smaller diff -- in other words less differences Can you give a *short* example, showing the output from Python's diff and what you would prefer instead? -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Replace in large text file ?
On Sat, 05 Jun 2010 00:53:23 -0700, Steve wrote:
> I am new to Python and am wanting to replace characters in a very large
> text file.6 GB
> In plain language what I wish to do is:
>
> Remove all comma's
> Replace all @ with comma's
> Save as a new file.
input_file = open("some_huge_file.txt", "r")
output_file = open("newfilename.txt", "w")
for line in input_file:
line = line.replace(",", "")
line = line.replace("@", ",")
output_file.write(line)
output_file.close()
input_file.close()
--
Steve
--
http://mail.python.org/mailman/listinfo/python-list
Moving Pickle data to Store in a Postgresql Database
Hi, I just need to know is there any existing module or library that could be used for storing pickle data fields into a Postgresql server database , I have found some answers to this about using with the SQlite3 for Python 2.5, but is it possible for Postgresql Server. ? Thanks Regards Jaideep -- http://mail.python.org/mailman/listinfo/python-list
Re: General questions - where & how
On 4 jun, 14:54, Terry Reedy wrote: > On 6/4/2010 1:35 PM, Philip Semanchuk wrote: > > On Jun 4, 2010, at 1:22 PM, Uriah Eisenstein wrote: > > >> I'm relatively new to Python and have a few questions. Frankly, it > >> took me a > >> while to find on python.org what seems like a suitable place to post my > >> questions. However, I'd like to check the archives and see if they > >> haven't > >> been discussed already... > > > I use Google's "site" keyword search. e.g. to search the archives for > > "banana": > >http://www.google.com/search?q=site%3Amail.python.org%2Fpipermail%2Fp... > > One can also search mailing lists mirrored and archived by > gmanehttp://search.gmane.org/ > This is gmane.comp.python.general. There are about 200 other g.c.python > lists/groups that are more specialized. I have no idea about comparitive > performance. In addition, Google Groups mirrors this list too as http://groups.google.com/group/comp.lang.python/ -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Python treats non-breaking space wrong?
It seems that Python treats non-breaking space (\xa0) as a normal whitespace character, e.g. when splitting a string. See below: >>> s='hello\xa0there' >>> s.split() ['hello', 'there'] Surely this is not intended behaviour? -- http://mail.python.org/mailman/listinfo/python-list
Re: Missing DLL in win98
On 4 jun, 19:47, Spyder42 wrote: > On Fri, 04 Jun 2010 14:03:48 -0400, Terry Reedy > wrote: > >On 6/4/2010 9:08 AM, Spyder42 wrote: > >> On Fri, 04 Jun 2010 14:50:28 +0200, Christian Heimes > > >>> Python 2.6 is not supported on Windows 98 and earlier. You need at least > >>> Windows 2000 with a recent service pack. > >> So your response is either, you don't know if there is a fix, or 'No > >> way in h377.' You couldn't figure out by my post that I already knew > >> that? > > >It was not obvious, without closely reading your original post, and even > >then it is not clear, that you *knew* than 2.6 was not supported on > >Win98. You could have asked 'I know 2.6+ is not officially supported in > >win98. Does anyone know a workaround other than upgrading windows or > >sticking with 2.5?". *That* would have been clear. > > I had a specific question and I got a non-specific non-answer. > If they didn't know, they should not have answered. You didn't state your question as clearly as you appear to think. > >It was not obvious, without closely reading your original post... > > So it WAS obvious to anyone who was PAYING ATTENTION? It is not obvious to me at least, even after closely reading your post. All I can deduce from it is that you assumed you would have a better chance upgrading your OS, not that you *knew* your current OS was officially unsupported. Christian Heimes gave you the right answer, even if it was not the answer you expected. -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Computer Tricks,Untold Shortcuts and Important facts
Know About Computer Tricks, Working Tricks, Untold Shortcuts and all Computer Facts .. . Visit my Website http://computertipsnfacts.blogspot.com http://amazingworld4all.blogspot.com Thank You, Santhosh... http://forexindianew.blogspot.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Python treats non-breaking space wrong?
On Sat, 05 Jun 2010 01:30:40 -0700, [email protected] wrote: > It seems that Python treats non-breaking space (\xa0) as a normal > whitespace character, e.g. when splitting a string. See below: > s='hello\xa0there' s.split() > ['hello', 'there'] > > Surely this is not intended behaviour? Yes it is. str.split() breaks on whitespace, and \xa0 is whitespace according to the Unicode standard. To put it another way, str.split() is not a word- wrapping split. This has been reported before, and rejected as a won't- fix. http://mail.python.org/pipermail/python-bugs-list/2006-January/031531.html -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: reading help() - newbie question
On 31 mayo, 07:19, Payal wrote: > When I type help(something) e.g. help(list), I see many methods like, > __methodname__(). Are these something special? How do I use them and why > put "__" around them? You may want to install and use "see", a human-friendly replacement of dir() So instead of this mess: py> dir(pencil_case) ['__add__', '__class__', '__contains__', '__delattr__', '__delitem__', ' __delslice__', '__doc__', '__eq__', '__ge__', '__getattribute__', '__get item__', '__getslice__', '__gt__', '__hash__', '__iadd__', '__imul__', ' __init__', '__iter__', '__le__', '__len__', '__lt__', '__mul__', '__ne__ ', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__rmul__', '__setattr__', '__setitem__', '__setslice__', '__str__', 'a ppend', 'count', 'extend', 'index', 'insert', 'pop', 'remove', 'reverse' , 'sort'] you get this instead: py> see(pencil_case) [] in ++= **= <<= == != >>= hash() help() iter() len()repr() reversed() str().append().count() .extend().index() .insert().pop() .remove().reverse() .sort() For us mere mortals, it's a lot more readable. "see" is available at http://github.com/inky/see -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: Py_single_input and the side-effects...
On 31 mayo, 08:11, moerchendiser2k3 wrote: > you are right, Python still holds the last > reference. I just set a dummy and thats it :) > > Can you tell me where did you get the information from? Do you mean the _ variable? It's in the tutorial: http://docs.python.org/tutorial/introduction.html#using-python-as-a-calculator -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: reading help() - newbie question
On 05/31/10 20:19, Payal wrote: > Hi, > I am trying to learn Python (again) and have some basic doubts which I > hope someone in the list can address. (English is not my first language and I > have no CS background except I can write decent shell scripts) > > When I type help(something) e.g. help(list), I see many methods like, > __methodname__(). Are these something special? How do I use them and why > put "__" around them? Yes, the double-underscore are hooks to the various python protocols. They defines, among all, operator overloading, class construction and initialization, iterator protocol, descriptor protocol, type-casting, etc. A typical usage of these double-underscore is to create a class that overrides these functions, e.g.: class Comparable(object): def __init__(self, value): self.value = value def __lt__(self, other): return self.value > other.value def __gt__(self, other): return self.value < other.value def __str__(self): return "Value: " + self.value You should never create your own double-underscore method, just override/use the ones that Python provide. > One more simple query. Many times I see something like this, > | D.iteritems() -> an iterator over the (key, value) items of D > What is this iterator they are talking about and how do I use these > methods because simly saying D.iteritems() does not work? > read about iterator protocol. Basically, the iterator protocol allows for-looping over a user-defined class (e.g. for emulating a collection). D.iteritems() returns an iterator object, which for-loop knows how to iterate over to generate the stream of (key, value) pairs. -- http://mail.python.org/mailman/listinfo/python-list
error in importing numpy
Hi, I am using ubuntu 9.10 . I just installed python 2.6.1 in /opt/python2.6 for using it with wingide for debugging symbols. I also installed numpy in python 2.6.1 using -- prefix method. but when i import numpy i get following error : ImportError: undefined symbol: _PyUnicodeUCS4_IsWhitespace please help me. Thanks in advance. Michell _ Hotmail: Powerful Free email with security by Microsoft. https://signup.live.com/signup.aspx?id=60969-- http://mail.python.org/mailman/listinfo/python-list
Re: error in importing numpy
On 12:26 pm, [email protected] wrote: Hi, I am using ubuntu 9.10 . I just installed python 2.6.1 in /opt/python2.6 for using it with wingide for debugging symbols. I also installed numpy in python 2.6.1 using -- prefix method. but when i import numpy i get following error : ImportError: undefined symbol: _PyUnicodeUCS4_IsWhitespace please help me. Your numpy is compiled for a UCS4 build of Python. But you have a UCS2 build of Python. Rebuild one of them to match up with the other. Jean-Paul -- http://mail.python.org/mailman/listinfo/python-list
is there a way to warn about missing modules *without* running python?
folks, hi, although i know the answer to this question, i'm having difficulty explaining it, to a user on the pyjamas list. i was therefore wondering if somebody could also provide an answer, on this list, to which i can refer the user. to make it clear: the user is confused as to why the pyjamas compiler (which strictly speaking isn't actually a compiler it's a "language translator", translating from one dynamic language into another dynamic language, feature-for-feature, concept-for-concept) is _not_ making any effort to help or advise the user at *COMPILE TIME* when they make the mistake of trying to import a module that does not exist. they are confused as to why they have to go to all the trouble of executing the [translated] javascript code in a web browser engine, where web browser engines are known to be a complete pain as far as debugging is concerned, just to find out that they made a mistake in mis-naming an import module. in other words, they are confusing the concept of "python as a dynamic language" with the concept of "compiler in the traditional static- language-such-as-c-or-c++ sense". i've tried explaining that the answer is because this has absolutely -all to do with pyjamas, and has everything to do with the http://python.org programming language, but i'm not getting through to them. if someone could perhaps explain this (in a different way from me), in the context of "python the programming language" and "python the http://python.org interpreter", i.e. having absolutely nothing to do with pyjamas, i would be most grateful, and it would benefit that user's understanding of python. many thanks, l. ref: http://groups.google.com/group/pyjamas-dev/browse_thread/thread/64cf948082bfec52?hl=en_US -- http://mail.python.org/mailman/listinfo/python-list
Tkinter help - Why this behavior ? (py3)
Hi,
let's consider this exemple :
from tkinter import *
from tkinter.ttk import *
class First:
def __init__(self):
self.root = Tk()
B = Button(self.root, command=self.op)
B.pack()
self.root.mainloop()
def op(self):
Second(self)
print("print")
class Second:
def __init__(self, parent):
root = Toplevel(parent.root)
root.grab_set()
root.mainloop()
First()
when I close the second window, the print is NOT executed. It's done
when I close the first window.
Why do it "freeze" my function?
Dorian
--
http://mail.python.org/mailman/listinfo/python-list
Re: is there a way to warn about missing modules *without* running python?
On Sat, 05 Jun 2010 06:42:44 -0700, lkcl wrote: > to make it clear: the user is confused as to why the pyjamas compiler > (which strictly speaking isn't actually a compiler it's a "language > translator", translating from one dynamic language into another dynamic > language, feature-for-feature, concept-for-concept) is _not_ making any > effort to help or advise the user at *COMPILE TIME* when they make the > mistake of trying to import a module that does not exist. Just because a module doesn't exist at compile time doesn't mean it won't exist later on when the code is run. And visa versa: just because the module exists when pyjamas "compiles" the code doesn't mean it will still be there when you actually try to run the code. [...] > in other words, they are confusing the concept of "python as a dynamic > language" with the concept of "compiler in the traditional static- > language-such-as-c-or-c++ sense". Neither Python, nor Javascript (as far as I know -- I welcome corrections) do static linking. If your Python code relies on a library, that library must be there *at runtime*, or you will get an error. This is no different from C or C++ code using dynamic linking. This is very common in e.g. Windows applications that use DLLs. Python modules are, conceptually speaking, like DLLs. They are loaded at runtime, not at compile time. This has many advantages over static linking, but some disadvantages as well. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Replace in large text file ?
Steve writes: > Remove all comma's > Replace all @ with comma's > Save as a new file. The simplest way is just copy the file one character at a time, making replacements to commas and @'s as stated. That will be a bit slow (especially in Python) but if you only have to do it once, just wait it out. -- http://mail.python.org/mailman/listinfo/python-list
Re: Replace in large text file ?
A module designed to do this is fileinput: http://docs.python.org/library/fileinput.html The approach is the same as the other except that it's in the standard library. 2010/6/5 Paul Rubin > Steve writes: > > Remove all comma's > > Replace all @ with comma's > > Save as a new file. > > The simplest way is just copy the file one character at a time, making > replacements to commas and @'s as stated. That will be a bit slow > (especially in Python) but if you only have to do it once, just wait it > out. > -- > http://mail.python.org/mailman/listinfo/python-list > -- http://mail.python.org/mailman/listinfo/python-list
Re: Replace in large text file ?
Steven D'Aprano wrote:
On Sat, 05 Jun 2010 00:53:23 -0700, Steve wrote:
I am new to Python and am wanting to replace characters in a very large
text file.6 GB
In plain language what I wish to do is:
Remove all comma's
Replace all @ with comma's
Save as a new file.
input_file = open("some_huge_file.txt", "r")
output_file = open("newfilename.txt", "w")
for line in input_file:
line = line.replace(",", "")
line = line.replace("@", ",")
output_file.write(line)
output_file.close()
input_file.close()
I'd probably process it in larger chunks:
CHUNK_SIZE = 1024 ** 2 # 1MB at a time
input_file = open("some_huge_file.txt", "r")
output_file = open("newfilename.txt", "w")
while True:
chunk = input_file.read(CHUNK_SIZE)
if not chunk:
break
chunk = chunk.replace(",", "")
chunk = chunk.replace("@", ",")
output_file.write(chunk)
output_file.close()
input_file.close()
--
http://mail.python.org/mailman/listinfo/python-list
Re: Plotting in batch with no display
On 2010-06-05, Hans Georg Schaathun wrote: > Raster graphics is not good enough, I will need a backend which > does vector graphics and pdf output. AFAICS from the FAQ at > sourceforge, agg only supports raster and png. Cairo supports > vector graphics and PDF, but I cannot find any information about > how/if it deals with X11... 1. I am not familiar with matlab; but, I have written python programs that generate plots using GNUplot (which can create PDFs). There are python modules available for working with GNUplot or you can open it yourself under a subprocess and send it commands through a pipe. GNUplot allows you to save files directly and does not therefore require X. 2. If you can generate a raster file with Matlab, you can use ImageMagick (which has python bindings or could be execed directly) or other conversion software to convert the file to whatever format you choose. 3. Depending on your requirements, you could generate a plot using postscript. Postscript is easy to convert directly to PDF as the two are closely related. As postscript is a powerful language itself you can actually write the graphing routines in postscript and use python to simply embed the data into the postscript. -- http://mail.python.org/mailman/listinfo/python-list
Re: Tkinter help - Why this behavior ? (py3)
* Dodo, on 05.06.2010 15:46:
Hi,
let's consider this exemple :
from tkinter import *
from tkinter.ttk import *
class First:
def __init__(self):
self.root = Tk()
B = Button(self.root, command=self.op)
B.pack()
self.root.mainloop()
def op(self):
Second(self)
print("print")
class Second:
def __init__(self, parent):
root = Toplevel(parent.root)
root.grab_set()
root.mainloop()
First()
when I close the second window, the print is NOT executed. It's done
when I close the first window.
Why do it "freeze" my function?
First, sorry about Thunderbird 3.x messing up the quoting of the code.
Don't know what they did to introduce all those bugs, but anyway, Thunderbird
3.x is an example that even seasoned programmers introduce an unbelievable
number of bugs, I think mostly just by repeating code patterns blindly.
In your code above you're doing as the TB programmers presumably did, repeating
a code pattern that you've seen has worked, without fully grokking it. The call
to 'mainloop' enters a loop. A button press causes your callback to be invoked
from within that loop, but your code then enters a new 'mainloop'.
Don't.
Except for modal dialogs the single top level 'mainloop' suffices (all it does
is to dispatch "messages" to "handlers", such as your button press callback).
So, just place a single call to 'mainloop' at the end of your program. Remove
the calls in 'First' and 'Second'.
Cheers & hth.,
- Alf
--
blog at http://alfps.wordpress.com>
--
http://mail.python.org/mailman/listinfo/python-list
Re: FIle transfer over network - with Pyro?
On Thu, 03 Jun 2010 20:05:15 +, exarkun wrote: > On 06:58 pm, [email protected] wrote: >>On Jun 3, 10:47 am, Nathan Huesken wrote: >>>Hi, >>> >>>I am writing a network application which needs from time to time do >>>file transfer (I am writing the server as well as the client). For >>>simple network messages, I use pyro because it is very comfortable. >>>But I suspect, that doing a file transfer is very inefficient over >>>pyro, am I right (the files are pretty big)? >>> >>>I somehow need to ensure, that the client requesting a file transfer is >>>the same client getting the file. So some sort of authentication is >>>needed. >>> >>>What library would you use to do the file transfer? Regards, >>>Nathan >> >>I've never used Pyro, but for a fast network file transfer in Python, >>I'd probably use the socket module directly, with a cache oblivious >>algorithm: >> http://en.wikipedia.org/wiki/Cache-oblivious_algorithm >> >>It doesn't use sockets, it uses files, but I recently did a Python >>progress meter application that uses a cache oblivious algorithm that >>can get over 5 gigabits/second throughput (that's without the network in >>the picture, though if it were used on 10 Gig-E with a suitable >>transport it could probably do nearly that), on a nearly-modern PC >>running Ubuntu with 2 cores It's at: >> http://stromberg.dnsalias.org/~strombrg/gprog/ . > > This seems needlessly complicated. Do you have a hard drive that can > deliver 5 gigabits/second to your application? More than likely not. Most such programs aren't optimized well for one machine, let alone adapting well to the cache-related specifics of about any transfer - so the thing you're using to measure performance, instead becomes the bottleneck itself. I don't think I'd use an oral thermometer that gave a patient a temporarily higher fever, and it'd be nice if I didn't have to retune the thermometer for each patient, too. Besides, it's a _conceptually_ simple algorithm - keep the n best- performing block sizes, and pick the best one historically for subsequent writes, trying a different, random blocksize once in a while even if things are going well with the current blocksize. It's actually something I learned about as an undergrad from a favorite professor, who was a little insistent that hard coding a "good" block size for the specifics of a single machine was short sighted when you care about performance, as code almost always moves to a different machine (or a different disk, or a different network peer) eventually. Come to think of it, she taught two of my 3 algorithms classes. Naturally, she also said that you shouldn't tune for performance unnecessarily. > A more realistic answer is probably to use something based on HTTP. This > solves a number of real-world problems, like the exact protocol to use > over the network, and detecting network issues which cause the transfer > to fail. It also has the benefit that there's plenty of libraries > already written to help you out. Didn't the OP request something fast? HTTP code is prone to be "optimized" for small transfers (if that), as most of the web is small files. OP: I should mention: If you're on gigabit or better, you probably should speak with your sysadmin about enabling Jumbo Frames and Path MTU Discovery - otherwise, even a cache oblivious algorithm likely won't be able to help much - the CPU would likely get pegged too early. If, on the other hand, you only care about 10BaseT speeds, or perhaps even 100BaseT speeds, HTTP would probably be fine (a typical CPU today can keep up with that fine), especially if you're doing a single transfer at a time. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Forum
Lie Ryan writes: > On 06/05/10 12:34, John Bokma wrote: [..] >> http://stackoverflow.com/questions/tagged/python >> >> But to be honest I mostly end up on Stack Overflow when I google for a >> specific problem, and most of the time I find a nice concise answer >> without much noise. > > Same here. But the point is, since Google bypasses the voting system, > that's why I don't see much added value in having a voting system. There is also voting on the answers ;-). -- John Bokma j3b Hacking & Hiking in Mexico - http://johnbokma.com/ http://castleamber.com/ - Perl & Python Development -- http://mail.python.org/mailman/listinfo/python-list
Re: Another MySQL Question
On Thu, Jun 3, 2010 at 4:09 PM, John Nagle wrote: > The real problem with this is not the Python. It's the approach > to SQL. What's going on here is that you have some collection of > named options that come in from some external source, and you're > trying to handle that by dynamically constructing the table schema. > > It would be easier to store the "option" values in a separate > table, with something like > >CREATE TABLE temp01 ( >entryid BIGINT NOT NULL AUTO_INCREMENT UNIQUE PRIMARY KEY, >Store VARCHAR(40) NOT NULL, >PatientID VARCHAR(40) NOT NULL, >ProdID VARCHAR(40) NOT NULL, >Pkg VARCHAR(10) NOT NULL) >ENGINE=INNODB; > >CREATE TABLE ouroptions ( >FOREIGN KEY (parententry) REFERENCES temp01(entryid) >ON DELETE CASCADE, >optionname VARCHAR(40) NOT NULL, >optionvalue VARCHAR(40) NOT NULL) >ENGINE=INNODB; > > This creates two tables which are locked together. For any entry > in "temp01", you can create any "option" entries you need in "ouroptions". > If you delete the entry in "temp01", the related "options" entries will go > away. > (That's what ON DELETE CASCADE does.) When inserting, insert the record in > temp01 first, then the recrods in "ouroptions", in a single transaction. > > Incidentally, if you don't specify some indices, lookups will > take a very long time. > Your solution is very elegant; however, the application here is inappropriate. I'd been contemplating creating a temp table like you suggest, but in this application, there is only need for one table. However, you've given me an idea that will no doubt come in helpful in the future ;) Thanks, beno -- http://mail.python.org/mailman/listinfo/python-list
Re: tallying occurrences in list
On Jun 4, 11:14 am, kj wrote:
> Task: given a list, produce a tally of all the distinct items in
> the list (for some suitable notion of "distinct").
>
> Example: if the list is ['a', 'b', 'c', 'a', 'b', 'c', 'a', 'b',
> 'c', 'a'], then the desired tally would look something like this:
>
> [('a', 4), ('b', 3), ('c', 3)]
>
> I find myself needing this simple operation so often that I wonder:
>
> 1. is there a standard name for it?
> 2. is there already a function to do it somewhere in the Python
> standard library?
>
> Granted, as long as the list consists only of items that can be
> used as dictionary keys (and Python's equality test for hashkeys
> agrees with the desired notion of "distinctness" for the tallying),
> then the following does the job passably well:
>
> def tally(c):
> t = dict()
> for x in c:
> t[x] = t.get(x, 0) + 1
> return sorted(t.items(), key=lambda x: (-x[1], x[0]))
>
> But, of course, if a standard library solution exists it would be
> preferable. Otherwise I either cut-and-paste the above every time
> I need it, or I create a module just for it. (I don't like either
> of these, though I suppose that the latter is much better than the
> former.)
>
> So anyway, I thought I'd ask. :)
>
> ~K
How about this one liner, if you prefer them;
set([(k,yourList.count(k)) for k in yourList])
--
http://mail.python.org/mailman/listinfo/python-list
Re: tallying occurrences in list
Sreenivas Reddy Thatiparthy writes: > How about this one liner, if you prefer them; > set([(k,yourList.count(k)) for k in yourList]) That has a rather bad efficiency problem if the list is large. -- http://mail.python.org/mailman/listinfo/python-list
Re: is there a way to warn about missing modules *without* running python?
On 6/5/2010 9:42 AM, lkcl wrote: if someone could perhaps explain this (in a different way from me), in the context of "python the programming language" and "python the http://python.org interpreter", i.e. having absolutely nothing to do with pyjamas, i would be most grateful, and it would benefit that user's understanding of python. I do not know the background of your user, but here is my try: Loading a Python module at runtime is like loading a song, picture, or web page. An import statement is like a web page link. If the name of the resource is misspelled, or if the resource is not available when requested, it cannot be loaded and the requesting progran can only report an error. ImportError is equivalent to http's "40x: not found" message. A web link that works one time may not work another time. Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Where's the List?
Hi;
I have this:
for order in order_details:
store = order[0]
prodid = order[1]
pkg = order[2]
quantity = order[3]
if 'PatientID' in order_fields:
patientID = order[4]
try:
option_values = order[5:]
except TypeError:
option_values = []
sql = 'insert into orders%s values (Null, %s)' % (store, ",
".join("%s" * (4 + len(option_values
cursor.execute(sql, tuple([pkg, prodid, tmpTable, quantity] +
option_values))
else:
patientID = ''
try:
option_values = order[4:]
except TypeError:
option_values = []
sql = 'insert into orders%s values (Null, %s)' % (store, ",
".join("%s" * (4 + len(option_values
#cursor.execute(sql, tuple([pkg, prodid, tmpTable, quantity,
order[4], order[5]]))
cursor.execute(sql, tuple([pkg, prodid, tmpTable, quantity] +
option_values))
It throws this error:
*TypeError*: can only concatenate list (not "tuple") to list
args = ('can only concatenate list (not "tuple") to list',)
Where's the list? They're both tuples in that last line of code.
TIA,
beno
--
http://mail.python.org/mailman/listinfo/python-list
Re: Where's the List?
Victor Subervi wrote:
> Where's the list? They're both tuples in that last line of code.
> for order in order_details:
> store = order[0]
> prodid = order[1]
> pkg = order[2]
> quantity = order[3]
> if 'PatientID' in order_fields:
> patientID = order[4]
> try:
> option_values = order[5:]
> except TypeError:
> option_values = []
Here...
> sql = 'insert into orders%s values (Null, %s)' % (store, ",
> ".join("%s" * (4 + len(option_values
> cursor.execute(sql, tuple([pkg, prodid, tmpTable, quantity] +
> option_values))
> else:
> patientID = ''
> try:
> option_values = order[4:]
> except TypeError:
> option_values = []
there...
> sql = 'insert into orders%s values (Null, %s)' % (store, ",
> ".join("%s" * (4 + len(option_values
> #cursor.execute(sql, tuple([pkg, prodid, tmpTable, quantity,
> order[4], order[5]]))
> cursor.execute(sql, tuple([pkg, prodid, tmpTable, quantity] +
> option_values))
>
> It throws this error:
>
> *TypeError*: can only concatenate list (not "tuple") to list
> args = ('can only concatenate list (not "tuple") to list',)
everywhere?
--
http://mail.python.org/mailman/listinfo/python-list
Re: Where's the List?
On 6/5/2010 1:17 PM Victor Subervi said...
cursor.execute(sql, tuple([pkg, prodid, tmpTable, quantity] +
option_values))
It throws this error:
*TypeError*: can only concatenate list (not "tuple") to list
args = ('can only concatenate list (not "tuple") to list',)
Where's the list? They're both tuples in that last line of code.
No they're not. That's what the error is saying.
--
http://mail.python.org/mailman/listinfo/python-list
Re: FIle transfer over network - with Pyro?
On Sat, Jun 5, 2010 at 10:14 AM, Dan Stromberg wrote: >> A more realistic answer is probably to use something based on HTTP. This >> solves a number of real-world problems, like the exact protocol to use >> over the network, and detecting network issues which cause the transfer >> to fail. It also has the benefit that there's plenty of libraries >> already written to help you out. > > Didn't the OP request something fast? Nope. He pointed out that pyro is not efficient and asked what libraries we would use. OP: HTTP is a reasonable choice unless you need really extreme performance. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Where's the List?
On Sat, Jun 5, 2010 at 1:17 PM, Victor Subervi wrote: > option_values = [] > Here you create a list. You also can get option_values by slicing your 'order' variable, and I assume that is actually producing a tuple. Otherwise, you wouldn't get the error you're getting. Either way, you go on to do some strangeness: cursor.execute(sql, tuple([pkg, prodid, tmpTable, quantity] + > option_values)) > I'm not at all sure why you're building a list, adding option_values to it, then trying to convert it all into a tuple. That's sorta a lot of wasted conversion going on. Splitting that single line up into multiple lines to clarify exactly what is going on: foo1 = [pkg, prodid, tmpTable, quantity] foo2 = foo1 + option_values foo3 = tuple(my_list) cur.execute(sql, foo3) Although its all on one line, those three separate actions are done in that order when said line is executed. The problem is on foo2. During the execution of foo2, my_list -- a list, not a tuple -- is there that you created, and you're adding option_values to it. The problem: option_values is a tuple, and you can't add a tuple to the end of a list. Why, you may ask? Because Python isn't sure what the result should be in such a case: a mutable list, or an immutable tuple? The operations "foo1.extend(option_values)" or foo1 += option_values would make it very clear that you're just adding stuff to the end of an existing list. But a + b returns a /new/ object, and Python won't guess if you want it to be a list or a tuple. Now, that said. The fix is easy. Broken down: foo1 = [pkg, prodid, tmpTable, quantity] foo2 = foo1 + list(option_values) cur.execute(sql, foo2) Alternately: foo1 = (pkg, prodid, tmpTable, quantity) foo2 = foo1 + option_values cur.execute(sql, foo2) And then condensed back into a single line: cur.execute(sql, [pkg, prodid, tmpTable, quantity] + list(option_values)) Or: cur.execute(sql, (pkg, prodid, tmpTable, quantity) + option_values) I removed the explicit conversion-to-tuple in the first, because... you don't need to. Execute takes a sequence. It doesn't have to be a tuple. A list is fine. In the second one, I just build my first arguments as a tuple instead of as a list, and then add option_values to it... You'll just have to go up top and do: option_values = () In both places where you currently use []. As you can't add a list to a tuple anymore then a tuple to a list, for the same reasons. HTH, --S -- http://mail.python.org/mailman/listinfo/python-list
pythonic web markup languages & templating systems & site generators
I started to review static blog-like HTML generators, and found myself overwhelmed with how many there were. I narrowed it down to pythonic ones, and then realized I had to pick a markup language and templating language, of which there are several pythonic implementations... The results of my inquiry (still ongoing) are here: http://www.subspacefield.org/~travis/static_blog_generators.html It's a little disorganized, but does represent a fair amount of work, and might be a nice intro for a python programmer who isn't up to his neck in web authoring tools. Suggestions on how to organize it are welcome. Of course, while writing it, I was struck by how much I actually needed one of these systems to write the reviews of the systems :-) writing raw HTML is teh suck. -- A Weapon of Mass Construction My emails do not have attachments; it's a digital signature that your mail program doesn't understand. | http://www.subspacefield.org/~travis/ If you are a spammer, please email [email protected] to get blacklisted. pgpnVFtgT60rk.pgp Description: PGP signature -- http://mail.python.org/mailman/listinfo/python-list
Re: pythonic web markup languages & templating systems & site generators
[email protected] writes: > systems to write the reviews of the systems :-) writing raw HTML > is teh suck. You might want to check out: http://code.google.com/p/zen-coding/ And thanks for your work, marked, and will read it later. -- John Bokma j3b Hacking & Hiking in Mexico - http://johnbokma.com/ http://castleamber.com/ - Perl & Python Development -- http://mail.python.org/mailman/listinfo/python-list
Re: reading help() - newbie question
On 06/05/10 21:24, Lie Ryan wrote: > On 05/31/10 20:19, Payal wrote: >> Hi, >> I am trying to learn Python (again) and have some basic doubts which I >> hope someone in the list can address. (English is not my first language and I >> have no CS background except I can write decent shell scripts) >> >> When I type help(something) e.g. help(list), I see many methods like, >> __methodname__(). Are these something special? How do I use them and why >> put "__" around them? > > Yes, the double-underscore are hooks to the various python protocols. > They defines, among all, operator overloading, class construction and > initialization, iterator protocol, descriptor protocol, type-casting, etc. > > A typical usage of these double-underscore is to create a class that > overrides these functions, e.g.: > > class Comparable(object): > def __init__(self, value): > self.value = value > def __lt__(self, other): > return self.value > other.value > def __gt__(self, other): > return self.value < other.value > def __str__(self): > return "Value: " + self.value > > You should never create your own double-underscore method, just > override/use the ones that Python provide. Ok, I just read what I wrote again and I noticed that the example isn't complete enough to illustrate what I'm talking about, so: class Comparable(object): def __init__(self, value): self.value = value def __lt__(self, other): return self.value > other.value def __gt__(self, other): return self.value < other.value def __str__(self): return "Value: " + self.value a = Comparable(10) # a.value = 10 b = Comparable(20) # b.value = 20 # the < operator calls __lt__ special method and this # prints False, because a.value > other.value is False print a < b # prints "Value: 10" since 'print' statement calls str() builtin # function which calls __str__ to turn objects into a string print a -- http://mail.python.org/mailman/listinfo/python-list
modify XMP data (Python/Windows)
I want to modify XMP data for a bunch of JPEG files, using Python if possible, on Windows. I expected PIL would support this. But no? I found the Python XMP Toolkit http://www.spacetelescope.org/static/projects/python-xmp-toolkit/docs/installation.html#requirements but no reports of successful use on Windows. I found Chilkat's XMP library http://www.chilkatsoft.com/python-xmp.asp but it is not open source and I found no encouraging reviews. Is there a Python package providing functinonality comparable to Image::ExifTool (perl)? Thanks, Alan Isaac -- http://mail.python.org/mailman/listinfo/python-list
Handling text lines from files with some (few) starnge chars
I need to read text files and process each line using string comparisions and regexp. I have a python2 program that uses .readline to read each line as a string. Then, processing it was a trivial job. With python3 I got error messagew like: File "./pp1.py", line 93, in RL line=inf.readline() File "/usr/lib64/python3.1/codecs.py", line 300, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) UnicodeDecodeError: 'utf8' codec can't decode bytes in position 4963-4965: invalid data How do I handle this? If I use .read from an open as binary file I got a object. Then how do I handle it? Reg exps, comparisions with strings, ?... Thanks for any help. -- http://mail.python.org/mailman/listinfo/python-list
Re: problems with CSV module
Carlos Grohmann wrote: > >Hi all, I'm using csv to read text files, and its working fine, except >in two cases: > >- when there is only one line of text (data) in the file >- when there is a blank line after the last data line >dialect = csv.Sniffer().sniff(sample) # Check for file format with >sniffer. csv.Sniffer uses heuristics to try to guess what the file format is. In order for those heuristics to work, the file must contain a sufficient number of lines, and the lines must actually have information that allows it to tell the difference between formats. It has been my experience that csv.Sniffer is NEVER worth the trouble. You know what the format is. Just embed the dialect yourself. -- Tim Roberts, [email protected] Providenza & Boekelheide, Inc. -- http://mail.python.org/mailman/listinfo/python-list
Re: Handling text lines from files with some (few) starnge chars
On Sat, Jun 5, 2010 at 4:03 PM, Paulo da Silva
wrote:
> I need to read text files and process each line using string
> comparisions and regexp.
>
> I have a python2 program that uses .readline to read each
> line as a string. Then, processing it was a trivial job.
>
> With python3 I got error messagew like:
> File "./pp1.py", line 93, in RL
> line=inf.readline()
> File "/usr/lib64/python3.1/codecs.py", line 300, in decode
> (result, consumed) = self._buffer_decode(data, self.errors, final)
> UnicodeDecodeError: 'utf8' codec can't decode bytes in position
> 4963-4965: invalid data
>
> How do I handle this?
Specify the encoding of the text when opening the file using the
`encoding` parameter. For Windows-1252 for example:
your_file = open("path/to/file.ext", 'r', encoding='cp1252')
Cheers,
Chris
--
http://blog.rebertia.com
--
http://mail.python.org/mailman/listinfo/python-list
Re: Handling text lines from files with some (few) starnge chars
Chris,
> Specify the encoding of the text when opening the file using the `encoding`
> parameter. For Windows-1252 for example:
>
> your_file = open("path/to/file.ext", 'r', encoding='cp1252')
This looks similar to the codecs module's functionality. Do you know if
the codecs module is still required in Python 3.x?
Thank you,
Malcolm
--
http://mail.python.org/mailman/listinfo/python-list
save xls to csv/dbf without Excel/win32com.client
Is there a way to save a .xls file (the first worksheet) as a .dbf or .csv without opening an instance of Excel with win32com.client (been awhile, is this the best module these days for v2.5)? In case a computer does not have Excel (2007) installed. -- http://mail.python.org/mailman/listinfo/python-list
vector addition
Hi, I am looking for a fast internal vector representation so that (a1,b2,c1)+(a2,b2,c2)=(a1+a2,b1+b2,c1+c2). So I have a list l = ['a'a,'bb','ca','de'...] I want to count all items that start with an 'a', 'b', and 'c'. What I can do is: count_a = sum(int(x[1]=='a') for x in l) count_b = sum(int(x[1]=='b') for x in l) count_c = sum(int(x[1]=='c') for x in l) But this loops through the list three times, which can be slow. I'd like to have something like this: count_a, count_b, count_c = sum( (int(x[1]=='a',int(x[1]=='b',int(x[1]=='c') for x in l) I hesitate to use numpy array, because that will literally create and destroy a ton of the arrays, and is likely to be slow. -- http://mail.python.org/mailman/listinfo/python-list
Re: save xls to csv/dbf without Excel/win32com.client
On 06/05/2010 06:47 PM, noydb wrote:
Is there a way to save a .xls file (the first worksheet) as a .dbf
or .csv without opening an instance of Excel with win32com.client
(been awhile, is this the best module these days for v2.5)? In case a
computer does not have Excel (2007) installed.
Use the "xlrd" module[1]
import csv
import xlrd
FILE_NAME = 'example.xls'
wb = xlrd.open_workbook(FILE_NAME)
for name in wb.sheet_names():
out = file('%s.csv' % name, 'wb')
writer = csv.writer(out)
sheet = wb.sheet_by_name(name)
for row in xrange(sheet.nrows):
writer.writerow([
sheet.cell_value(row, col)
for col in xrange(sheet.ncols)
])
out.close()
#
You say you only want the first sheet, so adjust accordingly.
-tkc
[1]
http://pypi.python.org/pypi/xlrd/
--
http://mail.python.org/mailman/listinfo/python-list
Re: Handling text lines from files with some (few) starnge chars
Em 06-06-2010 00:41, Chris Rebert escreveu:
> On Sat, Jun 5, 2010 at 4:03 PM, Paulo da Silva
> wrote:
...
>
> Specify the encoding of the text when opening the file using the
> `encoding` parameter. For Windows-1252 for example:
>
> your_file = open("path/to/file.ext", 'r', encoding='cp1252')
>
OK! This fixes my current problem. I used encoding="iso-8859-15". This
is how my text files are encoded.
But what about a more general case where the encoding of the text file
is unknown? Is there anything like "autodetect"?
--
http://mail.python.org/mailman/listinfo/python-list
Re: Diff of Text
Hi Lie, On Jun 5, 2:53 am, Lie Ryan wrote: > On 06/05/10 15:43, GZ wrote: > > > > > > > On Jun 4, 8:37 pm, Lie Ryan wrote: > >> On06/05/10 07:51, GZ wrote: > >>> No, rsync does not solve my problem. > > >>> I want a library that does unix 'diff' like function, i.e. compare two > >>> strings line by line and output the difference. Python's difflib does > >>> not work perfectly for me, because the resulting differences are > >>> pretty big. I would like an algorithm that generates the smallest > >>> differences. > > >> is n=0 not short enough? > > >> pprint.pprint(list(difflib.context_diff(s, t, n=0))) > > > This still does not do what I want it to do. It only displays the diff > > results in a different format. I want a different algorithm to > > generate a smaller diff -- in other words less differences > > No, I meant I was confirming that you already have turned off context > lines (i.e. the n=0 part), right? > > Also, what's the nature of the changes? You might be able to minimize > difflib's output by using word-based or character-based diff-ing instead > of traditional line-based diff-ing. > > diff output is fairly compressable, so you might want to look at zipping > the output.- Hide quoted text - > > - Show quoted text - Thanks for your response. The verboseness of the format is not really my problem and I only care about line by line comparison for now. Let me think of a better way to express what I mean by a "smaller diff." After I diff the two strings, I will have something like this: AAA - BBB + CCC + DDD - EEE It means the first line does not change, the second line is replaced by the third line, the forth line is new, and the fifth line is deleted. I define the "smallness" of the diff algorithm as "the sum of the total number of minuses and pluses". In my above example, it is 4 (two minuses and 2 pluses). Note that no matter what format we use to represent the diff, this number is the same. Python's difflib does not really minimize this number. It tries to make this number small, but also tries to yield matches that “look right” to people at the cost of increasing this number. (http:// docs.python.org/library/difflib.html). What I am looking for is an algo that can really minimize this number. -- http://mail.python.org/mailman/listinfo/python-list
Re: Diff of Text
GZ writes: > Let me think of a better way to express what I mean by a "smaller > diff." After I diff the two strings, I will have something like this: > > AAA > - BBB > + CCC > + DDD > - EEE > > It means the first line does not change, the second line is replaced > by the third line, the forth line is new, and the fifth line is > deleted. Are you drawing a distinction between: * “line FOO is replaced by line BAR” * “line FOO is deleted, line BAR is added” Your wording seems to make that distinction, but I don't see how it's useful or meaningful in a discussion about diff. Are they not exactly the same? -- \ “Injustice is relatively easy to bear; what stings is justice.” | `\ —Henry L. Mencken | _o__) | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: vector addition
On Sat, Jun 5, 2010 at 6:20 PM, GZ wrote:
> Hi,
>
> I am looking for a fast internal vector representation so that
> (a1,b2,c1)+(a2,b2,c2)=(a1+a2,b1+b2,c1+c2).
>
> So I have a list
>
> l = ['a'a,'bb','ca','de'...]
>
> I want to count all items that start with an 'a', 'b', and 'c'.
>
> What I can do is:
>
> count_a = sum(int(x[1]=='a') for x in l)
> count_b = sum(int(x[1]=='b') for x in l)
> count_c = sum(int(x[1]=='c') for x in l)
>
> But this loops through the list three times, which can be slow.
I don't really get how that relates to vectors or why you'd use that
representation, and it looks like you're forgotten that Python uses
0-based indexing, but anyway, here's my crack at something more
efficient:
from collections import defaultdict
cared_about = set('abc')
letter2count = defaultdict(int)
for item in l:
initial = item[0]
if initial in cared_about:
letter2count[initial] += 1
count_a = letter2count['a']
count_b = letter2count['b']
count_c = letter2count['c']
Cheers,
Chris
--
http://blog.rebertia.com
--
http://mail.python.org/mailman/listinfo/python-list
Re: vector addition
GZ wrote: Hi, I am looking for a fast internal vector representation so that (a1,b2,c1)+(a2,b2,c2)=(a1+a2,b1+b2,c1+c2). So I have a list l = ['a'a,'bb','ca','de'...] I want to count all items that start with an 'a', 'b', and 'c'. What I can do is: count_a = sum(int(x[1]=='a') for x in l) count_b = sum(int(x[1]=='b') for x in l) count_c = sum(int(x[1]=='c') for x in l) But this loops through the list three times, which can be slow. I'd like to have something like this: count_a, count_b, count_c = sum( (int(x[1]=='a',int(x[1]=='b',int(x[1]=='c') for x in l) I hesitate to use numpy array, because that will literally create and destroy a ton of the arrays, and is likely to be slow. If you want to do vector addition then numpy is the way to go. However, first you could try: from collections import defaultdict counts = defaultdict(int) for x in l: counts[x[0]] += 1 (Note that in Python indexes are zero-based.) -- http://mail.python.org/mailman/listinfo/python-list
[RELEASE] Python 2.7 release candidate 1 released
On behalf of the Python development team, I'm effusive to announce the first release candidate of Python 2.7. Python 2.7 is scheduled (by Guido and Python-dev) to be the last major version in the 2.x series. However, 2.7 will have an extended period of bugfix maintenance. 2.7 includes many features that were first released in Python 3.1. The faster io module, the new nested with statement syntax, improved float repr, set literals, dictionary views, and the memoryview object have been backported from 3.1. Other features include an ordered dictionary implementation, unittests improvements, a new sysconfig module, and support for ttk Tile in Tkinter. For a more extensive list of changes in 2.7, see http://doc.python.org/dev/whatsnew/2.7.html or Misc/NEWS in the Python distribution. To download Python 2.7 visit: http://www.python.org/download/releases/2.7/ While this is a preview release and is thus not suitable for production use, we strongly encourage Python application and library developers to test the release with their code and report any bugs they encounter to: http://bugs.python.org/ This helps ensure that those upgrading to Python 2.7 will encounter as few bumps as possible. 2.7 documentation can be found at: http://docs.python.org/2.7/ Enjoy! -- Benjamin Peterson Release Manager benjamin at python.org (on behalf of the entire python-dev team and 2.7's contributors) -- http://mail.python.org/mailman/listinfo/python-list
Re: Diff of Text
GZ wrote: I want a library that does unix 'diff' like function, i.e. compare two strings line by line and output the difference. Python's difflib does not work perfectly for me, because the resulting differences are pretty big. I would like an algorithm that generates the smallest differences. Thanks for your response. The verboseness of the format is not really my problem and I only care about line by line comparison for now. Let me think of a better way to express what I mean by a "smaller diff." After I diff the two strings, I will have something like this: AAA - BBB + CCC + DDD - EEE It means the first line does not change, the second line is replaced by the third line, the forth line is new, and the fifth line is deleted. I define the "smallness" of the diff algorithm as "the sum of the total number of minuses and pluses". In my above example, it is 4 (two minuses and 2 pluses). Note that no matter what format we use to represent the diff, this number is the same. Python's difflib does not really minimize this number. It tries to make this number small, but also tries to yield matches that “look right” to people at the cost of increasing this number. (http:// docs.python.org/library/difflib.html). What I am looking for is an algo that can really minimize this number. The - lines aren't needed, any more than the context lines are needed, so that will typically cut your results in half. But perhaps the real algorithm you're describing is one that permits lines to be moved as well as inserted and deleted. If you then consider that information to be free (it's not part of the measure you're specifying), you may do best with the following algorithm: Scan the two files looking for lines that appear exactly once in each file. Consider those lines the first invariants, and everything else a potential change. Now, starting with each pair (which I call a bridge between islands), look at the line previous and the line following, and if either or both are also matched, expand the two islands to include them. As an island grows to butt up against another island, merge them. Now, each bridge is a "move" operation, and every line left over in either file is either a plus or a minus, an insert or a delete. For most editing transactions, there will be relatively few of these. For example, if three functions were reordered, from A, B, C, to A, C, B, there would be no plus or minus lines at all. DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: An empty object with dynamic attributes (expando)
Right. >>> m = lambda:expando >>> m.myattr = 1 >>> print m.myattr 1 -- Cheers, Dmitry -- http://mail.python.org/mailman/listinfo/python-list
Re: Handling text lines from files with some (few) starnge chars
Paulo da Silva wrote:
Em 06-06-2010 00:41, Chris Rebert escreveu:
On Sat, Jun 5, 2010 at 4:03 PM, Paulo da Silva
wrote:
...
Specify the encoding of the text when opening the file using the
`encoding` parameter. For Windows-1252 for example:
your_file = open("path/to/file.ext", 'r', encoding='cp1252')
OK! This fixes my current problem. I used encoding="iso-8859-15". This
is how my text files are encoded.
But what about a more general case where the encoding of the text file
is unknown? Is there anything like "autodetect"?
>
An encoding like 'cp1252' uses 1 byte/character, but so does 'cp1250'.
How could you tell which was the correct encoding?
Well, if the file contained words in a certain language and some of the
characters were wrong, then you'd know that the encoding was wrong. This
does imply, though, that you'd need to know what the language should
look like!
You could try different encodings, and for each one try to identify what
could be words, then look them up in dictionaries for various languages
to see whether they are real words...
--
http://mail.python.org/mailman/listinfo/python-list
Announcing: WHIFF 1.0
WHIFF 1.0 RELEASED WHIFF 1.0 is the first stable release of WHIFF intended to be production ready. PROJECT PAGE: http://whiff.sourceforge.net/ DOCUMENTATION: http://whiffdoc.appspot.com DOWNLOAD: https://sourceforge.net/projects/whiff/ REPOSITORY: http://code.google.com/p/whiff/source/checkout WHIFF [WSGI HTTP Integrated Filesystem Frames] is a collection of support services for Python/WSGI Web applications which allows applications to be composed by "dropping" dynamic pages into container directories. This mode of development will be familiar to developers who have created PHP applications, vanilla CGI scripts, Apache/modpy Publisher applications, JSP pages, or static web content. WHIFF 1.0 adds support for Guarded Resources to implement information access control in web applications. http://whiffdoc.appspot.com/docs/W1100_2400.AuthorizedResources In addition to a number of bugfixes and minor enhancements the release also includes from previous releases: Test drive http://whiffdoc.appspot.com/docs/W1100_0500.TestDrive Mako template support http://whiffdoc.appspot.com/docs/W1100_1075.MakoGrading Drop down menus middlewares http://whiffdoc.appspot.com/docs/W1100_1300.menus AJAX callback support http://whiffdoc.appspot.com/docs/W1100_1400.calc Jquery helpers http://whiffdoc.appspot.com/docs/W1100_1450.jQueryUI Integration support for repoze.who authentication http://whiffdoc.appspot.com/docs/W1100_1500.who Open Flash Charts http://whiffdoc.appspot.com/docs/W1100_1600.openFlashCharts Internationalization support http://whiffdoc.appspot.com/docs/W1100_1700.international Tree views with AJAX callbacks http://whiffdoc.appspot.com/docs/W1100_2200.TreeView Google App Engine compatibility http://whiffdoc.appspot.com/docs/W1100_2300.GAEDeploy And much more. Please try it out and let me know what you think -- Aaron Watters === This one goes to eleven. -- http://mail.python.org/mailman/listinfo/python-list
GUIs - A Modest Proposal
I get the strong feeling that nobody is really happy with the state of Python GUIs. Tkinter is not widely liked, but is widely distributed. WxPython and PyGtk are both powerful, but quirky in different ways. PyQt is tied to one platform. And there are dozens more. Whether or not we like graphics programming, it's not going to go away. I get the uneasy feeling whenever I start a new project that there should be a 'better' GUI than the ones I currently use (WxPython and PyGtk). Fragmentation is our enemy. Our resources are being dissipated. Is it not time to start again? We have shown that it is possible to do the right thing, by creating Python3. I ask the group; should we try to create a new GUI for Python, with the following properties?: - Pythonic - The default GUI (so it replaces Tkinter) - It has the support of the majority of the Python community - Simple and obvious to use for simple things - Comprehensive, for complicated things - Cross-platform - Looks good (to be defined) - As small as possible in its default form If so, what are the next steps? The Python SIG on GUIs closed years ago. Should that be revived? This is "A Modest Proposal" (J. Swift). In a sense, I am suggesting that we eat our own babies. But don't we owe it to the community? -- http://mail.python.org/mailman/listinfo/python-list
Re: Replace in large text file ?
On 5 June, 08:53, Steve wrote: > I am new to Python and am wanting to replace characters in a very > large text file.6 GB > In plain language what I wish to do is: > > Remove all comma's > Replace all @ with comma's > Save as a new file. > > Any of you clever people know the best way to do this..idiot guide > please. > > Thanks > > Steve Many thanks for your suggestions. sed -i 's/Hello/hello/g' file Run twice on the CL..with the hello's changed for my needs did it in a few minutes , Again thanks Steve -- http://mail.python.org/mailman/listinfo/python-list
Re: GUIs - A Modest Proposal
On Sat, Jun 5, 2010 at 7:22 PM, ant wrote: > I get the strong feeling that nobody is really happy with the state of > Python GUIs. > Tkinter is not widely liked, but is widely distributed. WxPython and > PyGtk are both > powerful, but quirky in different ways. PyQt is tied to one platform. > And there are > dozens more. > > Whether or not we like graphics programming, it's not going to go > away. I get the > uneasy feeling whenever I start a new project that there should be a > 'better' GUI > than the ones I currently use (WxPython and PyGtk). > > Fragmentation is our enemy. Our resources are being dissipated. Is it > not time to > start again? We have shown that it is possible to do the right thing, > by creating Python3. > > I ask the group; should we try to create a new GUI for Python, with > the following > properties?: > > - Pythonic > - The default GUI (so it replaces Tkinter) > - It has the support of the majority of the Python community > - Simple and obvious to use for simple things > - Comprehensive, for complicated things > - Cross-platform > - Looks good (to be defined) > - As small as possible in its default form > > If so, what are the next steps? > > The Python SIG on GUIs closed years ago. Should that be revived? > > This is "A Modest Proposal" (J. Swift). In a sense, I am suggesting > that > we eat our own babies. > > But don't we owe it to the community? No. http://www.cosc.canterbury.ac.nz/greg.ewing/python_gui/ Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Diff of Text
On Jun 5, 8:42 pm, Ben Finney wrote: > GZ writes: > > Let me think of a better way to express what I mean by a "smaller > >diff." After Idiffthe two strings, I will have something like this: > > > AAA > > - BBB > > + CCC > > + DDD > > - EEE > > > It means the first line does not change, the second line is replaced > > by the third line, the forth line is new, and the fifth line is > > deleted. > > Are you drawing a distinction between: > > * “line FOO is replaced by line BAR” > * “line FOO is deleted, line BAR is added” > > Your wording seems to make that distinction, but I don't see how it's > useful or meaningful in a discussion aboutdiff. Are they not exactly > the same? > > -- > \ “Injustice is relatively easy to bear; what stings is justice.” | > `\ —Henry L. Mencken | > _o__) | > Ben Finney I should distinguish between modifications and additions. In my above example, one line is modified/replaced, one line is added and one line is deleted. There are a total of 3 edits. I am looking for an alternative python library other than difflib that minimizes this number (edit distance). -- http://mail.python.org/mailman/listinfo/python-list
Re: GUIs - A Modest Proposal
ant uklinux.net> writes: > PyQt is tied to one platform. What do you mean one platform? -- http://mail.python.org/mailman/listinfo/python-list
Re: Handling text lines from files with some (few) starnge chars
On Jun 6, 12:14 pm, MRAB wrote:
> Paulo da Silva wrote:
> > Em 06-06-2010 00:41, Chris Rebert escreveu:
> >> On Sat, Jun 5, 2010 at 4:03 PM, Paulo da Silva
> >> wrote:
> > ...
>
> >> Specify the encoding of the text when opening the file using the
> >> `encoding` parameter. For Windows-1252 for example:
>
> >> your_file = open("path/to/file.ext", 'r', encoding='cp1252')
>
> > OK! This fixes my current problem. I used encoding="iso-8859-15". This
> > is how my text files are encoded.
> > But what about a more general case where the encoding of the text file
> > is unknown? Is there anything like "autodetect"?
>
> >
> An encoding like 'cp1252' uses 1 byte/character, but so does 'cp1250'.
> How could you tell which was the correct encoding?
>
> Well, if the file contained words in a certain language and some of the
> characters were wrong, then you'd know that the encoding was wrong. This
> does imply, though, that you'd need to know what the language should
> look like!
>
> You could try different encodings, and for each one try to identify what
> could be words, then look them up in dictionaries for various languages
> to see whether they are real words...
This has been automated (semi-successfully, with caveats) by the
chardet package ... see http://chardet.feedparser.org/
--
http://mail.python.org/mailman/listinfo/python-list
Re: Handling text lines from files with some (few) starnge chars
Em 06-06-2010 04:05, John Machin escreveu: > On Jun 6, 12:14 pm, MRAB wrote: >> Paulo da Silva wrote: ... >>> OK! This fixes my current problem. I used encoding="iso-8859-15". This >>> is how my text files are encoded. >>> But what about a more general case where the encoding of the text file >>> is unknown? Is there anything like "autodetect"? >> ... > > This has been automated (semi-successfully, with caveats) by the > chardet package ... see http://chardet.feedparser.org/ This seems nice! Thanks -- http://mail.python.org/mailman/listinfo/python-list
Re: GUIs - A Modest Proposal
On Sat, 2010-06-05 at 19:22 -0700, ant wrote: > I get the strong feeling that nobody is really happy with the state of > Python GUIs. > WxPython and PyGtk are both powerful, but quirky in different ways. All widget libraries are quirky - because sophisticated user interfaces are complicated. If you make a new one that is sophisticated enough to be really useful - it will be "quirky". > PyQt is tied to one platform. No it isn't. > Whether or not we like graphics programming, it's not going to go > away. I get the > uneasy feeling whenever I start a new project that there should be a > 'better' GUI > than the ones I currently use (WxPython and PyGtk). > > Fragmentation is our enemy. So fragment some more? And at least PyGtk is a wrapper around Gtk, so in a sense that is anti-fragmentation. It is reusing Gtk which is also reused as Gtk# by Mono/.NET and other bindings. > Our resources are being dissipated. Is it not time to start again? No. > I ask the group; should we try to create a new GUI for Python, with > the following properties?: > - Pythonic > - The default GUI (so it replaces Tkinter) > - It has the support of the majority of the Python community > - Simple and obvious to use for simple things > - Comprehensive, for complicated things > - Cross-platform > - Looks good (to be defined) > - As small as possible in its default form Good luck. Seems pointless to me. > But don't we owe it to the community? Seems like there are already several very mature options. -- http://mail.python.org/mailman/listinfo/python-list
Re: GUIs - A Modest Proposal
On 06/05/2010 08:22 PM, ant wrote: > WxPython and PyGtk are both powerful, but quirky in different ways. > PyQt is tied to one platform. And there are dozens more. In what way is PyQt (or the new PySide bindings) tied to one platform? PyQt is "native" on Win32, Mac, and Linux. Would your universal GUI be any less quirky? > I ask the group; should we try to create a new GUI for Python, with > the following properties?: > > - Comprehensive, for complicated things - Cross-platform Most GUI toolkits currently are, to some degree or another. Qt is the most comprehensive cross-platform toolkit that I know of. You can pretty much do any application operation using its API. > - Looks good (to be defined) Does that mean it looks native? Should it be native? Does not the Tkinter gui look "good?" I can think of at least the following reasons why a new universal GUI for Python will have acceptance issues: - stuck with the lowest common denominator of functionality on each platform if you thunk to native widgets (a la wxWidgets) - often look and feel is not quite native even when using native themes (GTK on windows, for example) - if you take the Java Swing approach, you'll look out of place everywhere, which is kind of where tkinter is now. -- http://mail.python.org/mailman/listinfo/python-list
Re: Diff of Text
GZ wrote: > I should distinguish between modifications and additions. In my above > example, one line is modified/replaced, one line is added and one line > is deleted. There are a total of 3 edits. I am looking for an > alternative python library other than difflib that minimizes this > number (edit distance). Since you know the lingo, "edit distance", I expect you've already Googled up the general results. Depending on what kinds of edits we count, the problem of finding a minimal diff can be low-order poly- time, NP-hard, or incomputable. Gonzalo Navarro's 2001 survey, "A Guided Tour to Approximate String Matching", is now available on-line for free: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.96.7225&rep=rep1&type=pdf The paper notes, "issues have been put aside to keep a reasonable scope," but presents more than most of us ever wanted to know about the subject. I don't know of a module that does what GZ asks. There are scores of line-oriented diff implementations, and there are minimal-edit- distance diffs, but the combination is rare at best. Problem domains that call for a true minimal diff tend not to work in units of lines of text. -- --Bryan -- http://mail.python.org/mailman/listinfo/python-list
Python Imaging Library available for Python 3.1 ? Fractals
On the site http://code.activestate.com/recipes/langs/python/ there are several scripts for fractals. See page five. These begin from PIL import Image This fails in my python 3.1.2 Google reveals PIL is Python Imaging Library from pythonware.com According to their website PIL is not available beyond 2.6 : ( Do I need a full GUI like tkinter to image a fractal ( on my python 3.1.2 ) ?Is there any simpler option ? Thanks, Dave WB3DWE -- http://mail.python.org/mailman/listinfo/python-list
"python setup.py sdist" does pick one directory
Hello All, The Selenium setup.py can be found at http://code.google.com/p/selenium/source/browse/trunk/setup.py. When running "python setup.py sdist" the "firefox/test/py" directory is ignored for some reason though it's mentioned in the "package_dir" and in "packages". Any ideas why it's ignored? Thanks, -- Miki -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Forum
In article , Monte Milanuk wrote: > >Decent NNTP access is harder to find. Not impossible, but no longer >a 'free' part of most standard ISP access any more. This seems like a good time to promote my ISP: panix.com -- Aahz ([email protected]) <*> http://www.pythoncraft.com/ "If you don't know what your program is supposed to do, you'd better not start writing it." --Dijkstra -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Forum
In article <[email protected]>, Steven D'Aprano wrote: > >I'm sorry for all you people who don't live in a place with a genuinely >free market, and instead have to suffer with the lack of competition and >poor service of a monopoly or duopoly masquerading as a free market. But >*my* point was that your woes are not universal, and Usenet is alive and >well. It might be declining, but it's a long, slow decline and, like >Cobol, it will probably still be around a decade after the cool kids >declared it dead. Your position is the same as mine as of about two weeks ago, before someone sent this to me: http://news.duke.edu/2010/05/usenet.html Now I think that if even a top-tier educational institution isn't willing to serve as a living museum for a technology it created, maybe the death of Usenet is closer than I'd like to think. :-( Sucks because nothing replaces a good netnews client. -- Aahz ([email protected]) <*> http://www.pythoncraft.com/ "If you don't know what your program is supposed to do, you'd better not start writing it." --Dijkstra -- http://mail.python.org/mailman/listinfo/python-list
