Re: Simple addition to random module - Student's t

[Terry Reedy]

Robert Kern wrote:

On 2009-09-02 14:15 PM, Raymond Hettinger wrote:

On Sep 2, 6:51 am, Thomas Philips  wrote:

While the random module allows one to generate randome numbers with a
variety of distributions, some useful distributions are omitted - the
Student's t being among them.


I'm curious to hear what your use cases are.

My understanding is that t-distribution is an estimation tool
used with small samples of a population where the variance or
standard deviation is unknown.

So, when do you ever need to generate random variables with
this distribution?  ISTM that this is akin to wanting
a generator for a Kolmogorov distribution -- usually the
distribution is used to test empirical data, not to generate it.


In more complicated models, estimates of one parameter need to be 
propagated through the model, particularly if you are looking at 
sensitivity to parameters. Student's t describes the variation of an 
estimate of a mean of a sample from a Gaussian distribution. If I were 
modeling a processing wherein someone makes an estimate of a mean and 
then acts on that estimate, I would want to generate random t variates 
to feed that model.



I think most of the existing generators were chosen because they
are useful in simulation programs.  AFAICT, the Student's t-
distribution
doesn't fall into that category (usually, you know the population
standard deviation when you're the one generating data).


Student's t distribution is also used as a sort of generic fat-tailed 
distribution in some models and is not tied to the "estimate of a mean" 
description.


Yes. One can run the simulation with various df's to see the effect of 
such data and possibly how and when a model breaks.



ISTM, there ought to be a statistics module that can calculate
cumulative distribution functions for a variety of distributions.
This would be far more helpful than creating more generators.


Yes, scipy.stats.


Is that stable enough so that all or part could be added to stdlib?

tjr


--
http://mail.python.org/mailman/listinfo/python-list

Re: Why does this group have so much spam?

[Terry Reedy]

Steven D'Aprano wrote:

On Wed, 02 Sep 2009 15:22:08 -0400, Terry Reedy wrote:


the conclusion you do. But I read your argument as being that having an 
open wi-fi connection was prima facie evidence of intent to commit crime 
regardless of whether you were a public advocate or not. Perhaps I 
misunderstood.


Yes, as you realized later.

So it's the *advocacy* (for the purposes of alibi) which is evidence of 
wrong-doing?


I said 'reason for me to be suspicious' rather than 'courtroom evidence'.


Not the open windows themselves?


Correct. The vast majority of open WiFi is due to ignorance or 
insufficient motivation to jump through the hoops needed to add units to 
a closed network. (I believe this can and should be easier, but that is 
another topic.)


The other advocated reason is basically to 'stick it to the 
corporation', under the delusion that it is possible to hurt the 
fictitious 'legal person' rather than the real people how are owners, 
workers, and other customers.  ISP's price residential service based on 
average fixed cost and average usage. Multiple homes using one 
connection push those averages up.


Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: An iteration idiom (Was: Re: [Guppy-pe-list] loading files containing multiple dumps)

[Chris Withers]

Raymond Hettinger wrote:

In the first case, you would write:
   sets.extend(h.load(f))


yes, what I had was:

for s in iter(h.load(f)): sets.append(s)

...which I mistakenly thought was working, but in in fact boils down to 
Raymond's code.


The problem is that each item that h.load(f) returns *is* actually an 
iterable, so either of the above just ends up the contents of each set 
being extended onto `sets` rather than the sets themselved.


It's all really rather confusing, apologies if there's interspersed rant 
in here:


>>> from guppy import hpy
>>> h = hpy()

Minor rant, why do I have to instantiate a

to do anything with heapy?
Why doesn't heapy just expose load, dump, etc?

(oh, and reading the code for guppy.heapy.Use and its ilk made me go 
temporarily blind!) ;-)


>>> f = open('copy.hpy')
>>> s = h.load(f)

Less minor rant: this applies to most things to do with heapy... Having 
__repr__ return the same as __str__ and having that be a long lump of 
text is rather annoying. If you really must, make __str__ return the big 
lump of text but have __repr__ return a simple, short, item containing 
the class, the id, and maybe the number of contained objects...


Anyway...

>>> id(s)
13905272
>>> len(s)
192
>>> s.__class__

>>> i = s[0]
>>> id(i)
13904112
>>> len(i)
1
>>> i.__class__


Hmmm, I'm sure there's a good reason why an item in a set has the exact 
same class and iterface as a whole set?


It feels like some kind of filtering, where are the docs that explain 
all this?


cheers,

Chris

--
Simplistix - Content Management, Batch Processing & Python Consulting
   - http://www.simplistix.co.uk
--
http://mail.python.org/mailman/listinfo/python-list

Re: string find mystery

[Tim Chase]

I have come across this very strange behaviour. Check this code:

if file_str.find('Geometry'):


While the "anser" is to compare the results of .find() with -1, 
but the more Pythonic answer is just to use "in":


  if "Geometry" in file_str:

which reads a lot more cleanly, IMHO.

-tkc




--
http://mail.python.org/mailman/listinfo/python-list

match braces?

[lallous]

Hello

In C/C++ you use the braces where as in Python you use the indentation
levels.
Most editors offer a Ctrl+[ to match the braces so that you can easily
identify the scopes (more correctly "statements blocks").

I am finding it difficult to see blocks and/or jump from end to start
with some IDE hotkeys, when coding in Python. Any advise?


Thanks,
Elias
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: match braces?

[Chris Rebert]

On Thu, Sep 3, 2009 at 2:38 AM, lallous wrote:
> Hello
>
> In C/C++ you use the braces where as in Python you use the indentation
> levels.
> Most editors offer a Ctrl+[ to match the braces so that you can easily
> identify the scopes (more correctly "statements blocks").
>
> I am finding it difficult to see blocks

Erm, how does the indentation itself not make it plainly and explicitly clear?
Perhaps you need to set your tabstops wider?

Cheers,
Chris
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: match braces?

[Tim Chase]

In C/C++ you use the braces where as in Python you use the indentation
levels.
Most editors offer a Ctrl+[ to match the braces so that you can easily
identify the scopes (more correctly "statements blocks").

I am finding it difficult to see blocks and/or jump from end to start
with some IDE hotkeys, when coding in Python. Any advise?


Any editor worth its salt will offer indentation-based folding (I 
know vim does, and I would be astonished if emacs didn't.  With 
other editors, YMMV).  You can just collapse the indented section 
to get a big-picture view.


-tkc

--
http://mail.python.org/mailman/listinfo/python-list

Modifying a textfile

[Olli Virta]

Hi!

So I got this big textfile. It's full of data from a database. About
150 or
more rows or lines in a textfile.
There's three first rows that belong to the same subject. And then
next
three rows belong to another subject and so on, to the end of the
file.

What I need to do, is put the three rows that goes together and belong
to
certain subject, on a one line in the output textfile. And the next
three
rows again on a one new line. And that goes with the rest of the data
to
the end of the new file.

Can't figure out a working loop structure to handle this.

 Thanks! OV
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Modifying a textfile

[Chris Rebert]

On Thu, Sep 3, 2009 at 3:21 AM, Olli Virta wrote:
> Hi!
>
> So I got this big textfile. It's full of data from a database. About
> 150 or
> more rows or lines in a textfile.
> There's three first rows that belong to the same subject. And then
> next
> three rows belong to another subject and so on, to the end of the
> file.
>
> What I need to do, is put the three rows that goes together and belong
> to
> certain subject, on a one line in the output textfile. And the next
> three
> rows again on a one new line. And that goes with the rest of the data
> to
> the end of the new file.
>
> Can't figure out a working loop structure to handle this.

#completely untested
from itertools import  izip_longest
#from itertools recipes
def grouper(n, iterable, fillvalue=None):
"grouper(3, 'ABCDEFG', 'x') --> ABC DEF Gxx"
args = [iter(iterable)] * n
return izip_longest(fillvalue=fillvalue, *args)

output = file("output_file.whatever", 'w')
f = file("input_file.whatever", 'r')
for triple in grouper(3, f):
triple = triple.replace('\n', '')
output.write(triple)
output.write('\n')

Cheers,
Chris
--
http://blog.rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: your favorite debugging tool?

[Marius Gedminas]

On Aug 25, 2:55 pm, Esmail  wrote:
> Re pdb, if you have a 'pointer' (ie reference) to an object, is there
> an easy way to dump out its contents, ie all of its members short of
> writing a method that does that and then calling it?

Usually

  pp vars(your_object)

does what you want
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Simple addition to random module - Student's t

[Daniel Stutzbach]

On Wed, Sep 2, 2009 at 2:15 PM, Raymond Hettinger  wrote:

> ISTM, there ought to be a statistics module that can calculate
> cumulative distribution functions for a variety of distributions.
> This would be far more helpful than creating more generators.
>

Many of the formulas for cumulative distribution functions require math
functions not currently provided by Python (erf, gamma, etc.).

(http://bugs.python.org/issue3366 includes a patch to provide them)

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Problem w/ mysqldump

[Nitebirdz]

On Wed, Sep 02, 2009 at 04:45:02PM -0400, Victor Subervi wrote:
>
> I tried running it like you said, got this error:
> 'mysqldump' is not a recognized internal or external command.
> If I could just figure out in what file the data were stored, I could copy
> it and try it in another computer. Any ideas?
> 

OK, that makes more sense now.  It's not truly a Python problem at all,
but rather an issue with the MySQL install/config on Windows.  As
somebody else pointed out, you are supposed to run the 'mysqldump'
command from the command prompt, which in the case of Windows is the
"cmd.exe", as far as I remember:

  Start --> Run --> command


I'm not familiar enough with Windows though.  I haven't run it in years.
However, the following URL may be of some help:

http://www.vbulletin.com/forum/showthread.php?t=68822


Judging by that, MySQL's binaries may not be in your path by default
when you run it on Windows.  If that's the case, you may have to add it
(search around for info on how to add new directories to your path in
Windows' terminal) or simply specify the absolute path to the command in
the Python script, instead of just passing 'mysqldump'.  

This other exchange from a forum may also help:

http://forums.mysql.com/read.php?35,144934,144960#msg-144960


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Entry Level Python Jobs

[Nitebirdz]

On Wed, Sep 02, 2009 at 08:31:20AM -0700, JonathanB wrote:
>
> I am a self-taught Python programmer with a liberal arts degree (Cross-
> cultural studies). I have been programming for several years now and
> would like to get a job as a python programmer. Unfortunately most of
> the job posts I have seen are for CS Majors or people with experience.
> 
> Is there a place I can look for job posts for entry level positions
> requiring no experience? For the hiring managers, if the job post said
> "CS Major" in the requirements, would you consider a liberal arts
> major at all?
> 

I have a liberal arts degree and have been working in the field for
years now, not as a programmer but as a high-level technical support
engineer (doing core dump analysis and the like).  While I opted for not
working as a programmer, other co-workers without a CompSci degree have
managed to do so without major problems.

It seems to me that most managers are willing to hire someone based on
his/her experience and proven knowledge, and not so much on the actual
degree you have.  Obviously, this means you will need to get some
experience before moving into actual programming.  

So, what would I recommend?  

First of all, make sure you get your foot in the door.  Apply for an
entry-level position at a company that works in the technology field,
even if it's doing technical support or writing documentation.  Once you
are in, work hard, show an interest in learning programming skills, talk
to the developers in the company, survey people around and try to figure
out where there is a need that can be met with a not-yet-written
application and put it together yourself, then show it to your manager
and try to convince him/her to deploy it as an official tool for your
team.  I've seen this work many times.  

Second, search around for open source projects that may look interesting
to you, download the source code, study it, subscribe to their
development mailing list, check out standing bugs and see if you can fix
them.  This is something you can definitely add to your resume.  


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Modifying a textfile

[Steven D'Aprano]

On Thu, 03 Sep 2009 03:21:25 -0700, Olli Virta wrote:

> What I need to do, is put the three rows that goes together and belong
> to
> certain subject, on a one line in the output textfile. And the next
> three
> rows again on a one new line. And that goes with the rest of the data to
> the end of the new file.
> 
> Can't figure out a working loop structure to handle this.

The basic algorithm is:
- grab three lines from the input file
- write to the output file
- repeat until done

There are lots of ways to do this. Here's one:

# untested
out = file(outfile, 'w')
in_ = file(infile, 'r')
while True:
# Grab three lines.
a = in_.readline()
if a == '':
# Nothing more to read, we're done.
break
b = in_.readline()
c = in_.readline()
out.write(a.rstrip() + ' ')
out.write(b.rstrip() + ' ')
out.write(c.rstrip() + '\n')
out.close()
in_.close()


Here's another version:

# untested
out = file(outfile, 'w')
accumulator = []
for line in file(infile, 'r'):
if len(accumulator) == 3:
out.write("%s %s %s\n" % tuple(accumulator))
accumulator = []
accumulator.append(line.rstrip())
if accumulator:
out.write("%s %s %s\n" % tuple(accumulator))
out.close()


Chris has posted another method, using itertools. That's probably faster 
for very large files, but less readable. 



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Annoying octal notation

[Albert van der Horst]

In article <6031ba08-08c8-416b-91db-ce8ff57ae...@w6g2000yqw.googlegroups.com>,
James Harris   wrote:

>
>So you are saying that Smalltalk has r where
>r is presumably for radix? That's maybe best of all. It preserves the
>syntactic requirement of starting a number with a digit and seems to
>have greatest flexibility. Not sure how good it looks but it's
>certainly not bad.
>
>  0xff & 0x0e | 0b1101
>  16rff & 16r0e | 2r1101
>
>Hmm. Maybe a symbol would be better than a letter.

Like 0#ff  16#ff ?

That is ALGOL68. It is incredible how many of it has become
vindicated over time. (Yes, nineteen hundred sixty eight was
the year that language was conceived.)

>
>James

Groetjes Albert

--
-- 
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
alb...@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: recursive decorator

[Michele Simionato]

On Sep 3, 12:19 am, Ethan Furman  wrote:
> Greetings, List!
>
> The recent thread about a recursive function in a class definition led
> me back to a post about bindfunc from Arnaud, and from there I found
> Michele Simionato's decorator module (many thanks! :-), and from there I
> began to wonder...
>
> from decorator import decorator
>
> @decorator
> def recursive1(func, *args, **kwargs):
>      return func(func, *args, **kwargs)
>
> @recursive1
> def factorial1(recurse, n):
>      if n < 2:
>          return 1
>      return n * recurse(n-1)
>
> factorial(4)
> TypeError: factorial1() takes exactly 2 arguments (1 given)

What are you trying to do here? I miss why you don't use the usual
definition of factorial.
If you have a real life use case which is giving you trouble please
share. I do not see
why you want to pass a function to itself (?)

M. Simionato
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Annoying octal notation

[Albert van der Horst]

In article ,
MRAB   wrote:
>Steven D'Aprano wrote:

>> Obviously I can't speak for Ken Thompson's motivation in creating this
>> feature, but I'm pretty sure it wasn't to save typing or space on
>> punchcards. Even in 1969, hex was more common than octal, and yet hex
>> values are written with 0x. My guess is that he wanted all numbers to
>> start with a digit, to simplify parsing, and beyond that, it was just his
>> programming style -- why call the copy command `copy` when you could call
>> it `cp`? (Thompson was the co-inventor of Unix.)
>>
>Maybe it was because they were working on minicomputers, not mainframes,
>so there was less processing power and storage available.

Not just any minicomputers: PDP11. Octal notation is friendly with
the PDP11 instruction set.

Groetjes Albert

--
-- 
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
alb...@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: string find mystery

[Hendrik van Rooyen]

On Thursday 03 September 2009 07:10:37 Helvin wrote:
> Hi,
>
> I have come across this very strange behaviour. Check this code:
>
> if file_str.find('Geometry'):
> #if file_str.endswith('Data_Input_Geometry.txt'):
> print 'I found geometry'
> elif file_str.find('Material'):
> print 'I found material'
> The amazing thing is when file_str  = 'C:\Qt\SimLCM\Default
> \Data_Input_Material.txt',
> the first if statement if fulfilled, that seemingly says that in this
> file_str, python actually finds the word 'Geometry'.
> I know this, because the line: 'I found geometry' is printed. However,
> if instead of using file_str.find(), I use file_str.endswith(), it
> does not exhibit this strange behaviour.
>
> Obviously, I want the elif line to be true, instead of the first if
> statement.
>
> Does anyone know why this is happening?
> Help much appreciated!

The interactive Interpreter is your friend:

s = "a;kljghkahklahdfgkjahdfhadafjd;l"
s.find("banana")
-1
bool(_)
True

- Hendrik


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Entry Level Python Jobs

[Michele Simionato]

On Sep 2, 5:31 pm, JonathanB  wrote:
> I am a self-taught Python programmer with a liberal arts degree (Cross-
> cultural studies). I have been programming for several years now and
> would like to get a job as a python programmer. Unfortunately most of
> the job posts I have seen are for CS Majors or people with experience.
>
> Is there a place I can look for job posts for entry level positions
> requiring no experience? For the hiring managers, if the job post said
> "CS Major" in the requirements, would you consider a liberal arts
> major at all?

Requiring a CS Major does not make sense. Sensible employers asks for
programmers
that know how to program, not for a piece of paper. For instance, at
work here
nobody has a CS degree, but still I would say that we have very
competent programmers.

You need a way to prove that you are a competent programmer.
Partecipating to
Open Source projects, writing articles about programming or having a
technical blog are ways to show your expertise. Here when we hire
people
we look at their posts in public newsgroups. I find the quality of the
posts a very good indicator of the quality of the perspective
employees.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Entry Level Python Jobs

[koranthala]

On Sep 3, 9:19 am, steve  wrote:
> On 09/03/2009 09:36 AM, steve wrote:
>
> > Hi Jonathan,
> > [...snip...]
>
> I feel stupid replying to my own post but just one more thing i thought about
> mentioning but forgot to add:
> - Look at your Liberal Arts major as an advantage. Every field has a 
> 'computing
> gap' that needs to be filled but cannot be done because they aren't any too 
> many
> good people who have the relevant cross-domain knowledge. For instance, one of
> the reasons I think this month's sourceforge.net project of the month is 
> really
> great is because the lead dev. has a CS degree and is listed as being a 
> medicine
> student:http://sourceforge.net/community/potm-200909/
>
> So, look for these gaps in your domain which can be filled using your 
> technical
> knowledge.
>
> again, ..
>
>
>
> > Wish you the best,
> > regards,
> > - steve
>
> --
> random non tech spiel:http://lonetwin.blogspot.com/
> tech randomness:http://lonehacks.blogspot.com/
> what i'm stumbling into:http://lonetwin.stumbleupon.com/

Also, I think topcoder.com is a good place for him. I have not used
them much, but their business plan -- of asking medium to difficult
questions every week, and contacting people who solves them with jobs
-- is quite sound.
Try that too.
-- 
http://mail.python.org/mailman/listinfo/python-list

obscure problem using elementtree to make xhtml website

[Lee]

Elementtree (python xml parser) will transform markup like



into



which is a reasonable thing to do for xml (called minimization, I
think).

But this caused an obscure problem when I used it to create the xhtml
parts of my website,
causing Internet Explorer to display nearly blank pages. I explain the
details at

http://lee-phillips.org/scripttag/

and am writing here as a heads-up to anyone who might be using a
workflow similar to mine: writing documents in xml and using python
and elementtree to transform those into xhtml webpages, and using the
standard kludge of serving them as text/html to IE, to get around the
latter's inability to handle xml. I can't be the only one (and I doubt
this problem is confined to elementtree).


Lee Phillips
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Annoying octal notation

[sjm]

On Aug 21, 2:45 pm, John Nagle  wrote:

>      In 2009, Unisys finally exited the mainframe hardware business, and the
> last of the 36-bit machines, the ClearPath servers, are being phased out.
> That line of machines goes back to the UNIVAC 2200 series, and the UNIVAC
> 1100 series, all the way back to the vacuum-tube UNIVAC 1103 from 1952.
> It's the longest running series of computers in history, and code for all
> those machines used octal heavily.

You're right that the 36-bit machines rely heavily on octal notation.
However, Unisys has not exited the hardware business.

The descendants of the original 36-bit 1100-series machines are now
called Dorado.  Unisys announced new models as recently as May 2009
(see http://unisys.com/unisys/news/detail.jsp?id=1694).

I have the extreme pleasure of supporting a Dorado 180 and writing
Python code on Windows--the best of both worlds!

Cheers,
  Steve J. Martin

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: match braces?

[Ben Finney]

Tim Chase  writes:

> Any editor worth its salt will offer indentation-based folding (I know
> vim does, and I would be astonished if emacs didn't.

Emacs calls that “hide/show”, and the ‘hs-minor-mode’ can be enabled for
any buffer (and can thus of course be automatically enabled on defined
conditions, e.g. whenever a Python buffer is detected).

Learn more at http://www.emacswiki.org/cgi-bin/wiki/HideShow>.

-- 
 \ “Creativity can be a social contribution, but only in so far as |
  `\society is free to use the results.” —Richard Stallman |
_o__)  |
Ben Finney
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python daemon - compress data and load data into MySQL by pyodbc

[MacRules]

Sean DiZazzo wrote:

On Sep 2, 8:36 pm, MacRules  wrote:

Hi,

I installed Python daemon, pyodbc module to access the back-end DB server.

My setup is like this

load data job -> Python Daemon A, port 6000 -> Python Daemon B, port
7000 -> MySQL

Daemon A will perform data compression, such as GZIP, and send over data
to Daemon B.
Daemon B will perform data uncompression, GUNZIP, and insert records to
MySQL or MSSQL or Oracle.

Where should I start this to code this?
Can someone give me a hint, as detail as possible here?

I am a python newbie.

Thanks for all the help I can get,


Start by reading the tutorial.  http://docs.python.org/tutorial/

~Sean

Are you a Python expert?
Can you show me the basic coding to get a Daemon (pass through insert 
data) up and insert to the backend MySQL?


--
http://mail.python.org/mailman/listinfo/python-list

Re: obscure problem using elementtree to make xhtml website

[David Smith]

Lee wrote:
> Elementtree (python xml parser) will transform markup like
> 
> 
> 
> into
> 
> 
> 
> which is a reasonable thing to do for xml (called minimization, I
> think).
> 
> But this caused an obscure problem when I used it to create the xhtml
> parts of my website,
> causing Internet Explorer to display nearly blank pages. I explain the
> details at
> 
> http://lee-phillips.org/scripttag/
> 
> and am writing here as a heads-up to anyone who might be using a
> workflow similar to mine: writing documents in xml and using python
> and elementtree to transform those into xhtml webpages, and using the
> standard kludge of serving them as text/html to IE, to get around the
> latter's inability to handle xml. I can't be the only one (and I doubt
> this problem is confined to elementtree).
> 
> 
> Lee Phillips

It's not just Elementtree that does this .. I've seen others libraries
(admittedly in other languages I won't mention here) transform empty
tags to the self-terminating form.  A whitespace text node or comment
node in between *should* prevent that from happening.  AFAIK, the only
tag in IE xhtml that really doesn't like to be reduced like that is the
 tag.  Firefox seems to be fine w/ self-terminating