Re: [Tutor] data analysis with python

[David Martins]

Thanks Andre and Ryan

At first glance Pytables looks certainly a lot better than sql... I also found 
vitables which seems to be a nice GUI interface and will play around with both 
tomorrow.

I remember having looked at R a while ago but did never pick it up. I found a 
nice tutorial and will give it a go.

Cheers
Chris

Date: Tue, 13 Nov 2012 21:13:46 -0800
Subject: Re: [Tutor] data analysis with python
From: ryan.wap...@gmail.com
To: walksl...@gmail.com
CC: awesome.me...@outlook.com; tutor@python.org

Not sure how stuck you are to python (I have no doubt it can tackle this) but 
this is very much the sort of thing that 'R' is *really* good at.

Just FYI.

Good luck

Ryan

  ___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] data analysis with python

[Alan Gauld]

On 14/11/12 08:13, David Martins wrote:


I remember having looked at R a while ago but did never pick it up. I
found a nice tutorial and will give it a go.


There is an interface to R from Python too.
So you can combine the two..

However, given your stated aims SQL does look like the most natural 
choice and is worth learning, it's really not that hard, especially if 
performance is not critical. But if you want to really crunch the data 
rather than just mine it then R or numpy might be better suited.


BTW There are graphical GUI front tends to SQLite that hide the SQL from 
you so if you managed to get the data into SQLite you could use those to 
get it out again... Try a Google search.


--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] data analysis with python

[Bjorn Madsen]

Hi David,

I have found happiness with http://ipython.org/ which can do stuff like
this:
[image: _images/ipy_0.13.png]


SQLite is embedded in python's database API, and gives an easy data import
and handling. The syntax is extremely well described here:
http://www.sqlite.org/lang.html and I've been handling just short of
300million records on a EEE-netbook.

And in context with iPython's visual display and HTML comment you should be
able to save the print-out from iPython directly onto your web-server.

Feel free to write if you get stuck.

Kind Regards,
Bjorn

On 14 November 2012 05:13, Ryan Waples  wrote:

> Not sure how stuck you are to python (I have no doubt it can tackle this)
> but this is very much the sort of thing that 'R' is *really* good at.
> Just FYI.
> Good luck
> Ryan
>
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] data analysis with python

[Oscar Benjamin]

On 14 November 2012 03:17, David Martins  wrote:
> Hi All
>
> I'm trying to use python for analysing data from building energy simulations
> and was wondering whether there is way to do this without using anything sql
> like.

There are many ways to do this.

>
> The simulations are typically run for a full year, every hour, i.e. there
> are 8760 rows and about 100+ variables such as external air temperature,
> internal air temperature, humidity, heating load, ... making roughly a
> million data points. I've got the data in a csv file and also managed to
> write it in a sqlite db.

This dataset is not so big that you can't just load it all into memory.

>
> I would like to make requests like the following:
>
> Show the number of hours the aircon is running at 10%, 20%, ..., 100%
> Show me the average, min, max air temperature, humidity, solar gains,
> when the aircon is running at 10%, 20%,...,100%
>
> Eventually I'd also like to generate an automated html or pdf report with
> graphs. Creating graphs is actually somewhat essential.

Do you mean graphs or plots? I would use matplotlib for plotting. It
can automatically generate image files of plots. There are also ways
to generate output for visualising graphs but I guess that's not what
you mean. Probably I would create a pdf report using latex and
matplotlib but that's not the only way.
http://en.wikipedia.org/wiki/Graph_(mathematics)
http://en.wikipedia.org/wiki/Plot_(graphics)

> I tried sql  and find it horrible, error prone, too much to write, the logic
> somehow seems to work different than my brain and I couldn't find
> particulary good documentation (particulary the documentation of the api is
> terrible, in my humble opinion). I heard about zope db which might be an
> alternative. Would you mind pointing me towards an appropriate way to solve
> my problem? Is there a way for me to avoid having to learn sql or am I
> doomed?

There are many ways to avoid learning SQL. I'll suggest the simplest
one: Can you not just read all the data into memory and then perform
the computations you want?

For example:

$ cat tmp.csv
Temp,Humidity
23,85
25,87
26,89
23,90
24,81
24,80

$ cat tmp.py
#!/usr/bin/env python

import csv

with open('tmp.csv', 'rb') as f:
reader = csv.DictReader(f)
data = []
for row in reader:
row = dict((k, float(v)) for k, v in row.items())
data.append(row)

maxtemp = max(row['Temp'] for row in data)
mintemp = min(row['Temp'] for row in data)
meanhumidity = sum(row['Humidity'] for row in data) / len(data)

print('max temp is: %d' % maxtemp)
print('min temp is: %d' % mintemp)
print('mean humidity is: %f' % meanhumidity)

$ ./tmp.py
max temp is: 26
min temp is: 23
mean humidity is: 85.33

This approach can also be extended to the case where you don't read
all the data into memory.


Oscar
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

[Tutor] LCM

[Selby Rowley Cannon]

Hey,

I've been trying to write a function to find the Lowest Common 
Multiple of two numbers, but it isn't working and I've kinda hit a dead 
end on the thought-process end of things. Anyone mind looking at it, and 
tell me what's wrong? (I hop you don't think it's too long to put in an 
email)


Code:

def lowestCommonMultiple(a, b, amount): #k
float(amount)
if a == b:
print 'You cannot find the LCM of the same number twice.'
else:
numbersList = list(range(amount))
aMultiples = []
bMultiples = []
for aNumber in numbersList:
aMultiples.append(a*aNumber)
for bNumber in numbersList:
bMultiples.append(b*bNumber)
if aMultiples[1] == bMultiples[1]:
print 'The LCM of ', a, ' and ', b, ' is ', aMultiple[1]
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] LCM

[Dave Angel]

On 11/14/2012 12:52 PM, Selby Rowley Cannon wrote:
> Hey,
>
> I've been trying to write a function to find the Lowest Common
> Multiple of two numbers, but it isn't working and I've kinda hit a
> dead end on the thought-process end of things. Anyone mind looking at
> it, and tell me what's wrong? (I hop you don't think it's too long to
> put in an email)
>
> Code:
>
> def lowestCommonMultiple(a, b, amount): #k
> float(amount)
> if a == b:
> print 'You cannot find the LCM of the same number twice.'
> else:
> numbersList = list(range(amount))
> aMultiples = []
> bMultiples = []
> for aNumber in numbersList:
> aMultiples.append(a*aNumber)
> for bNumber in numbersList:
> bMultiples.append(b*bNumber)
> if aMultiples[1] == bMultiples[1]:
> print 'The LCM of ', a, ' and ', b, ' is ', aMultiple[1]

Which are you interested in, finding what's wrong with the code, or a
simpler way to actually find an LCM ?

Is this an assignment, or just part of a bigger program you're writing?

-- 
DaveA
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] LCM

[Selby Rowley Cannon]

On 14/11/12 18:27, Dave Angel wrote:

On 11/14/2012 12:52 PM, Selby Rowley Cannon wrote:

Hey,

 I've been trying to write a function to find the Lowest Common
Multiple of two numbers, but it isn't working and I've kinda hit a
dead end on the thought-process end of things. Anyone mind looking at
it, and tell me what's wrong? (I hop you don't think it's too long to
put in an email)

Code:

def lowestCommonMultiple(a, b, amount): #k
 float(amount)
 if a == b:
 print 'You cannot find the LCM of the same number twice.'
 else:
 numbersList = list(range(amount))
 aMultiples = []
 bMultiples = []
 for aNumber in numbersList:
 aMultiples.append(a*aNumber)
 for bNumber in numbersList:
 bMultiples.append(b*bNumber)
 if aMultiples[1] == bMultiples[1]:
 print 'The LCM of ', a, ' and ', b, ' is ', aMultiple[1]

Which are you interested in, finding what's wrong with the code, or a
simpler way to actually find an LCM ?

Is this an assignment, or just part of a bigger program you're writing?


a.) Finding what's wrong with the code;
b.) Part of a bigger program.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] LCM

[Dave Angel]

On 11/14/2012 01:34 PM, Selby Rowley Cannon wrote:
> On 14/11/12 18:27, Dave Angel wrote:
>> On 11/14/2012 12:52 PM, Selby Rowley Cannon wrote:
>>> Hey,
>>>

Tell us what version of Python you're targeting.  I'm going to assume
2.x, since you have print without parens.

>>>  I've been trying to write a function to find the Lowest Common
>>> Multiple of two numbers, but it isn't working and I've kinda hit a
>>> dead end on the thought-process end of things. Anyone mind looking at
>>> it, and tell me what's wrong? (I hop you don't think it's too long to
>>> put in an email)
>>>
>>> Code:
>>>
>>> def lowestCommonMultiple(a, b, amount): #k

Why would you require the caller to specify the "amount" ?  It's easier
for you to calculate the two ranges below, as needed.

>>>  float(amount)

This statement does nothing, which is good, since range doesn't work on
floats.

>>>  if a == b:
>>>  print 'You cannot find the LCM of the same number twice.'

here you should just return a

>>>  else:
>>>  numbersList = list(range(amount))

In Python 2.x, range already returns a list.  Besides, you'd be better
off with a sequence, so I'd use xrange.

>>>  aMultiples = []
>>>  bMultiples = []
>>>  for aNumber in numbersList:
>>>  aMultiples.append(a*aNumber)
>>>  for bNumber in numbersList:
>>>  bMultiples.append(b*bNumber)
>>>  if aMultiples[1] == bMultiples[1]:

This will never be true, since lhs is 2*a, and rhs is 2*b, and a != b
What you really need is some form of loop, which compares every item in
the first list to every item in the second, returning the lowest number
that's in both.  Try something like:

for item in aMultiples:
if item in bMultiples:
return item

>>>  print 'The LCM of ', a, ' and ', b, ' is ', aMultiple[1]

It's generally better to return a calculated result, and print it
separately, than always to print it.

>> Which are you interested in, finding what's wrong with the code, or a
>> simpler way to actually find an LCM ?
>>
>> Is this an assignment, or just part of a bigger program you're writing?
>>
> a.) Finding what's wrong with the code;
> b.) Part of a bigger program.
> 
> 

This algorithm is fine for small numbers, but the loop you'll have to
write at the end is going to be very slow for large ones.  It'll also
use up lots of memory for those lists.

You could speed it up a tiny bit by using different ranges for the two
lists.  After all the first list needn't have more than b items in it,
and the second needn't have more than a items in it.

You could cut the memory usage a lot by making the first list simply an
xrange.

The second, though, has to be traversed multiple times, so you
presumably need a list.  But a set would be MUCH faster.


Anyway, if all you care about is the result, then use Euclid's method,
and the % operator to calculate the gcd.  A few loops around, and even
pretty large numbers will crumble.  Then the lcm is simply the product
of a and b, divided by the gcd of a and b.

The heart of such a gcd loop is something like:
while b > 0:
a, b = b, a%b

-- 

DaveA
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] LCM

[eryksun]

On Wed, Nov 14, 2012 at 12:52 PM, Selby Rowley Cannon
 wrote:
>
> I've been trying to write a function to find the Lowest Common Multiple
> of two numbers, but it isn't working and I've kinda hit a dead end on the
> thought-process end of things.

Since the LCM is the smallest multiple of both numbers, you want to
factor out what they have in common from one of them:

x = x_unique * gcd
y = y_unique * gcd
lcm = x_unique * y_unique * gcd
= (x / gcd) * y

For example, say you have x = 21 = 3*7 and y = 35 = 5*7. The factor in
common, i.e. the greatest common divisor (GCD), is 7. So the LCM is
105 = 3*5*7.

I'm sure you can search for a good GCD algorithm with no problem since
it's over 2,000 years old. But there's an "implementation" (it's not
much work) in the fractions module:

>>> 21 / fractions.gcd(21, 35) * 35
105

>>> print inspect.getsource(fractions.gcd)
def gcd(a, b):
"""Calculate the Greatest Common Divisor of a and b.

Unless b==0, the result will have the same sign as b (so that when
b is divided by it, the result comes out positive).
"""
while b:
a, b = b, a%b
return a
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] unicode help

[Marilyn Davis]

Hi,

Last year, I was helped so that this ran nicely on my 2.6:

#! /usr/bin/env python
# -*- coding: utf-8 -*-
# necessary for python not to complain about "¥"

symbol = unichr(165)
print unicode(symbol)

--- end of code ---

But, now on my 2.7, and on 2.6 when I tried reinstalling it, I get:

bash-3.2$ ./uni_test.py
./uni_test.py
Traceback (most recent call last):
  File "./uni_test.py", line 6, in 
print unicode(symbol)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xa5' in
position 0: ordinal not in range(128)
bash-3.2$

Can anyone please help?  It says 'ascii' codec?  Shouldn't it be seeing
'utf-8'?

I can't imagine it matters but I'm on an imac now and before I was on Ubuntu.

Thank you,

Marilyn Davis








On Sat, May 28, 2011 2:17 pm, Marilyn Davis wrote:

> Thank you Martin,
>
>
> This:
>
>
> #!/usr/bin/env python
> # -*- coding: utf8 -*-
> '''Unicode handling for 2.6.
> '''
> [rest of module deleted]
>
>
> produces an emacs warning:
>
> Warning (mule): Invalid coding system `utf8' is specified
> for the current buffer/file by the :coding tag. It is highly recommended to
> fix it before writing to a file.
>
> But, if I save anyway, and run, I get this:
>
>
> ./uni.py
> File "./uni.py", line 13
> SyntaxError: 'utf8' codec can't decode byte 0xa5 in position 0: unexpected
>  code byte
>
> but, on a hunch, I tried
>
> # -*- coding: utf-8 -*-
>
>
> and emacs and python were very happy.  Thank you thank you thank you.
>
> Now I can enjoy my Saturday.
>
>
> Marilyn
>
>
>
>
>
> On Sat, May 28, 2011 3:00 pm, Martin A. Brown wrote:
>
>
>> Hello there,
>>
>>
>>
>> : I'm still on Python 2.6 and I'm trying to work some unicode
>> : handling.
>> :
>> : I've spent some hours on this snippet of code, trying to follow
>> : PEP 0263, since the error tells me to see it.  I've tried other
>> : docs too and I am still clueless.
>>
>>
>>
>> OK, so this is PEP 0263.  http://www.python.org/dev/peps/pep-0263/
>>
>>
>>
>> Did you miss these lines?
>>
>>
>>
>> To define a source code encoding, a magic comment must
>> be placed into the source files either as first or second line in the
>> file, such as:
>>
>> Or was it the lack of an explicit example for UTF-8 in the PEP?
>>
>>
>>
>> Try adding a single line to your script, as the second line.  That
>> should make your script look like:
>>
>> #! /usr/bin/env python
>> # -*- coding: utf8 -*-
>>
>>
>>
>> You might wonder why on earth you have to do this.  The interpreter
>> cannot safely assume that your editor (any arbitrary text editor) knows
>> how to create/save anything other than ASCII without this (slightly
>> hackish) hint.
>>
>> Good luck,
>>
>>
>>
>> -Martin
>>
>>
>>
>> --
>> Martin A. Brown
>> http://linux-ip.net/

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] unicode help

[Dave Angel]

On 11/14/2012 03:10 PM, Marilyn Davis wrote:
> Hi,
>
> Last year, I was helped so that this ran nicely on my 2.6:
>
> #! /usr/bin/env python
> # -*- coding: utf-8 -*-
> # necessary for python not to complain about "¥"
>
> symbol = unichr(165)
> print unicode(symbol)
>
> --- end of code ---
>
> But, now on my 2.7, and on 2.6 when I tried reinstalling it, I get:
>
> bash-3.2$ ./uni_test.py
> ./uni_test.py
> Traceback (most recent call last):
>   File "./uni_test.py", line 6, in 
> print unicode(symbol)
> UnicodeEncodeError: 'ascii' codec can't encode character u'\xa5' in
> position 0: ordinal not in range(128)
> bash-3.2$
>
> Can anyone please help?  It says 'ascii' codec?  Shouldn't it be seeing
> 'utf-8'?
>
> I can't imagine it matters but I'm on an imac now and before I was on Ubuntu.
>
> Thank you,
>
> Marilyn Davis
>
>
>
>

You top-posted your message.  If you need to quote something, please put
your new text after what you're quoting.

Try the following in your 2.7 interpreter:

>>> import sys
>>> print sys.stdout.encoding
UTF-8

If you don't see UTF-8, then there's a discrepancy between what you
think the terminal is doing, and what Python thinks.

Somebody familiar with the Mac might be able to tell you how to fix it
right, but you could try
sys.stdout.encoding = "UTF-8"

and see if it changes your symptoms.


-- 

DaveA

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] unicode help

[Marilyn Davis]

Thank you, Dave, for looking at my problem, and for correcting me on my
top posting.

See below:

On Wed, November 14, 2012 12:34 pm, Dave Angel wrote:

> On 11/14/2012 03:10 PM, Marilyn Davis wrote:
>
>> Hi,
>>
>>
>> Last year, I was helped so that this ran nicely on my 2.6:
>>
>>
>> #! /usr/bin/env python
>> # -*- coding: utf-8 -*-
>> # necessary for python not to complain about "¥"
>>
>>
>> symbol = unichr(165) print unicode(symbol)
>>
>> --- end of code ---
>>
>>
>> But, now on my 2.7, and on 2.6 when I tried reinstalling it, I get:
>>
>>
>> bash-3.2$ ./uni_test.py ./uni_test.py
>> Traceback (most recent call last):
>> File "./uni_test.py", line 6, in 
>> print unicode(symbol) UnicodeEncodeError: 'ascii' codec can't encode
>> character u'\xa5' in position 0: ordinal not in range(128) bash-3.2$
>>
>> Can anyone please help?  It says 'ascii' codec?  Shouldn't it be seeing
>>  'utf-8'?
>>
>>
>> I can't imagine it matters but I'm on an imac now and before I was on
>> Ubuntu.
>>
>>
>> Thank you,
>>
>>
>> Marilyn Davis
>>
>>
>>
>>
>>
>
> You top-posted your message.  If you need to quote something, please put
> your new text after what you're quoting.
>
> Try the following in your 2.7 interpreter:
>
>
 import sys print sys.stdout.encoding
> UTF-8
>
>
> If you don't see UTF-8, then there's a discrepancy between what you
> think the terminal is doing, and what Python thinks.

You're right, I get this:

bash-3.2$ python
Python 2.7.3 (v2.7.3:70274d53c1dd, Apr  9 2012, 20:52:43)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
import sys
>>> print sys.stdout.encoding
print sys.stdout.encoding
US-ASCII
>>>


>
> Somebody familiar with the Mac might be able to tell you how to fix it
> right, but you could try sys.stdout.encoding = "UTF-8"
>
> and see if it changes your symptoms.

and, your good idea didn't work:

>>> sys.stdout.encoding="UTF-8"
sys.stdout.encoding="UTF-8"
Traceback (most recent call last):
  File "", line 1, in 
TypeError: readonly attribute
>>>

Goodness!  I didn't expect it to be a Mac thing.

So, on a Windows machine, running Python 2.6.6, sys.stdout.encoding is
'cp1252', yet the code runs fine.

On Ubuntu with 2.7, it's 'UTF-8' and  it runs just fine.

I find this most mysterious.

Thank you for any help to get it running on my Mac.

Marilyn





>
> --
>
>
> DaveA


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] unicode help

[Marilyn Davis]

On Wed, November 14, 2012 1:07 pm, Marilyn Davis wrote:

> Thank you, Dave, for looking at my problem, and for correcting me on my
> top posting.
>
> See below:
>
>
> On Wed, November 14, 2012 12:34 pm, Dave Angel wrote:
>
>
>> On 11/14/2012 03:10 PM, Marilyn Davis wrote:
>>
>>
>>> Hi,
>>>
>>>
>>>
>>> Last year, I was helped so that this ran nicely on my 2.6:
>>>
>>>
>>>
>>> #! /usr/bin/env python
>>> # -*- coding: utf-8 -*-
>>> # necessary for python not to complain about "¥"
>>>
>>>
>>>
>>> symbol = unichr(165) print unicode(symbol)
>>>
>>> --- end of code ---
>>>
>>>
>>>
>>> But, now on my 2.7, and on 2.6 when I tried reinstalling it, I get:
>>>
>>>
>>>
>>> bash-3.2$ ./uni_test.py ./uni_test.py Traceback (most recent call
>>> last):
>>> File "./uni_test.py", line 6, in 
>>> print unicode(symbol) UnicodeEncodeError: 'ascii' codec can't encode
>>> character u'\xa5' in position 0: ordinal not in range(128) bash-3.2$
>>>
>>> Can anyone please help?  It says 'ascii' codec?  Shouldn't it be
>>> seeing 'utf-8'?
>>>
>>>
>>>
>>> I can't imagine it matters but I'm on an imac now and before I was on
>>>  Ubuntu.
>>>
>>>
>>>
>>> Thank you,
>>>
>>>
>>>
>>> Marilyn Davis
>>>
>>>
>>>
>>>
>>>
>>>
>>
>> You top-posted your message.  If you need to quote something, please
>> put your new text after what you're quoting.
>>
>> Try the following in your 2.7 interpreter:
>>
>>
>>
> import sys print sys.stdout.encoding
>> UTF-8
>>
>>
>>
>> If you don't see UTF-8, then there's a discrepancy between what you
>> think the terminal is doing, and what Python thinks.
>
> You're right, I get this:
>
>
> bash-3.2$ python Python 2.7.3 (v2.7.3:70274d53c1dd, Apr  9 2012, 20:52:43)
>  [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
>
 import sys
> import sys
 print sys.stdout.encoding
> print sys.stdout.encoding US-ASCII
>

>
>
>>
>> Somebody familiar with the Mac might be able to tell you how to fix it
>> right, but you could try sys.stdout.encoding = "UTF-8"
>>
>> and see if it changes your symptoms.
>
> and, your good idea didn't work:
>
 sys.stdout.encoding="UTF-8"
> sys.stdout.encoding="UTF-8" Traceback (most recent call last):
> File "", line 1, in 
> TypeError: readonly attribute
>

>
> Goodness!  I didn't expect it to be a Mac thing.
>
>
> So, on a Windows machine, running Python 2.6.6, sys.stdout.encoding is
> 'cp1252', yet the code runs fine.
>
>
> On Ubuntu with 2.7, it's 'UTF-8' and  it runs just fine.
>
>
> I find this most mysterious.
>
>
> Thank you for any help to get it running on my Mac.
>
>
> Marilyn
>

I found this site:
http://hints.macworld.com/article.php?story=20100713130450549

and that fixes it.

I would never guessed that!

Marilyn

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] unicode help

[Dave Angel]

On 11/14/2012 04:07 PM, Marilyn Davis wrote:
> 
>
> Goodness!  I didn't expect it to be a Mac thing.
>
> So, on a Windows machine, running Python 2.6.6, sys.stdout.encoding is
> 'cp1252', yet the code runs fine.
>
> On Ubuntu with 2.7, it's 'UTF-8' and  it runs just fine.
>
> I find this most mysterious.
>
> Thank you for any help to get it running on my Mac.
>
>

To resolve something like this, I'd start searching the internet (and
especially the python.org site) for a string like:
 python  sys.stdout.encoding

https://drj11.wordpress.com/2007/05/14/python-how-is-sysstdoutencoding-chosen/

According to this page, the encoding is chosen by the environment variable:
 LC_CTYPE

set LC_CTYPE=en_GB.utf-8


I have no idea if that's correct, or always correct, but it could be
worth experimenting.  (Later note:  I think this affects more than just
the terminal, so I'd skip it)

In the meantime, you  posted a link to

http://hints.macworld.com/article.php?story=20100713130450549

which suggests two different environment variables:  PYTHONIOENCODING and 
LC-CTYPE


Looking at the python.org site is presumably more authoritative.  See

http://docs.python.org/2/whatsnew/2.6.html?highlight=pythonioencoding

which says """
The encoding used for standard input, output, and standard error can be 
specified by setting the PYTHONIOENCODING 
 
environment variable before running the interpreter. The value should be a 
string in the form  or :. The /encoding/ part 
specifies the encoding’s name, e.g. utf-8 or latin-1; the optional 
/errorhandler/ part specifies what to do with characters that can’t be handled 
by the encoding, and should be one of “error”, “ignore”, or “replace”. 
(Contributed by Martin von Loewis.)"""


Also:  
http://docs.python.org/2/using/cmdline.html?highlight=pythonioencoding#PYTHONIOENCODING
 


Anyway, just realize that this does NOT change the terminal.  If it's not 
really utf-8, you'll sometimes get garbage characters. But at least you 
shouldn't get the encoding errors.

The *right* answer I figure would be to experiment to find out what the 
terminal on your Mac actually uses, and use that in your environment variable.


-- 

DaveA

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] unicode help

[Sander Sweers]

Marilyn Davis schreef op wo 14-11-2012 om 13:23 [-0800]:
> I found this site:
> http://hints.macworld.com/article.php?story=20100713130450549
> 
> and that fixes it.

Short answer: It is not a fix but a workaround. Try:
print symbol.encode('utf-8')

Longer answer: It is not really a fix, it is a workaround for the
implicit encode/decode python does. Setting this environment variable
will make the errors go away for *you* but what about the other people
running the code. What you are seeing is the *implicit* encode from a
unicode string to a byte string using sys.getdefaultencoding. What you
need to do is encode your *unicode* string to a *byte* string by using
encode() method. You should watch/read
http://nedbatchelder.com/text/unipain.html and see how to fix this
properly.

This quick interactive session shows how this error is properly solved.

$ python
Python 2.7.3 (default, Sep 22 2012, 18:13:33) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.stdout.encoding
'ascii'
>>> unicode_symbol = unichr(165)
>>> print unicode_symbol
Traceback (most recent call last):
  File "", line 1, in 
UnicodeEncodeError: 'ascii' codec can't encode character u'\xa5' in
position 0: ordinal not in range(128)
>>> byte_symbol = unicode_symbol.encode('utf-8')
>>> print byte_symbol
¥

I do not use macosx but I suspect it will support UTF-8, however if not
you need to find an encoding that is supported and has your character.

This can be quite confusing so I really suggest strongly to watch Ned's
talk ;-).

Greets
Sander

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] data analysis with python

[David Martins]

Thanks again for all the useful tips.
I settled with R for now. As Oscar said, the dataset is not massive so I could 
have done it using a dictionary. However some of the more frequent requests 
will include to find data during certain times during certain days, for 
specific months or weekdays vs. weekends, etc. I believe this would mean that I 
would have needed some indexing (which made me think of using databases in the 
first place). All of this seems to be quite easy in R as well
# The weather[3] column stores the string for the weekday
wkdays = which(weather[3] !="Sat"& weather[3] !="Sun")
I guess that would be easy enough with a list comprehension in python too. 
Binning looks like this:
heatcut = 
cut(as.matrix(lib[15]),breaks=c(0,max(lib[15])*0.1,max(lib[15])*0.2,max(lib[15])*0.3,max(lib[15])*0.4,max(lib[15])*0.5,max(lib[15])*0.6,max(lib[15])*0.7,max(lib[15])*0.8,max(lib[15])*0.9,max(lib[15])*1.0),labels=c('0%','10%','20%','30%','40%','50%','60%','80%','90%','100%'))
This can be added to a function. So the call will look something like 
bin_me(lib[15], breaks=default, labels=default).
To get one bin in sqlite I wrote this for a sqlite db (not sure if there is an 
easier way):
select count(Heating_plant_sensible_load) from LibraryMainwhere 
Heating_plant_sensible_load > (select max(Heating_plant_sensible_load)*0.3 from 
LibraryMain ANDHeating_plant_sensible_load < (select 
max(Heating_plant_sensible_load)*0.4 from LibraryMain;
Indexing for certain times, using my approach, would add even more lines; on 
top of this, I believe you would have to add this either to a view or a new 
table...
So are seems to be clearer and more concise compared to sql/sqlite (at least 
for me). On top of that it provides the possibility to do additional analysis 
later on for specific cases. That it can connect with python is another plus.
Thanks again for everyones ideasdm




> Date: Wed, 14 Nov 2012 13:59:25 +
> Subject: Re: [Tutor] data analysis with python
> From: oscar.j.benja...@gmail.com
> To: awesome.me...@outlook.com
> CC: tutor@python.org
> 
> On 14 November 2012 03:17, David Martins  wrote:
> > Hi All
> >
> > I'm trying to use python for analysing data from building energy simulations
> > and was wondering whether there is way to do this without using anything sql
> > like.
> 
> There are many ways to do this.
> 
> >
> > The simulations are typically run for a full year, every hour, i.e. there
> > are 8760 rows and about 100+ variables such as external air temperature,
> > internal air temperature, humidity, heating load, ... making roughly a
> > million data points. I've got the data in a csv file and also managed to
> > write it in a sqlite db.
> 
> This dataset is not so big that you can't just load it all into memory.
> 
> >
> > I would like to make requests like the following:
> >
> > Show the number of hours the aircon is running at 10%, 20%, ..., 100%
> > Show me the average, min, max air temperature, humidity, solar gains,
> > when the aircon is running at 10%, 20%,...,100%
> >
> > Eventually I'd also like to generate an automated html or pdf report with
> > graphs. Creating graphs is actually somewhat essential.
> 
> Do you mean graphs or plots? I would use matplotlib for plotting. It
> can automatically generate image files of plots. There are also ways
> to generate output for visualising graphs but I guess that's not what
> you mean. Probably I would create a pdf report using latex and
> matplotlib but that's not the only way.
> http://en.wikipedia.org/wiki/Graph_(mathematics)
> http://en.wikipedia.org/wiki/Plot_(graphics)
> 
> > I tried sql  and find it horrible, error prone, too much to write, the logic
> > somehow seems to work different than my brain and I couldn't find
> > particulary good documentation (particulary the documentation of the api is
> > terrible, in my humble opinion). I heard about zope db which might be an
> > alternative. Would you mind pointing me towards an appropriate way to solve
> > my problem? Is there a way for me to avoid having to learn sql or am I
> > doomed?
> 
> There are many ways to avoid learning SQL. I'll suggest the simplest
> one: Can you not just read all the data into memory and then perform
> the computations you want?
> 
> For example:
> 
> $ cat tmp.csv
> Temp,Humidity
> 23,85
> 25,87
> 26,89
> 23,90
> 24,81
> 24,80
> 
> $ cat tmp.py
> #!/usr/bin/env python
> 
> import csv
> 
> with open('tmp.csv', 'rb') as f:
> reader = csv.DictReader(f)
> data = []
> for row in reader:
> row = dict((k, float(v)) for k, v in row.items())
> data.append(row)
> 
> maxtemp = max(row['Temp'] for row in data)
> mintemp = min(row['Temp'] for row in data)
> meanhumidity = sum(row['Humidity'] for row in data) / len(data)
> 
> print('max temp is: %d' % maxtemp)
> print('min temp is: %d' % mintemp)
> print('mean humidity is: %f' % meanhumidity)
> 
> $ ./tmp.py
> max temp is: 26
> min temp is: 23
> mean humidity is: 85.33
> 
> This approa

[Tutor] Tkinter, how to retrieve information about an object on canvas

[Matheus Soares da Silva]

Hello, I would like to be able to get information from a Tkinter canvas
object. (color, width, tags, root points, etc),

 I wrote the following function that, with a canvas bind, returns me the
widget that has been clicked on, the widget is returned as a tuple by the
find_overlapping method.

# detecting click
def Hasclicked(e):
global obj
global lastClick
lastClick = [e.x, e.y]
obj = e.widget.find_overlapping(e.x, e.y, e.x, e.y)

  So, there's any method I can use on 'obj' to get the attributes?

(Tkinter Documentation: http://www.tkdocs.com/tutorial/index.html)
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor