[Tutor] Masked arrays & scipy.ndimage.
Dear tutors, Given an image, I want to generate another image with the mean values of the pixels in their neighbourhood, thereby ignoring some of the neighbouring pixels (e.g. the padded boundary). I hoped I could use masked arrays for that, but apparently, this does not work. -- def fnc(buffer) return numpy.mean(buffer) mean = scipy.ndimage.generic_filter(masked_array, fnc, footprint = f, mode = 'constant', cval = 0.0) -- Is there a way to get around this issue, or do I have to accept the filter function can not handle masked arrays? Thanks in advance! Sincerely, Koen ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Masked arrays & scipy.ndimage.
Koen De Munter wrote: Dear tutors, Given an image, I want to generate another image with the mean values of the pixels in their neighbourhood, thereby ignoring some of the neighbouring pixels (e.g. the padded boundary). I hoped I could use masked arrays for that, but apparently, this does not work. Define "does not work". Do you get an exception? Your computer crashes? You get a result, but it's not the result you were expecting? Something else? -- def fnc(buffer) return numpy.mean(buffer) mean = scipy.ndimage.generic_filter(masked_array, fnc, footprint = f, mode = 'constant', cval = 0.0) scipy, even more than numpy, is a fairly specialised library. You may have better luck asking this question on a dedicated scipy mailing list. Is there a way to get around this issue, or do I have to accept the filter function can not handle masked arrays? What does the documentation for generic_filter say? What happens if you test it with a really small, simple example, say an array of just five or six values? Can you show us what you expect to get and what you actually get? -- Steven ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
[Tutor] Benefit/Disadvantage to storing data as dictionary vs. in a class
I'm working on a class that handles multiple rooms that generate a large
amount of data. Currently my class model looks something like this (more
apologies for any indentation errors):
Class Model:
rooms= {}
for z in range(num_of_zones):
for i in range(24):
tmp[i] = { VAR:0, SD:0, AVG:0, READINGS:[] }
tmp[i]['updated'] = datetime.utcnow()
for j in OT_RANGE:
tmp[i][j] = { VAR:0, SD:0, AVG:0, READINGS:[] }
rooms[z] = tmp
In case that gets complicated, I'm looking to store readings based off
the current hour and current outside temperature. The Model class has
other variables and functions that it uses to compare and manipulate the
data as it comes in.
My question is in regards to the storing all this data in a dictionary,
which makes it easy to reference self.rooms[z][12][85][AVG]; however is
this better/worse or the same as creating a 'Room' class to store the
data? Since a 'Room' only contains a large amount of data, but doesn't
contain any functions, which form of storing the data is considered
'better'?
Thank you,
Adam
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Masked arrays & scipy.ndimage.
> Date: Fri, 1 Jun 2012 18:50:15 +1000 > From: st...@pearwood.info > To: tutor@python.org > Subject: Re: [Tutor] Masked arrays & scipy.ndimage. > > Koen De Munter wrote: >> Dear tutors, >> >> Given an image, I want to generate another image with the mean values of >> the pixels in their neighbourhood, thereby ignoring some of the >> neighbouring pixels (e.g. the padded boundary). I hoped I could use >> masked arrays for that, but apparently, this does not work. > > Define "does not work". > > Do you get an exception? Your computer crashes? You get a result, but it's not > the result you were expecting? Something else? > > > >> -- >> def fnc(buffer) >> return numpy.mean(buffer) > >> mean = scipy.ndimage.generic_filter(masked_array, fnc, footprint = f, mode = >> 'constant', cval = 0.0) > > > scipy, even more than numpy, is a fairly specialised library. You may have > better luck asking this question on a dedicated scipy mailing list. > >> Is there a way to get around this issue, or do I have to accept the filter >> function can not handle masked arrays? > > What does the documentation for generic_filter say? > > What happens if you test it with a really small, simple example, say an array > of just five or six values? Can you show us what you expect to get and what > you actually get? > > > > -- > Steven > > ___ > Tutor maillist - Tutor@python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor Hi Steven, Thanks for your reply. 1) "does not work" -- With this I mean that each time, all pixels belonging to the neighbourhood are involved in the calculation of the mean value. My neighbourhood is a circle, defined in a binary square footprint array (1 = neighbourhood, 0 = not neighbourhood). With a masked array, I aim to command the filter method as follows: "ignore those pixels that are identified by the mask array as 'invalid'". But, it seems the generic_method doesn't work with masked arrays: it only takes account for the data part of a masked array, not the mask part. So: result is not what I expect to get. For the bigger story I can refer to the following link: http://stackoverflow.com/questions/10683596/efficiently-calculating-boundary-adapted-neighbourhood-average 2) A small example -- d = array([[0, 0, 0, 0],[0, 1, 6, 0],[0, 0, 0, 0]]) #this represents an image array [1, 6], surrounded by zeroes d = d.astype(float) m = array([[1, 1, 1, 1],[1, 0, 0, 1],[1, 1, 1, 1]]) #the ones in the mask have a meaning of 'invalid' md = numpy.ma.masked_array(d, mask = m) f = array([[0, 1, 0],[1, 1, 1],[0, 1, 0]]) #small footprint def fnc(buffer): return numpy.mean(buffer) avg = scipy.ndimage.generic_filter(md, fnc, footprint = f, mode = 'constant', cval = 0.0) --> what I expect to get: array([[ 0. , 1. , 6. , 0. ], [ 1. , 3.5, 3.5, 6. ], [ 0. , 1. , 6. , 0. ]]) --> what I actually get: array([[ 0. , 0.2, 1.2, 0. ], [ 0.2, 1.4, 1.4, 1.2], [ 0. , 0.2, 1.2, 0. ]]) Footprint f has 5 elements. I expect this number to change according to the number of elements that should be ignored. But that doesn't. The code as I presented above always calculates the mean of the full 5 members of the footprint/neighbourhood. Regards, Koen ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Benefit/Disadvantage to storing data as dictionary vs. in a class
On Fri, Jun 1, 2012 at 9:12 AM, Adam wrote:
> I'm working on a class that handles multiple rooms that generate a large
> amount of data. Currently my class model looks something like this (more
> apologies for any indentation errors):
> Class Model:
>rooms= {}
>for z in range(num_of_zones):
>for i in range(24):
>tmp[i] = { VAR:0, SD:0, AVG:0, READINGS:[] }
>tmp[i]['updated'] = datetime.utcnow()
>for j in OT_RANGE:
>tmp[i][j] = { VAR:0, SD:0, AVG:0, READINGS:[] }
>rooms[z] = tmp
>
> In case that gets complicated, I'm looking to store readings based off the
> current hour and current outside temperature. The Model class has other
> variables and functions that it uses to compare and manipulate the data as
> it comes in.
>
> My question is in regards to the storing all this data in a dictionary,
> which makes it easy to reference self.rooms[z][12][85][AVG]; however is
> this better/worse or the same as creating a 'Room' class to store the data?
> Since a 'Room' only contains a large amount of data, but doesn't contain
> any functions, which form of storing the data is considered 'better'?
>
> Thank you,
> Adam
> __**_
> Tutor maillist - Tutor@python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/**mailman/listinfo/tutor
>
If you have functions (or in this case methods) which will be used on a
certain 'objects' data structure, a class representation of the data is the
natural fit. Otherwise, you will need to pass the dictionary around to
various functions and this will be seriously confusing.
However, you probably should consider how to represent that data within a
class body, that is, it is extremely confusing to access data such
as self.rooms[z][12][85][AVG].
Instead, you should break down what the data structure into like-things and
how they are related, for example, you might consider something like this:
class Weather(object):
def __init__(self, temp, precipitation, humidity, bara_pres):
self. temp = temp
self. temp = precipitation
self. temp = humidity
self. temp = bara_pres
def my_func(self, *args,**kwargs):
pass #do stuff here
However, if you need to either further define the datapoints of if you want
the datapoints to be able to "do things" then you could do something like
this:
class Temperature(object):
def __init__(self, temp):
self. temp = temp
self.other_stuff_about_temp = stuff
def my_temp_func(self, *args,**kwargs):
pass #do stuff here
class Weather(object):
def __init__(self, temp, precipitation, humidity, bara_pres):
self. temp = Temperature(temp)
self.precipitation = precipitation
self.humidity = humidity
self.bara_pres = bara_pres
def my_func(self, *args,**kwargs):
pass #do stuff here
from here you could either put your instantiated objects into a list and do
math on the elements thereof, or you could put them into another class
object like this:
class WeatherOverTime(object):
def __init__(self, my_list):
self.my_list = my_list
def avg(self, attr): #attr is the attribute you want to take an average
of, so humidity, temp, whatever.
temp_list = map(lambda x: getattr(x, attr), my_list)
return sum(temp_list) / count(temp_list)
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Benefit/Disadvantage to storing data as dictionary vs. in a class
At least in my own case, the 'self.rooms[z][12][85]' is a means to
accessing a 'bucket' array for a given room (z), during a given hour
(12), at a given temperature(85). Each bucket holds, the average,
variance, standard deviation and an array of previous readings (note,
readings are not temperatures, these are actually timestamps).
Is there any benefit to creating a class for each bucket (with an
'update' method) vs. storing those variables as a dictionary with the
'Room' class as {'var':0, 'avg': 0, 'sd':0, 'readings':[]} and having
the Room class hold the update function; not so much from good
programming practices, but more attempting to number of resources the
class will be using.
Thank you,
Adam
On 6/1/2012 10:24 AM, James Reynolds wrote:
If you have functions (or in this case methods) which will be used on
a certain 'objects' data structure, a class representation of the data
is the natural fit. Otherwise, you will need to pass the dictionary
around to various functions and this will be seriously confusing.
However, you probably should consider how to represent that data
within a class body, that is, it is extremely confusing to access data
such as self.rooms[z][12][85][AVG].
Instead, you should break down what the data structure into
like-things and how they are related, for example, you might consider
something like this:
class Weather(object):
def __init__(self, temp, precipitation, humidity, bara_pres):
self. temp = temp
self. temp = precipitation
self. temp = humidity
self. temp = bara_pres
def my_func(self, *args,**kwargs):
pass #do stuff here
However, if you need to either further define the datapoints of if you
want the datapoints to be able to "do things" then you could do
something like this:
class Temperature(object):
def __init__(self, temp):
self. temp = temp
self.other_stuff_about_temp = stuff
def my_temp_func(self, *args,**kwargs):
pass #do stuff here
class Weather(object):
def __init__(self, temp, precipitation, humidity, bara_pres):
self. temp = Temperature(temp)
self.precipitation = precipitation
self.humidity = humidity
self.bara_pres = bara_pres
def my_func(self, *args,**kwargs):
pass #do stuff here
from here you could either put your instantiated objects into a list
and do math on the elements thereof, or you could put them into
another class object like this:
class WeatherOverTime(object):
def __init__(self, my_list):
self.my_list = my_list
def avg(self, attr): #attr is the attribute you want to take an
average of, so humidity, temp, whatever.
temp_list = map(lambda x: getattr(x, attr), my_list)
return sum(temp_list) / count(temp_list)
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] iPython check if user running script is root user (Linux)
> > Since this is my first question to the list, is it appropriate to reply > > with a "thanks, that solved it" or is that considered unnecessary? > > Its not necessary but its not frowned on either. It is most useful where > numerous different options are being suggested since it indicates to > future googlers which option worked best for the OP... I think everyone likes to be thanked. Ramit Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology 712 Main Street | Houston, TX 77002 work phone: 713 - 216 - 5423 -- This email is confidential and subject to important disclaimers and conditions including on offers for the purchase or sale of securities, accuracy and completeness of information, viruses, confidentiality, legal privilege, and legal entity disclaimers, available at http://www.jpmorgan.com/pages/disclosures/email. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Benefit/Disadvantage to storing data as dictionary vs. in a class
Hi Adam,
On 1 June 2012 15:45, Adam wrote:
> Is there any benefit to creating a class for each bucket (with an 'update'
> method) vs. storing those variables as a dictionary with the 'Room' class as
> {'var':0, 'avg': 0, 'sd':0, 'readings':[]} and having the Room class hold
> the update function; not so much from good programming practices, but more
> attempting to number of resources the class will be using.
I'd submit that worrying about resource usage above/beyond/to the
exclusion of worrying about the easy maintenance and readability of
your code at this apparently early stage may be a sign of premature
optimisation. IMHO the difference in resource usage is going to be
fairly negligible either way and you should lean towards prioritising
keeping your code easy to read and easy to maintain at this point.
How many rooms/buckets are we talking about anyway? (I might add I
like Adam also find the triple indexed self.rooms[z][12][85] obscure
and would prefer even just say a method with positional and/or names
parameters. But I suppose I may feel differently if I worked with
your API for a while. :) )
Walter
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
[Tutor] (no subject)
In python, why does 17/-10= -2? Shouldn't it be -1? Godsend221 ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] (no subject)
Jason Barrett wrote: > In python, why does 17/-10= -2? Shouldn't it be -1? http://docs.python.org/faq/programming.html#why-does-22-10-return-3 ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
[Tutor] Connecting to MySQLdb
Hello, I have downloaded MySQLdb 1.2.3 from: http://pypi.python.org/pypi/MySQL-python Also, I'm using Python 2.7, but I can't even get past here: >>> import MySQLdb Traceback (most recent call last): File "", line 1, in ImportError: No module named MySQLdb Thanks ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Connecting to MySQLdb
On Fri, 1 Jun 2012 19:01:00 -0400 Nicholas Picciano wrote: > Hello, > > I have downloaded MySQLdb 1.2.3 from: > > http://pypi.python.org/pypi/MySQL-python > > Also, I'm using Python 2.7, but I can't even get past here: > > >>> import MySQLdb > Traceback (most recent call last): > File "", line 1, in > ImportError: No module named MySQLdb > > Thanks Did you install it? Assuming UNIX(-like): I highly, *highly* recommend you use a virtualenv [1] and, if you want a fancy UI, virtualenvwrapper [2]. From there you can install all the packages you want with pip, without having to worry about mucking up the system, versions of things, etc. Assuming Windows: Good luck! -- Corey Richardson ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Connecting to MySQLdb
Virtualenv works just fine in windows On Jun 1, 2012 7:20 PM, "Corey Richardson" wrote: > On Fri, 1 Jun 2012 19:01:00 -0400 > Nicholas Picciano wrote: > > > Hello, > > > > I have downloaded MySQLdb 1.2.3 from: > > > > http://pypi.python.org/pypi/MySQL-python > > > > Also, I'm using Python 2.7, but I can't even get past here: > > > > >>> import MySQLdb > > Traceback (most recent call last): > > File "", line 1, in > > ImportError: No module named MySQLdb > > > > Thanks > > Did you install it? > > Assuming UNIX(-like): > > I highly, *highly* recommend you use a virtualenv [1] and, if you want > a fancy UI, virtualenvwrapper [2]. From there you can install all the > packages you want with pip, without having to worry about mucking up > the system, versions of things, etc. > > Assuming Windows: > > Good luck! > > -- > Corey Richardson > ___ > Tutor maillist - Tutor@python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Benefit/Disadvantage to storing data as dictionary vs. in a class
Adam wrote:
I'm working on a class that handles multiple rooms that generate a large
amount of data. Currently my class model looks something like this (more
apologies for any indentation errors):
In my experience, what people consider "a large amount of data" and what the
computer considers large is often very, very different.
You talk about "multiple rooms" with 24 records per day (one per hour). If
each room has 100 readings per hour, and 100 outside temperatures each with
100 readings, I would expect you to need under 50 MB per room. That's not that
much these days.
My question is in regards to the storing all this data in a dictionary,
which makes it easy to reference self.rooms[z][12][85][AVG]; however is
this better/worse or the same as creating a 'Room' class to store the
data? Since a 'Room' only contains a large amount of data, but doesn't
contain any functions, which form of storing the data is considered
'better'?
In modern object-oriented languages like Python, the fact that a class doesn't
contain any methods (functions) is not a problem. Objects can be used as the
equivalent of Pascal records or C structs too.
Besides, Python dicts and lists are already objects. The amount of overhead in
putting them together into programmer-friendly classes with named attributes
is trivial. Assuming my figure of 50 MB above is accurate, I'd expect the
equivalent figure for a class-based solution would be 50.2 MB. Big deal.
Write your code to make it is easy and simple to read and write as possible.
Program as if the next guy who maintains the program will be a violent
psychopath who knows where you live.
Your aim is to have code that is so simple that it is obviously correct,
rather than so complicated that there are no obvious bugs. Expressions like:
self.rooms[z][12][85][AVG]
strike me as tending towards the later ("no obvious bugs") than the first.
--
Steven
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
