Eric's probably right and it's indexing with a masked array that's causing you trouble. Since you seem to say your NaN values correspond to your mask, you should be able to simply do:
modelData[modeData.mask] = dataMin Note that in further processing it may then make more sense to remove the mask, since your array is now full with valid data: modelData = modelData.data -=- Olivier Le 27 janvier 2012 17:37, Howard <how...@renci.org> a écrit : > On 1/27/12 5:21 PM, Eric Firing wrote: > > On 01/27/2012 11:18 AM, Howard wrote: > > Hi all > > I am a fairly recent convert to python and I have got a question that's > got me stumped. I hope this is the right mailing list: here goes :) > > I am reading some time series data out of a netcdf file a single > timestep at a time. If the data is NaN, I want to reset it to the > minimum of the dataset over all timesteps (which I already know). The > data is in a variable of type numpy.ma.core.MaskedArray called modelData. > > If I do this: > > for i in range(len(modelData)): > if math.isnan(modelData[i]): > modelData[i] = dataMin > > I get the effect I want, If I do this: > > modelData[np.isnan(modelData)] = dataMin > > it doesn't seem to be working. Of course I could just do the first one, > but len(modelData) is about 3.5 million, and it's taking about 20 > seconds to run. This is happening inside of a rendering loop, so I'd > like it to be as fast as possible, and I thought the second one might be > faster, and maybe it is, but it doesn't seem to be working! :) > > It would help if you would say explicitly what you mean by "doesn't seem > to be working", ideally by providing a minimal complete example > illustrating the problem. > > Hi Eric > > Thanks for the reply. Yes, I can be a little more specific about the > issue. I am reading data from a storm surge model out of a NetCDF file so > I can render it with tricontourf. The model data has both a triangulation > and a set of lat, lon points that are invariant for the entire model run, > as well as data for each time step. As the model runs, triangles in the > coastal plain wet and dry: the dry values are indicated by NaN values in > the data and should not be rendered. Those I mask off previous to this > code. I have found, in using tricontourf, that in the mapping from data > values to color values, the range of the data seems to include even the > data from the masked triangles. This causes the data to be either > monochromatic or bi-chromatic (the high and low colors in the map). > However, once the triangles are masked, if I set the corresponding data > values to the known dataMin (or in fact, any value in the valid data range) > the render proceeds correctly. So in the case of the first piece of code, > I get reasonable images: using the second I do not. > > > Does modelData have masked values that you want to keep separate from > your NaN values? If not, you can do this: > > > No I don't think so. > > y = np.ma.masked_invalid(modelData).filled(dataMin) > > Then y will be an ordinary ndarray. If this is not satisfactory because > you need to keep separate some initially masked values, then you may > need to save the initial mask and use it to turn y back into a masked array. > > You may be running into trouble with your initial approach because using > np.isnan on a masked array is giving a masked array, and I think trying > to index with a masked array is not advised. > > This could certainly be be the issue. I will look into this Monday. > > Thanks very much for taking the time to reply. > Howard > > > In [2]: np.isnan(np.ma.array([1.0, np.nan, 2.0], mask=[False, False, True])) > Out[2]: > masked_array(data = [False True --], > mask = [False False True], > fill_value = True) > > Eric > > > Any ideas would be much appreciated. > > Thanks > Howard > > -- > Howard Lander <mailto:how...@renci.org> <how...@renci.org> > Senior Research Software Developer > Renaissance Computing Institute (RENCI) <http://www.renci.org> > <http://www.renci.org> > The University of North Carolina at Chapel Hill > Duke University > North Carolina State University > 100 Europa Drive > Suite 540 > Chapel Hill, NC 27517 > 919-445-9651 > > > > _______________________________________________ > NumPy-Discussion mailing > listNumPy-Discussion@scipy.orghttp://mail.scipy.org/mailman/listinfo/numpy-discussion > > _______________________________________________ > NumPy-Discussion mailing > listNumPy-Discussion@scipy.orghttp://mail.scipy.org/mailman/listinfo/numpy-discussion > > > > -- > Howard Lander <how...@renci.org> > > Senior Research Software Developer > Renaissance Computing Institute (RENCI) <http://www.renci.org> > The University of North Carolina at Chapel Hill > Duke University > North Carolina State University > 100 Europa Drive > Suite 540 > Chapel Hill, NC 27517 > 919-445-9651 > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > >
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
