I would say pandas is really cool. More people need to know it. and we should have better documentation.
cheers, Chao 2011/10/18 Bruce Southey <bsout...@gmail.com> > ** > On 10/18/2011 09:12 AM, Chao YUE wrote: > > thanks. Olivier. I see. > > Chao > > 2011/10/18 Olivier Delalleau <sh...@keba.be> > >> As far as I can tell ma.mean() is working as expected here: it computes >> the mean only over non-masked values. >> If you want to get rid of any mean that was computed over a series >> containing masked value you can do: >> >> b = a.mean(0) >> b.mask[a.mask.any(0)] = True >> >> Then b will be: >> >> masked_array(data = [5.0 -- -- 8.0 9.0 -- 11.0 12.0 -- 14.0], >> mask = [False True True False False True False False True >> False], >> fill_value = 1e+20) >> >> -=- Olivier >> >> 2011/10/18 Chao YUE <chaoyue...@gmail.com> >> >>> Dear all, >>> >>> previoulsy I think np.ma.mean() will automatically filter the masked >>> (missing) value but it's not? >>> In [489]: a=np.arange(20.).reshape(2,10) >>> >>> In [490]: >>> a=np.ma.masked_array(a,(a==2)|(a==5)|(a==11)|(a==18),fill_value=np.nan) >>> >>> In [491]: a >>> Out[491]: >>> masked_array(data = >>> [[0.0 1.0 -- 3.0 4.0 -- 6.0 7.0 8.0 9.0] >>> [10.0 -- 12.0 13.0 14.0 15.0 16.0 17.0 -- 19.0]], >>> mask = >>> [[False False True False False True False False False False] >>> [False True False False False False False False True False]], >>> fill_value = nan) >>> >>> In [492]: a.mean(0) >>> Out[492]: >>> masked_array(data = [5.0 1.0 12.0 8.0 9.0 15.0 11.0 12.0 8.0 14.0], >>> mask = [False False False False False False False False >>> False False], >>> fill_value = 1e+20) >>> >>> In [494]: np.ma.mean(a,0) >>> Out[494]: >>> masked_array(data = [5.0 1.0 12.0 8.0 9.0 15.0 11.0 12.0 8.0 14.0], >>> mask = [False False False False False False False False >>> False False], >>> fill_value = 1e+20) >>> >>> In [495]: np.ma.mean(a,0)==a.mean(0) >>> Out[495]: >>> masked_array(data = [ True True True True True True True True >>> True True], >>> mask = False, >>> fill_value = True) >>> >>> only use a.filled().mean(0) can I get the result I want: >>> In [496]: a.filled().mean(0) >>> Out[496]: array([ 5., NaN, NaN, 8., 9., NaN, 11., 12., NaN, >>> 14.]) >>> >>> I am doing this because I tried to have a small fuction from the web to >>> do moving average for data: >>> >>> import numpy as np >>> def rolling_window(a, window): >>> if window < 1: >>> raise ValueError, "`window` must be at least 1." >>> if window > a.shape[-1]: >>> raise ValueError, "`window` is too long." >>> shape = a.shape[:-1] + (a.shape[-1] - window + 1, window) >>> strides = a.strides + (a.strides[-1],) >>> return np.lib.stride_tricks.as_strided(a, shape=shape, >>> strides=strides) >>> >>> def move_ave(a,window): >>> temp=rolling_window(a,window) >>> pre=int(window)/2 >>> post=int(window)-pre-1 >>> return >>> np.concatenate((a[...,0:pre],np.mean(temp,-1),a[...,-post:]),axis=-1) >>> >>> >>> In [489]: a=np.arange(20.).reshape(2,10) >>> >>> In [499]: move_ave(a,4) >>> Out[499]: >>> masked_array(data = >>> [[ 0. 1. 1.5 2.5 3.5 4.5 5.5 6.5 7.5 9. ] >>> [ 10. 11. 11.5 12.5 13.5 14.5 15.5 16.5 17.5 19. ]], >>> mask = >>> False, >>> fill_value = 1e+20) >>> >>> thanks, >>> >>> Chao >>> >>> -- >>> >>> *********************************************************************************** >>> Chao YUE >>> Laboratoire des Sciences du Climat et de l'Environnement (LSCE-IPSL) >>> UMR 1572 CEA-CNRS-UVSQ >>> Batiment 712 - Pe 119 >>> 91191 GIF Sur YVETTE Cedex >>> Tel: (33) 01 69 08 29 02; Fax:01.69.08.77.16 >>> >>> ************************************************************************************ >>> >>> >>> _______________________________________________ >>> NumPy-Discussion mailing list >>> NumPy-Discussion@scipy.org >>> http://mail.scipy.org/mailman/listinfo/numpy-discussion >>> >>> >> >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion@scipy.org >> http://mail.scipy.org/mailman/listinfo/numpy-discussion >> >> > > > -- > > *********************************************************************************** > Chao YUE > Laboratoire des Sciences du Climat et de l'Environnement (LSCE-IPSL) > UMR 1572 CEA-CNRS-UVSQ > Batiment 712 - Pe 119 > 91191 GIF Sur YVETTE Cedex > Tel: (33) 01 69 08 29 02; Fax:01.69.08.77.16 > > ************************************************************************************ > > > _______________________________________________ > NumPy-Discussion mailing > listNumPy-Discussion@scipy.orghttp://mail.scipy.org/mailman/listinfo/numpy-discussion > > Looked at pandas for your rolling window functionality: > http://pandas.sourceforge.net > > *"Time series*-specific functionality: date range generation and frequency > conversion, moving window statistics, moving window linear regressions, date > shifting and lagging, etc." > > Bruce > > > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > > -- *********************************************************************************** Chao YUE Laboratoire des Sciences du Climat et de l'Environnement (LSCE-IPSL) UMR 1572 CEA-CNRS-UVSQ Batiment 712 - Pe 119 91191 GIF Sur YVETTE Cedex Tel: (33) 01 69 08 29 02; Fax:01.69.08.77.16 ************************************************************************************
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
