Re: Quickie - Regexp for a string not at the beginning of the line
On Thu, 25 Oct 2012 18:08:53 -0700 (PDT), Rivka Miller wrote in <73f60cf3-d932-4366-a405-676748856...@q16g2000yqc.googlegroups.com>: >no one has really helped yet. We regret that you are not a satisfied customer. Please take your receipt to the cashier and you will receive double your money back according to our "you must be satisfied" guarantee. -- http://mail.python.org/mailman/listinfo/python-list
SAX parsing problem
So I've encountered a strange behavior that I'm hoping someone can fill me in on. i've written a simple handler that works with one small exception, when the parser encounters a line with '&' in it, it only returns the portion that follows the occurence. For example, parsing a file with the line : mykeysome%20&%20value results in getting "%20value" back from the characters method, rather than "some%20&%20value". After looking into this a bit, I found that SAX supports entities and that it is probably believing the & to be an entity and processing it in some way that i'm unware of. I'm using the default EntityResolver. Any help/info would be much appreciated. gh -- http://mail.python.org/mailman/listinfo/python-list
[Newbie] How do I get better at Python programming?
I've gotten off to a good start for programming using Python (my first programming language). I can do the basics (different kinds of loops, variables, functions), but I'm not too good with classes yet. I've written some really small programs just for kicks (most involve statistics of different situations using random numbers). I'd like to get to the next level, but I'm not sure how. Are there any suggestions for continuing to learn? How did you guys learn? -- http://mail.python.org/mailman/listinfo/python-list
Importing a class, please help...
Hi, Newbie to Python and I have a question please. I am using Windows XP, SPE 0.8.2.a and Python24. I have done this: import sys print sys.path no problem there, sys imports just fine. I have a folder that I called c\JavaProjects\PythonTesting and it shows up in the output from sys.path above. I have dropped a very simple Jar in that folder and I have attempted to do an Import on it. This is where things fail. I cannot see the contents of my Jar. What am I missing here? Would somebody please drop me a hint, please? I would HIGHLY appreciate an example; does anybody know of a website or page that would show me how to do this? I realize this is a simple question and I appreciate any help somebody would give me. Regards, joe -- http://mail.python.org/mailman/listinfo/python-list
Re: Importing a class, please help...
Xavier Yes I meant Python. I have also been using Jython and I am VERY WELL aware what a JAR is. Jython has the ability to call methods stored in classes in a JAR. I was only asking if this could be done with Python also. You ask "why the hell would a Python script be able to read a JAR..". Well, I was simply asking a newbie question. The only way to gather knowladge is by asking questions. You know, an old college proffesor I had was very knowladgeable and wise. He always took the time to answer questions, no matter how 'stupid' they would sound. One can make a bigger impression on folks by stepping down a couple of steps and give a polite answer. I mean no disrespect, just a suggestion when you post answers to folks like me who are just getting started with a new language... regards "Xavier Morel" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] > anon wrote: >> Would somebody please drop me a hint, please? >> > Yeah, the definition of "JAR" is Java ARchive, why the hell would a Python > script be able to read a JAR in the first place (truth is it is, a JAR > file is nothing but a renamed ZIP, therefore the zipfile module allows you > to read it's content) and -- more importantly -- import Java classes. > > Notice the difference between *Python* script and *Java* class? That's > because they're two different languages. > If you want to use Java classes and modules in a Pythonic context, you > want Jython (http://www.jython.org/) not Python (http://www.python.org/) -- http://mail.python.org/mailman/listinfo/python-list
Re: Threading change, 2.5.4 -> 2.6.1
Gib Bogle wrote: The code below runs with Python 2.5.4, but gives the following error messages with Python 2.6.1. What needs to be done to make it work? Thanks. C:\Summer09\Tutorials>python url_queue.pyw Traceback (most recent call last): File "url_queue.pyw", line 3, in import threading File "C:\Summer09\Tutorials\threading.py", line 9, in class ProcessingThread(threading.Thread, QtCore.QObject): AttributeError: 'module' object has no attribute 'Thread' Nothing to do with the version of python You have a file called threading.py in C:\Summer09\Tutorials (the same folder as url_queue.pyw). The line 'import threading' is finding this module before python's threading module. Rename this file and you should be fine. -- http://mail.python.org/mailman/listinfo/python-list
Re: One function calling another defined in the same file being exec'd
Rather than exec the files, why not import them? I can get both your examples to work using the 'imp' module. http://docs.python.org/3.1/library/imp.html#module-imp I used python 2.6.4. Note that 3.1 also has 'importlib' module. import imp # the name of the python file written by a user name = 'test1' fp, pathname, description = imp.find_module(name) test1 = imp.load_module(name, fp, pathname, description) print test1.result # remember to close file (see docs) fp.close() -- http://mail.python.org/mailman/listinfo/python-list
Re: A question about making a sort-of-counter.
Justin Park wrote: Suppose I have a list. a = list() And suppose allowed digits as the element are 1,2,3,4,5. What can I do in order to iterate over all possible values for each element? For instance, the sequence of the list I want to have would be [1,1,1,1,1] [1,1,1,1,2] [1,1,1,1,3] [5,5,5,5,4] [5,5,5,5,5] How can I make it happen? Thanks, Justin. http://docs.python.org/library/itertools.html#itertools.product from itertools import product for item in product([1, 2], repeat=2): print item -- http://mail.python.org/mailman/listinfo/python-list
Re: escape character / csv module
V N wrote:
string "\x00" has a length of 1. When I use the csv module to write
that to a file
csv_f = csv.writer(file("test.csv","wb"),delimiter="|")
csv_f.writerow(["\x00","zz"])
The output file looks like this:
|zz
Is it possible to force the writer to write that string?
This will do what you want:
"\\x00"
--
http://mail.python.org/mailman/listinfo/python-list
turbogears 1
Hey guys, someone asked me to maintain his old website, trouble is, it's in python, more trouble is it's in turbogears 1. I'm not fluent in python but all right, I can learn, but this turbogears thing.. First of all, is it still alive? Looks like turbogears 2 is the most recent version but even that is being abandoned. Am I basically given vaporware? Does anyone have any up to date info? Best, Hung -- Viktor Orban Prime minister of Hungary http://spreadingviktororban.weebly.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Komodo 7 release (Python development tools)
>> My name is Todd. I'm the lead developer for Komodo IDE (Interactive >> Development Environment) and Komodo Edit (a free, open-source editor) at >> ActiveState. I wanted to announce that the newest version, Komodo 7, has >> been released: > > This is a pretty good release announcement, but a few questions. > ... > >http://www.activestate.com/komodo-ide/python-editor > > It would seem that the Professional Python Editor is the same as Komodo > Edit, but it is unclear why only Python editing would be featured for > Komodo IDE. > > http://www.activestate.com/komodo-edit > > is the page with the link people need to download just the editor. > > Does K.Edit let me run a program with one key, like F5 in IDLE? AFAIK it sure does! > If so, does it leave me in interactive mode (python -i) as IDLE does? Hmmm, I don't know about that. Best, anonhung -- http://spreadingviktororban.weebly.com";>Viktor Orban Prime minister of Hungary -- http://mail.python.org/mailman/listinfo/python-list
Re: turbogears 1
>> Hey guys, someone asked me to maintain his old website, trouble is, >> it's in python, more trouble is it's in turbogears 1. I'm not fluent >> in python but all right, I can learn, but this turbogears >> thing.. >> >> First of all, is it still alive? Looks like turbogears 2 is the most >> recent version but even that is being abandoned. > > Yup, looks dead to me. Hasn't had a release in almost 2 months or a > commit to the GIT repo in 2 days. Must be nailed to its perch or > something. > > http://turbogears.org/en/current-status > http://sourceforge.net/p/turbogears2/tg2/commit_browser All right, what I got confused about is that they talk about pyramid these days which will not be turbogears 2 based. >> Am I basically given vaporware? Does anyone have any up to date info? > > Have you considered Ruby on Rails? This is joke, right? :) Best, anonhung -- Viktor Orban Prime minister of Hungary http://spreadingviktororban.weebly.com -- http://mail.python.org/mailman/listinfo/python-list
Re: turbogears 1
>>> Hey guys, someone asked me to maintain his old website, trouble is, >>> it's in python, more trouble is it's in turbogears 1. I'm not fluent in >>> python but all right, I can learn, but this turbogears thing.. >>> >>> First of all, is it still alive? Looks like turbogears 2 is the most >>> recent version but even that is being abandoned. >> >> Yup, looks dead to me. Hasn't had a release in almost 2 months or a >> commit to the GIT repo in 2 days. Must be nailed to its perch or >> something. >> >> http://turbogears.org/en/current-status >> http://sourceforge.net/p/turbogears2/tg2/commit_browser > > Ohhh, sarcasm... > > To the original poster: what makes you think that Turbogears 2 is > abandoned? As I've said in another reply they keep talking about pyramid as if that will be a totally new framework. I'm not saying I know things definitely but surely my impression was that the team is out for something new not based on turbogears 2. >>> Am I basically given vaporware? Does anyone have any up to date info? >> >> Have you considered Ruby on Rails? > > Now that's just cruel. Yep, it is :) Best, anonhung -- Viktor Orban Prime minister of Hungary http://spreadingviktororban.weebly.com -- http://mail.python.org/mailman/listinfo/python-list
Fwd: turbogears 1
>>> Hey guys, someone asked me to maintain his old website, trouble is, >>> it's in python, more trouble is it's in turbogears 1. I'm not fluent in >>> python but all right, I can learn, but this turbogears thing.. >>> >>> First of all, is it still alive? Looks like turbogears 2 is the most >>> recent version but even that is being abandoned. >> >> Yup, looks dead to me. Hasn't had a release in almost 2 months or a >> commit to the GIT repo in 2 days. Must be nailed to its perch or >> something. >> >> http://turbogears.org/en/current-status >> http://sourceforge.net/p/turbogears2/tg2/commit_browser > > Ohhh, sarcasm... > > To the original poster: what makes you think that Turbogears 2 is > abandoned? The development team keeps talking about pyramid which is supposed to be a totally new framework. I don't pretend to understand the work flow but surely they got me confused about switching gears (no pun intended!). >>> Am I basically given vaporware? Does anyone have any up to date info? >> >> Have you considered Ruby on Rails? > > Now that's just cruel. Yep, it is :) Best, anonhung -- Viktor Orban Prime minister of Hungary http://spreadingviktororban.weebly.com -- Viktor Orban Prime minister of Hungary http://spreadingviktororban.weebly.com -- http://mail.python.org/mailman/listinfo/python-list
Re: standalone python web server
>> I am building a small intranet website and I would like to use Python. I >> was wondering if there was a easy and medium performance python based web >> server available. I would like to run it on port :8080 since I wont have >> root access also I prefer something easy to deploy meaning I would like >> to >> move the server from one physical host to another without too much fuss. >> >> Currently, I am using flask (development server) and everything is ok but >> the performance is really bad. How about cherrypy? Best, anonhung -- Viktor Orban Prime minister of Hungary http://spreadingviktororban.weebly.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Issue with Scrapping Data from a webpage- Noob Question
> Hi Fellow Pythoners,
>
> I'm trying to collect table data from an authenticated webpage (Tool) to
> which I have access.
>
> I will have the required data after 'click'ing a submit button on the tool
> homepage.
> When I inspect the submit button i see
>
>
> Thus the tool's homepage is of the form www.example.com/Tool and on
> clicking the submit button the data I need will be at
> www.example.com/Tool/index.do
>
> The problem that I'm running into is in my below code is giving me the
> source of homepage(www.example.com/Tool) and not the of the submitted page(
> www.example.com/Tool/index.do)
>
> url="www.example.com/Tool/index.do"
> request = urllib2.Request(url, data, {'Authorization': "Basic " +
> base64.b64encode("%s:%s" % (username, password))})
> Response_Page=urllib2.urlopen(request).read()
>
> Is there a way I can access the source of the submitted page?
>
> PS: Sorry for laying out very tiny details on what I'm trying to do, I just
> wanted to explain myself clearly :)
>
> Thanks in advance for your time on this one.
Have you checked beautifulsoup?
Best,
anonhung
--
Viktor Orban Prime minister of Hungary
http://spreadingviktororban.weebly.com
--
http://mail.python.org/mailman/listinfo/python-list
Re: frozendict
> I've been trying for a few days (only a little bit at a time) to come up > with a way of implementing a frozendict that doesn't suck. I'm gradually > converging to a solution, but I can't help but think that there's some > subtlety that I'm probably missing which is why it's not already provided. The freezing temperature of typical dicts is around - 10 Celsius. HTH, anonhung -- Viktor Orban Prime minister of Hungary http://spreadingviktororban.weebly.com -- http://mail.python.org/mailman/listinfo/python-list
Re: ANN: eGenix mxODBC Zope Database Adapter 2.0.2
Thanks a bunch for the whole team! Best, anonhung On 2/9/12, eGenix Team: M.-A. Lemburg wrote: > > ANNOUNCEMENT > > mxODBC Zope Database Adapter > > Version 2.0.2 > > for Zope and the Plone CMS > > Available for Zope 2.10 and later on > Windows, Linux, Mac OS X, FreeBSD and other platforms > > This announcement is also available on our web-site for online reading: > http://www.egenix.com/company/news/eGenix-mxODBC-Zope-DA-2.0.2-GA.html > > > INTRODUCTION > > The eGenix mxODBC Zope Database Adapter allows you to easily connect > your Zope or Plone installation to just about any database backend on > the market today, giving you the reliability of the commercially > supported eGenix product mxODBC and the flexibility of the ODBC > standard as middle-tier architecture. > > The mxODBC Zope Database Adapter is highly portable, just like Zope > itself and provides a high performance interface to all your ODBC data > sources, using a single well-supported interface on Windows, Linux, > Mac OS X, FreeBSD and other platforms. > > This makes it ideal for deployment in ZEO Clusters and Zope hosting > environments where stability and high performance are a top priority, > establishing an excellent basis and scalable solution for your Plone > CMS. > > Product page: > > http://www.egenix.com/products/zope/mxODBCZopeDA/ > > > NEWS > > We are pleased to announce a new version 2.0.2 of our mxODBC Zope DA > product. > > With the patch level 2.0.2 release we have updated the integrated > mxODBC Python Extension to the latest 3.1.1 release, which > includes a number of important workarounds for these ODBC drivers: > > * Oracle 10gR1 and 10gR2 > * Oracle 11gR1 and 11gR2 > * Teradata 13 > * Netezza > > Due to popular demand, we have also added instructions on how to > install mxODBC Zope DA 2.0 with Plone 4.1 and Zope 2.13 - even though > this combination is not officially supported by the mxODBC Zope DA 2.0 > series: > > http://www.egenix.com/products/zope/mxODBCZopeDA/#Installation > > > UPGRADING > > Licenses purchased for version 2.0.x of the mxODBC Zope DA will continue > to work with the 2.0.2 patch level release. > > Licenses purchased for version 1.0.x of the mxODBC Zope DA will not > work with version 2.0. More information about available licenses > is available on the product page: > > http://www.egenix.com/products/zope/mxODBCZopeDA/#Licensing > > Compared to the popular mxODBC Zope DA 1.0, version 2.0 offers > these enhancements: > > * Includes mxODBC 3.1 with updated support for many current ODBC >drivers, giving you more portability and features for a wider >range of database backends. > > * Mac OS X 10.6 (Snow Leopard) support. > > * Plone 3.2, 3.3, 4.0 support. Plone 4.1 works as well. > > * Zope 2.10, 2.11, 2.12 support. Zope 2.13 works as well. > > * Python 2.4 - 2.6 support. > > * Zero maintenance support to automatically reconnect the >Zope connection after a network or database problem. > > * More flexible Unicode support with options to work with >pure Unicode, plain strings or mixed setups - even for >databases that don't support Unicode > > * Automatic and transparent text encoding and decoding > > * More flexible date/time support including options to work >with Python datetime objects, mxDateTime, strings or tuples > > * New decimal support to have the Zope DA return decimal >column values using Python's decimal objects. > > * Fully eggified to simplify easy_install and zc.buildout based >installation > > > MORE INFORMATION > > For more information on the mxODBC Zope Database Adapter, licensing > and download instructions, please visit our web-site: > > http://www.egenix.com/products/zope/mxODBCZopeDA/ > > You can buy mxODBC Zope DA licenses online from the eGenix.com shop at: > > http://shop.egenix.com/ > > > > Thank you, > -- > Marc-Andre Lemburg > eGenix.com > > Professional Python Services directly from the Source (#1, Feb 09 2012) Python/Zope Consulting and Support ...http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ > > > ::: Try our new mxODBC.Connect Python Database Interface for free ! > > >eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 > D-40764 Langenfeld, Germany.
memory error
Hello All,
I keep coming across a memory error when processing many netcdf files. I
assume it has something to do with how I loop things and maybe need to close
things off properly.
In the code below I am looping through a bunch of netcdf files (each file is
hourly data for one month) and within each netcdf file I am outputting a
*png file every three hours.
This works for one netcdf file but when it begins to process the next netcdf
file I receive this memory error:
*Traceback (most recent call last):
File
"d:/plot_netcdf_merc_multiplot_across_multifolders_mkdirs_memoryerror.py",
line 44, in
TSFC=ncfile.variables['T_SFC'][:]
File "netCDF4.pyx", line 2473, in netCDF4.Variable.__getitem__
(netCDF4.c:23094)
MemoryError*
To reduce processing requirements I have tried making the LAT and LON to
only use [0] but I also receive an error:
*Traceback (most recent call last):
File
"d:/plot_netcdf_merc_multiplot_across_multifolders_mkdirs_memoryerror.py",
line 75, in
x,y=map(*N.meshgrid(LON,LAT))
File "C:\Python27\lib\site-packages\numpy\lib\function_base.py", line
3256, in meshgrid
numRows, numCols = len(y), len(x) # yes, reversed
TypeError: len() of unsized object*
finally I have added gc.collect() in a couple of places but that doesn't
seem to do anything to help.
I am using :*Python 2.7.2 |EPD 7.1-2 (32-bit)| (default, Jul 3 2011,
15:13:59) [MSC v.1500 32 bit (Intel)] on win32*
Any feedback will be greatly appreciated!
from netCDF4 import Dataset
import numpy
import numpy as N
import matplotlib.pyplot as plt
from numpy import ma as MA
from mpl_toolkits.basemap import Basemap
from netcdftime import utime
from datetime import datetime
import os
import gc
print "start processing"
inputpath=r'E:/GriddedData/Input/'
outputpath=r'E:/GriddedData/Validation/'
shapefile1="E:/test_GIS/DSE_REGIONS"
for (path, dirs, files) in os.walk(inputpath):
for dir in dirs:
print dir
sourcepath=os.path.join(path,dir)
relativepath=os.path.relpath(sourcepath,inputpath)
newdir=os.path.join(outputpath,relativepath)
if not os.path.exists(newdir):
os.makedirs(newdir)
for ncfile in files:
if ncfile[-3:]=='.nc':
print "dealing with ncfiles:", ncfile
ncfile=os.path.join(sourcepath,ncfile)
#print ncfile
ncfile=Dataset(ncfile, 'r+', 'NETCDF4')
TSFC=ncfile.variables['T_SFC'][:,:,:]
TIME=ncfile.variables['time'][:]
LAT=ncfile.variables['latitude'][:]
LON=ncfile.variables['longitude'][:]
fillvalue=ncfile.variables['T_SFC']._FillValue
TSFC=MA.masked_values(TSFC, fillvalue)
ncfile.close()
gc.collect()
print "garbage collected"
for TSFC, TIME in zip((TSFC[1::3]),(TIME[1::3])):
print TSFC, TIME
#convert time from numbers to date and prepare it to have no
symbols for saving to filename
cdftime=utime('seconds since 1970-01-01 00:00:00')
ncfiletime=cdftime.num2date(TIME)
print ncfiletime
timestr=str(ncfiletime)
d = datetime.strptime(timestr, '%Y-%m-%d %H:%M:%S')
date_string = d.strftime('%Y%m%d_%H%M')
#Set up basemap using mercator projection
http://matplotlib.sourceforge.net/basemap/doc/html/users/merc.html
map = Basemap(projection='merc',llcrnrlat=-40,urcrnrlat=-33,
llcrnrlon=139.0,urcrnrlon=151.0,lat_ts=0,resolution='i')
# compute map projection coordinates for lat/lon grid.
x,y=map(*N.meshgrid(LON,LAT))
map.drawcoastlines(linewidth=0.5)
map.readshapefile(shapefile1, 'DSE_REGIONS')
map.drawstates()
plt.title('Surface temperature at %s UTC'%ncfiletime)
ticks=[-5,0,5,10,15,20,25,30,35,40,45,50]
CS = map.contourf(x,y,TSFC, ticks, cmap=plt.cm.jet)
l,b,w,h =0.1,0.1,0.8,0.8
cax = plt.axes([l+w+0.025, b, 0.025, h], )
cbar=plt.colorbar(CS, cax=cax, drawedges=True)
#save map as *.png and plot netcdf file
plt.savefig((os.path.join(newdir,'TSFC'+date_string+'UTC.png')))
plt.close()
gc.collect()
print "garbage collected again"
print "end of processing"
--
http://mail.python.org/mailman/listinfo/python-list
Re: memory error
efile1,
'DSE_REGIONS')
map.drawstates()
plt.title(Title+' %s
UTC'%ncfiletime)
CS = map.contourf(x,y,variable,
ticks, cmap=cmap)
l,b,w,h =0.1,0.1,0.8,0.8
cax = plt.axes([l+w+0.025, b,
0.025, h], )
cbar=plt.colorbar(CS, cax=cax,
drawedges=True)
#save map as *.png and plot
netcdf file
plt.savefig((os.path.join(OutputFolder,
ncvariablename+date_string+'UTC.png')))
#plt.show()
plt.close()
##
On Wed, Sep 14, 2011 at 4:08 PM, questions anon wrote:
> Hello All,
> I keep coming across a memory error when processing many netcdf files. I
> assume it has something to do with how I loop things and maybe need to close
> things off properly.
> In the code below I am looping through a bunch of netcdf files (each file
> is hourly data for one month) and within each netcdf file I am outputting a
> *png file every three hours.
> This works for one netcdf file but when it begins to process the next
> netcdf file I receive this memory error:
>
> *Traceback (most recent call last):
> File
> "d:/plot_netcdf_merc_multiplot_across_multifolders_mkdirs_memoryerror.py",
> line 44, in
> TSFC=ncfile.variables['T_SFC'][:]
> File "netCDF4.pyx", line 2473, in netCDF4.Variable.__getitem__
> (netCDF4.c:23094)
> MemoryError*
>
> To reduce processing requirements I have tried making the LAT and LON to
> only use [0] but I also receive an error:
>
> *Traceback (most recent call last):
> File
> "d:/plot_netcdf_merc_multiplot_across_multifolders_mkdirs_memoryerror.py",
> line 75, in
> x,y=map(*N.meshgrid(LON,LAT))
> File "C:\Python27\lib\site-packages\numpy\lib\function_base.py", line
> 3256, in meshgrid
> numRows, numCols = len(y), len(x) # yes, reversed
> TypeError: len() of unsized object*
>
> finally I have added gc.collect() in a couple of places but that doesn't
> seem to do anything to help.
> I am using :*Python 2.7.2 |EPD 7.1-2 (32-bit)| (default, Jul 3 2011,
> 15:13:59) [MSC v.1500 32 bit (Intel)] on win32*
> Any feedback will be greatly appreciated!
>
>
> from netCDF4 import Dataset
> import numpy
> import numpy as N
> import matplotlib.pyplot as plt
> from numpy import ma as MA
> from mpl_toolkits.basemap import Basemap
> from netcdftime import utime
> from datetime import datetime
> import os
> import gc
>
> print "start processing"
>
> inputpath=r'E:/GriddedData/Input/'
> outputpath=r'E:/GriddedData/Validation/'
> shapefile1="E:/test_GIS/DSE_REGIONS"
> for (path, dirs, files) in os.walk(inputpath):
> for dir in dirs:
> print dir
> sourcepath=os.path.join(path,dir)
> relativepath=os.path.relpath(sourcepath,inputpath)
> newdir=os.path.join(outputpath,relativepath)
> if not os.path.exists(newdir):
> os.makedirs(newdir)
>
> for ncfile in files:
> if ncfile[-3:]=='.nc':
> print "dealing with ncfiles:", ncfile
> ncfile=os.path.join(sourcepath,ncfile)
> #print ncfile
> ncfile=Dataset(ncfile, 'r+', 'NETCDF4')
> TSFC=ncfile.variables['T_SFC'][:,:,:]
> TIME=ncfile.variables['time'][:]
> LAT=ncfile.variables['latitude'][:]
> LON=ncfile.variables['longitude'][:]
> fillvalue=ncfile.variables['T_SFC']._FillValue
> TSFC=MA.masked_values(TSFC, fillvalue)
> ncfile.close()
> gc.collect()
> print "garbage collected"
>
>
> for TSFC, TIME in zip((TSFC[1::3]),(TIME[1::3])):
> print TSFC, TIME
> #convert time from numbers to date and prepare it to have no
> symbols for saving to filename
> cdftime=utime('seconds since 1970-01-01 00:00:00')
> ncfiletime=cdftime.num2date(TIME)
> print ncfiletime
> timestr=str(ncfiletime)
> d = datetime.strptime(timestr, '%Y-%m-%d %H:%M:%S')
> date_string = d.strftime('%Y%m%d_%H%M')
>
> #Set up basemap using mercator projection
> http://matplotlib.sourceforge.net/basemap/doc/html/user
mask one array using another array
I am trying to mask one array using another array. I have created a masked array using mask=MA.masked_equal(myarray,0), that looks something like: [1 - - 1, 1 1 - 1, 1 1 1 1, - 1 - 1] I have an array of values that I want to mask whereever my mask has a a '-'. how do I do this? I have looked at http://www.cawcr.gov.au/bmrc/climdyn/staff/lih/pubs/docs/masks.pdf but the command: d = array(a, mask=c.mask() results in this error: TypeError: 'numpy.ndarray' object is not callable I basically want to do exactly what that article does in that equation. Any feedback will be greatly appreciated. -- http://mail.python.org/mailman/listinfo/python-list
Re: mask one array using another array
thank you, that makes sense. I should have posted this on another list (which I have now). and the change required is: If your new array is x, you can use: numpy.ma.masked_array(x, mask=mask.mask) On Tue, Nov 22, 2011 at 11:48 AM, MRAB wrote: > On 21/11/2011 21:42, questions anon wrote: > >> I am trying to mask one array using another array. >> >> I have created a masked array using >> mask=MA.masked_equal(myarray,**0), >> that looks something like: >> [1 - - 1, >> 1 1 - 1, >> 1 1 1 1, >> - 1 - 1] >> >> I have an array of values that I want to mask whereever my mask has a a >> '-'. >> how do I do this? >> I have looked at >> http://www.cawcr.gov.au/bmrc/**climdyn/staff/lih/pubs/docs/**masks.pdf<http://www.cawcr.gov.au/bmrc/climdyn/staff/lih/pubs/docs/masks.pdf>but >> the command: >> >> d = array(a, mask=c.mask() >> >> results in this error: >> TypeError: 'numpy.ndarray' object is not callable >> >> I basically want to do exactly what that article does in that equation. >> >> Any feedback will be greatly appreciated. >> >> The article is using the Numeric module, but your error says that you're > using > the numpy module. They're not the same. > -- > http://mail.python.org/**mailman/listinfo/python-list<http://mail.python.org/mailman/listinfo/python-list> > -- http://mail.python.org/mailman/listinfo/python-list
loop through arrays and find maximum
I would like to calculate the max and min across many netcdf files. I know how to create one big array and then concatenate and find the numpy.max but when I run this on 1000's of arrays I have a memory error. What I would prefer is to loop through the arrays and produce the maximum without having the make a big array. Does anyone have any ideas as to how I could achieve this? My idea goes something like: netCDF_list=[] maxarray=[] for dir in glob.glob(MainFolder + '*/01/')+ glob.glob(MainFolder + '*/02/')+ glob.glob(MainFolder + '*/12/'): for ncfile in glob.glob(dir + '*.nc'): netCDF_list.append(ncfile) for filename in netCDF_list: ncfile=netCDF4.Dataset( filename) TSFC=ncfile.variables['T_SFC'][:] fillvalue=ncfile.variables['T_SFC']._FillValue TSFC=MA.masked_values(TSFC, fillvalue) for i in TSFC: if i == N.max(TSFC, axis=0): maxarray.append(i) else: pass print maxarray -- http://mail.python.org/mailman/listinfo/python-list
