There has been recent work on incorporating netcdf support for point and trajectory data with Talend, maybe that can help?
http://www.neogeo-online.net/blog/archives/1350/ http://talendforge.org/svn/sdi/trunk/org.talend.sdi.designer.components.sandbox/components/sNetCDFInput/ Etienne On Tue, Aug 30, 2011 at 10:18 AM, Etienne Tourigny <etienne...@yahoo.com>wrote: > Brian, > > I am not familiar with madis data. I assume that it is point-data for > meteorological stations? > > If you like, please post your ideas and a short description in a new topic > in the wiki entry at > http://trac.osgeo.org/gdal/wiki/NetCDF_Improvements . > > Etienne > > > > ----- Original Message ----- > From: Brian Case <r...@winkey.org> > To: Etienne Tourigny <etienne...@yahoo.com> > Cc: "gdal-dev@lists.osgeo.org" <gdal-dev@lists.osgeo.org> > Sent: Sunday, August 28, 2011 7:54:13 PM > Subject: Re: [gdal-dev] Re: discussion on improvements to the NetCDF driver > and CF-1 convention > > any thoughts on a ogr netcdf driver? I have a simple one for madis data > that needs work in my sandbox > > Brian > > > > On Sun, 2011-08-28 at 14:56 -0700, Etienne Tourigny wrote: > > I have created a new wiki page at > http://trac.osgeo.org/gdal/wiki/NetCDF_Improvements , following Even's > suggestion (Thanks Even). > > > > The following topics have been included: > > > > > > Topics: > > > > 1 - UNIDATA NetCDF Conventions > > > > 2 - Metadata > > 3 - Datum issues > > 4 - Dimension issues > > 5 - Compatibility with other software > > 6 - Test files > > 7 - Contribution > > Discussion > > > > I encourage interested parties in participating in this discussion in the > wiki and in the mailing list. > > > > > > In particular, it would be useful to collect information on contributors > and obtain test files with issues and the expected outcome. > > > > regards, Etienne > > > > > > ----- Original Message ----- > > From: Even Rouault <even.roua...@mines-paris.org> > > To: gdal-dev@lists.osgeo.org; Etienne Tourigny <etienne...@yahoo.com> > > Cc: > > Sent: Friday, August 26, 2011 4:20:06 PM > > Subject: Re: [gdal-dev] Re: discussion on improvements to the NetCDF > driver and CF-1 convention > > > > Le vendredi 26 août 2011 17:15:47, Etienne Tourigny a écrit : > > > > My thoughts, just from a process point of view, not being myself deeply > > involved in the NetCDF format : > > > > * A wiki page (I'd suggest either a new page - > > http://trac.osgeo.org/gdal/wiki/NetCDF_Improvements - or at least a > clearly > > identified section in the current page, to avoid confusion for outsiders > about > > what is currently implemented and what is not) can be a convenient way of > > structuring and editing the content by anyone with a osgeo id. But > discussions > > on the mailing list are also OK, at least to notify new content has been > added > > (provided that it doesn't cause flooding on the mailing list ;-)) > > > > * A http://download.osgeo.org/gdal/data/netcdf directory would be the > right > > place to store NetCDF sample files, provided they are free under the > terms of > > http://download.osgeo.org/gdal/data/COPYING . As write access to it is > > restricted to people with appropriate rights, having those rights, I can > help > > uploading files over there. Having small (typically < or ~ 10 KB ) sample > files > > is also good so they can be directly included in SVN under the > > autotest/gdrivers/data directory and thus easily used by regression > tests. > > > > * Tickets can be used to track the implementation of specific items and > > bugfixes. But I've found they are not generally suitable for > brainstorming when > > the discussion is lasting, as you cannot edit already posted content. > Wiki is > > clearly better for this. > > > > * Looking at past history, I see we haven't yet used the RFC formalism > for > > work specific to one driver, while it doesn't require changes or > extensions to > > the GDAL API. But it wouldn't hurt to create one if the need arises. > Anyway, a > > point that might require care and some consensus among the interested > parties > > is about how backward compatibility issues, in particular with respect to > files > > produced by older GDAL versions, are adressed. > > http://trac.osgeo.org/gdal/wiki/rfc1_pmc details when a vote is > required. > > > > Best regards, > > > > Even > > > > > Hi all, > > > > > > Since a few people have shown an interest in helping out, I would > suggest > > > to set up space where we can post ideas, goals and test files. This > could > > > take the form of one of the following: > > > > > > - a new wiki entry in trac, or the existing one at > > > http://trac.osgeo.org/gdal/wiki/NetCDF - a file repository at > > > http://download.osgeo.org/gdal/data/netcdf as suggested some time ago > by > > > warmerdam > > > > > > and/or > > > > > > - a new ticket in trac (which allows to post files) > > > > > > What are you thoughts about this? Would it make sense to eventually > > > prepare an RFC? > > > > > > > > > > > > I would suggest that each group identify the problems with the current > > > driver in terms of importing and exporting, post the files that they > would > > > like to be supported, which other software they use with the netcdf > files, > > > and their proposed contribution (code, test, ideas, etc.). > > > > > > Those that cannot participate in code development can help out in > testing > > > and discussion. Personally, I can write and test code, given example > files > > > and expected outcome. > > > > > > > > > Now to your specific comments, Knut-Frode: > > > > > > 1) Yes CF-1.5 should be the goal, although I'm not sure we need to > support > > > upgrading/downgrading. > > > > > > 5) The WGS84 issue needs further research. Weather and ocean data are > in > > > lat/lon, but what is the sphere radius assumed by the various data > > > providers and the models which output/import netcdf files? I agree > that a > > > small error does not make a difference with low-resolution dataset, but > > > I'd like to be sure in any case. We also need a means to translate the > > > CF-1.x datum information into EPSG/Proj.4 values. > > > > > > > > > 7) There is some support of a z-axis right now, in fact they driver > assumes > > > that a third dimension is vertical if I understand correctly. However, > I > > > have not tested nor looked into that. This issue definitely needs come > > > planning and an RFC, IMHO. > > > > > > > > > Kyle, thanks for you help and interest! > > > > > > > > > cheers, > > > Etienne > > > > > > > > > ________________________________ > > > From: Knut-Frode Dagestad <knutfrodesop...@hotmail.com> > > > To: gdal-dev@lists.osgeo.org > > > Sent: Friday, August 26, 2011 4:41:02 AM > > > Subject: [gdal-dev] Re: discussion on improvements to the NetCDF driver > and > > > CF-1 convention > > > > > > Etienne, > > > > > > My group is also very much in support of the suggested improvements, as > > > NetCDF/CF is becoming the most important standard for data > > > storage/exchange in our community (climate, meteorology, oceanography). > My > > > group is not much into GDAL driver development (at least not yet), but > we > > > are now building our future very much around GDAL, so we are interested > to > > > join the discussion around improved support for NetCDF/CF reading and > > > writing. > > > > > > About some of the mentioned issues: > > > > > > 1) The latest version of CF is 1.5, so it would be better to target 1.5 > (or > > > 1.x) than 1.0. Ideally the driver could be made generic for smooth > upgrade > > > to newer (or downgrade to older) versions of the CF conventions. > > > > > > 5) It sounds meaningful to assume WGS84 datum if not specified. I > believe > > > at least 99% of data typically stored in NetCDF/CF will have this > datum, > > > and even when that is not the case, positioning errors less than 100 m > > > should normally have little impact for the types of data. Precision on > > > datum level is probably more relevant for more typical GIS-data with > > > roads, borders etc. A warning could however be issued each time the > driver > > > has to assume WGS84. > > > > > > 7) We agree in particular that proper handling of time axis is of high > > > value. This is one of the issues which is highly relevant for > climate-data > > > (often 4-dimensional), but not well supported by typical GIS-software > or > > > standards designed with 2-dimensional data in mind. A general > > > representation of a time-axis (or even vertical axis) in GDAL would be > > > ideal, but that would probably be too much an effort for anyone to > > > develop. > > > > > > > > > Knut-Frode > > > Nansen Environmental and Remote Sensing Center > > > > > > On 24/08/2011 22:37, Etienne Tourigny wrote: > > > > Hi all, > > > > > > > > I would like to start a discussion with those interested about fixing > > > > various issues in the NetCDF driver. > > > > > > > > As it stands, it is not fully CF-1.0 compliant, and produces > > > > geographical grids that are not valid for other software. > > > > Also, the infamous "up-side down" problem has been addressed for > > > > importing netcdfs, but needs to be addressed for exporting. > > > > > > > > Related ticket: http://trac.osgeo.org/gdal/ticket/2129 > > > > I have been submitting patches for some time to fix the related > issues. > > > > > > > > I have identified a number of issues, of which the following have > been > > > > fixed partially in my proposed patch. > > > > > > > > 1- conform to the cf-1.0 standard for geographical grids (creating > lat > > > > and lon dimensions and variables) > > > > 2- make netcdf grids "bottom-up" by default, as all software out > there > > > > uses that convention > > > > 3- fix problems related to floating-point / string conversions (also > > > > reported in issue #4200 ) > > > > 4- fix metadata handling and duplication (as reported in issue > #4204), > > > > and keep band (variable) metadata outside of the global metadata > > > > > > > > Pending issues: > > > > > > > > 5- how to deal with netcdfs which have no datum? Can we assume WGS84? > > > > These files are extremely common, and CF-1.0 has provisions for > > > > identifying the key variables, but no Proj/Wkt codes. > > > > 6- fix proper export of projected CRS (this is not fully necessary > > > > though) with lat/lon values respecting CF-1.0 > > > > 7- handle time axis properly (at least when creating a netcdf) - that > > > > would be great for import/export of netcdf > > > > > > > > Regards, > > > > Etienne > > > > > > > > > > > > _______________________________________________ > > > > gdal-dev mailing list > > > > gdal-dev@lists.osgeo.org > > > > http://lists.osgeo.org/mailman/listinfo/gdal-dev > > > > > > _______________________________________________ > > > gdal-dev mailing list > > > gdal-dev@lists.osgeo.org > > > http://lists.osgeo.org/mailman/listinfo/gdal-dev > > > _______________________________________________ > > > gdal-dev mailing list > > > gdal-dev@lists.osgeo.org > > > http://lists.osgeo.org/mailman/listinfo/gdal-dev > > _______________________________________________ > > gdal-dev mailing list > > gdal-dev@lists.osgeo.org > > http://lists.osgeo.org/mailman/listinfo/gdal-dev > _______________________________________________ > gdal-dev mailing list > gdal-dev@lists.osgeo.org > http://lists.osgeo.org/mailman/listinfo/gdal-dev >
_______________________________________________ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev
