Let's not cross-post! I'll pick up only one point of general relevance on this list (R-help).

On Tue, 5 Aug 2008, Matt Oliver wrote:

you can try

memory.limit(size=4000)

only if you have 4GB of memory on the system

This is not guaranteed to solve your problem though

Assuming this is Windows (it looks like it) .... The limiting issue on a 32-bit system is not memory (RAM) but address space: using memory.limit(size=4000) will still only get you 2GB of address space unless you use the /3GB flag. (See the rw-FAQ Q2.9.)

With big datasets like lidar, you are much better off getting access to a
64bit system with a ton of RAM (>64GB).

Well, you are better off even with 4GB of RAM because of address-space issues (and even with 2GB if the 32-bit OS is Windows -- I used to run a dual-boot Windows XP / 64-bit Linux box with 2GB, so this is first-hand experience).


Cheers

Matt


On Tue, Aug 5, 2008 at 1:47 PM, Alessandro <[EMAIL PROTECTED]>wrote:



Hi All,



I am a PhD student in forestry science and I am working with LiDAR data
set (huge data set). I am a brand-new in R and geostatistic (SORRY, my
background it's in forestry) but I wish improve my skill for improve
myself.  I wish to develop a methodology to processing a large data-set of
points (typical in LiDAR) but there is a problem with memory. I had created
a subsample data-base but the semivariogram is periodic shape and I am not
to able to try a fit the model. This is a maximum of two weeks of work (day
bay day) SORRY. Is there a geostatistical user I am very happy to listen his
suggests. Data format is X, Y and Z (height to create the DEM) in txt format

I have this questions:



1.       After the random selection (10000 points) and fit.semivariogram
model is it possible to use all LiDAR points? Because the new LiDAR power is
to use huge number of points (X;Y; Z value) to create a very high resolution
map of DEM and VEGETATION. The problem is the memory, but we can use a
cluster-linux network to improve the capacity of R



2.       Is it possible to improve the memory capacity?



3.       The data has a trend and the qqplot shows a Gaussian trend. Is it
possible to normalize the data (i.e. with log)?



4.       When I use this R code "subground.uk = krige(log(Z)~X+Y,
subground, new.grid, v.fit, nmax=40)" to appear an Error massage: Error in
eval(expr, envir, enclos) : oggetto "X" non trovato



I send you a report and attach the image to explain better.



all procedure is write in R-software and to improve in gstat . The number
of points of GROUND data-set (4x2 km) is 5,459,916.00. The random sub- set
from original data-set is 10000 (R code is:  > samplerows
<-sample(1:nrow(testground),size=10000,replace=FALSE) > subground
<-testground[samplerows,])



1.       Original data-set Histogram: show two populations;

2.       original data-set density plot: show again two populations of
data;

3.        Original data-set Boxplot: show there aren't outlayers un the
data-set (the classification with terrascan is good and therefore there
isn't a problem with original data)

4.        ordinary kriging: show a trend in the space (hypothesis: the
points are very close in the space)

5.       de-trend dataset with:  v <- variogram (log(Z)~X+Y, subground,
cutoff=1800, width=100))

6.       map of semi-variogram: show an anisotropy in the space (0? is
Nord= 135? major radius 45? minus radius)

7.       semi-variogram with anisotropy (0?, 45?, 90?, 135?), shows a  best
shape is from 135?

8. semi-variogram fit with Gaussian Model. R code is (see the fig):

v = variogram(Z~X+Y, subground, cutoff=1800, width=200, alpha=c(135))

v.fit = fit.variogram(v, vgm(psill = 1, model="Gau", range=1800, nugget=
0, anis=c(135, 0.5)))



R code:



testground2 <-
read.table(file="c:/work_LIDAR_USA/R_kriging_new_set/ground_26841492694149_xyz.txt",
header=T)

class (testground2)

coordinates (testground2)=~X+Y # this makes testground a
SpatialPointsDataFrame

class (testground2)

str(as.data.frame(testground))



hist(testground$Z,nclass=20) #this makes a histogram

plot(density(testground$Z)) #this makes a plot density

boxplot(testground$Z)#this makes a boxplot



samplerows<-sample(1:nrow(testground),size=10000,replace=FALSE) #select n.
points from all data-base

subground <-testground[samplerows,]

hist(subground$Z,nclass=20) #this makes a histogram

plot(density(subground$Z)) #this makes a plot density

boxplot(subground$Z)#this makes a boxplot

spplot(subground["Z"], col.regions=bpy.colors(), at = seq(850,1170,10))



library(maptools)

library(gstat)

plot(variogram(Z~1, subground)) #Ordinary Kriging (without detrend)

# if there is a trend we must use a detrend fuction Z~X+Y

x11(); plot(variogram(log(Z)~X+Y, subground, cutoff=1800, width=80))
#Universal Kriging (with detrend)

x11(); plot(variogram(log(Z)~X+Y, subground, cutoff=1800, width=80, map=T))

x11(); plot(variogram(log(Z)~X+Y, subground, cutoff=1800, width=80,
alpha=c(0, 45, 90, 135)))

v = variogram(log(Z)~X+Y, subground, cutoff=1800, width=80, alpha=c(135,
45))

v.fit = fit.variogram(v, vgm(psill = 1, model="Gau", range=1800, nugget= 0,
anis=c(135, 0.5)))

plot(v, v.fit, plot.nu=F, pch="+")

# create the new grid

new.grid <- spsample(subground, type="regular", cellsize=c(1,1))

gridded(new.grid) <- TRUE

fullgrid(new.grid) <- TRUE

[EMAIL PROTECTED]

#using Universal Kriging

subground.uk = krige(log(Z)~X+Y, subground, new.grid, v.fit, nmax=40)
 #ERROR


--
Brian D. Ripley,                  [EMAIL PROTECTED]
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Reply via email to