[Rd] Why does my RPy2 program run faster on Windows?
Hi This is my function. It serves an HTML page after the calculations. I'm connecting to a MSSQL DB using pyodbc. def CAPM(self,client): r=self.r cds="1590" bm="20559" d1 = [] v1 = [] v2 = [] print"Parsing GET Params" params=client.g[1].split("&") for items in params: item=items.split("=") if(item[0]=="cds"): cds=unquote(item[1]) elif(item[0]=="bm"): bm=unquote(item[1]) print "cds: %s bm: %s" % (cds,bm) print "Fetching data" t3=datetime.now() for row in self.cursor.execute("select * from (select * from ( select co_code,dlyprice_date,dlyprice_close from feed_dlyprice P where co_code in (%s,%s) ) DataTable PIVOT ( max(dlyprice_close) FOR co_code IN ([%s],[%s]) )PivotTable ) a order by dlyprice_date" %(cds,bm,cds,bm)): d1.append(str(row[0])) v1.append(row[1]) v2.append(row[2]) t4=datetime.now() t1=datetime.now() print "Calculating" d1.pop(0) d1vec = robjects.StrVector(d1) v1vec = robjects.FloatVector(v1) v2vec = robjects.FloatVector(v2) r1 = r('Return.calculate(%s)' %v1vec.r_repr()) r2 = r('Return.calculate(%s)' %v2vec.r_repr()) tl = robjects.rlc.TaggedList([r1,r2],tags=('Geo','Nifty')) df = robjects.DataFrame(tl) ts2 = r.timeSeries(df,d1vec) tsa = r.timeSeries(r1,d1vec) tsb = r.timeSeries(r2,d1vec) robjects.globalenv["ta"] = tsa robjects.globalenv["tb"] = tsb robjects.globalenv["t2"] = ts2 a = r('table.CAPM(ta,tb)') t2=datetime.now() page="CAPMResult:%sTime taken by DB:%sTime taken by R:%sTotal time elapsed:%s" %(str(a),str(t4-t3),str(t2-t1),str(t2-t3)) print "Serving page:" #print page self.serveResource(page,"text",client) On Linux Time taken by DB:0:00:00.024165 Time taken by R:0:00:05.572084 Total time elapsed:0:00:05.596288 On Windows Time taken by DB:0:00:00.112000 Time taken by R:0:00:02.355000 Total time elapsed:0:00:02.467000 Why is there such a huge difference in the time taken by R on the two platforms? Am I doing something wrong? It's my first Rpy2 code so I guess it's badly written. I'm loading the following libraries: 'PerformanceAnalytics','timeSeries','fPortfolio','fPortfolioBacktest' I'm using Rpy2 2.1.0 and R 2.11 Regards Abhijit Bera [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Why does my RPy2 program run faster on Windows?
Update: it appears that the time taken isn't so much on the Data conversion. The maximum time taken is in CAPM calculation. :( Anyone know why the CAPM calculation would be faster on Windows? On Wed, May 19, 2010 at 5:51 PM, Abhijit Bera wrote: > Hi > > This is my function. It serves an HTML page after the calculations. I'm > connecting to a MSSQL DB using pyodbc. > > def CAPM(self,client): > > r=self.r > > cds="1590" > bm="20559" > > d1 = [] > v1 = [] > v2 = [] > > > print"Parsing GET Params" > > params=client.g[1].split("&") > > for items in params: > item=items.split("=") > > if(item[0]=="cds"): > cds=unquote(item[1]) > elif(item[0]=="bm"): > bm=unquote(item[1]) > > print "cds: %s bm: %s" % (cds,bm) > > print "Fetching data" > > t3=datetime.now() > > for row in self.cursor.execute("select * from (select * from ( > select co_code,dlyprice_date,dlyprice_close from feed_dlyprice P where > co_code in (%s,%s) ) DataTable PIVOT ( max(dlyprice_close) FOR co_code IN > ([%s],[%s]) )PivotTable ) a order by dlyprice_date" %(cds,bm,cds,bm)): > d1.append(str(row[0])) > v1.append(row[1]) > v2.append(row[2]) > > t4=datetime.now() > > t1=datetime.now() > > print "Calculating" > > d1.pop(0) > d1vec = robjects.StrVector(d1) > v1vec = robjects.FloatVector(v1) > v2vec = robjects.FloatVector(v2) > > r1 = r('Return.calculate(%s)' %v1vec.r_repr()) > r2 = r('Return.calculate(%s)' %v2vec.r_repr()) > > tl = robjects.rlc.TaggedList([r1,r2],tags=('Geo','Nifty')) > df = robjects.DataFrame(tl) > > ts2 = r.timeSeries(df,d1vec) > tsa = r.timeSeries(r1,d1vec) > tsb = r.timeSeries(r2,d1vec) > > robjects.globalenv["ta"] = tsa > robjects.globalenv["tb"] = tsb > robjects.globalenv["t2"] = ts2 > a = r('table.CAPM(ta,tb)') > > t2=datetime.now() > > > page="CAPMResult:%sTime taken by > DB:%sTime taken by R:%sTotal time elapsed:%s" > %(str(a),str(t4-t3),str(t2-t1),str(t2-t3)) > print "Serving page:" > #print page > > self.serveResource(page,"text",client) > > > > On Linux > Time taken by DB:0:00:00.024165 > Time taken by R:0:00:05.572084 > Total time elapsed:0:00:05.596288 > > On Windows > Time taken by DB:0:00:00.112000 > Time taken by R:0:00:02.355000 > Total time elapsed:0:00:02.467000 > > Why is there such a huge difference in the time taken by R on the two > platforms? Am I doing something wrong? It's my first Rpy2 code so I guess > it's badly written. > > I'm loading the following libraries: > 'PerformanceAnalytics','timeSeries','fPortfolio','fPortfolioBacktest' > > I'm using Rpy2 2.1.0 and R 2.11 > > Regards > > Abhijit Bera > > > > > [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Why does my RPy2 program run faster on Windows?
Here is an updated bench mark: Linux Time taken by DB:0:00:00.226888 Time taken by R:0:00:05.536973 Time taken for vector conversions:0:00:00.001799 Total time taken for return calculation:0:00:00.090062 Total time taken for making Tagged list and Data Frame:0:00:00.015424 Total time taken for making TimeSeries:0:00:00.887713 Total time to assign global vars:0:00:00.000364 Total time taken for CAPM:0:00:04.541611 Total time elapsed:0:00:05.763861 Windows Time taken by DB:0:00:00.139000 Time taken by R:0:00:02.593000 Time taken for vector conversions:0:00:00.037000 Total time taken for return calculation:0:00:00.114000 Total time taken for making Tagged list and Data Frame:0:00:00.021000 Total time taken for making TimeSeries:0:00:00.201000 Total time to assign global vars:0:00:00.002000 Total time taken for CAPM:0:00:02.218000 Total time elapsed:0:00:02.732000 On Wed, May 19, 2010 at 6:34 PM, Abhijit Bera wrote: > Update: it appears that the time taken isn't so much on the Data > conversion. The maximum time taken is in CAPM calculation. :( Anyone know > why the CAPM calculation would be faster on Windows? > > > On Wed, May 19, 2010 at 5:51 PM, Abhijit Bera wrote: > >> Hi >> >> This is my function. It serves an HTML page after the calculations. I'm >> connecting to a MSSQL DB using pyodbc. >> >> def CAPM(self,client): >> >> r=self.r >> >> cds="1590" >> bm="20559" >> >> d1 = [] >> v1 = [] >> v2 = [] >> >> >> print"Parsing GET Params" >> >> params=client.g[1].split("&") >> >> for items in params: >> item=items.split("=") >> >> if(item[0]=="cds"): >> cds=unquote(item[1]) >> elif(item[0]=="bm"): >> bm=unquote(item[1]) >> >> print "cds: %s bm: %s" % (cds,bm) >> >> print "Fetching data" >> >> t3=datetime.now() >> >> for row in self.cursor.execute("select * from (select * from ( >> select co_code,dlyprice_date,dlyprice_close from feed_dlyprice P where >> co_code in (%s,%s) ) DataTable PIVOT ( max(dlyprice_close) FOR co_code IN >> ([%s],[%s]) )PivotTable ) a order by dlyprice_date" %(cds,bm,cds,bm)): >> d1.append(str(row[0])) >> v1.append(row[1]) >> v2.append(row[2]) >> >> t4=datetime.now() >> >> t1=datetime.now() >> >> print "Calculating" >> >> d1.pop(0) >> d1vec = robjects.StrVector(d1) >> v1vec = robjects.FloatVector(v1) >> v2vec = robjects.FloatVector(v2) >> >> r1 = r('Return.calculate(%s)' %v1vec.r_repr()) >> r2 = r('Return.calculate(%s)' %v2vec.r_repr()) >> >> tl = robjects.rlc.TaggedList([r1,r2],tags=('Geo','Nifty')) >> df = robjects.DataFrame(tl) >> >> ts2 = r.timeSeries(df,d1vec) >> tsa = r.timeSeries(r1,d1vec) >> tsb = r.timeSeries(r2,d1vec) >> >> robjects.globalenv["ta"] = tsa >> robjects.globalenv["tb"] = tsb >> robjects.globalenv["t2"] = ts2 >> a = r('table.CAPM(ta,tb)') >> >> t2=datetime.now() >> >> >> page="CAPMResult:%sTime taken >> by DB:%sTime taken by R:%sTotal time elapsed:%s" >> %(str(a),str(t4-t3),str(t2-t1),str(t2-t3)) >> print "Serving page:" >> #print page >> >> self.serveResource(page,"text",client) >> >> >> >> On Linux >> Time taken by DB:0:00:00.024165 >> Time taken by R:0:00:05.572084 >> Total time elapsed:0:00:05.596288 >> >> On Windows >> Time taken by DB:0:00:00.112000 >> Time taken by R:0:00:02.355000 >> Total time elapsed:0:00:02.467000 >> >> Why is there such a huge difference in the time taken by R on the two >> platforms? Am I doing something wrong? It's my first Rpy2 code so I guess >> it's badly written. >> >> I'm loading the following libraries: >> 'PerformanceAnalytics','timeSeries','fPortfolio','fPortfolioBacktest' >> >> I'm using Rpy2 2.1.0 and R 2.11 >> >> Regards >> >> Abhijit Bera >> >> >> >> >> > [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Why does my RPy2 program run faster on Windows?
Dear Abhijit, If you think that table.CAPM is the culprit, you could run the call to such function in R on both platforms using Rprof to check which part of the function is producing the bottleneck. Best regards, Carlos J. Gil Bellosta http://www.datanalytics.com 2010/5/19 Abhijit Bera : > Update: it appears that the time taken isn't so much on the Data conversion. > The maximum time taken is in CAPM calculation. :( Anyone know why the CAPM > calculation would be faster on Windows? > > On Wed, May 19, 2010 at 5:51 PM, Abhijit Bera wrote: > >> Hi >> >> This is my function. It serves an HTML page after the calculations. I'm >> connecting to a MSSQL DB using pyodbc. >> >> def CAPM(self,client): >> >> r=self.r >> >> cds="1590" >> bm="20559" >> >> d1 = [] >> v1 = [] >> v2 = [] >> >> >> print"Parsing GET Params" >> >> params=client.g[1].split("&") >> >> for items in params: >> item=items.split("=") >> >> if(item[0]=="cds"): >> cds=unquote(item[1]) >> elif(item[0]=="bm"): >> bm=unquote(item[1]) >> >> print "cds: %s bm: %s" % (cds,bm) >> >> print "Fetching data" >> >> t3=datetime.now() >> >> for row in self.cursor.execute("select * from (select * from ( >> select co_code,dlyprice_date,dlyprice_close from feed_dlyprice P where >> co_code in (%s,%s) ) DataTable PIVOT ( max(dlyprice_close) FOR co_code IN >> ([%s],[%s]) )PivotTable ) a order by dlyprice_date" %(cds,bm,cds,bm)): >> d1.append(str(row[0])) >> v1.append(row[1]) >> v2.append(row[2]) >> >> t4=datetime.now() >> >> t1=datetime.now() >> >> print "Calculating" >> >> d1.pop(0) >> d1vec = robjects.StrVector(d1) >> v1vec = robjects.FloatVector(v1) >> v2vec = robjects.FloatVector(v2) >> >> r1 = r('Return.calculate(%s)' %v1vec.r_repr()) >> r2 = r('Return.calculate(%s)' %v2vec.r_repr()) >> >> tl = robjects.rlc.TaggedList([r1,r2],tags=('Geo','Nifty')) >> df = robjects.DataFrame(tl) >> >> ts2 = r.timeSeries(df,d1vec) >> tsa = r.timeSeries(r1,d1vec) >> tsb = r.timeSeries(r2,d1vec) >> >> robjects.globalenv["ta"] = tsa >> robjects.globalenv["tb"] = tsb >> robjects.globalenv["t2"] = ts2 >> a = r('table.CAPM(ta,tb)') >> >> t2=datetime.now() >> >> >> page="CAPMResult:%sTime taken by >> DB:%sTime taken by R:%sTotal time elapsed:%s" >> %(str(a),str(t4-t3),str(t2-t1),str(t2-t3)) >> print "Serving page:" >> #print page >> >> self.serveResource(page,"text",client) >> >> >> >> On Linux >> Time taken by DB:0:00:00.024165 >> Time taken by R:0:00:05.572084 >> Total time elapsed:0:00:05.596288 >> >> On Windows >> Time taken by DB:0:00:00.112000 >> Time taken by R:0:00:02.355000 >> Total time elapsed:0:00:02.467000 >> >> Why is there such a huge difference in the time taken by R on the two >> platforms? Am I doing something wrong? It's my first Rpy2 code so I guess >> it's badly written. >> >> I'm loading the following libraries: >> 'PerformanceAnalytics','timeSeries','fPortfolio','fPortfolioBacktest' >> >> I'm using Rpy2 2.1.0 and R 2.11 >> >> Regards >> >> Abhijit Bera >> >> >> >> >> > > [[alternative HTML version deleted]] > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel > __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R in sandbox/jail (long question)
I think you'll find it's a bit more complicated than that. Firstly, R --sandbox is pretty crippled, since as far as I can tell it can't load packages, since package loading uses gzfile(). This would include the 'stats' package. If you can load packages you would need to sanitize all those packages, since they may contain functions that directly talk to the operating system (for example, the 'foreign' package does). Also, most functions called by .C() and many called by .Call() can be made to overwrite memory they don't own, by passing invalid arguments, so the sandbox would only protect you from mistakes by the user and from incompetent attacks, but not from competent attacks. -thomas On Tue, 18 May 2010, Assaf Gordon wrote: Hello, I have a setup similar to Rweb ( http://www.math.montana.edu/Rweb/ ): I get R scripts from users and need to execute them in in a safe manner (they are executed automatically, without human inspection). I would like to limit the user's script to reading from STDIN and writing to STDOUT/ERR. Specifically, preventing any kind of interaction with the underlying operating system (files, sockets, system(), etc.). I've found this old thread: http://r.789695.n4.nabble.com/R-in-a-sandbox-jail-td921991.html But for technical reasons I'd prefer not to setup a chroot jail. I have written a patch that adds a "--sandbox" parameter. When this parameter is used, the user's script can't create any kind of connection object or run "system()". My plan is to run R like this: cat INPUT | R --vanila --slave --sandbox --file SCRIPT.R > OUTPUT Where 'INPUT' is my chosen input and 'SCRIPT.R' is the script submitted by the user. If the script tries to create a conncetion or run a disabled function, an error is printed. This is the patch: http://cancan.cshl.edu/labmembers/gordon/files/R_2.11.0_sandbox.patch So my questions are: 1. Would you be willing to consider this feature for inclusion ? 2. Are there any other 'dangerous' functions I need to intercept ( ".Internal" perhaps ?) All comments and suggestions are welcomed, thanks, -gordon __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel Thomas Lumley Assoc. Professor, Biostatistics tlum...@u.washington.eduUniversity of Washington, Seattle __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R in sandbox/jail (long question)
How about some "computing on the language", something like this: exprs <- parse("SCRIPT.R") invalids <- c(".Internal", ".Primitive") if( any( invalids %in% all.names(exprs) ) ) stop("sandbox check failed") I believe this would prevent evaluating any direct calls to '.Primitive' and '.Internal'. Of course, you could extend the 'invalids' vector to include any names. If you want to consider arguments to calls (i.e. argument to 'file' or 'library') or something more sophisticated, check out the functions in the codetools package, something like this: library(codetools) walkerCall <- function(e, w) { for( ee in as.list(e)) { if(!missing(ee)) { if(is.call(ee)) { #stop .Internal calls if(ee[1] == call(".Internal")) stop("invalid \'.Internal()\' call") #restrict file to STDIN if(ee[1] == call("file")) { mc <- match.call(file, ee) if(mc[[2]] != "stdin") stop("\'file()\' only valid with \'description=\"stdin\"\'") } } walkCode(ee, w) } } } walker <- makeCodeWalker(call=walkerCall, leaf=function(e,w){}) exprs <- parse("SCRIPT.R") for( expr in exprs ) walkCode(expr,walker) I'm a little surprised this there isn't a 'sandbox' package or something similar to this. A reverse depends check on the codetools package indicates there is not. However, I believe there is some demand for it. Matt Shotwell http://biostatmatt.com On Tue, 2010-05-18 at 22:38 -0400, Assaf Gordon wrote: > Hello, > > I have a setup similar to Rweb ( http://www.math.montana.edu/Rweb/ ): > I get R scripts from users and need to execute them in in a safe manner (they > are executed automatically, without human inspection). > > I would like to limit the user's script to reading from STDIN and writing to > STDOUT/ERR. > Specifically, preventing any kind of interaction with the underlying > operating system (files, sockets, system(), etc.). > > I've found this old thread: > http://r.789695.n4.nabble.com/R-in-a-sandbox-jail-td921991.html > But for technical reasons I'd prefer not to setup a chroot jail. > > I have written a patch that adds a "--sandbox" parameter. > When this parameter is used, the user's script can't create any kind of > connection object or run "system()". > > My plan is to run R like this: > > cat INPUT | R --vanila --slave --sandbox --file SCRIPT.R > OUTPUT > > Where 'INPUT' is my chosen input and 'SCRIPT.R' is the script submitted by > the user. > If the script tries to create a conncetion or run a disabled function, an > error is printed. > > This is the patch: > http://cancan.cshl.edu/labmembers/gordon/files/R_2.11.0_sandbox.patch > > So my questions are: > 1. Would you be willing to consider this feature for inclusion ? > 2. Are there any other 'dangerous' functions I need to intercept ( > ".Internal" perhaps ?) > > All comments and suggestions are welcomed, > thanks, >-gordon > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] pretty.Date(): new "halfmonth" time step
>Much better to implement directly what this is trying to do: i.e. to >have a "halfmonth" time step. This is just the union of two "monthly" >sequences, one on the 1st of each month and another on the 15th of >each month. For some applications that might be true. But not for others. For a month with 31 days, there are 14 days before the 15th of the month and 16 days after the 15th, so, for example, March 16th (specifically noon) rather than March 15th would be the halfway point if you define "halfway" in terms of the distances to the beginning and end of the month. For a month with 30 days -- like April -- the halfway point would be the instant between the 15th and the 16th of the month. Do you label that instant April 15 or April 16? (I prefer "15".) Don't get me started on February. - Dan Murphy [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] pretty.Date(): new "halfmonth" time step
On 20 May 2010 11:56, Daniel Murphy wrote: >>Much better to implement directly what this is trying to do: i.e. to >>have a "halfmonth" time step. This is just the union of two "monthly" >>sequences, one on the 1st of each month and another on the 15th of >>each month. > > For some applications that might be true. But not for others. For a month > with 31 days, there are 14 days before the 15th of the month and 16 days > after the 15th, so, for example, March 16th (specifically noon) rather than > March 15th would be the halfway point if you define "halfway" in terms of > the distances to the beginning and end of the month. For a month with 30 > days -- like April -- the halfway point would be the instant between the > 15th and the 16th of the month. Do you label that instant April 15 or April > 16? (I prefer "15".) Don't get me started on February. Dan, you are correct: the midpoint of a 30 day month is the 16th at 00:00. That instant is called the 16th according to print.POSIXt. junstart <- as.POSIXct("2000-06-01 00:00", tz="GMT") julstart <- as.POSIXct("2000-07-01 00:00", tz="GMT") junstart + ((julstart - junstart) / 2) #[1] "2000-06-16 GMT" How embarassing... So I think it would be better to use 16 rather than 15 for the "halfmonth" time step. Yes, months have variable lengths, but I think it is best to use a consistent date (the 16th) than to calculate exact midpoints, just as a normal monthly series has a consistent date (the 1st) and has variable lengths. Regards -Felix > > - Dan Murphy > > [[alternative HTML version deleted]] > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel > -- Felix Andrews / 安福立 Postdoctoral Fellow Integrated Catchment Assessment and Management (iCAM) Centre Fenner School of Environment and Society [Bldg 48a] The Australian National University Canberra ACT 0200 Australia M: +61 410 400 963 T: + 61 2 6125 4670 E: felix.andr...@anu.edu.au CRICOS Provider No. 00120C -- http://www.neurofractal.org/felix/ __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Use of R and Rscript in configure/Makevars in packages
We have seen problems with a number of packages which use R/Rscript to run R code in configure or makefiles. (a) You must give a full path: there need be no version of R in the path, and if there is it might not be the version/build of R under which package installation is being done. So the general form is to use ${R_HOME}/bin/R to select the right version. And since ${R_HOME} might contain spaces, you need something like "${R_HOME}/bin/R". There appear to be bare uses of Rscript in Rcpp RQuantLib bifactorial mvabund, of bare R in ROracle pgfSweave rcom and many more packages without quotes. (b) There are further issues with platforms which use sub-architectures (mainly Mac OS X and R-devel Windows). On Windows the architecture-dependent executables will (as from R 2.12.0) be in subdirectories of ${R_HOME}/bin, so the general form is to use one of "${R_HOME}/bin${R_ARCH_BIN}/Rscript.exe" "${R_HOME}/bin${R_ARCH_BIN}/Rterm.exe" "${R_HOME}/bin${R_ARCH_BIN}/Rcmd.exe" On R-devel Windows ${R_HOME}/bin/R.exe and ${R_HOME}/bin/Rscript.exe do exist and are 32-bit executables whose sole task is to launch the appropriate executable from a sub-directory. Since process creation is expensive on Windows, this intermediate step is best avoided. On Mac OS X, ${R_HOME}/bin/R is a script that launches the architecture-dependent executables from a subdirectory, and on CRAN builds ${R_HOME}/bin/Rscript is a 'fat' (multi-arch) executable, so the issues have been worked around (but not necessarily for user installs). (c) Calling R in configure.win will typically call 32-bit R with R-devel Windows. If the result needs to depend on the architecture (and e.g. the library dirs may well) then the work needs to be done in src/Makevars.win: see the R-devel 'Writing R Extensions' manual for how to achieve this. (The reason is that configure.win is called once, and then src/Makevars.win is called for each architecture.) BDR -- Brian D. Ripley, rip...@stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595 __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel