Re: [Rd] sparse.model.matrix Generates Non-Existent Factor Levels if Ord.factor Columns Present
color and clarity are ordered factors, so sparse.model.matrix is generating orthogonal-polynomial contrasts (see ?contr.poly). This is by design ... what are you trying to do? Are you interested in fac2sparse? On 18-02-07 11:00 PM, Dario Strbenac wrote: > Good day, > > Sometimes, sparse.model.matrix outputs a dgCMatrix which has column names > consisting of factor levels that were not in the original dataset. The first > factor appears to be correctly transformed, but the following factors don't. > For example: > > diamonds <- as.data.frame(ggplot2::diamonds) >> colnames(sparse.model.matrix(~ . -1, diamonds)) > [1] "carat""cutFair" "cutGood" "cutVery Good" "cutPremium" > "cutIdeal" "color.L" "color.Q" "color.C" "color^4" > "color^5" > [12] "color^6" "clarity.L""clarity.Q""clarity.C""clarity^4" > "clarity^5""clarity^6""clarity^7""depth""table" > "price" > [23] "x""y""z" > > The variables color and clarity don't have factor levels which have been > suffixed to them in the transformed matrix. The values in those columns are > also wrong. Changing the Ord.factor columns into simply being factors fixes > the problem. > >> diamonds[, "cut"] <- factor(as.character(diamonds[, "cut"])) >> diamonds[, "color"] <- factor(as.character(diamonds[, "color"])) >> diamonds[, "clarity"] <- factor(as.character(diamonds[, "clarity"])) > >> colnames(sparse.model.matrix(~ . -1, diamonds)) # No more invented factor >> levels. > [1] "carat""cutFair" "cutGood" "cutIdeal" "cutPremium" > "cutVery Good" "colorE" "colorF" "colorG" "colorH" > [11] "colorI" "colorJ" "clarityIF""claritySI1" "claritySI2" > "clarityVS1" "clarityVS2" "clarityVVS1" "clarityVVS2" "depth" > [21] "table""price""x""y""z" > > Can it be made to work correctly for both plain and ordered factors? > >> sessionInfo() > R Under development (unstable) (2018-02-06 r74231) > Platform: i386-w64-mingw32/i386 (32-bit) > > other attached packages: > [1] Matrix_1.2-12 > > loaded via a namespace (and not attached): > [1] colorspace_1.3-2 scales_0.5.0 compiler_3.5.0 lazyeval_0.2.1 > [5] plyr_1.8.4 pillar_1.1.0 gtable_0.2.0 tibble_1.4.2 > [9] Rcpp_0.12.15 ggplot2_2.2.1grid_3.5.0 rlang_0.1.6 > [13] munsell_0.4.3lattice_0.20-35 > > -- > Dario Strbenac > University of Sydney > Camperdown NSW 2050 > Australia > > __ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel > __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Warning from Sys.junction when using network drive.
Unfortunately, junctions cannot link to a network drive, they only can link directories on the same computer (possibly on different local volumes). This is a limitation imposed by Windows. I have updated the documentation for Sys.junction in R-devel accordingly. Tomas On 02/06/2018 10:50 PM, MARK BANGHART wrote: I am running 3.4.3 on a windows server and I ran the code in a new session. I get a warning when running packrat::init() on a project that is located on a windows network drive. The warning I get is Warning message: cannot set reparse point 'U:/packrat5/packrat/lib-R/base', reason 'Access is denied' The error is created based inside the function .Internal(mkjunction(fr, link)) which is called from Sys.junction(). I have run Sys.junction inside the RStudio debugger and I checked that the 'U:/packrat5/packrat/lib-R/base' could be accessed via the windows file explorer before the .Internal(mkjunction(fr, link)) call is made. Looking at the code for do_mkjunction(), the warning looks to be thrown based on the return status from DeviceIoControl. I setup a project on the C: drive and tried the same packrat::init() code. The call to .Internal(mkjunction(fr, link)) did not produce an error. I would appreciate any help you can provide on this issue. Thanks, Mark [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] saveRDS() overwrites file when object is not found
Thanks, this has been already reported as bug 17358. Addressed in R-devel 74238. R may still create a corrupt file, though, in other circumstances (e.g. if it runs out of memory or is interrupted during serialization, etc). Tomas On 02/07/2018 04:14 PM, Kenny Bell wrote: I ran into this behaviour when accidentally running a line of code that I shouldn't have. When saving over an rds with an object that's not found, I would have expected saveRDS to not touch the file. saveRDS(iris, "test.rds") file.size("test.rds") #> [1] 1080 saveRDS(no_object_here, "test.rds") #> Error in saveRDS(no_object_here, "test.rds"): object 'no_object_here' not found file.size("test.rds") #> [1] 20 file.remove("test.rds") [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] sparse.model.matrix Generates Non-Existent Factor Levels if Ord.factor Columns Present
Good day, The intention is to convert the dataset into a format suitable for the random forest classifier implemented by the CRAN package xgboost. The input data is required to be transformed into one-hot format using the sparse.discrim.matrix function, as specified by the package's vignette of URL https://cran.r-project.org/web/packages/xgboost/vignettes/discoverYourData.html I did not know to read the help page for contr.poly after reading the sparse.discrim.matrix help page. Perhaps there could be a helpful mention added to it? -- Dario Strbenac University of Sydney Camperdown NSW 2050 Australia __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] R Compilation gets stuck on Windows 64
Hi All, I am trying to compile R from source on a 64 bit Windows. I have downloaded and installed all the third party software as per the R - documentation. The compilation starts fine and after a while it stops with the following error message: D:/Rtools/mingw_64/bin/../lib/gcc/x86_64-w64-mingw32/4.9.3/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -lRgraphapp collect2.exe: error: ld returned 1 exit status make[3]: *** [R.dll] Error 1 make[2]: *** [../../bin/x64/R.dll] Error 2 make[1]: *** [rbuild] Error 2 make: *** [all] Error 2 The file Rgraphapp.dll is created in the folder *D:\R64\R-3.4.3\bin\x64* but somehow the process is not able to find it and gives an error. The purpose for going through this process was to compile R with OpenBlas which I had compiled earlier with MinGW and had created the file: *libopenblas_haswell-r0.2.20.a. * I have attached the MkRules.local and the output log along with this email. Any would be greatly appreciated. Regards, Indrajit make --no-print-directory -C front-ends Rpwd make -C ../../include -f Makefile.win version make Rpwd.exe D:/Rtools/mingw_64/bin/gcc -std=gnu99 -I../../include -DR_ARCH='"x64"' -O3 -Wall -pedantic -c rpwd.c -o rpwd.o D:/Rtools/mingw_64/bin/windres -DWN64 -i rcico.rc -o rcico.o D:/Rtools/mingw_64/bin/gcc -std=gnu99 -s -o Rpwd.exe rpwd.o rcico.o cp front-ends/Rpwd.exe Rpwd.exe Building ../../../library/base/R/Rprofile from ../../library/profile/Common.R ../../library/profile/Rprofile.windows mkdir -p ../../../library/base/R cp -p html/rwin.html ../../../doc/html/index.html cp -p ./etc/Makeconf ./etc/Rcmd_environ ./etc/Rconsole ./etc/Rdevga ./etc/Rprofile.site ./etc/rgb.txt ../../../etc rm -f ../../../etc/Makeconf mkdir -p ../../../etc//x64 sed -e 's/WIN = 32/WIN = 64/' \ -e "s/-O3/-O2/" \ -e "s/@EOPTS@/-mtune=native/" \ -e "s|BINPREF =|BINPREF ?= D:/Rtools/mingw_64/bin/|" \ -e "s|COMPILED_BY =|COMPILED_BY = gcc-4.9.3|" \ -e "s|IMPDIR = bin|IMPDIR = lib|" \ -e "s|LOCAL_SOFT =|LOCAL_SOFT = |" \ -e "s|R_ARCH =|R_ARCH = /x64|" \ -e "s|DT_ARCH =|DT_ARCH = -m i386:x86-64 --as-flags --64|" \ -e "s|RC_ARCH =|RC_ARCH = |" \ -e "s|M_ARCH =|M_ARCH = |" \ -e "s|@SYMPAT@|'s/^.* [BCDRT] / /p'|" \ -e "s|(TCL_HOME)/bin|(TCL_HOME)/bin64|" \ -e "s|@OPENMP@|-fopenmp|" \ -e "s|@PTHREAD@|-pthread|" \ -e "s@NM_FILTER =@NM_FILTER = | sed -e '/[.]refptr[.]/d' -e '/[.]weak[.]/d'@" \ etc/Makeconf > ../../../etc/x64/Makeconf sed -e "s+@BINDIR@+bin/x64+" Makeconf > ../../../Makeconf cp -p ../CHANGES0 ../CHANGES1 ../CHANGES2 ../../../doc make[4]: Nothing to be done for `svnonly'. installing C headers D:/Rtools/mingw_64/bin/dlltool --as D:/Rtools/mingw_64/bin/as -m i386:x86-64 --as-flags --64 -k --dllname Riconv.dll \ --input-def unicode/iconv.def --output-lib libRiconv.dll.a make[5]: Nothing to be done for `svnonly'. installing C headers make --no-print-directory -C ../extra/intl CFLAGS='-O3 -Wall -pedantic -mtune=native' -f Makefile.win sed -e 's,@''HAVE_POSIX_PRINTF''@,1,g' \ -e 's,@''HAVE_ASPRINTF''@,0,g' \ -e 's,@''HAVE_SNPRINTF''@,1,g' \ -e 's,@''HAVE_WPRINTF''@,0,g' \ < libgnuintl.h.in > libgnuintl.h D:/Rtools/mingw_64/bin/gcc -std=gnu99 -DIN_LIBINTL -DHAVE_CONFIG_H -I. -I../../include -O3 -Wall -pedantic -mtune=native -c bindtextdom.c -o bindtextdom.o D:/Rtools/mingw_64/bin/gcc -std=gnu99 -DIN_LIBINTL -DHAVE_CONFIG_H -I. -I../../include -O3 -Wall -pedantic -mtune=native -c dcgettext.c -o dcgettext.o D:/Rtools/mingw_64/bin/gcc -std=gnu99 -DIN_LIBINTL -DHAVE_CONFIG_H -I. -I../../include -O3 -Wall -pedantic -mtune=native -c dgettext.c -o dgettext.o D:/Rtools/mingw_64/bin/gcc -std=gnu99 -DIN_LIBINTL -DHAVE_CONFIG_H -I. -I../../include -O3 -Wall -pedantic -mtune=native -c gettext.c -o gettext.o D:/Rtools/mingw_64/bin/gcc -std=gnu99 -DIN_LIBINTL -DHAVE_CONFIG_H -I. -I../../include -O3 -Wall -pedantic -mtune=native -c finddomain.c -o finddomain.o D:/Rtools/mingw_64/bin/gcc -std=gnu99 -DIN_LIBINTL -DHAVE_CONFIG_H -I. -I../../include -O3 -Wall -pedantic -mtune=native -c loadmsgcat.c -o loadmsgcat.o D:/Rtools/mingw_64/bin/gcc -std=gnu99 -DIN_LIBINTL -DHAVE_CONFIG_H -I. -I../../include -O3 -Wall -pedantic -mtune=native -c textdomain.c -o textdomain.o D:/Rtools/mingw_64/bin/gcc -std=gnu99 -DIN_LIBINTL -DHAVE_CONFIG_H -I. -I../../include -O3 -Wall -pedantic -mtune=native -c l10nflist.c -o l10nflist.o D:/Rtools/mingw_64/bin/gcc -std=gnu99 -DIN_LIBINTL -DHAVE_CONFIG_H -I. -I../../include -O3 -Wall -pedantic -mtune=native -c explodename.c -o explodename.o D:/Rtools/mingw_64/bin/gcc -std=gnu99 -DIN_LIBINTL -DHAVE_CONFIG_H -I. -I../../include -DLOCALEDIR=\"\" -O3 -Wall -pedantic -mtune=native -c dcigettext.c -o dcigettext.o D:/Rtools/mingw_64/bin/gcc -std=gnu99 -DIN_LIBINTL -DHAVE_CONFIG_H -I. -I../../include -O3 -Wall -pedantic -mtune=native -c dcngettext.c -o dcngettext.o D:/Rtools/mingw_64/bin/gcc -std=gnu99 -DIN_LIBINTL -DHAVE_C
Re: [Rd] R Compilation gets stuck on Windows 64
On Thu, Feb 8, 2018 at 9:44 PM, Indrajit Sen Gupta wrote: > Hi All, > > I am trying to compile R from source on a 64 bit Windows. [snip] > I had compiled earlier with MinGW and had created the file: > *libopenblas_haswell-r0.2.20.a. * Hello, Indrajit. I don't see your MkRules.local attached. In any event, perhaps try following the directions here [1]. I've been building R with OpenBLAS on Windows 64 for years and it almost always works. In the past year or two, rarely, it will stop with an error. But if you restart the make process (by just typing "make" again) it finishes with no issues and passes make check-devel. I have not tried this with R-dev, though. R 3.4.3 Patched (2018-01-03 r74042) is the most recent I have built successfully. Good luck, Avraham [1] https://www.avrahamadler.com/r-tips/build-openblas-for-windows-r64/ __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] R Compilation gets stuck on Windows 64
Hi Avraham, What a coincidence, I have been following this post of yours: https://www.avrahamadler.com/2013/10/24/an-openblas-based-rblas-for-windows-64-step-by-step/ Looks like this post is slightly older than what you have shared previously. It is strange that you did not get the attachments. I am pasting the contents of the MkRules.local here: --- #-*- Makefile -*- ## This is only used when building R itself but it does customize ## etc/*/Makeconf using LOCAL_SOFT, BINPREF[64], IMPLIB and R_ARCH ## Customize by copying to MkRules.local and uncommenting and editing ## some of the definitions there. ## ## === configuration macros for building packages # Absolute path to '/usr/local' software collection. The versions used # on CRAN can be found at https://www.stats.ox.ac.uk/pub/Rtools/libs.html # It can be interrogated by 'R CMD config LOCAL_SOFT' # Use 'make rsync-extsoft' to populate the default directory. # LOCAL_SOFT = D:/R64/extsoft ## == configuration macros for building R === # Path of library directory containing zlib, bzlib, liblzma, pcre, # libpng, libjpeg, libtiff. # Use 'make rsync-extsoft' to populate the default directory. EXT_LIBS = D:/R64/extsoft # an alternative is to use -gstabs here, if the debugger supports only stabs. # G_FLAG = -gdwarf-2 # Set to YES and specify the path if you want to use the ATLAS BLAS. USE_ATLAS = YES ATLAS_PATH =D:/home/thread0 # Support for the ACML and Goto BLASes has been withdrawn: see R-admin.html # Define to use svnversion to set SVN-REVISION (slow, and requires a clean # checkout with no modifications). # USE_SVNVERSION = YES # With the previously recommended gcc 4.6.3 toolchain, set this to 32 or 64 # MULTI = 64 # If the toolchain's bin directory is not in your path, set this to the path # (including the trailing /, and use / not \). # TOOL_PATH = # for other toolchains leave these empty and set the more detailed options below # With the recommended gcc 4.9.3 toolchain or another toolchain, set # BINPREF and BINPREF64 (below) to the respective bin directories. # Include the trailing /, and use / not \. # Do this in the more detailed options below # Set this to 32 or 64 WIN = 64 ### BEGIN more detailed options # Some of the toolchains have prefixes for e.g. ar, gcc. # This can also be used to give the full path to the compiler, # including a trailing / . # BINPREF = c:/Rtools/mingw_32/bin/ # prefix for 64-bit: BINPREF64 = D:/Rtools/mingw_64/bin/ # Set this to indicate a non-gcc compiler and version # COMPILED_BY = # Others use a -m64 or -m32 option to select architectures # M_ARCH = -m64 # and for as (--32 or --64) # AS_ARCH = --64 # and for windres (-F pe-i386 or pe-x86-64) # RC_ARCH = pe-x86-64 # and for dlltool ("-m i386 --as-flags --32" vs "-m i386:x86-64 --as-flags --64") DT_ARCH = -m i386:x86-64 --as-flags --64 # 32- or 64-bit Windows? WIN = 64 # The gcc 4.9.3 64 bit toolchain is set up for the 'medium code' model and needs # to remove the .refptr and .weak entries from the exports list; this is the default # when WIN = 64, with blank for WIN = 32: NM_FILTER = | $(SED) -e '/[.]refptr[.]/d' -e '/[.]weak[.]/d' # We normally link directly against DLLs, # but this macro forces the use of import libs # Has been needed for some versions of MinGW-w64 USE_IMPLIBS = YES ### END more detailed options # set to use ICU USE_ICU = YES # path to parent of ICU headers ICU_PATH = D:/home/ICU ICU_LIBS = -lsicuin -lsicuuc -lsicudt -lstdc++ # set to use libcurl USE_LIBCURL = YES # path to parent of libcurl headers CURL_PATH = D:/home/curl # libs: for 32-bit # CURL_LIBS = -lcurl -lrtmp -lssl -lssh2 -lcrypto -lgdi32 -lcrypt32 -lz -lws2_32 -lgdi32 -lcrypt32 -lwldap32 -lwinmm -lidn # libs: for 64-bit CURL_LIBS = -lcurl -lrtmp -lssl -lssh2 -lcrypto -lgdi32 -lcrypt32 -lz -lws2_32 -lgdi32 -lcrypt32 -lwldap32 -lwinmm # For the cairographics devices # Optionally use a static build of cairographics from # https://www.rforge.net/Cairo/files/cairo-current-win.tar.gz # and set CAIRO_HOME to the parent of the win32/win64 directories # # If CAIRO_HOME is not set the devices are not built. CAIRO_HOME = D:/home/cairo/src # set this to YES to build static HTML help BUILD_HTML = YES # unset this if you are *not* using MiKTeX MIKTEX = TRUE # Recent MiKTEX does not provide texi2dvi and needs something like TEXI2DVI = TEXINDEX=texindex.exe texify # for texinfo >= 5.1. If the texinfo files are installed at /packages/texinfo, # TEXI2ANY = /path/to/perl -I/packages/texinfo /packages/texinfo/texi2any # if you do not have texinfo (default), # TEXI2ANY = missing # additional optimization flags (use -mtune=native for a private build) EOPTS = -mtune=native # define to -fopenmp if the toolchain has OpenMP support # OPENMP = -fopenmp # define to -pthread if the toolchain has pthreads support # PTHREAD = -pthread ## == configuration macros for buil
Re: [Rd] R Compilation gets stuck on Windows 64
Hi Avraham, A quick question - I realized I did not have *Perl* installed. So I installed *ActiveState Perl* right now. Also I see I need *texinfo* and *texi2any*. I was able to installed *texinfo* from here: http://gnuwin32.sourceforge.net/packages/texinfo.htm. But not sure where to get *texi2any*. Can you guide me in this step? Regards, Indrajit On Fri, Feb 9, 2018 at 11:58 AM, Indrajit Sen Gupta wrote: > Hi Avraham, > > What a coincidence, I have been following this post of yours: https://www. > avrahamadler.com/2013/10/24/an-openblas-based-rblas-for- > windows-64-step-by-step/ > > Looks like this post is slightly older than what you have shared > previously. It is strange that you did not get the attachments. I am > pasting the contents of the MkRules.local here: > > --- > > #-*- Makefile -*- > > ## This is only used when building R itself but it does customize > ## etc/*/Makeconf using LOCAL_SOFT, BINPREF[64], IMPLIB and R_ARCH > > ## Customize by copying to MkRules.local and uncommenting and editing > ## some of the definitions there. > ## > > ## === configuration macros for building packages > # Absolute path to '/usr/local' software collection. The versions used > # on CRAN can be found at https://www.stats.ox.ac.uk/pub/Rtools/libs.html > # It can be interrogated by 'R CMD config LOCAL_SOFT' > # Use 'make rsync-extsoft' to populate the default directory. > # LOCAL_SOFT = D:/R64/extsoft > > ## == configuration macros for building R === > > # Path of library directory containing zlib, bzlib, liblzma, pcre, > # libpng, libjpeg, libtiff. > # Use 'make rsync-extsoft' to populate the default directory. > EXT_LIBS = D:/R64/extsoft > > # an alternative is to use -gstabs here, if the debugger supports only > stabs. > # G_FLAG = -gdwarf-2 > > # Set to YES and specify the path if you want to use the ATLAS BLAS. > USE_ATLAS = YES > ATLAS_PATH =D:/home/thread0 > > # Support for the ACML and Goto BLASes has been withdrawn: see R-admin.html > > # Define to use svnversion to set SVN-REVISION (slow, and requires a clean > # checkout with no modifications). > # USE_SVNVERSION = YES > > # With the previously recommended gcc 4.6.3 toolchain, set this to 32 or 64 > # MULTI = 64 > # If the toolchain's bin directory is not in your path, set this to the > path > # (including the trailing /, and use / not \). > # TOOL_PATH = > # for other toolchains leave these empty and set the more detailed options > below > > # With the recommended gcc 4.9.3 toolchain or another toolchain, set > # BINPREF and BINPREF64 (below) to the respective bin directories. > # Include the trailing /, and use / not \. > # Do this in the more detailed options below > # Set this to 32 or 64 > WIN = 64 > > > ### BEGIN more detailed options > # Some of the toolchains have prefixes for e.g. ar, gcc. > # This can also be used to give the full path to the compiler, > # including a trailing / . > # BINPREF = c:/Rtools/mingw_32/bin/ > # prefix for 64-bit: > BINPREF64 = D:/Rtools/mingw_64/bin/ > # Set this to indicate a non-gcc compiler and version > # COMPILED_BY = > > # Others use a -m64 or -m32 option to select architectures > # M_ARCH = -m64 > # and for as (--32 or --64) > # AS_ARCH = --64 > # and for windres (-F pe-i386 or pe-x86-64) > # RC_ARCH = pe-x86-64 > # and for dlltool ("-m i386 --as-flags --32" vs "-m i386:x86-64 --as-flags > --64") > DT_ARCH = -m i386:x86-64 --as-flags --64 > > # 32- or 64-bit Windows? > WIN = 64 > > # The gcc 4.9.3 64 bit toolchain is set up for the 'medium code' model and > needs > # to remove the .refptr and .weak entries from the exports list; this is > the default > # when WIN = 64, with blank for WIN = 32: > NM_FILTER = | $(SED) -e '/[.]refptr[.]/d' -e '/[.]weak[.]/d' > > # We normally link directly against DLLs, > # but this macro forces the use of import libs > # Has been needed for some versions of MinGW-w64 > USE_IMPLIBS = YES > > ### END more detailed options > > > # set to use ICU > USE_ICU = YES > # path to parent of ICU headers > ICU_PATH = D:/home/ICU > ICU_LIBS = -lsicuin -lsicuuc -lsicudt -lstdc++ > > # set to use libcurl > USE_LIBCURL = YES > # path to parent of libcurl headers > CURL_PATH = D:/home/curl > # libs: for 32-bit > # CURL_LIBS = -lcurl -lrtmp -lssl -lssh2 -lcrypto -lgdi32 -lcrypt32 -lz > -lws2_32 -lgdi32 -lcrypt32 -lwldap32 -lwinmm -lidn > # libs: for 64-bit > CURL_LIBS = -lcurl -lrtmp -lssl -lssh2 -lcrypto -lgdi32 -lcrypt32 -lz > -lws2_32 -lgdi32 -lcrypt32 -lwldap32 -lwinmm > > # For the cairographics devices > # Optionally use a static build of cairographics from > # https://www.rforge.net/Cairo/files/cairo-current-win.tar.gz > # and set CAIRO_HOME to the parent of the win32/win64 directories > # > # If CAIRO_HOME is not set the devices are not built. > CAIRO_HOME = D:/home/cairo/src > > # set this to YES to build static HTML help > BUILD_HTML = YES > > # unset this if you