[Rd] readlines() truncates text file with Codepage 437 encoding
Hello r-devel,
The attached Code page 437-encoded file contains 245 characters
(including the final newline), but readLines only reads 242 of them:
> test_text <- readLines(file('437__characters.txt', encoding='437'))
Warning message:
In readLines(file("437__characters.txt", :
incomplete final line found on '437__characters.txt'
> test_text
[1]
"\v\f\016\017\020\021\022\023\024\025\026\027\030\031\032\033\034\035\036\037
!\"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_`abcdefghijklmnopqrstuvwxyz{|}~\177
¡¢£¥ª«¬°±²µ·º»¼½¿ÄÅÆÇÉÑÖÜßàáâäåæçèéêëìíîïñòóôö÷ùúûüÿƒΓΘΣΦΩαδεπστφⁿ₧∙√∞∩≈≡≤≥⌐⌠⌡─│┌┐└┘├┤┬┴┼═║╒╓╔╕╖╗╘╙╚╛╜╝╞╟╠╡╢╣╤╥╦╧╨╩╪╫╬▀▄█▌▐░▒"
> nchar(test_text)
[1] 242
You'll note that readLines hasn't read the final characters "▓■\n".
# Diagnostics
My best guess is that this is something to do with how readLines()
determines when it has reached EOF, because of the following:
- The file is terminated with an ASCII LF (0x0a), but R gives an
'incomplete final line found' warning. Note that in some implementations
of Code page 437, 0x0a is interpreted as a graphical character rather
than a control character, but this does not seem to be the problem here.
The same problem occurs if the file ends with 0x0d or 0x0d 0x0a.
- Adding seven or more characters to the end of the file makes it read
correctly
- Similarly, the file is read correctly if you remove three characters
from anywhere in the file
- The same issue seems to occur with reading files encoded in other DOS
code pages
# Additional information
> sessionInfo()
R version 3.2.3 (2015-12-10)
Platform: x86_64-apple-darwin14.5.0 (64-bit)
Running under: OS X 10.10.5 (Yosemite)
locale:
[1] en_GB.UTF-8/en_GB.UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
The same behaviour occurs under R 2.15.1 on a Linux server.
In case the attached file is somehow corrupted, here is a hexdump:
: 0b0c 0e0f 1011 1213 1415 1617 1819 1a1b
0010: 1c1d 1e1f 2021 2223 2425 2627 2829 2a2b !"#$%&'()*+
0020: 2c2d 2e2f 3031 3233 3435 3637 3839 3a3b ,-./0123456789:;
0030: 3c3d 3e3f 4041 4243 4445 4647 4849 4a4b <=>?@ABCDEFGHIJK
0040: 4c4d 4e4f 5051 5253 5455 5657 5859 5a5b LMNOPQRSTUVWXYZ[
0050: 5c5d 5e5f 6061 6263 6465 6667 6869 6a6b \]^_`abcdefghijk
0060: 6c6d 6e6f 7071 7273 7475 7677 7879 7a7b lmnopqrstuvwxyz{
0070: 7c7d 7e7f ffad 9b9c 9da6 aeaa f8f1 fde6 |}~.
0080: faa7 afac aba8 8e8f 9280 90a5 999a e185
0090: a083 8486 9187 8a82 8889 8da1 8c8b a495
00a0: a293 94f6 97a3 9681 989f e2e9 e4e8 eae0
00b0: ebee e3e5 e7ed fc9e f9fb ecef f7f0 f3f2
00c0: a9f4 f5c4 b3da bfc0 d9c3 b4c2 c1c5 cdba
00d0: d5d6 c9b8 b7bb d4d3 c8be bdbc c6c7 ccb5
00e0: b6b9 d1d2 cbcf d0ca d8d7 cedf dcdb ddde
00f0: b0b1 b2fe 0a .
Has anyone encountered something similar?
Kind regards,
Adam Obeng
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] getGraphicsEvent on X11 and event queuing
Hi R-Devel,
I've been working on an oscilloscope project using an Arduino
microcontroller board. I found that it's quite easy to get realtime
updates, e.g. 30+ frames per second, if I read data from the board in
a little Rcpp library. I have to use dev.hold() and dev.flush() to
keep the plot from flickering, which restricts me to the "cairo" X11
device.
I'd like to be able to add interactivity to the oscilloscope display,
for instance to bind a key to save the current plot to a file, or to
bind keys for adjusting the time scale etc.
However, I ran into two problems:
(1) setGraphicsEventHandlers only works on the "Xlib" X11 device,
which doesn't support buffering via dev.hold() - it flickers.
(2) getGraphicsEvent and friends lack some interface features which
are needed to use the functions in an asynchronous fashion. Typically,
event listener library functions have a "timeout" parameter, and the
ability to return already-queued events. But getGraphicsEvent() has
neither - it waits indefinitely (you can't set a timeout), and it
seems to ignore events which occurred before it was called (I can't
figure out why, from the code, I guess the normal R event processing
grabs events which occur between calls to getGraphicsEvent?).
It seems like it should be possible to set a handler for keyboard
events and have it execute in between plot updates when the user
presses a key - yet without blocking further updates if no key has
been pressed.
Is anyone interested in fixing (1) and (2)? Or is there some other
library or workaround to solve my problems?
Here is some code I used to play around with these functions:
X11(type="Xlib");
keydown = function(key) {
cat("Got key: ",key);
lastkey <<- key
}
setGraphicsEventHandlers(onKeybd = keydown);
plot(c(0,0));
getGraphicsEvent();
Well, I think it's great that getGraphicsEvent exists at all, and want
to salute Duncan Murdoch who is listed as the author. I hope I may
have helped by describing some new possible uses for these functions.
Thank you,
Frederick
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
