DejaGNU gurus ahoi! (Re: Testing use of STDIN redirected etc.)
Hi,
what would be the best way to test stuff like two binaries
communicating via a pipe, FIFO or such with DejaGNU? The gfortran
testsuite has by now quite extensive coverage of all the weird and
quirky corner cases of Fortran I/O behavior, but practically all these
tests are done using regular files. Over the years we've had some
embarrassing bugs in handling non-seekable files like terminals,
FIFO's, and pipes. I believe these would have been quickly caught if
only we had testing coverage.
(Something which works only on x86_64-unknown-linux-gnu and perhaps a
few other "mainstream" targets where we can expect the presence of a
"normal" shell environment etc., while not optimal, is certainly a lot
better than what we have now.)
Any ideas? See also the mail below which discusses the same issue.
On Mon, Oct 6, 2014 at 3:29 AM, Jerry DeLisle wrote:
> Hi all,
>
> Over the years we have had a number of regressions related to reading from
> STDIN.
>
> Would it be acceptable to add a sub-directory such as
> gfortran.dg/testscripts.
>
> In this location we put one or more scripts to be executed to perform tests
> outside of the dejagnu environment.
>
> Within gfortran.dg we place a { dg-run } program that invokes the test
> scripts and checks the results.
>
> Is this doable? Anyone have any better ideas?
>
> Regards,
>
> Jerry
--
Janne Blomqvist
Re: DejaGNU gurus ahoi! (Re: Testing use of STDIN redirected etc.)
On Oct 31 2014, Janne Blomqvist wrote: what would be the best way to test stuff like two binaries communicating via a pipe, FIFO or such with DejaGNU? The gfortran testsuite has by now quite extensive coverage of all the weird and quirky corner cases of Fortran I/O behavior, but practically all these tests are done using regular files. Over the years we've had some embarrassing bugs in handling non-seekable files like terminals, FIFO's, and pipes. I believe these would have been quickly caught if only we had testing coverage. It's worse. It's much, much worse :-( When the original f77 was perpetrated, er, written, it chose a bizarre unformatted format to make BACKSPACE efficient. Inter alia, this made it hard for implementations to support things like sockets (and magnetic tape!) for unformatted files. Previously, virtually every system supported such things. BACKSPACE has been a known abomination since the 1960s, and has caused and still causes a LOT of trouble over the years, but I failed to get it even deprecated. So the first question is to decide the priority of I/O variations, including types of file and permitted operations for each connection mode. The key characteristics (in OPEN) are ACCESS, FORM and ACTION. In particular, there is disagreement on WG5 exactly what constitutes a 'file' - I have been told that magnetic tape files and sockets are not files by someone who ought to have known better. For example, does gfortran intend to support the piping of unformatted files? Not all compilers do. My apologies if gfortran/DejaGNU has already done this, but I am pretty sure not. If anyone has, please could they point me to a reference? (Something which works only on x86_64-unknown-linux-gnu and perhaps a few other "mainstream" targets where we can expect the presence of a "normal" shell environment etc., while not optimal, is certainly a lot better than what we have now.) You can do a LOT better than THAT! It would be trivial to write something that would work on any reasonably Unix-like system (including anything that supports base POSIX 1996). I have done that for system scripts and still do that for my course tests, though I no longer have access to weird and wonderful Unices to test them. You are quite right that you need to assume a reasonable shell, though you can write such tests that will work on pretty well any modern Bourne, Korn or POSIX shell, and any version of bash. The original Bourne shell was beyond redemption, but I believe that it is now completely dead. That is from experience in writing portable system scripts a decade or so back, but I am pretty sure it still holds. Regards, Nick Maclaren.
Re: DejaGNU gurus ahoi! (Re: Testing use of STDIN redirected etc.)
On Fri, Oct 31, 2014 at 1:20 PM, N.M. Maclaren wrote: > On Oct 31 2014, Janne Blomqvist wrote: >> >> >> what would be the best way to test stuff like two binaries >> communicating via a pipe, FIFO or such with DejaGNU? The gfortran >> testsuite has by now quite extensive coverage of all the weird and >> quirky corner cases of Fortran I/O behavior, but practically all these >> tests are done using regular files. Over the years we've had some >> embarrassing bugs in handling non-seekable files like terminals, >> FIFO's, and pipes. I believe these would have been quickly caught if >> only we had testing coverage. > > > It's worse. It's much, much worse :-( I know, being one of the main libgfortran IO maintainers for several years, I know more about the quirkyness of Fortran I/O than I ever would have wanted to.. :( My main aim here really is to just get the basics right before worrying about corner cases. For instance, I recall we have failed in a simple sequential reading of a access="sequential", form="formatted" unit connected to a pipe or such, because libgfortran incorrectly issued a lseek() at some point, which returned an error (since the fd wasn't seekable), and that error was then duly propagated upwards. But since the testsuite didn't test against this, it took an embarrassingly long time before that bug was found and fixed. > So the first question is to decide the priority of I/O variations, > including types of file and permitted operations for each connection > mode. The key characteristics (in OPEN) are ACCESS, FORM and ACTION. > In particular, there is disagreement on WG5 exactly what constitutes a > 'file' - I have been told that magnetic tape files and sockets are not > files by someone who ought to have known better. For example, does > gfortran intend to support the piping of unformatted files? Not all > compilers do. My argument in this, which I have stated many times on the mailing list and in bugzilla over the years, is that there are many kinds of special files which are special in different ways. E.g. wrt. seeking, some special files allow seeking just fine, others report an error, and others always report an offset of 0. Instead of trying to enumerate in which ways special files are special on all supported targets, which is a sisyphean task, libgfortran should just do what it's being told to, no more, and if any error occurs, propagate it upwards to the calling program. The "no more" is the problematic part, e.g. a spurious lseek(fd, 0, SEEK_CUR) would have have no effect on a regular file, but cause an error on a non-seekable file. Hence the need for a testsuite for non-regular files. >> (Something which works only on x86_64-unknown-linux-gnu and perhaps a >> few other "mainstream" targets where we can expect the presence of a >> "normal" shell environment etc., while not optimal, is certainly a lot >> better than what we have now.) > > > You can do a LOT better than THAT! It would be trivial to write > something that would work on any reasonably Unix-like system (including > anything that supports base POSIX 1996). I have done that for system > scripts and still do that for my course tests, though I no longer have > access to weird and wonderful Unices to test them. > > You are quite right that you need to assume a reasonable shell, though > you can write such tests that will work on pretty well any modern Bourne, > Korn or POSIX shell, and any version of bash. The original Bourne shell > was beyond redemption, but I believe that it is now completely dead. > That is from experience in writing portable system scripts a decade or > so back, but I am pretty sure it still holds. For what I'm thinking of, I'm sure a pretty basic POSIX sh + maybe a few utilities such as mkfifo would do just fine. My reason for wanting to limit it, is that I, and AFAIK most gfortran developers, develop and test on x86_64-unknown-linux-gnu, and with the very limited time that I can afford to gfortran these days I don't want to spend that time trying to debug-over-email why a script fails on some weird target I don't have access to. If you want to make it work on Ultrix or something like that, be my guest, patches welcome. (For a similar reason, I think it wouldn't make sense to run these tests in a cross environment, more complexity for little gain (unless it's really simple?).) -- Janne Blomqvist
Performance for AArch64 in ILP32 ABI
Hi, According to this mail thread https://gcc.gnu.org/ml/gcc-patches/2013-12/msg00282.html GCC has ILP32 GNU/Linux support. 1. The question is: how reasonable it can be to use ILP32 mode for building of the *whole* Linux distribution from the side of view of performance? IIRC gcc built for i686 can work faster than gcc built for x86_64 architecture on the same hardware, because there are a lot of data structures with fields of pointer type, and if 32 pointers are used, less memory is allocated for these structures. As a result, smaller structures are loaded from memory faster and less cache misses happen. Is this the same case for AArch64 ILP32 ABI? 2nd idea is that if integers are of 32 bit size, than 2 times more integers can be saved in CPU registers than if they were of 64 bit size, and thus less loads/stores to the memory are needed. 2. What's the current status of ILP32 support implementation in GCC? 3. Did anybody try to benchmark AArch64 binaries ILP32 vs. LP64 builds? Is it possible to compare the performance of these modes? Best regards, Ilya Palachev
Re: libcc1 still breaks bootstrap (with clang as system compiler)
On 31 October 2014 06:51, Phil Muldoon wrote:
> On 30/10/14 21:47, Gerald Pfeifer wrote:
>> Now the error is gone on my nightly FreeBSD test systems,
>> I am getting the following:
>>
>> In file included from /scratch2/tmp/gerald/gcc-HEAD/libcc1/plugin.cc:58:
>> In file included from /usr/include/c++/v1/string:438:
>> In file included from /usr/include/c++/v1/cwchar:107:
>> In file included from /usr/include/c++/v1/cwctype:54:
>> /usr/include/c++/v1/cctype:51:72: error: use of undeclared identifier
>> 'do_not_use_isalnum_with_safe_ctype'
>> inline _LIBCPP_INLINE_VISIBILITY int __libcpp_isalnum(int __c) {return
>> isalnum(__c);}
>>^
>> /scratch2/tmp/gerald/gcc-HEAD/libcc1/../include/safe-ctype.h:126:20:
>> note: expanded from macro 'isalnum'
>> #define isalnum(c) do_not_use_isalnum_with_safe_ctype
>>^
>
> Sorry for the issues. Jakub has a patch pending that disables libcc1
> on bootstrap so that may alleviate the issue. However if you let me
> know your ./configure parameters and any other build parameters I will
> attempt to replicate the issue.
Maybe the same issue as
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63699 which has a patch.
Re: DejaGNU gurus ahoi! (Re: Testing use of STDIN redirected etc.)
On Oct 31 2014, Janne Blomqvist wrote: My main aim here really is to just get the basics right before worrying about corner cases. For instance, I recall we have failed in a simple sequential reading of a access="sequential", form="formatted" unit connected to a pipe or such, because libgfortran incorrectly issued a lseek() at some point, which returned an error (since the fd wasn't seekable), and that error was then duly propagated upwards. But since the testsuite didn't test against this, it took an embarrassingly long time before that bug was found and fixed. Yes. I agree. But see my next point. So the first question is to decide the priority of I/O variations, including types of file and permitted operations for each connection mode. The key characteristics (in OPEN) are ACCESS, FORM and ACTION. In particular, there is disagreement on WG5 exactly what constitutes a 'file' - I have been told that magnetic tape files and sockets are not files by someone who ought to have known better. For example, does gfortran intend to support the piping of unformatted files? Not all compilers do. My argument in this, which I have stated many times on the mailing list and in bugzilla over the years, is that there are many kinds of special files which are special in different ways. E.g. wrt. seeking, some special files allow seeking just fine, others report an error, and others always report an offset of 0. Instead of trying to enumerate in which ways special files are special on all supported targets, which is a sisyphean task, libgfortran should just do what it's being told to, no more, and if any error occurs, propagate it upwards to the calling program. The "no more" is the problematic part, e.g. a spurious lseek(fd, 0, SEEK_CUR) would have have no effect on a regular file, but cause an error on a non-seekable file. Hence the need for a testsuite for non-regular files. Again, I agree, but that wasn't quite my point. I fully agree that providing such a list in the release notes is insanity. What I feel needs doing is a generic priority list of which functionality should take priority over which, for the guidance of gfortran developers and maintainers. Naturally, it would be subject to change. The point is that the choice of open() options, data format and OS transfer calls constrains what will work, and so you have to choose before you know what the code will do. The most ancient and worst insanity is not knowing the initial direction of transfer, but (as you say) repositioning and truncation vs replacement are almost as ancient and bad :-( While I would much prefer such decisions being made at OPEN time, my attempt to get this supported (not required!) in the standard failed. But there ARE some simple decisions that could be taken; for example (and these are merely PROPOSALS): If it is not a 'regular' file, ACCESS='DIRECT' should be forbidden If it is a FIFO, at least BACKSPACE and POS= should be forbidden Important decisions of priority would include whether pipability should take precedence over positionability for (a) stream and (b) unformatted sequential files. The Unix lunacy of compounding stream I/O with direct-access I/O (i.e. ACCESS='STREAM') has always been a disaster for both RAS and portability. And I really don't know what most compilers' attitudes are to that one, despite having asked several developers! For what I'm thinking of, I'm sure a pretty basic POSIX sh + maybe a few utilities such as mkfifo would do just fine. My reason for wanting to limit it, is that I, and AFAIK most gfortran developers, develop and test on x86_64-unknown-linux-gnu, and with the very limited time that I can afford to gfortran these days I don't want to spend that time trying to debug-over-email why a script fails on some weird target I don't have access to. If you want to make it work on Ultrix or something like that, be my guest, patches welcome. (For a similar reason, I think it wouldn't make sense to run these tests in a cross environment, more complexity for little gain (unless it's really simple?).) Yes, but this is EXACTLY the sort of thing that fails on those systems, and such tests would be really, really useful for people working on such systems. What I would suggest is that the scripts be supported for a generic POSIX system, but not be enabled by default or in a way that J. Random User is likely to trip over. Regards, Nick Maclaren.
Support for architectures without hardware interlocks
Hello, I'm working on porting gcc to an architecture without hardware interlock support for floating point unit. I read that instruction latency time can be expressed in machine description file of gcc. I set the latency time of the instructions and built gcc. I expected that gcc would put the two dependent instructions apart automatically at least as many as the latency time of the first instruction. However, my gcc doesn't do that. I'm using a little old 4.7.3. I also expected that gcc may fill the gap with no-op when it cannot find other useful instructions to fill the gap. But, I don't see that, either. Does gcc support an architecture without hardware interlock automatically? Could anyone help me to understand how I can enforce the latency requirements of two dependent instructions in gcc? I saw that GCC didn't support architectures without hardware interlocks in the gcc mailing list which is dated in 2007. (https://gcc.gnu.org/ml/gcc/2007-07/msg00915.html) Is it still true? Thanks, David. -- -- Dr. Dong-In "David" Kang Computer Scientist USC/ISI
how to keep a hard register across multiple instrutions?
Hi, I'm newbie in gcc porting. The architecture that I'm porting gcc has hardware FPU. But the compiler has to generate code which builds a FPU instruction in a integer register at run-time and writes the value to the FPU command register. To make a single FPU instruction, three instructions are needed. Two instructions make the FPU instruction in 32 bit (cmd, operands[2], operands[1], operands[0]) format. Here operands are the FPU register numbers, which can be 0 ~ 32. As an example, f3 = f1 + 2 can be encoded as (code of 'add', 2, 1, 3). And the third instruction write it to a FPU command register. The architecture can issue up to 3 instructions at a time. The difficulty lies in that we need to know the FPU register number for those operands to generate the FPU instruction. The easiest but lowest performance implementation is to generate those three instruction from a single "define_insn" as three consecutive instructions. However, we lose all possible bundling of those 3 instructions with other instructions for optimization. So, I'm trying to find a better way. I used "define_insn_and_split" and split a single FPU instruction into 3 instructions like this: (Here I assume to use register r10, but it can be any integer register.) operands[0] = plus (operands[1], operands[2]) ==> (1) r10 <- lower half of FPU instruction using (code of 'add', operands[0], operands[1], operands[2]) (2) r10 <- r10 | upper half of FPU instruction using (code of 'add', operands[0], operands[1], operands[2]) (3) (FPU cmd register) <- r10 The problem is that gcc catches that operands[0] is used before the 3rd instruction, and allocates two different hard registers for (1,2) instructions and (3) instruction. So, when the code is generated, the first two instructions are assuming wrong register for operands[0]. This happens especially frequently when '-unroll' option is used. So, I think if there is a way to inform gcc to use the same hard registers for operands[0] across those three instructions. Is it possible? Or would there be any better way to generate efficient FPU code? I will appreciate any advice or pointer to further information. Thanks, David -- -- Dr. Dong-In "David" Kang Computer Scientist USC/ISI
