[RFC, Fortran] Avoid race on testsuite temporary files
Hi, as it was raised in https://gcc.gnu.org/ml/gcc-patches/2015-10/msg01540.html we experiment random failures in gfortran validation when it is run in parallel (-j 8). The issues occurs because of concurrent access to the same file, the first two patches fixed some of them by changing the file names used, but there are still remaining conflicts (6 usages of foo, 8 of test.dat). This is easy to fix and I've a patch for that, but there is another issue and I'd like to have your thoughts on it. There is a little bit more than 1000 testcases which use IO without explicit file names, ~150 use scratches (a temporary file named gfortrantmp + 6 extra char is created) and the others, which only specify the io unit number, use a file named fort.NN with NN the number of the io unit. We see conflicts on these generated files, as lot of testcases use the same number, the most used are: 10 => 150 99 => 70 6 => 31 11 => 27 1 => 23 I started to change the testcases to use scratches when it is possible and before finding that there is that many to fix, and I also had conflicts on the generated scratch names. The options I see to fix that are: 1- Move all these testcases into an IO subdir and change the testsuite to avoid parallelism in that directory. 2- Use scratches when possible and improve libgfortran file name generation, I don't know well fortran but is it possible to change the file name patterns for scratches and io unit files ? 3- Change the io unit numbers used, as it was suggested on irc, but I find it a bit painful to maintain. Any comments are welcome. Thanks Yvan
Re: [RFC, Fortran] Avoid race on testsuite temporary files
Hi Janne, On 11 December 2015 at 12:12, Janne Blomqvist wrote: > On Wed, Dec 9, 2015 at 4:27 PM, Yvan Roux wrote: >> Hi, >> >> as it was raised in >> https://gcc.gnu.org/ml/gcc-patches/2015-10/msg01540.html we experiment >> random failures in gfortran validation when it is run in parallel (-j >> 8). The issues occurs because of concurrent access to the same file, >> the first two patches fixed some of them by changing the file names >> used, but there are still remaining conflicts (6 usages of foo, 8 of >> test.dat). This is easy to fix and I've a patch for that, but there is >> another issue and I'd like to have your thoughts on it. >> >> There is a little bit more than 1000 testcases which use IO without >> explicit file names, ~150 use scratches (a temporary file named >> gfortrantmp + 6 extra char is created) and the others, which only >> specify the io unit number, use a file named fort.NN with NN the >> number of the io unit. We see conflicts on these generated files, as >> lot of testcases use the same number, the most used are: >> >> 10 => 150 >> 99 => 70 >> 6 => 31 >> 11 => 27 >> 1 => 23 >> >> I started to change the testcases to use scratches when it is possible >> and before finding that there is that many to fix, and I also had >> conflicts on the generated scratch names. The options I see to fix >> that are: >> >> 1- Move all these testcases into an IO subdir and change the testsuite >> to avoid parallelism in that directory. >> 2- Use scratches when possible and improve libgfortran file name >> generation, I don't know well fortran but is it possible to change the >> file name patterns for scratches and io unit files ? > > For the "io unit files", I don't think we can change it. fort.NN is a > common convention, and I suspect a lot of people rely on it. As for > scratch files, I'm surprised you're seeing conflicts. At least on > Linux/POSIX targets it should use mk(o)stmp(), which should make sure > a unique file name is chosen and opened with exclusive access > (O_CREAT|O_EXCL). Furthermore, on POSIX targets it unlinks the file > immediately after opening it, further reducing the possibility of any > conflict. So what target are you compiling for? OK if people rely on the fort.NN name pattern I understand it can't be changed. We're validating on x86, ARM and AArch64 linux targets and we experienced random issues with backslash_1.f90, given the nature of the testcase and the other issues with fort.NN conflict I assumed it was the same kind, but it's most likely due to some instabilities we had in our lab several months ago, as we have no more issues with it now, and my manual experiment of mk(o)stmp() usage didn't manage to create conflicts. So, Thanks for the clarification and sorry for the false alarm on scratches. >> 3- Change the io unit numbers used, as it was suggested on irc, but I >> find it a bit painful to maintain. > > Is it really that much more work than 1)? If you don't want to keep > track of a count, just generate a random number between 10 and 2^31, > and use that as the unit number? Should at least make conflicts rare. I don't think it is more work to fix the actual testsuite, my point of view is that 1) will guarantee that we will not have any conflicts no matter which IO units or temporary file names are or will be used, whereas if we keep them as it, with changing the numbers, maintainers and contributors will have to take care of choosing an IO unit or a temporary file name that is not already used in the testsuite. Thanks, Yvan
Romain Geissler copyright assignment
Hi, Romain is doing an internship at STMicroelectronics on GCC plugins, and as his mentor, I confirm and/or inform you that he is covered by the copyright assignement RT 211150 between ST and FSF. Regards. -- Yvan ROUX STMicroelectronics
[4.6 branch] Issue with C++ references.
Hi,
I have the small program below, which check the binding of a reference :
int main() {
const int Ci = 0;
const int &rCi = Ci;
if (!(&Ci == &rCi)) return 1;
return 0;
}
If my understanding of the standard (8.5.3) is correct, the reference is
an lvalue (rCi), the initializer expression (Ci) is an lvalue and the two
referenced types are the same => rCi is bound to Ci, my program is a
valid one. If this is not the case, the rest of this mail could be skip.
The program has the expected behaviour if you use a 4.5 gcc or the trunk, but
if you compile it with a 4.6 one (even with the branch's head) with -O0,
the program returns the 1 value, because rCi is not bound to Ci, but to an
anonymous temporary which cantains the value 0.
The problem was introduced during the integration of the constexpr support,
in the cp_parser_initializer_clause function of the parser.c file, with the
commit :
Git : ce984e5e401accb43a1c4bfee31a4743f4004118
Svn : trunk@166167
and was fixed on the trunk by the commit :
Git : 5e260adbb1f177a25a39691d3e4510d6bc3b898a
Svn : trunk@175284
In fact only the part of the commit which removes the call to
maybe_constant_value in cp_parser_initializer_clause fixes the bug.
I'm not really sure of what I have to do in such a case :
peparing the patch or asking to backport Jason's on the 4.6 branch ?
Many Thanks
Yvan
Re: [4.6 branch] Issue with C++ references.
On Wed, Oct 19, 2011 at 06:08:15AM +0200, Jason Merrill wrote: > I'll backport that hunk, thanks for catching this. Great ! You're welcome Jason. Yvan
