Re: [cfe-users] Clang9 UBSan and GMP
> On 31 Oct 2019, at 18:40, David Blaikie wrote: > >> Right, but that is something one would avoid when computing arithmetical >> results. > > One would try to, yes - but that's sort of what the whole discussion is > resolving around: Is the code correct? I don't know. I wouldn't assume it is > (I'm not assuming it isn't either) - but without a reduced test case that > gets to the root of the difference in behavior, we don't know if the code is > correct. Nor whether it is a compiler bug. ___ cfe-users mailing list cfe-users@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-users
Re: [cfe-users] Clang9 UBSan and GMP
> On 31 Oct 2019, at 21:40, David Blaikie wrote: > >> On Thu, Oct 31, 2019 at 12:00 PM Hans Åberg wrote: >> >> > On 31 Oct 2019, at 18:40, David Blaikie wrote: >> > >> >> Right, but that is something one would avoid when computing arithmetical >> >> results. >> > >> > One would try to, yes - but that's sort of what the whole discussion is >> > resolving around: Is the code correct? I don't know. I wouldn't assume it >> > is (I'm not assuming it isn't either) - but without a reduced test case >> > that gets to the root of the difference in behavior, we don't know if the >> > code is correct. >> >> Nor whether it is a compiler bug. > > Indeed - but you can imagine that, on average (just due to there being way > more code compiled by the compiler, than the code of the compiler itself) the > bug is in external code, not the compiler. GMP is not the average program, though. > Such that it's not practical for the compiler developers to do all the leg > work of investigating 3rd party code bugs to determine if it's a bug in the > compiler. It doesn't scale/we wouldn't have any time to work on the compiler > & most of the time we'd be finding user bugs, not compiler bugs. The GMP developers feel exactly the same, dropping Clang support. It is mostly a problem for MacOS users that do not have access to GCC. > Apologies for the snark in the title of this article, but it covers some of > the ideas: > https://blog.codinghorror.com/the-first-rule-of-programming-its-always-your-fault/ > & other articles around discuss similar ideas. This article is pretty naive: Yes, it is a good starting point to check ones own code first, but eventually one learns to identify compiler bugs as well. It is very time consuming, though. > Yes, there are compiler bugs - but you've sort of got to continue under the > assumption that that's not the issue until you've got some fairly compelling > evidence of one (very narrow test case where you can look at all the code & > visually inspect/discuss/reason about its standards conformance - currently > "all of GMP" is too big to apply that level of scrutiny). GMP is indeed very complex, not only from a programming point of view, but also the underlying algorithms. ___ cfe-users mailing list cfe-users@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-users
Re: [cfe-users] Clang9 UBSan and GMP
Indeed, very hard to figure out. If it is some hidden undefined behavior causing it, the UBSan should have caught it, but it does not. The link that Matthew gave says that the GMP developers experienced a number of such issues with Clang. One can though turn off the optimizer, and the tests pass. > On 30 Oct 2019, at 22:17, David Blaikie wrote: > > I ran the test & understand it a bit better now - so the abort is part of the > code, when the test fails, the test harness uses abort to fail. > > So this isn't "clang causes abort" (it didn't select a bad instruction, etc) > this is "clang causes test failure" - this could be any number of things in > terms of compiler optimizations due to overly dependent code (or due to > miscompiles, to be sure). It's possible the test relies on specific numeric > results that the C++ programming language doesn't guarantee (either overly > high precision, or overly low precision - C++ allows extra precision in > computations which can break numerical code that's relying on certain > precision, for instance). > > So, yeah, really hard to say where the fault lies without further > investigation. > > On Fri, Oct 25, 2019 at 4:13 PM Hans Åberg wrote: > The GMP developers felt it was a compiler bug, so I think I will leave it at > that. But thanks for the tips. > > > > On 26 Oct 2019, at 00:32, David Blaikie wrote: > > > > UBSan doesn't catch everything - you could also try ASan and/or valgrind, > > etc. (MSan if you want, but that's a bit fussier/more work to use) > ___ cfe-users mailing list cfe-users@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-users
Re: [cfe-users] Clang9 UBSan and GMP
> On 31 Oct 2019, at 01:53, David Blaikie wrote:
>
>> Yes, but assuming that the GMP adheres to the C standard, there should be no
>> difference in the arithmetical values produced.
>
> Not necessarily - C (well, I don't know the C standard as well as I know the
> C++ standard, admittedly) does allow various variations (implementation and
> unspecified behavior, for instance). eg: order of evaluation of function
> arguments (not that this is likely to vary due to optimizations - and doesn't
> with clang to the best of my knowledge, but as an example of the /sort/ of
> thing):
>
> int f() {
> static int i = 0;
> return i++;
> }
> int f2(int i, int j) {
> return i;
> }
> int main() {
> return f2(f(), f());
> }
>
> This program could exit with 0 or 1 - both results are valid interpretations
> of the program per the standard. (again, I don't know the C spec quite as
> well, so I'm not sure exactly what it says about this code)
Right, but that is something one would avoid when computing arithmetical
results.
___
cfe-users mailing list
cfe-users@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-users
Re: [cfe-users] Clang9 UBSan and GMP
> On 31 Oct 2019, at 00:28, David Blaikie wrote: > > > > On Wed, Oct 30, 2019 at 4:25 PM Hans Åberg wrote: > >> I believe that GMP is just using integer types, and then uses that to make >> multiprecision integers, rational numbers, and floating point numbers. At >> least MPFR uses only the integer and rational number part of GMP, and builds >> multiprecision floating point numbers on top of that, which is necessary >> because of special requirements of the standards they adhere to. > > Ah, fair enough - that narrows down the points of failure a little. In addition, they use assembly code, but that can be turned off with configure --disable-assembly, though I did not 'make check' with that. >> GMP has been used in three years in a sequenced operation that must be exact >> and without errors to solve the problem [1], and I would think it used GCC >> with optimizations. So that puts Clang in a tough spot. :-) > > Not as much as it would seem - again, the spec allows for a fair bit of > flexibility in a bunch of ways. (admittedly, within integer arithmetic > without invoking UB (but, again, that's not proven - UBSan isn't guaranteed > to catch everything)) Different compilers optimize code in different ways - > that the code "works"/produces the desired behavior on one compiler/under > some optimizations and not others doesn't give us much idea about whether the > code or the compiler is correct. Different behavior is acceptable in C++ in a > bunch of ways & compilers rely on that flexibility. Yes, but assuming that the GMP adheres to the C standard, there should be no difference in the arithmetical values produced. ___ cfe-users mailing list cfe-users@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-users
Re: [cfe-users] Clang9 UBSan and GMP
> On 30 Oct 2019, at 23:50, David Blaikie wrote: > >> On Wed, Oct 30, 2019 at 2:29 PM Hans Åberg wrote: >> Indeed, very hard to figure out. If it is some hidden undefined behavior >> causing it, the UBSan should have caught it, but it does not. >> > Right - but especially with numerics (especially floating point) there's > loads of room for valid but different behavior between different compilers - > behavior that isn't UB. How much precision a certain mathematical equation > maintains is really at the whim of the optimizers in a lot of ways. I believe that GMP is just using integer types, and then uses that to make multiprecision integers, rational numbers, and floating point numbers. At least MPFR uses only the integer and rational number part of GMP, and builds multiprecision floating point numbers on top of that, which is necessary because of special requirements of the standards they adhere to. >> The link that Matthew gave says that the GMP developers experienced a number >> of such issues with Clang. One can though turn off the optimizer, and the >> tests pass. > > Sure - most of the numeric effects would only appear with optimizations. > Without them every numeric operation's just done in registers, then written > right back to memory (so no chance of excess precision leaking in by storing > the value in an 80bit floating point register between multiple operations, or > any risk of fused operations that produces extra precision, etc). > > The only way to know is to trace down/reduce the point where the values > diverge & stare at the code to see who's right. GMP has been used in three years in a sequenced operation that must be exact and without errors to solve the problem [1], and I would think it used GCC with optimizations. So that puts Clang in a tough spot. :-) 1. https://gmplib.org/list-archives/gmp-discuss/2019-April/006319.html ___ cfe-users mailing list cfe-users@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-users
Re: [cfe-users] Clang9 UBSan and GMP
> On 31 Oct 2019, at 22:07, David Blaikie wrote: > >> On Thu, Oct 31, 2019 at 1:51 PM Hans Åberg wrote: >> >> > On 31 Oct 2019, at 21:40, David Blaikie wrote: >> > >> >> On Thu, Oct 31, 2019 at 12:00 PM Hans Åberg wrote: >> >> >> >> > On 31 Oct 2019, at 18:40, David Blaikie wrote: >> >> > >> >> >> Right, but that is something one would avoid when computing >> >> >> arithmetical results. >> >> > >> >> > One would try to, yes - but that's sort of what the whole discussion is >> >> > resolving around: Is the code correct? I don't know. I wouldn't assume >> >> > it is (I'm not assuming it isn't either) - but without a reduced test >> >> > case that gets to the root of the difference in behavior, we don't know >> >> > if the code is correct. >> >> >> >> Nor whether it is a compiler bug. >> > >> > Indeed - but you can imagine that, on average (just due to there being way >> > more code compiled by the compiler, than the code of the compiler itself) >> > the bug is in external code, not the compiler. >> >> GMP is not the average program, though. >> >> > Such that it's not practical for the compiler developers to do all the leg >> > work of investigating 3rd party code bugs to determine if it's a bug in >> > the compiler. It doesn't scale/we wouldn't have any time to work on the >> > compiler & most of the time we'd be finding user bugs, not compiler bugs. >> >> The GMP developers feel exactly the same, dropping Clang support. It is >> mostly a problem for MacOS users that do not have access to GCC. > > Yep, that's certainly their call - there's a cost to maintaining > compatibility with each compiler/toolchain/platform, etc. Yes, it involves hard study of the various CPUs used. > If you have a personal interest in GMP on MacOS, then perhaps the cost falls > to you, if you're willing to pay it, to investigate this sort of thing & help > support this particular library+compiler combination, if it's worth your time > to do so. Both GCC and Clang can be conveniently installed using MacPorts. The Apple inhouse clang is weird. >> > Apologies for the snark in the title of this article, but it covers some >> > of the ideas: >> > https://blog.codinghorror.com/the-first-rule-of-programming-its-always-your-fault/ >> > & other articles around discuss similar ideas. >> >> This article is pretty naive: Yes, it is a good starting point to check ones >> own code first, but eventually one learns to identify compiler bugs as well. >> It is very time consuming, though. > > Certainly - which is why it's not practical for compiler engineers to be > spending all that time on everyone's bugs, right? It depends, for example, I recently found bug in the GCC C++ regex library, and in the process found an algorithm do compute matches in linear time via some reverse NFA lookups, eliminating the need for backtracking. Regardless how they resolve on GCC, I may have use of it myself. >> > Yes, there are compiler bugs - but you've sort of got to continue under >> > the assumption that that's not the issue until you've got some fairly >> > compelling evidence of one (very narrow test case where you can look at >> > all the code & visually inspect/discuss/reason about its standards >> > conformance - currently "all of GMP" is too big to apply that level of >> > scrutiny). >> >> GMP is indeed very complex, not only from a programming point of view, but >> also the underlying algorithms. > > Yep - which makes it all the harder for me or someone else on the LLVM > project to likely be able to find any potential compiler bugs in it. I just noticed the issue, and can not give any good advice on how to attack it. ___ cfe-users mailing list cfe-users@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-users
[cfe-users] clang-format single line if
If I build clang-format from tip of master and run it
using this .clang-format file
AllowShortIfStatementsOnASingleLine: Always
it will change this single line if statement
if (x) { x++; }
into this three line if statement
if (x) {
x++;
}
How do I keep it as one line?
If I delete the braces it stays on one line, but the
style guide for the code I work on says the braces are
required. If I modify my .clang-format file to be
AllowShortIfStatementsOnASingleLine: Always
AllowShortBlocksOnASingleLine: Always
it stays on one line, but I don't want all short blocks
on a single line, just the if statement block. Any ideas?
___
cfe-users mailing list
cfe-users@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-users
Re: [cfe-users] Clang9 UBSan and GMP
> On Oct 31, 2019, at 14:30, Hans Åberg via cfe-users > wrote: > >> >> On 31 Oct 2019, at 22:07, David Blaikie wrote: >> >>> On Thu, Oct 31, 2019 at 1:51 PM Hans Åberg wrote: >>> On 31 Oct 2019, at 21:40, David Blaikie wrote: Such that it's not practical for the compiler developers to do all the leg work of investigating 3rd party code bugs to determine if it's a bug in the compiler. It doesn't scale/we wouldn't have any time to work on the compiler & most of the time we'd be finding user bugs, not compiler bugs. >>> >>> The GMP developers feel exactly the same, dropping Clang support. It is >>> mostly a problem for MacOS users that do not have access to GCC. >> >> Yep, that's certainly their call - there's a cost to maintaining >> compatibility with each compiler/toolchain/platform, etc. > > Yes, it involves hard study of the various CPUs used. > >> If you have a personal interest in GMP on MacOS, then perhaps the cost falls >> to you, if you're willing to pay it, to investigate this sort of thing & >> help support this particular library+compiler combination, if it's worth >> your time to do so. > > Both GCC and Clang can be conveniently installed using MacPorts. The Apple > inhouse clang is weird. I haven’t followed the rest of this thread closely, but do you have a reference for the GMP developers abandoning Clang on macOS? Or were you referring to their comment about Clang on the page I linked? Personally I regularly use GMP with a macOS-supplied Clang without any issues. Admittedly not for any extreme numerical computation, but my experience is that GMP works fine in this scenario.___ cfe-users mailing list cfe-users@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-users
