Re: [cfe-dev] ISO C3X proposal: nonnull qualifier

[Dmitri Gribenko via Gcc]

Hi Alejandro,

On Tue, Nov 16, 2021 at 1:34 PM Alejandro Colomar (man-pages) via
cfe-dev  wrote:
> First of all,
> I see unnecessary (probably over-engineered) qualifiers:
>
> - _Null_unspecified seems to me the same as nothing.
> If I didn't specify its nullability,
> it's by definition unspecified.  Right?
>
> - _Nullable seems to me also the same as nothing.
> The language allows for a pointer to be NULL,
> so if you don't specify if it can or not be null,
> you better stay on the safe side and consider it as nullable.

_Nullable is used in conjunction with the `#pragma clang
assume_nonnull begin/end` pragma that flips the default:

```
#pragma clang assume_nonnull begin
int *global_int_ptr; // implicitly _Nonnull
#pragma clang assume_nonnull end
```

Within these pragma brackets, you need to use _Nullable to get the
opposite behavior.

The pragma itself is useful because it reduces the amount of noise the
annotations introduce. When these annotations were adopted in Apple
SDKs, it was found that in practice most pointers are non-nullable. So
if we only had _Nonnull, we would have to annotate most pointers.
Instead, Apple's SDKs bracket every header contents with this pragma,
and instead annotate nullable pointers, significantly reducing the
amount of annotations.

_Null_unspecified is a way to say that nullability is complicated due
to legacy reasons (for example, a function may return NULL under
extremely rare circumstances that most users don't care about, so we
want to allow returning NULL, while suppressing warnings at the usage
site if the user assumes that the returned pointer is non-NULL). It
might have been useful for bringing certain legacy APIs into the
annotated world with more checks enabled. But right now it is used
extremely rarely in Apple's SDKs and therefore it is unclear to me
personally whether it really pulls its weight at this point.

Dmitri

-- 
main(i,j){for(i=2;;i++){for(j=2;j*/

Re: [cfe-dev] ISO C3X proposal: nonnull qualifier

[Dmitri Gribenko via Gcc]

Hi Alejandro,

On Wed, Nov 17, 2021 at 1:06 AM Alejandro Colomar (man-pages) via
cfe-dev  wrote:
> On 11/16/21 13:34, Alejandro Colomar (man-pages) wrote:
> > $ cat _Nonnull.c
> > #include 
> >
> > int *_Nonnull f(int *_Nullable p)
> > {
> >  if (!p)
> >  exit(1);
> >  return p;
> > }
> >
> >
> > - I get a warning from f().
> >Ideally,
> >a programmer should not need to cast
> >(casts are dangerous),
> >to convert a nullable pointer to a _Nonnull pointer.
> >For that,
> >appropriate checks should be in the preceeding code.
> >Otherwise, a diagnostic should be issued.
> >To be on the safe side,
> >if a compiler has doubts,
> >it should diagnose.
> >
> >There's some Clang document that talks about something similar.
> >I don't know its validity,
> >or if it was a draft before _Nonnull qualifiers.
> >
>
> That document suggests that I shouldn't get a diagnostic from f().
> Why did I get a diagnostic?  (I tried clang 11, 13 & 14(experimental))
>
>
> Is it talking about a different nonnull attribute/qualifier?
> Was it about a proposal prior to the current _Nonnull?
> Why is it not in use?  Was it too difficult to implement?

The false positive you're getting is from the Clang warning
-Wnullable-to-nonnull-conversion. It is a simple type-based check that
does not take dataflow information into account. In other words, the
reason for the nullability false positive in your example is identical
to the reason for the integer conversion false positive here:

```
#include 
#include 
#include 

uint8_t f(uint32_t x) {
  if (x > UINT8_MAX)
  exit(1);
 return x;
}
```

warning: implicit conversion loses integer precision: 'uint32_t' (aka
'unsigned int') to 'uint8_t' (aka 'unsigned char')
[-Wimplicit-int-conversion]

This webpage 
https://clang.llvm.org/docs/analyzer/developer-docs/nullability.html
describes Clang Static Analyzer, is a sophisticated path-sensitive
static analysis tool. It is unfortunately often too slow to enable in
regular compilation.

> Do you think Clang could be improved to not warn on f()?

Absolutely. We can implement a dataflow-based check that takes the
flow condition into account. Flow-sensitive diagnostics should scale a
lot better than path-sensitive, and they should be fast enough to
become a compiler warning. We are currently upstreaming a dataflow
analysis framework that should make implementing such diagnostics
easier. https://lists.llvm.org/pipermail/cfe-dev/2021-October/069098.html

Dmitri

-- 
main(i,j){for(i=2;;i++){for(j=2;j*/

Re: [cfe-dev] ISO C3X proposal: nonnull qualifier

[Alejandro Colomar (man-pages) via Gcc]

Hi Dmitry,

On 11/23/21 12:17, Dmitri Gribenko wrote:

Hi Alejandro,

On Tue, Nov 16, 2021 at 1:34 PM Alejandro Colomar (man-pages) via
cfe-dev  wrote:

First of all,
I see unnecessary (probably over-engineered) qualifiers:

- _Null_unspecified seems to me the same as nothing.
If I didn't specify its nullability,
it's by definition unspecified.  Right?

- _Nullable seems to me also the same as nothing.
The language allows for a pointer to be NULL,
so if you don't specify if it can or not be null,
you better stay on the safe side and consider it as nullable.


_Nullable is used in conjunction with the `#pragma clang
assume_nonnull begin/end` pragma that flips the default:

```
#pragma clang assume_nonnull begin
int *global_int_ptr; // implicitly _Nonnull
#pragma clang assume_nonnull end
```

Within these pragma brackets, you need to use _Nullable to get the
opposite behavior.

The pragma itself is useful because it reduces the amount of noise the
annotations introduce. When these annotations were adopted in Apple
SDKs, it was found that in practice most pointers are non-nullable. So
if we only had _Nonnull, we would have to annotate most pointers.
Instead, Apple's SDKs bracket every header contents with this pragma,
and instead annotate nullable pointers, significantly reducing the
amount of annotations.


That's interesting.  Most of my functions also tipically are full of 
[[gnu::nonnull]], so the _Nonnull default seems the best thing.


However, would that be viable in old code that relies on standard C?
I think that it would, but maybe you have more experience.  Do you agree 
with the following?


Let's imagine a scenario where C3X specifies that non-qualified pointers 
are nonnull.  And there's only a qualifier, _Nullable, to allow NULL. 
Asigning _Nullable to nonnull would issue a diagnostic.


Old code will stop compiling if it uses NULL, but that can easily be 
fixed by marking the pointers as _Nullable, and maybe while at that, 
programmers will find a few bugs.


Compilers will have to be carefull, because memcpy() will make NULL 
members of structures, so they'll need to know if that can be done or 
not, and many structure members will need to be marked as _Nullable, if 
the structure is expected to be bzero()ed.


Also, do you have any experience in avoiding to diagnose a _Nullable to 
nonnull assignment _after_ explicitly comparing to NULL?  I.e., allow 
the following:


int *_Nullable p;
int *q;

if (!p)
q = p;

Thanks,
Alex

Re: How to describe ‘earlyclobber’ explicitly for specific source operand ?

[Richard Earnshaw via Gcc]

On 22/11/2021 06:40, Jojo R via Gcc wrote:


— Jojo
在 2021年11月20日 +0800 AM6:11，Peter Bergner ，写道：

On 11/19/21 1:28 AM, Jojo R via Gcc wrote:

We know gcc supply earlyclobber function to avoid register overlap,

but it can not describe explicitly for specific source operand, is it right ?


You add the early clobber to the OUTPUT operand(s) that can clobber any of the
input source operands. You don't mark the source operands that could be 
clobbered.

Yes, so we need to enhance the early clobber to cover this scene ?


Peter


You can write alternatives that explicitly tie a source to the 
destination, provided that the source and destination are the same size. 
 See the Arm backend for examples.


R.

Re: [cfe-dev] ISO C3X proposal: nonnull qualifier

[Dmitri Gribenko via Gcc]

Hi Alejandro,

On Tue, Nov 23, 2021 at 12:45 PM Alejandro Colomar (man-pages)
 wrote:
>
> Hi Dmitry,
>
> On 11/23/21 12:17, Dmitri Gribenko wrote:
> > Hi Alejandro,
> >
> > On Tue, Nov 16, 2021 at 1:34 PM Alejandro Colomar (man-pages) via
> > cfe-dev  wrote:
> >> First of all,
> >> I see unnecessary (probably over-engineered) qualifiers:
> >>
> >> - _Null_unspecified seems to me the same as nothing.
> >> If I didn't specify its nullability,
> >> it's by definition unspecified.  Right?
> >>
> >> - _Nullable seems to me also the same as nothing.
> >> The language allows for a pointer to be NULL,
> >> so if you don't specify if it can or not be null,
> >> you better stay on the safe side and consider it as nullable.
> >
> > _Nullable is used in conjunction with the `#pragma clang
> > assume_nonnull begin/end` pragma that flips the default:
> >
> > ```
> > #pragma clang assume_nonnull begin
> > int *global_int_ptr; // implicitly _Nonnull
> > #pragma clang assume_nonnull end
> > ```
> >
> > Within these pragma brackets, you need to use _Nullable to get the
> > opposite behavior.
> >
> > The pragma itself is useful because it reduces the amount of noise the
> > annotations introduce. When these annotations were adopted in Apple
> > SDKs, it was found that in practice most pointers are non-nullable. So
> > if we only had _Nonnull, we would have to annotate most pointers.
> > Instead, Apple's SDKs bracket every header contents with this pragma,
> > and instead annotate nullable pointers, significantly reducing the
> > amount of annotations.
>
> That's interesting.  Most of my functions also tipically are full of
> [[gnu::nonnull]], so the _Nonnull default seems the best thing.
>
> However, would that be viable in old code that relies on standard C?
> I think that it would, but maybe you have more experience.  Do you agree
> with the following?
>
> Let's imagine a scenario where C3X specifies that non-qualified pointers
> are nonnull.  And there's only a qualifier, _Nullable, to allow NULL.
> Asigning _Nullable to nonnull would issue a diagnostic.

I think C3X specifying that non-qualified pointers are nonnnull would
be a showstopper, I don't think it is likely to happen given how the
users and the committee value backward compatibility that C has
offered throughout the decades.

If I were to speculate what would happen if C3X did flip the default,
I think it would be treated by the community as a language fork.
Pre-C3X headers won't work correctly when included in C3X programs,
making incremental adoption of C3X syntax, as it was intended to be
used, impossible. Projects would likely invent a NULLABLE macro, which
would expand to _Nullable in C3X and nothing in earlier versions, to
enable an incremental transition.

That's why Clang introduced the pragma, enabling new rules to be
adopted incrementally.

> Also, do you have any experience in avoiding to diagnose a _Nullable to
> nonnull assignment _after_ explicitly comparing to NULL?  I.e., allow
> the following:
>
> int *_Nullable p;
> int *q;
>
> if (!p)
> q = p;

Internally at Google we have a checker based on the dataflow analysis
framework (https://lists.llvm.org/pipermail/cfe-dev/2021-October/069098.html)
that diagnoses usages of std::optional::value() not guarded by
has_value(). We are planning to upstream it after we finish
upstreaming the dataflow framework itself. Ensuring guarded usage of
std::optional::value() is very similar to diagnosing dereferences
of nullable pointers. I think a lot of the experience is transferable.
However, we haven't attempted to implement a pointer nullability check
yet, so I don't yet understand all corner cases that arise in real
world software.

However, fundamentally, there are a few questions that you need to answer:

- does _Nullable create a distinct type or not? It is extremely
important when you consider C++.

- do you want nullability-related diagnostics to be mandatory, or
optional? For example, a compiler is not required to issue a
diagnostic about a program that violates the constraints of
`restrict`.

If _Nullable does not create a distinct type, and `T*` is the same
type as `T* _Nullable`, then we can't rely on the regular type system
mechanisms to issue diagnostics.

If _Nullable creates a distinct type, then according to regular C type
checking rules, you would get a warning on the `q = p` assignment
regardless of the `if (!p)` check. That's how the C type system works,
it is not flow-sensitive.

If we want the diagnostics to be fllow-sensitive like in your example
(I think it would be the best choice), then we need to add a new
flow-sensitive component to the C type system. I don't think there is
a precedent for this in C right now. I'm not sure how the committee or
implementors would react to such a proposal.

Dmitri

-- 
main(i,j){for(i=2;;i++){for(j=2;j*/

Re: How to get started with contribution

[Jonathan Wakely via Gcc]

On Tue, 23 Nov 2021 at 13:22, Om Kenge via Gcc  wrote:
>
>
> Respected Sir/Madam,
> I am Om Kenge, a Second Year IT Student, I have just entered my second year 
> at MMCOE Pune. I am new to open source contributions but I am well aware of 
> C++. I would love to contribute to your Organisation

Thanks for your interest in GCC!

A good place to start is by reading the project's mailing list (the
one you've contacted!) and if you did that, you would find this
question has already been asked and answered several times this month.
You could start by reading those previous threads, so we don't need to
keep repeating ourselves.

See these threads (and others):

https://gcc.gnu.org/pipermail/gcc/2021-November/237719.html
https://gcc.gnu.org/pipermail/gcc/2021-November/237745.html
https://gcc.gnu.org/pipermail/gcc/2021-November/237754.html
https://gcc.gnu.org/pipermail/gcc/2021-November/237771.html
https://gcc.gnu.org/pipermail/gcc/2021-November/237773.html

Re: New ThreadSanitizer runtime (v3)

[Florian Weimer via Gcc]

* Dmitry Vyukov via Gcc:

> I wanted to give heads up regarding a significant re-design of the
> ThreadSanitizer runtime:
> https://reviews.llvm.org/D112603
> Currently it's submitted:
> https://github.com/llvm/llvm-project/commit/1784fe0532a69ead17793bced060a9bf9d232027
> but can well be rolled back if too many buildbots fail, but should be
> submitted again soon anyway.
>
> It was extensively tested and lots of bugs were fixed, but it's still
> possible it will cause some issues just because of the size of the
> change and OS/arch sensitivity.

Have there been changes to the glibc integration?  Does it still
hard-code the size of (private) struct pthread?

Thanks,
Florian

Re: New ThreadSanitizer runtime (v3)

[Dmitry Vyukov via Gcc]

On Tue, 23 Nov 2021 at 14:49, Florian Weimer  wrote:
>
> * Dmitry Vyukov via Gcc:
>
> > I wanted to give heads up regarding a significant re-design of the
> > ThreadSanitizer runtime:
> > https://reviews.llvm.org/D112603
> > Currently it's submitted:
> > https://github.com/llvm/llvm-project/commit/1784fe0532a69ead17793bced060a9bf9d232027
> > but can well be rolled back if too many buildbots fail, but should be
> > submitted again soon anyway.
> >
> > It was extensively tested and lots of bugs were fixed, but it's still
> > possible it will cause some issues just because of the size of the
> > change and OS/arch sensitivity.
>
> Have there been changes to the glibc integration?

Hi Florian,

No changes to direct glibc integration.
Or what kind of integration do you mean? Tsan did not have any direct
integration and worked with unmodified glibc.

> Does it still hard-code the size of (private) struct pthread?

I am not aware of any such hard-code.

Re: New ThreadSanitizer runtime (v3)

[Florian Weimer via Gcc]

* Dmitry Vyukov:

> Or what kind of integration do you mean? Tsan did not have any direct
> integration and worked with unmodified glibc.

I thought there is a false-positive data race report if an initial-exec
or local-exec TLS variable is reused (whose memory is not managed by
malloc).

Thanks,
Florian

Re: New ThreadSanitizer runtime (v3)

[Dmitry Vyukov via Gcc]

On Tue, 23 Nov 2021 at 14:59, Florian Weimer  wrote:
>
> * Dmitry Vyukov:
>
> > Or what kind of integration do you mean? Tsan did not have any direct
> > integration and worked with unmodified glibc.
>
> I thought there is a false-positive data race report if an initial-exec
> or local-exec TLS variable is reused (whose memory is not managed by
> malloc).

I don't remember any such open issues.

Re: New ThreadSanitizer runtime (v3)

[Florian Weimer via Gcc]

* Dmitry Vyukov:

> On Tue, 23 Nov 2021 at 14:59, Florian Weimer  wrote:
>>
>> * Dmitry Vyukov:
>>
>> > Or what kind of integration do you mean? Tsan did not have any direct
>> > integration and worked with unmodified glibc.
>>
>> I thought there is a false-positive data race report if an initial-exec
>> or local-exec TLS variable is reused (whose memory is not managed by
>> malloc).
>
> I don't remember any such open issues.

This glibc patch submission mentions tsan:

  [PATCH] elf: Add __libc_get_static_tls_bounds [BZ #16291]
  

It's currently stalled (adding an interface for something that may stop
working in the future is problematic).

Thanks,
Florian

Re: New ThreadSanitizer runtime (v3)

[Dmitry Vyukov via Gcc]

On Tue, 23 Nov 2021 at 17:16, Florian Weimer  wrote:
>
> * Dmitry Vyukov:
>
> > On Tue, 23 Nov 2021 at 14:59, Florian Weimer  wrote:
> >>
> >> * Dmitry Vyukov:
> >>
> >> > Or what kind of integration do you mean? Tsan did not have any direct
> >> > integration and worked with unmodified glibc.
> >>
> >> I thought there is a false-positive data race report if an initial-exec
> >> or local-exec TLS variable is reused (whose memory is not managed by
> >> malloc).
> >
> > I don't remember any such open issues.
>
> This glibc patch submission mentions tsan:
>
>   [PATCH] elf: Add __libc_get_static_tls_bounds [BZ #16291]
>   
>
> It's currently stalled (adding an interface for something that may stop
> working in the future is problematic).

I see. Nothing has changed with respect to
__libc_get_static_tls_bounds with my patches.

GNU OMPD implementation

[Mohamed Atef via Gcc]

Hello everyone,
I need to remind you that we are working on implementation of OMPD, so
you don't make it open for GSoC this year.

our progress so far,
We are working on a GDB extension using python so we can provide OMPD with
callbacks.
Jakub said that we need GDB support, but the GDB community didn't reply to
us although we mailed them more than one time, so is it okay to build the
plugin and extend GDB with python for testing purposes ?

to enable OMPD the runtime must provide some routines like:
ompd_dll_locations_valid(void)
void ompd_bp_parallel_begin(void)
void ompd_bp_parallel_end(void)
void ompd_bp_task_begin(void)
void ompd_bp_task_end(void)
and define the variable const char **ompd_dll_locations.
so far so good , BUT OMPD can not access debug information at the runtime,
so it needs to access them. We will write some macros for that.
the macros will generate the sizes and offsets for the following structs:
gomp_team
gomp_task_icv

Should we generate all structs sizes?

Are we on the right path?

I wish to hear from you soon.

Thanks,
Mohamed