Re: [Mesa-dev] [EXTERNAL] Cross-compile from Linux to Windows

2020-11-10 Thread Federico Dossena

Thank you for your reply.

That's where I got stuck on Windows, I can't get it to find LLVM.

I built LLVM without problems for both x86 and x86_64 in 2 separate 
folders, but when I try to build mesa, even with --cmake-prefix-path, it 
doesn't find it, it always say "Neither a subproject directory nor a 
llvm.wrap file was found".


I tried moving my LLVM files in the subprojects/llvm folder and that 
didn't fix it, I don't know what to do.


Any suggestion?

On 2020-11-09 16:06, Jesse Natalie wrote:

Have you tried using Meson on Windows, instead of cross-compiling? If you run 
it from a Visual Studio command prompt, it should just work out of the box, at 
least for x86. I believe you'll need to use a cross file for amd64, but the 
only thing you'd need to specify is the host architecture.

Though I guess if you're building LLVMPipe you'll need to point it to LLVM. In my 
experience, adding "--cmake-prefix-path" to the Meson command line should work 
well.

-Jesse

-Original Message-
From: mesa-dev  On Behalf Of Federico 
Dossena
Sent: Saturday, November 7, 2020 4:14 AM
To: mesa-dev@lists.freedesktop.org
Subject: [EXTERNAL] [Mesa-dev] Cross-compile from Linux to Windows

I'm sorry to bother you in the dev list but it seems to be the only one with 
activity so it seems like the best place to ask.

I maintain some Mesa builds for Windows. These are builds of libgl-gdi with 
llvmpipe, for both x86 and x86_64; in other words, they're opengl32.dll files 
that users can drop into their game/application to have software rendering for 
OpenGL as a workaround for broken drivers or as a fallback for older systems.
I've always used scons on Windows and msvc as a compiler to make these builds, 
but now that scons is being deprecated I've been trying to use meson and ninja 
to cross-compile from Linux (amd64) to Windows (both x86 and x86_64) without 
much success. I can get regular Linux builds without issues though.

I tried reading the Mesa documentation on cross-compiling but it's very minimal 
and it's partially obsolete so I thought it would be best to ask you directly: 
how do I make these builds? Explain like I'm 5, basically.

Thanks,

Federico

___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fmesa-dev&data=04%7C01%7Cjenatali%40microsoft.com%7C7ed90f213a9a4204b25108d8848e0ef1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637405092966689938%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=A%2BvRfw0Y4C4NufTtNWOwo4COLOOJAw%2F3H9rttkk5jA4%3D&reserved=0

___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev


[Mesa-dev] Static thread safety checking with clang

2020-11-10 Thread Kristian Høgsberg
Hi,

I wanted to call attention to

  https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7529

which shows how we can use a new clang __attribute__ to statically
check locking invariants. It's probably not perfect, but more like
static type checking - there will always be cases where you can't
statically determine if the code is right or wrong, but it will be
useful and correct in most cases (like in the MR).  The attributes
have a few quirks and I suspect that they were mostly designed and
tested to work for C++ classes, but I found a way to make it work for
C.

For each type of lock, declare a lock capability:

  extern lock_cap_t fd_screen_lock_cap;

Functions that take and release that lock get annotated with:

  static inline void
  fd_screen_lock(struct fd_screen *screen)
acquire_cap(fd_screen_lock_cap);

  static inline void
  fd_screen_unlock(struct fd_screen *screen)
release_cap(fd_screen_lock_cap)

where acquire_cap and release_cap are #defines for an __attribute__.
One of the quirks is that the function doing the locking triggers a
warning about how it doesn't actually take the lock, so I silenced
that by adding a no_thread_safety_analysis in there.

Now whenever we have a function that expects to be called with that
lock held (we often call them foo_locked) we can add the requires_cap
annotation:

  static struct gmem_key *
  gmem_key_init(struct fd_batch *batch, bool assume_zs, bool no_scis_opt)
requires_cap(fd_screen_lock_cap)

and calls to this function will warn or error if called from a point
where it's not statically determinable that you've taken the lock:

../../master/src/gallium/drivers/freedreno/freedreno_gmem.c:532:25:
error: calling function 'gmem_key_init' requires holding lock
'fd_screen_lock_cap' exclusively [-Werror,-Wthread-safety-analysis]
struct gmem_key *key = gmem_key_init(batch, assume_zs, no_scis_opt);
   ^
1 error generated.

Many functions that assert a lock is taken are better off using the
requires_cap annotation, which makes the check a compile failure
instead. For cases where it's not possible to determine statically, we
can still assert at runtime:

  static inline void
  fd_screen_assert_locked(struct fd_screen *screen)
assert_cap(fd_screen_lock_cap)

which tells the checker to assume the lock is taken.  Finally, it's
possible to annotate struct members:

  struct fd_gmem_cache gmem_cache guarded_by(fd_screen_lock_cap);

such that any access to that field can only happen with the lock taken:

../../master/src/gallium/drivers/freedreno/freedreno_gmem.c:277:20:
error: reading variable 'gmem_cache' requires holding lock
'fd_screen_lock_cap' [-Werror,-Wthread-safety-analysis]
rzalloc(screen->gmem_cache.ht, struct fd_gmem_stateobj);
^
Having these annotations also helps human readers of the code by
spelling out the conventions explicitly instead of relying on _locked
suffix conventions or documentation that call out which parts of a
struct are protected by which lock. All in all this seems really
useful.

Kristian
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev


Re: [Mesa-dev] Static thread safety checking with clang

2020-11-10 Thread Chia-I Wu
On Tue, Nov 10, 2020 at 8:09 AM Kristian Høgsberg  wrote:
>
> Hi,
>
> I wanted to call attention to
>
>   https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7529
>
> which shows how we can use a new clang __attribute__ to statically
> check locking invariants. It's probably not perfect, but more like
> static type checking - there will always be cases where you can't
> statically determine if the code is right or wrong, but it will be
> useful and correct in most cases (like in the MR).  The attributes
> have a few quirks and I suspect that they were mostly designed and
> tested to work for C++ classes, but I found a way to make it work for
> C.
>
> For each type of lock, declare a lock capability:
>
>   extern lock_cap_t fd_screen_lock_cap;
>
> Functions that take and release that lock get annotated with:
>
>   static inline void
>   fd_screen_lock(struct fd_screen *screen)
> acquire_cap(fd_screen_lock_cap);
>
>   static inline void
>   fd_screen_unlock(struct fd_screen *screen)
> release_cap(fd_screen_lock_cap)
>
> where acquire_cap and release_cap are #defines for an __attribute__.
> One of the quirks is that the function doing the locking triggers a
> warning about how it doesn't actually take the lock, so I silenced
> that by adding a no_thread_safety_analysis in there.
>
> Now whenever we have a function that expects to be called with that
> lock held (we often call them foo_locked) we can add the requires_cap
> annotation:
>
>   static struct gmem_key *
>   gmem_key_init(struct fd_batch *batch, bool assume_zs, bool no_scis_opt)
> requires_cap(fd_screen_lock_cap)
>
> and calls to this function will warn or error if called from a point
> where it's not statically determinable that you've taken the lock:
>
> ../../master/src/gallium/drivers/freedreno/freedreno_gmem.c:532:25:
> error: calling function 'gmem_key_init' requires holding lock
> 'fd_screen_lock_cap' exclusively [-Werror,-Wthread-safety-analysis]
> struct gmem_key *key = gmem_key_init(batch, assume_zs, no_scis_opt);
>^
> 1 error generated.

fd_screen_lock_cap seems to be a placeholder here.  Is it one of the
quirks?  It would be nice if the error message reads "... requiring
hold lock '&screen->lock' exclusively".

From https://clang.llvm.org/docs/ThreadSafetyAnalysis.html#mutex-h,
wrapping of the threading api is expected.  I wonder if we can
annotate simple_mtx_t (and other in-tree locking wrappers), and
perhaps kepp lock_cap_t only for code that uses mtx_t or
pthread_mutex_t directly.


>
> Many functions that assert a lock is taken are better off using the
> requires_cap annotation, which makes the check a compile failure
> instead. For cases where it's not possible to determine statically, we
> can still assert at runtime:
>
>   static inline void
>   fd_screen_assert_locked(struct fd_screen *screen)
> assert_cap(fd_screen_lock_cap)
>
> which tells the checker to assume the lock is taken.  Finally, it's
> possible to annotate struct members:
>
>   struct fd_gmem_cache gmem_cache guarded_by(fd_screen_lock_cap);
>
> such that any access to that field can only happen with the lock taken:
>
> ../../master/src/gallium/drivers/freedreno/freedreno_gmem.c:277:20:
> error: reading variable 'gmem_cache' requires holding lock
> 'fd_screen_lock_cap' [-Werror,-Wthread-safety-analysis]
> rzalloc(screen->gmem_cache.ht, struct 
> fd_gmem_stateobj);
> ^
> Having these annotations also helps human readers of the code by
> spelling out the conventions explicitly instead of relying on _locked
> suffix conventions or documentation that call out which parts of a
> struct are protected by which lock. All in all this seems really
> useful.
100% agreed.

>
> Kristian
> ___
> mesa-dev mailing list
> mesa-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev


Re: [Mesa-dev] Static thread safety checking with clang

2020-11-10 Thread Kristian Høgsberg
On Tue, Nov 10, 2020 at 8:00 PM Chia-I Wu  wrote:
>
> On Tue, Nov 10, 2020 at 8:09 AM Kristian Høgsberg  wrote:
> >
> > Hi,
> >
> > I wanted to call attention to
> >
> >   https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7529
> >
> > which shows how we can use a new clang __attribute__ to statically
> > check locking invariants. It's probably not perfect, but more like
> > static type checking - there will always be cases where you can't
> > statically determine if the code is right or wrong, but it will be
> > useful and correct in most cases (like in the MR).  The attributes
> > have a few quirks and I suspect that they were mostly designed and
> > tested to work for C++ classes, but I found a way to make it work for
> > C.
> >
> > For each type of lock, declare a lock capability:
> >
> >   extern lock_cap_t fd_screen_lock_cap;
> >
> > Functions that take and release that lock get annotated with:
> >
> >   static inline void
> >   fd_screen_lock(struct fd_screen *screen)
> > acquire_cap(fd_screen_lock_cap);
> >
> >   static inline void
> >   fd_screen_unlock(struct fd_screen *screen)
> > release_cap(fd_screen_lock_cap)
> >
> > where acquire_cap and release_cap are #defines for an __attribute__.
> > One of the quirks is that the function doing the locking triggers a
> > warning about how it doesn't actually take the lock, so I silenced
> > that by adding a no_thread_safety_analysis in there.
> >
> > Now whenever we have a function that expects to be called with that
> > lock held (we often call them foo_locked) we can add the requires_cap
> > annotation:
> >
> >   static struct gmem_key *
> >   gmem_key_init(struct fd_batch *batch, bool assume_zs, bool no_scis_opt)
> > requires_cap(fd_screen_lock_cap)
> >
> > and calls to this function will warn or error if called from a point
> > where it's not statically determinable that you've taken the lock:
> >
> > ../../master/src/gallium/drivers/freedreno/freedreno_gmem.c:532:25:
> > error: calling function 'gmem_key_init' requires holding lock
> > 'fd_screen_lock_cap' exclusively [-Werror,-Wthread-safety-analysis]
> > struct gmem_key *key = gmem_key_init(batch, assume_zs, no_scis_opt);
> >^
> > 1 error generated.
>
> fd_screen_lock_cap seems to be a placeholder here.  Is it one of the
> quirks?  It would be nice if the error message reads "... requiring
> hold lock '&screen->lock' exclusively".
>
> From https://clang.llvm.org/docs/ThreadSafetyAnalysis.html#mutex-h,
> wrapping of the threading api is expected.  I wonder if we can
> annotate simple_mtx_t (and other in-tree locking wrappers), and
> perhaps kepp lock_cap_t only for code that uses mtx_t or
> pthread_mutex_t directly.

I couldn't get the guarded_by attribute to work with anything other
than a global variable. I suspect for C++ there's an implicit 'this'
in play that allows it to refer to class members. Similar for the
other attributes that refer to capabilities.

> > Many functions that assert a lock is taken are better off using the
> > requires_cap annotation, which makes the check a compile failure
> > instead. For cases where it's not possible to determine statically, we
> > can still assert at runtime:
> >
> >   static inline void
> >   fd_screen_assert_locked(struct fd_screen *screen)
> > assert_cap(fd_screen_lock_cap)
> >
> > which tells the checker to assume the lock is taken.  Finally, it's
> > possible to annotate struct members:
> >
> >   struct fd_gmem_cache gmem_cache guarded_by(fd_screen_lock_cap);
> >
> > such that any access to that field can only happen with the lock taken:
> >
> > ../../master/src/gallium/drivers/freedreno/freedreno_gmem.c:277:20:
> > error: reading variable 'gmem_cache' requires holding lock
> > 'fd_screen_lock_cap' [-Werror,-Wthread-safety-analysis]
> > rzalloc(screen->gmem_cache.ht, struct 
> > fd_gmem_stateobj);
> > ^
> > Having these annotations also helps human readers of the code by
> > spelling out the conventions explicitly instead of relying on _locked
> > suffix conventions or documentation that call out which parts of a
> > struct are protected by which lock. All in all this seems really
> > useful.
> 100% agreed.
>
> >
> > Kristian
> > ___
> > mesa-dev mailing list
> > mesa-dev@lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/mesa-dev
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev


Re: [Mesa-dev] [EXTERNAL] Cross-compile from Linux to Windows

2020-11-10 Thread Dylan Baker
cmake_prefix_path isn't like pkg_config_path, it expects to find an
installed prefix, on linux it expects to see something like
path_to_prefix/usr/{lib,bin}.

Dylan

Quoting Federico Dossena (2020-11-09 07:33:58)
> Thank you for your reply.
> 
> That's where I got stuck on Windows, I can't get it to find LLVM.
> 
> I built LLVM without problems for both x86 and x86_64 in 2 separate 
> folders, but when I try to build mesa, even with --cmake-prefix-path, it 
> doesn't find it, it always say "Neither a subproject directory nor a 
> llvm.wrap file was found".
> 
> I tried moving my LLVM files in the subprojects/llvm folder and that 
> didn't fix it, I don't know what to do.
> 
> Any suggestion?
> 
> On 2020-11-09 16:06, Jesse Natalie wrote:
> > Have you tried using Meson on Windows, instead of cross-compiling? If you 
> > run it from a Visual Studio command prompt, it should just work out of the 
> > box, at least for x86. I believe you'll need to use a cross file for amd64, 
> > but the only thing you'd need to specify is the host architecture.
> >
> > Though I guess if you're building LLVMPipe you'll need to point it to LLVM. 
> > In my experience, adding "--cmake-prefix-path" to the Meson command line 
> > should work well.
> >
> > -Jesse
> >
> > -Original Message-
> > From: mesa-dev  On Behalf Of 
> > Federico Dossena
> > Sent: Saturday, November 7, 2020 4:14 AM
> > To: mesa-dev@lists.freedesktop.org
> > Subject: [EXTERNAL] [Mesa-dev] Cross-compile from Linux to Windows
> >
> > I'm sorry to bother you in the dev list but it seems to be the only one 
> > with activity so it seems like the best place to ask.
> >
> > I maintain some Mesa builds for Windows. These are builds of libgl-gdi with 
> > llvmpipe, for both x86 and x86_64; in other words, they're opengl32.dll 
> > files that users can drop into their game/application to have software 
> > rendering for OpenGL as a workaround for broken drivers or as a fallback 
> > for older systems.
> > I've always used scons on Windows and msvc as a compiler to make these 
> > builds, but now that scons is being deprecated I've been trying to use 
> > meson and ninja to cross-compile from Linux (amd64) to Windows (both x86 
> > and x86_64) without much success. I can get regular Linux builds without 
> > issues though.
> >
> > I tried reading the Mesa documentation on cross-compiling but it's very 
> > minimal and it's partially obsolete so I thought it would be best to ask 
> > you directly: how do I make these builds? Explain like I'm 5, basically.
> >
> > Thanks,
> >
> > Federico
> >
> > ___
> > mesa-dev mailing list
> > mesa-dev@lists.freedesktop.org
> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fmesa-dev&data=04%7C01%7Cjenatali%40microsoft.com%7C7ed90f213a9a4204b25108d8848e0ef1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637405092966689938%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=A%2BvRfw0Y4C4NufTtNWOwo4COLOOJAw%2F3H9rttkk5jA4%3D&reserved=0
> ___
> mesa-dev mailing list
> mesa-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev

signature.asc
Description: signature
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev


Re: [Mesa-dev] [EXTERNAL] Cross-compile from Linux to Windows

2020-11-10 Thread Dylan Baker
You can build for x86 or x86_64, just open the appropriate command
prompt, meson should properly detect whether you're using an x86 or an
x86_64 toolchain.

Quoting Jesse Natalie (2020-11-09 07:06:31)
> Have you tried using Meson on Windows, instead of cross-compiling? If you run 
> it from a Visual Studio command prompt, it should just work out of the box, 
> at least for x86. I believe you'll need to use a cross file for amd64, but 
> the only thing you'd need to specify is the host architecture.
> 
> Though I guess if you're building LLVMPipe you'll need to point it to LLVM. 
> In my experience, adding "--cmake-prefix-path" to the Meson command line 
> should work well.
> 
> -Jesse
> 
> -Original Message-
> From: mesa-dev  On Behalf Of Federico 
> Dossena
> Sent: Saturday, November 7, 2020 4:14 AM
> To: mesa-dev@lists.freedesktop.org
> Subject: [EXTERNAL] [Mesa-dev] Cross-compile from Linux to Windows
> 
> I'm sorry to bother you in the dev list but it seems to be the only one with 
> activity so it seems like the best place to ask.
> 
> I maintain some Mesa builds for Windows. These are builds of libgl-gdi with 
> llvmpipe, for both x86 and x86_64; in other words, they're opengl32.dll files 
> that users can drop into their game/application to have software rendering 
> for OpenGL as a workaround for broken drivers or as a fallback for older 
> systems.
> I've always used scons on Windows and msvc as a compiler to make these 
> builds, but now that scons is being deprecated I've been trying to use meson 
> and ninja to cross-compile from Linux (amd64) to Windows (both x86 and 
> x86_64) without much success. I can get regular Linux builds without issues 
> though.
> 
> I tried reading the Mesa documentation on cross-compiling but it's very 
> minimal and it's partially obsolete so I thought it would be best to ask you 
> directly: how do I make these builds? Explain like I'm 5, basically.
> 
> Thanks,
> 
> Federico
> 
> ___
> mesa-dev mailing list
> mesa-dev@lists.freedesktop.org
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fmesa-dev&data=04%7C01%7Cjenatali%40microsoft.com%7C7ed90f213a9a4204b25108d8848e0ef1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637405092966689938%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=A%2BvRfw0Y4C4NufTtNWOwo4COLOOJAw%2F3H9rttkk5jA4%3D&reserved=0
> ___
> mesa-dev mailing list
> mesa-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev

signature.asc
Description: signature
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev


[Mesa-dev] clover's interface to clang

2020-11-10 Thread Dave Airlie
Hey all (mostly Tom).

I've been learning new things today since Matt pushed a patch to clang
to remove "-cl-denorms-are-zero" from cc1 options. I thought this was
a regression or we should hack things to pass a different flag (which
I did locally for testing), but Matt informed me clover is likely
interfacing to clang wrong.

The correct way to do things seems to be to build up a set of command
line args pass those to the Driver, create a Compilation object, with
jobs, and then execute each job in turns, one of the jobs would be a
cc1 job and the driver would have all the correct arguments for it.

Now I'll likely dig into this a bit more, but I was wondering if
anyone knows historically why this wasn't done. I know for example
with clover we really only want to use a the cc1 pass since at least
for the NIR backend we just want to emit LLVM bytecode and pass it to
the spirv conversion, so using the driver might be overkill.

Dave.
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev