Re: Replacing NIR with SPIR-V?

[Abel Bernabeu]

>
> Yes, NIR arrays and struct and nir_deref to deal with them but, by the
> time you get into the back-end, all the nir_derefs are gone and you're left
> with load/store messages with actual addresses (either a 64-bit memory
> address or a index+offset pair for a bound resource).  Again, unless you're
> going to dump straight into LLVM, you really don't want to handle that in
> your back-end unless you really have to.
>

That is the thing: there is already a community maintained LLVM backend for
RISC-V and I need to see how to get value from that effort. And that is a
very typical escenario for new architectures. There is already an LLVM
backend for a programmable device and someone asks: could you do some
graphics around this without spending millions?

Then your options as an engineer are:

- Use Mesa as a framework and translate NIR to assembly (most likely
choice).

- Use Mesa as a framework and translate NIR to LLVM IR with some
intrinsics, then feed the pre-existing LLVM backend.

- Use some new alternative, possibly a Mesa fork relying on the Khronos
SPIR-V to LLVM IR translator. Start fixing the tool for supporting
graphics... Make SPIR-V the IR that communicates frontend and backend :-)

I am not thinking in terms of what is best for Mesa, but in terms of how
could the RISC-V community organize its effort given that an LLVM backend
is a given thing.

I see the current reasons why NIR is preferred over SPIR-V in Mesa. So far
you have given me three

- There is a well designed library for traversing NIR, whereas SPIR-V
defines nothing.
- The arrays and structs are lowered before the shader is passed to the
backend.
- You see SPIR-V as a "serializing" format for IR to be exchanged through
the network (like a .PNG for shaders), whereas NIR's focus is more about
how the data structures are represented in memory while in use.

My takeaway messages are two:

- Advise to support NIR on the RISC-V plan.
- If I have a chance, suggest to Khronos making SPIR-V more like NIR, so in
the future it is considered beyond a serializing format.

Thanks for your comments so far.



On Fri, Jan 21, 2022 at 4:24 AM Jason Ekstrand  wrote:

> On Thu, Jan 20, 2022 at 5:49 PM Abel Bernabeu <
> abel.berna...@esperantotech.com> wrote:
>
>> In principle, all the properties you highlight in your blog
>>  as key
>> points of NIR also apply to SPIR-V.
>>
>
> First off, that blog post is truly ancient.  Based on the quote from
> nir_opt_algebraic.c, it looks like less than 6 months after the original
> NIR patches landed which puts it at 5-6 years old.  A lot has changed since
> then.
>
>
>> I was curious to know where in the details that I miss, NIR starts
>> shining as a more suitable IR than SPIR-V for the task of communicating
>> front-end and back-end. By the way, thanks for putting together that blog
>> post.
>>
>
> In terms of what they're capable of communicating, yes, SPIR-V and NIR can
> express many of the same things.  But that's not the point.  The point is
> that there's a lot that happens between coming out of GLSL or SPIR-V and
> going into the back-end.  A lot of what we do with NIR is share as much of
> that lowering across drivers as possible.  Yes, we could convert back to
> SPIR-V before going into back-ends but there's really no point since they
> need their own IRs anyway.  If you're dumping straight into LLVM or
> similar, then maybe you don't need any of that, but if you're building a
> custom back-end, you really want to let NIR do that lowering and you don't
> want to handle it all on your own.
>
>
>> As it seems clear that the NIR question is well settled within the mesa
>> community and I really see value in having mesa drivers, I promise to pay
>> as much attention to the NIR use cases as I did with SPIR-V :-)
>>
>> By the way, we are not planning on supporting with specific RISC-V
>> instructions everything that has an instruction on SPIR-V. Regarding the
>> two areas you mention:
>>
>> - Arrays and structs: SPIR-V's OpAccessChain would need to be processed
>> by a backend and translated to pointer arithmetic plus dereferencing (kind
>> of the same thing as having to process a nir_deref). This translation can
>> be done in RISC-V with no issue, whether it is OpAccessChain or nir_deref.
>>
>
> A big part of the point of NIR is to get rid of these things so that
> drivers don't have to deal with them.  Yes, NIR arrays and struct and
> nir_deref to deal with them but, by the time you get into the back-end, all
> the nir_derefs are gone and you're left with load/store messages with
> actual addresses (either a 64-bit memory address or a index+offset pair for
> a bound resource).  Again, unless you're going to dump straight into LLVM,
> you really don't want to handle that in your back-end unless you really
> have to.
>
> Over-all, I think you're asking the wrong set of questions.  If you're
> trying to understand Mesa GPU compilers, looking at N

Re: Replacing NIR with SPIR-V?

[Timur Kristóf]

Hi Abel,

On Sun, 2022-01-23 at 13:58 +0100, Abel Bernabeu wrote:
> 
> That is the thing: there is already a community maintained LLVM
> backend for RISC-V and I need to see how to get value from that
> effort. And that is a very typical escenario for new architectures.
> There is already an LLVM backend for a programmable device and
> someone asks: could you do some graphics around this without spending
> millions?

While LLVM is a very good tool, it is not perfect and may not be the
best tool for compiling shaders. For example, ACO (the Valve-funded
compiler for AMD GPUs) was created despite the existence of an LLVM
backend. Our main concerns were:

1. Games stuttering due to slow compilation speed.
2. We are part of the Mesa source tree so we can fix bugs without
waiting for a new LLVM release.

> 
> - Use Mesa as a framework and translate NIR to assembly (most likely
> choice).

I think this is the optimal solution. See below for a writeup about how
this works in practice.

> 
> - Use Mesa as a framework and translate NIR to LLVM IR with some
> intrinsics, then feed the pre-existing LLVM backend.

A viable choice if you want to benefit from the infrastructure in Mesa,
but don't want to invest in a custom backend yet. You can always choose
to create a custom backend later, if you reached the limits of what
LLVM can do for you.

> 
> - Use some new alternative, possibly a Mesa fork relying on the
> Khronos SPIR-V to LLVM IR translator. Start fixing the tool for
> supporting graphics... Make SPIR-V the IR that communicates frontend
> and backend :-)

I'm not familiar with those tools so I can't judge their viability.

> 
> I am not thinking in terms of what is best for Mesa, but in terms of
> how could the RISC-V community organize its effort given that an LLVM
> backend is a given thing.
> 
> I see the current reasons why NIR is preferred over SPIR-V in Mesa.
> So far you have given me three
> 
> - There is a well designed library for traversing NIR, whereas SPIR-V
> defines nothing.
> - The arrays and structs are lowered before the shader is passed to
> the backend.
> - You see SPIR-V as a "serializing" format for IR to be exchanged
> through the network (like a .PNG for shaders), whereas NIR's focus is
> more about how the data structures are represented in memory while in
> use.

SPIR-V and NIR are designed for two different purposes:

- SPIR-V is meant to be a high-level platform-independent format and
the input to a compiler.
- NIR is an intermediate representation to be used internally by a
shader compiler. NIR's job is to represent a shader in such a way that
it will be fast and easy to modify and optimize.

The process of compilation is roughly this:

1. SPIR-V or GLSL (or another representation) is parsed into NIR.
2. Various optimizations are applied. We have a wide range of
optimizations and thanks to the nature of NIR, it's rather easy to
create more if the need arises.
3. Various lowerings are applied. NIR can understand which instructions
are available in your backend and which aren't. It transforms the
unsupported instructions to use supported ones.
4. A backend compiler gets the NIR and parses it into its own backend-
specific IR. This is easy because the backend will only see
instructions that it actually supports on the target hardware.
5. The backend may do further optimization and finally emit machine
code that the GPU can understand.

If you are interested how to build an OpenGL driver in Mesa, I
recommend watching the XDC 2018 talk by Kenneth Graunke who talks about
why and how he made the Intel Iris driver:
https://www.youtube.com/watch?v=XUis_0lMUBI

If you are interested in learning more about how to build a backend
that uses NIR, I recommend these talks:
https://www.youtube.com/watch?v=7C4PmtqBVqo
https://www.youtube.com/watch?v=XoGmKhb5vpg

Hope this helps,
Timur

Re: Replacing NIR with SPIR-V?

[Connor Abbott]

On Sun, Jan 23, 2022 at 1:58 PM Abel Bernabeu
 wrote:
>>
>> Yes, NIR arrays and struct and nir_deref to deal with them but, by the time 
>> you get into the back-end, all the nir_derefs are gone and you're left with 
>> load/store messages with actual addresses (either a 64-bit memory address or 
>> a index+offset pair for a bound resource).  Again, unless you're going to 
>> dump straight into LLVM, you really don't want to handle that in your 
>> back-end unless you really have to.
>
>
> That is the thing: there is already a community maintained LLVM backend for 
> RISC-V and I need to see how to get value from that effort. And that is a 
> very typical escenario for new architectures. There is already an LLVM 
> backend for a programmable device and someone asks: could you do some 
> graphics around this without spending millions?
>
> Then your options as an engineer are:
>
> - Use Mesa as a framework and translate NIR to assembly (most likely choice).
>
> - Use Mesa as a framework and translate NIR to LLVM IR with some intrinsics, 
> then feed the pre-existing LLVM backend.

Using the pre-existing backend probably isn't a real option, because
it's designed for different things. The biggest hurdle is that for
many years now, vendors have realized that SIMT-style parallelism is
the most appropriate for GPUs, and in order to do SIMT effectively
your whole compiler stack has to be aware of it, from the frontend to
the backend. There are two "views" of your program, the "thread-level
view" which is what the programmer wrote where SIMD lanes are separate
threads and you're specifying what happens to one thread, and the
"wave-level view" which is what the machine actually executes. In the
backend you have to be aware of both the thread-level view and
wave-level view of your program in order to effectively register
allocate, and that's something that LLVM's backend infrastructure just
can't do. AMDGPU has some hacks, but they're not 100% effective and to
use them in RISC-V you'd probably have to rewrite the whole backend
anyway. For example, in the AMDGPU backend vector registers aren't
exposed as actual vector registers to LLVM's machinery, but it still
models control-flow at the "wave-level" which causes some inaccuracy
in liveness. So, the existing investment isn't really worth as much as
you think it is.

>
> - Use some new alternative, possibly a Mesa fork relying on the Khronos 
> SPIR-V to LLVM IR translator. Start fixing the tool for supporting 
> graphics... Make SPIR-V the IR that communicates frontend and backend :-)
>
> I am not thinking in terms of what is best for Mesa, but in terms of how 
> could the RISC-V community organize its effort given that an LLVM backend is 
> a given thing.
>
> I see the current reasons why NIR is preferred over SPIR-V in Mesa. So far 
> you have given me three
>
> - There is a well designed library for traversing NIR, whereas SPIR-V defines 
> nothing.
> - The arrays and structs are lowered before the shader is passed to the 
> backend.
> - You see SPIR-V as a "serializing" format for IR to be exchanged through the 
> network (like a .PNG for shaders), whereas NIR's focus is more about how the 
> data structures are represented in memory while in use.
>
> My takeaway messages are two:
>
> - Advise to support NIR on the RISC-V plan.

Sure.

> - If I have a chance, suggest to Khronos making SPIR-V more like NIR, so in 
> the future it is considered beyond a serializing format.

No, this is a terrible idea. Serialization formats like SPIR-V and IRs
intended to be consumed by a backend like NIR (and LLVM) have very
different needs - SPIR-V has to be backwards compatible and have very
broad support, NIR has to represent lower-level constructs that SPIR-V
doesn't, etc. If you created an in-memory IR datastructure that
adhered slavishly to SPIR-V you'd have a terrible replacement for NIR,
and that's just by necessity - they're solving different problems.

There's some history to this, because before SPIR-V there was SPIR
which was just LLVM bitcode with some graphics stuff on top. It failed
for the reasons above and was abandoned, and that's why we have SPIR-V
today.

>
> Thanks for your comments so far.
>
>
>
> On Fri, Jan 21, 2022 at 4:24 AM Jason Ekstrand  wrote:
>>
>> On Thu, Jan 20, 2022 at 5:49 PM Abel Bernabeu 
>>  wrote:
>>>
>>> In principle, all the properties you highlight in your blog as key points 
>>> of NIR also apply to SPIR-V.
>>
>>
>> First off, that blog post is truly ancient.  Based on the quote from 
>> nir_opt_algebraic.c, it looks like less than 6 months after the original NIR 
>> patches landed which puts it at 5-6 years old.  A lot has changed since then.
>>
>>>
>>> I was curious to know where in the details that I miss, NIR starts shining 
>>> as a more suitable IR than SPIR-V for the task of communicating front-end 
>>> and back-end. By the way, thanks for putting together that blog post.
>>
>>
>> In terms of what they're capable of communicating, 

Re: Replacing NIR with SPIR-V?

[Dave Airlie]

On Sun, 23 Jan 2022 at 22:58, Abel Bernabeu
 wrote:
>>
>> Yes, NIR arrays and struct and nir_deref to deal with them but, by the time 
>> you get into the back-end, all the nir_derefs are gone and you're left with 
>> load/store messages with actual addresses (either a 64-bit memory address or 
>> a index+offset pair for a bound resource).  Again, unless you're going to 
>> dump straight into LLVM, you really don't want to handle that in your 
>> back-end unless you really have to.
>
>
> That is the thing: there is already a community maintained LLVM backend for 
> RISC-V and I need to see how to get value from that effort. And that is a 
> very typical escenario for new architectures. There is already an LLVM 
> backend for a programmable device and someone asks: could you do some 
> graphics around this without spending millions?

No.

If you want something useful, it's going to cost millions over the
lifetime of creating it. This stuff is hard, it needs engineers who
understand it and they usually have to be paid.

RISC-V as-is isn't going to make a good compute core for a GPU. I
don't think any of the implementations are the right design. as long
as people get sucked into thinking it might, there'll be millions
wasted. SIMT vs SIMD is just making SSE-512 type decisions or
recreating Intel Larrabee efforts. Nobody has made an effective GPU in
this fashion. You'd need someone to create a new GPU with it's own
instruction set (maybe dervied from RISC-V), but with it's own
specialised compute core.

The alternate more tractable project is to possibly make sw rendering
(with llvmpipe) on RISC-V more palatable, but that's really just
optimising llvmpipe and the LLVM backend and maybe finding a few
instructions to enhance things. It might be possible to use a texture
unit to speed things up and really for software rendering and hw
rendering, memory bandwidth is a lot of the problem to solve.

Dave.

Re: Replacing NIR with SPIR-V?

[Abel Bernabeu]

Dave,

Am glad for the Mesa community to support the RISC-V effort with the advice
given so far.

I hear your concerns regarding performance. I am familiar with the Larabee
case and know some of the people who worked on that. However, I am not here
to discuss what is the RISC-V strategy for graphics beyond the fact that
there is a NIR backend planned. That would be offtopic for this list.

Join the Graphics and ML SIG at RISC-V and let us discuss it there.

First join RISC-V as an individual contributor, strategic member or
community member:
https://riscv.org/membership/

Then join the Graphics and ML SIG through the Working Groups Portal. Feel
free to ask for details privately.

Thanks.



On Sun, Jan 23, 2022 at 9:10 PM Dave Airlie  wrote:

> On Sun, 23 Jan 2022 at 22:58, Abel Bernabeu
>  wrote:
> >>
> >> Yes, NIR arrays and struct and nir_deref to deal with them but, by the
> time you get into the back-end, all the nir_derefs are gone and you're left
> with load/store messages with actual addresses (either a 64-bit memory
> address or a index+offset pair for a bound resource).  Again, unless you're
> going to dump straight into LLVM, you really don't want to handle that in
> your back-end unless you really have to.
> >
> >
> > That is the thing: there is already a community maintained LLVM backend
> for RISC-V and I need to see how to get value from that effort. And that is
> a very typical escenario for new architectures. There is already an LLVM
> backend for a programmable device and someone asks: could you do some
> graphics around this without spending millions?
>
> No.
>
> If you want something useful, it's going to cost millions over the
> lifetime of creating it. This stuff is hard, it needs engineers who
> understand it and they usually have to be paid.
>
> RISC-V as-is isn't going to make a good compute core for a GPU. I
> don't think any of the implementations are the right design. as long
> as people get sucked into thinking it might, there'll be millions
> wasted. SIMT vs SIMD is just making SSE-512 type decisions or
> recreating Intel Larrabee efforts. Nobody has made an effective GPU in
> this fashion. You'd need someone to create a new GPU with it's own
> instruction set (maybe dervied from RISC-V), but with it's own
> specialised compute core.
>
> The alternate more tractable project is to possibly make sw rendering
> (with llvmpipe) on RISC-V more palatable, but that's really just
> optimising llvmpipe and the LLVM backend and maybe finding a few
> instructions to enhance things. It might be possible to use a texture
> unit to speed things up and really for software rendering and hw
> rendering, memory bandwidth is a lot of the problem to solve.
>
> Dave.
>

Re: Replacing NIR with SPIR-V?

[Dave Airlie]

On Mon, 24 Jan 2022 at 06:52, Abel Bernabeu
 wrote:
>
> Dave,
>
> Am glad for the Mesa community to support the RISC-V effort with the advice 
> given so far.
>
> I hear your concerns regarding performance. I am familiar with the Larabee 
> case and know some of the people who worked on that. However, I am not here 
> to discuss what is the RISC-V strategy for graphics beyond the fact that 
> there is a NIR backend planned. That would be offtopic for this list.

We are pretty flexible on our topics since we moved most of the day to
day work to gitlab :-)

With the current chip architectures and possible vector extensions to
RISC-V, I think using llvmpipe and your current LLVM backend would be
as efficient as you can get, modulo optimising things more for
possible use cases you have. You might be able to add some fixed
function blocks to enhance things like texturing or blending.

I would think having a NIR backend would only really show usefulness
if you moved to a SIMT architecture, now you may have plans in that
area, and that might give you more reason to do NIR. Having a SIMT
architecture would be more around designing a GPU and you'd no longer
be considering it a RISC-V cpu at all that case and running general
purpose tasks would not be a goal.

>
> Join the Graphics and ML SIG at RISC-V and let us discuss it there.
>
> First join RISC-V as an individual contributor, strategic member or community 
> member:
> https://riscv.org/membership/
>
> Then join the Graphics and ML SIG through the Working Groups Portal. Feel 
> free to ask for details privately.

Thanks for the info, I'll see if I can manage that, but my time is
pretty stretched out here.

Dave.

Re: Replacing NIR with SPIR-V?

[Abel Bernabeu]

I can tease you all with the promise of SIMT and fixed function units.

In the meantime you can hear me talking about the work we do at the
Graphics and ML Special Interest Group within RISC-V:
https://www.youtube.com/watch?v=kM0lsWjqOaw

I still need to make our site a bit more useful, but here is the GitHub
repo where I put our meeting minutes:
https://github.com/riscv-admin/graphics

Regards.

On Sun, Jan 23, 2022 at 11:07 PM Ian Romanick <
i...@paranormal-entertainment.com> wrote:

> On 1/23/22 12:10 PM, Dave Airlie wrote:
> > On Sun, 23 Jan 2022 at 22:58, Abel Bernabeu
> >  wrote:
> >>>
> >>> Yes, NIR arrays and struct and nir_deref to deal with them but, by the
> time you get into the back-end, all the nir_derefs are gone and you're left
> with load/store messages with actual addresses (either a 64-bit memory
> address or a index+offset pair for a bound resource).  Again, unless you're
> going to dump straight into LLVM, you really don't want to handle that in
> your back-end unless you really have to.
> >>
> >>
> >> That is the thing: there is already a community maintained LLVM backend
> for RISC-V and I need to see how to get value from that effort. And that is
> a very typical escenario for new architectures. There is already an LLVM
> backend for a programmable device and someone asks: could you do some
> graphics around this without spending millions?
> >
> > No.
> >
> > If you want something useful, it's going to cost millions over the
> > lifetime of creating it. This stuff is hard, it needs engineers who
> > understand it and they usually have to be paid.
> >
> > RISC-V as-is isn't going to make a good compute core for a GPU. I
> > don't think any of the implementations are the right design. as long
> > as people get sucked into thinking it might, there'll be millions
> > wasted. SIMT vs SIMD is just making SSE-512 type decisions or
> > recreating Intel Larrabee efforts. Nobody has made an effective GPU in
> > this fashion. You'd need someone to create a new GPU with it's own
> > instruction set (maybe dervied from RISC-V), but with it's own
> > specialised compute core.
> >
> > The alternate more tractable project is to possibly make sw rendering
> > (with llvmpipe) on RISC-V more palatable, but that's really just
> > optimising llvmpipe and the LLVM backend and maybe finding a few
> > instructions to enhance things. It might be possible to use a texture
> > unit to speed things up and really for software rendering and hw
> > rendering, memory bandwidth is a lot of the problem to solve.
>
> For the love of all that is good in the world, no! :) That was my
> original master's project that I gave up on.
>
> Executive summary: There's a reason GPUs have huge piles of
> fixed-function blocks.  It's the only way to get enough power
> efficiency, and power consumption (and the heat it generates) is *the*
> problem.
>
> > Dave.
>
>