[Bug c/79177] New: use the register keyword with ymm0 register

[alexander.kjall at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=79177

Bug ID: 79177
   Summary: use the register keyword with ymm0 register
   Product: gcc
   Version: 6.2.0
Status: UNCONFIRMED
  Severity: normal
  Priority: P3
 Component: c
  Assignee: unassigned at gcc dot gnu.org
  Reporter: alexander.kjall at gmail dot com
  Target Milestone: ---

I'm trying to represent the ymm0 register as a variable with the help of the
register key word.

Code that i have tried:

register int __attribute__ ((vector_size (32))) _YMM0_r asm ("ymm0") = _YMM0;

and

register __m256 _YMM0_r asm ("ymm0") = _YMM0;

And this is the error that I get:

encryptor.c:72:17: error: data type of ‘_YMM0_r’ isn’t suitable for a register
 register __m256 _YMM0_r asm ("ymm0") = _YMM0;

I can't understand in what way the __m256 datatype is unsuitable for the ymm0
register.

[Bug target/79177] use the register keyword with ymm0 register

[alexander.kjall at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=79177

--- Comment #2 from Alexander Kjäll  ---
Sorry for the sparse information.

I'm trying to rewrite some output from the decompiler in IDA Pro, and that is a
total mixed of C and ASM. The output generated C code that was in visual studio
syntax.

I don't think I can post the complete source before monday, as it's an ongoing
CTF challenge in the insomnia'hach CTF competition.

Doing roughly same thing for xmm1 works fine:
register __int128 _XMM1_r asm ("xmm1");

I also tried to just comment that section out, and just work on a different
section of the code base, and got this error:

encryptor.c:762:1: internal compiler error: in expand_stack_alignment, at
cfgexpand.c:6004
 }

I presume it's because of all the strange asm stuff, but should i report that
as a separate error after monday?

[Bug target/79177] use the register keyword with ymm0 register

[alexander.kjall at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=79177

--- Comment #3 from Alexander Kjäll  ---
I wrote a small testcase that reproduced the structure of the error, with just
some dummy asm:

#include 

int main() {
  __m256 _YMM0;
  register int p1 asm ("r12") = 1;
  //register int p2 asm ("r13") = 2;
  register __m256 p2 asm ("ymm0") = _YMM0;
  register int result asm ("rax");
  asm ("CLI" : "=r" (result) : "0" (p1), "r" (p2));
}

[Bug target/79177] use the register keyword with ymm0 register

[alexander.kjall at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=79177

Alexander Kjäll  changed:

   What|Removed |Added

 Status|UNCONFIRMED |RESOLVED
 Resolution|--- |INVALID

--- Comment #5 from Alexander Kjäll  ---
Thanks for the pointer to the -mavx2 flag, I didn't realize those registers
needed a compiler flag.