[llvm-branch-commits] [llvm] d4ce062 - [RISCV][PrologEpilogInserter] "Float" emergency spill slots to avoid making them immediately unreachable from the stack pointer
Author: Roger Ferrer Ibanez
Date: 2021-01-23T09:10:03Z
New Revision: d4ce062340064c3f73b8f6136c7350a5abe83cac
URL:
https://github.com/llvm/llvm-project/commit/d4ce062340064c3f73b8f6136c7350a5abe83cac
DIFF:
https://github.com/llvm/llvm-project/commit/d4ce062340064c3f73b8f6136c7350a5abe83cac.diff
LOG: [RISCV][PrologEpilogInserter] "Float" emergency spill slots to avoid
making them immediately unreachable from the stack pointer
In RISC-V there is a single addressing mode of the form imm(reg) where
imm is a signed integer of 12-bit with a range of [-2048..2047] bytes
from reg.
The test MultiSource/UnitTests/C++11/frame_layout of the LLVM test-suite
exercises several scenarios with the stack, including function calls
where the stack will need to be realigned to to a local variable having
a large alignment of 4096 bytes.
In situations of large stacks, the RISC-V backend (in
RISCVFrameLowering) reserves an extra emergency spill slot which can be
used (if no free register is found) by the register scavenger after the
frame indexes have been eliminated. PrologEpilogInserter already takes
care of keeping the emergency spill slots as close as possible to the
stack pointer or frame pointer (depending on what the function will
use). However there is a final alignment step to honour the maximum
alignment of the stack that, when using the stack pointer to access the
emergency spill slots, has the side effect of setting them farther from
the stack pointer.
In the case of the frame_layout testcase, the net result is that we do
have an emergency spill slot but it is so far from the stack pointer
(more than 2048 bytes due to the extra alignment of a variable to 4096
bytes) that it becomes unreachable via any immediate offset.
During elimination of the frame index, many (regular) offsets of the
stack may be immediately unreachable already. Their address needs to be
computed using a register. A virtual register is created and later
RegisterScavenger should be able to find an unused (physical) register.
However if no register is available, RegisterScavenger will pick a
physical register and spill it onto an emergency stack slot, while we
compute the offset (restoring the chosen register after all this). This
assumes that the emergency stack slot is easily reachable (this is,
without requiring another register!).
This is the assumption we seem to break when we perform the extra
alignment in PrologEpilogInserter.
We can "float" the emergency spill slots by increasing (in absolute
value) their offsets from the incoming stack pointer. This way the
emergency spill slots will remain close to the stack pointer (once the
function has allocated storage for the stack, including the needed
realignment). The new size computed in PrologEpilogInserter is padding
so it should be OK to move the emergency spill slots there. Also because
we're increasing the alignment, the new location should stay aligned for
the purpose of the emergency spill slots.
Note that this change also impacts other backends as shown by the tests.
Changes are minor adjustments to the emergency stack slot offset.
Differential Revision: https://reviews.llvm.org/D89239
Added:
llvm/test/CodeGen/RISCV/out-of-reach-emergency-slot.mir
Modified:
llvm/lib/CodeGen/PrologEpilogInserter.cpp
llvm/test/CodeGen/AArch64/framelayout-scavengingslot.mir
llvm/test/CodeGen/AArch64/framelayout-sve-scavengingslot.mir
llvm/test/CodeGen/AArch64/swiftself-scavenger.ll
llvm/test/CodeGen/AMDGPU/pei-scavenge-vgpr-spill.mir
llvm/test/CodeGen/Thumb/emergency-spill-slot.ll
Removed:
diff --git a/llvm/lib/CodeGen/PrologEpilogInserter.cpp
b/llvm/lib/CodeGen/PrologEpilogInserter.cpp
index 65b2165bf2a0..378aaba2a65f 100644
--- a/llvm/lib/CodeGen/PrologEpilogInserter.cpp
+++ b/llvm/lib/CodeGen/PrologEpilogInserter.cpp
@@ -1077,7 +1077,26 @@ void PEI::calculateFrameObjectOffsets(MachineFunction
&MF) {
// If the frame pointer is eliminated, all frame offsets will be relative
to
// SP not FP. Align to MaxAlign so this works.
StackAlign = std::max(StackAlign, MaxAlign);
+int64_t OffsetBeforeAlignment = Offset;
Offset = alignTo(Offset, StackAlign, Skew);
+
+// If we have increased the offset to fulfill the alignment constrants,
+// then the scavenging spill slots may become harder to reach from the
+// stack pointer, float them so they stay close.
+if (OffsetBeforeAlignment != Offset && RS && !EarlyScavengingSlots) {
+ SmallVector SFIs;
+ RS->getScavengingFrameIndices(SFIs);
+ LLVM_DEBUG(if (!SFIs.empty()) llvm::dbgs()
+ << "Adjusting emergency spill slots!\n";);
+ int64_t Delta = Offset - OffsetBeforeAlignment;
+ for (SmallVectorImpl::iterator I = SFIs.begin(), IE = SFIs.end();
+ I != IE; ++I) {
+LLVM_DEBUG(llvm::dbgs() << "Adjusting offset of emergency spill slot #"
+
[llvm-branch-commits] [llvm] 524d8fa - [RISCV] Do not grow the stack a second time when we need to realign the stack
Author: Roger Ferrer Ibanez
Date: 2021-01-09T16:51:09Z
New Revision: 524d8fa9a5a5428628a21a91016a52a54a9fe838
URL:
https://github.com/llvm/llvm-project/commit/524d8fa9a5a5428628a21a91016a52a54a9fe838
DIFF:
https://github.com/llvm/llvm-project/commit/524d8fa9a5a5428628a21a91016a52a54a9fe838.diff
LOG: [RISCV] Do not grow the stack a second time when we need to realign the
stack
This is a first change needed to fix a crash in which the emergency
spill splot ends being out of reach. This happens when we run the
register scavenger after we have eliminated the frame indexes. The fix
for the actual crash will come in a later change.
This change removes an extra stack size increase we do in
RISCVFrameLowering::determineFrameLayout.
We don't have to change the size of the stack here as
PEI::calculateFrameObjectOffsets is already doing this with the right
size accounting the extra alignment.
Differential Revision: https://reviews.llvm.org/D89237
Added:
Modified:
llvm/lib/Target/RISCV/RISCVFrameLowering.cpp
llvm/test/CodeGen/RISCV/stack-realignment-with-variable-sized-objects.ll
llvm/test/CodeGen/RISCV/stack-realignment.ll
Removed:
diff --git a/llvm/lib/Target/RISCV/RISCVFrameLowering.cpp
b/llvm/lib/Target/RISCV/RISCVFrameLowering.cpp
index 26ff6ddcd048..564d97f47d9e 100644
--- a/llvm/lib/Target/RISCV/RISCVFrameLowering.cpp
+++ b/llvm/lib/Target/RISCV/RISCVFrameLowering.cpp
@@ -235,18 +235,12 @@ bool RISCVFrameLowering::hasBP(const MachineFunction &MF)
const {
// Determines the size of the frame and maximum call frame size.
void RISCVFrameLowering::determineFrameLayout(MachineFunction &MF) const {
MachineFrameInfo &MFI = MF.getFrameInfo();
- const RISCVRegisterInfo *RI = STI.getRegisterInfo();
// Get the number of bytes to allocate from the FrameInfo.
uint64_t FrameSize = MFI.getStackSize();
// Get the alignment.
Align StackAlign = getStackAlign();
- if (RI->needsStackRealignment(MF)) {
-Align MaxStackAlign = std::max(StackAlign, MFI.getMaxAlign());
-FrameSize += (MaxStackAlign.value() - StackAlign.value());
-StackAlign = MaxStackAlign;
- }
// Set Max Call Frame Size
uint64_t MaxCallSize = alignTo(MFI.getMaxCallFrameSize(), StackAlign);
diff --git
a/llvm/test/CodeGen/RISCV/stack-realignment-with-variable-sized-objects.ll
b/llvm/test/CodeGen/RISCV/stack-realignment-with-variable-sized-objects.ll
index edfa4b4200d3..7f7e1c4c7e13 100644
--- a/llvm/test/CodeGen/RISCV/stack-realignment-with-variable-sized-objects.ll
+++ b/llvm/test/CodeGen/RISCV/stack-realignment-with-variable-sized-objects.ll
@@ -9,15 +9,15 @@ declare void @callee(i8*, i32*)
define void @caller(i32 %n) {
; RV32I-LABEL: caller:
; RV32I: # %bb.0:
-; RV32I-NEXT:addi sp, sp, -128
-; RV32I-NEXT:.cfi_def_cfa_offset 128
-; RV32I-NEXT:sw ra, 124(sp) # 4-byte Folded Spill
-; RV32I-NEXT:sw s0, 120(sp) # 4-byte Folded Spill
-; RV32I-NEXT:sw s1, 116(sp) # 4-byte Folded Spill
+; RV32I-NEXT:addi sp, sp, -64
+; RV32I-NEXT:.cfi_def_cfa_offset 64
+; RV32I-NEXT:sw ra, 60(sp) # 4-byte Folded Spill
+; RV32I-NEXT:sw s0, 56(sp) # 4-byte Folded Spill
+; RV32I-NEXT:sw s1, 52(sp) # 4-byte Folded Spill
; RV32I-NEXT:.cfi_offset ra, -4
; RV32I-NEXT:.cfi_offset s0, -8
; RV32I-NEXT:.cfi_offset s1, -12
-; RV32I-NEXT:addi s0, sp, 128
+; RV32I-NEXT:addi s0, sp, 64
; RV32I-NEXT:.cfi_def_cfa s0, 0
; RV32I-NEXT:andi sp, sp, -64
; RV32I-NEXT:mv s1, sp
@@ -25,26 +25,26 @@ define void @caller(i32 %n) {
; RV32I-NEXT:andi a0, a0, -16
; RV32I-NEXT:sub a0, sp, a0
; RV32I-NEXT:mv sp, a0
-; RV32I-NEXT:addi a1, s1, 64
+; RV32I-NEXT:mv a1, s1
; RV32I-NEXT:call callee@plt
-; RV32I-NEXT:addi sp, s0, -128
-; RV32I-NEXT:lw s1, 116(sp) # 4-byte Folded Reload
-; RV32I-NEXT:lw s0, 120(sp) # 4-byte Folded Reload
-; RV32I-NEXT:lw ra, 124(sp) # 4-byte Folded Reload
-; RV32I-NEXT:addi sp, sp, 128
+; RV32I-NEXT:addi sp, s0, -64
+; RV32I-NEXT:lw s1, 52(sp) # 4-byte Folded Reload
+; RV32I-NEXT:lw s0, 56(sp) # 4-byte Folded Reload
+; RV32I-NEXT:lw ra, 60(sp) # 4-byte Folded Reload
+; RV32I-NEXT:addi sp, sp, 64
; RV32I-NEXT:ret
;
; RV64I-LABEL: caller:
; RV64I: # %bb.0:
-; RV64I-NEXT:addi sp, sp, -128
-; RV64I-NEXT:.cfi_def_cfa_offset 128
-; RV64I-NEXT:sd ra, 120(sp) # 8-byte Folded Spill
-; RV64I-NEXT:sd s0, 112(sp) # 8-byte Folded Spill
-; RV64I-NEXT:sd s1, 104(sp) # 8-byte Folded Spill
+; RV64I-NEXT:addi sp, sp, -64
+; RV64I-NEXT:.cfi_def_cfa_offset 64
+; RV64I-NEXT:sd ra, 56(sp) # 8-byte Folded Spill
+; RV64I-NEXT:sd s0, 48(sp) # 8-byte Folded Spill
+; RV64I-NEXT:sd s1, 40(sp) # 8-byte Folded Spill
; RV64I-NEXT:.cfi_offset ra, -8
; RV64I-NEXT:.cfi_offset s0, -16
; RV64I-NEXT:.cfi_offset s1, -24
-; RV64I-NEXT:addi s0, sp
