Nanmur opened a new issue, #20007:
URL: https://github.com/apache/tvm/issues/20007
### Expected behavior
Embedding `float32` parameter values as Relax constants, or binding
equivalent values with `Function.bind_params`, should preserve the numerical
result of a Relax function compiled for LLVM.
Passing the same values as explicit VM arguments should be numerically
equivalent to the embedded-constant form.
### Actual behavior
On Windows, a pure Relax program produces an incorrect result when its
weight and twelve broadcast bias vectors are embedded as `relax.const` values.
The same program is correct when those values are passed as explicit VM
arguments.
For the minimal reproducer below, the embedded-constant path produces
`-2.010380268096924`, while the NumPy reference is `-2.289207935333252`.
The explicit-parameter path has a maximum absolute difference of `0` for the
same input values.
The minimal reproducer does not involve a model frontend.
### Environment
- OS: Windows 11
- Python: 3.11.14
- TVM: 0.25.0
- Build: local Release build with LLVM `22.1.8`, target `llvm`
- Device: CPU
### Steps to reproduce
Save the following as `repro_relax_const.py` and run it with the Python
environment that imports the matching TVM source and compiled libraries.
```python
import numpy as np
import tvm
from tvm import relax
from tvm.runtime import tensor as tvm_tensor
def run(embed_as_constants):
rng = np.random.default_rng(0)
x_value = rng.normal(size=(1, 1)).astype("float32")
weight_value = rng.normal(size=(1, 1)).astype("float32")
bias_values = [rng.normal(size=(1,)).astype("float32") for _ in
range(12)]
expected = x_value @ weight_value
for bias_value in bias_values:
expected = expected + bias_value
sinfo_x = relax.TensorStructInfo((1, 1), "float32")
sinfo_w = relax.TensorStructInfo((1, 1), "float32")
sinfo_b = relax.TensorStructInfo((1,), "float32")
x = relax.Var("x", sinfo_x)
weight = relax.Var("weight", sinfo_w)
biases = [relax.Var(f"bias_{i}", sinfo_b) for i in range(12)]
if embed_as_constants:
weight_expr = relax.const(weight_value)
bias_exprs = [relax.const(value) for value in bias_values]
function_params = [x]
else:
weight_expr = weight
bias_exprs = biases
function_params = [x, weight, *biases]
bb = relax.BlockBuilder()
with bb.function("main", function_params):
with bb.dataflow():
result = bb.emit(relax.op.matmul(x, weight_expr))
for bias_expr in bias_exprs:
result = bb.emit(relax.op.add(result, bias_expr))
output = bb.emit_output(result)
bb.emit_func_output(output)
mod = bb.finalize()
executable = relax.build(mod, target="llvm")
vm = relax.VirtualMachine(executable, tvm.cpu(0))
arguments = [tvm_tensor(x_value, tvm.cpu(0))]
if not embed_as_constants:
arguments.extend([tvm_tensor(weight_value, tvm.cpu(0))])
arguments.extend(tvm_tensor(value, tvm.cpu(0)) for value in
bias_values)
actual = vm["main"](*arguments).numpy()
print(f"embed_as_constants={embed_as_constants}")
print("expected:", expected)
print("actual: ", actual)
np.testing.assert_allclose(actual, expected, rtol=1e-5, atol=1e-5)
run(False) # Passes: maximum absolute difference is 0.
run(True) # Fails: maximum absolute difference is about 0.27882767.
```
`run(False)` passes.
`run(True)` fails with:
```text
ACTUAL: [[-2.0103803]]
DESIRED: [[-2.289208]]
Max absolute difference: 0.27882767
```
I also reproduced the same parameter-mode distinction with an ONNX
DenseNet-121 model on the same Windows environment.
`from_onnx(..., keep_params_in_input=False)` produced a maximum absolute
difference of `0.848955` against ONNX Runtime, while
`keep_params_in_input=True` passed with a maximum absolute difference of
`3.58e-06`.
This ONNX result is supporting evidence only; the pure Relax script above is
the intended minimal reproducer.
### Triage
* needs-triage
* type: bug
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
