[Apache TVM Discuss] [Questions] [C++ RPC + AOT] Error "Can not pass in local device" when deploying AOT-compiled model via C++ RPC client

[Mark via Apache TVM Discuss]

Hi TVM community,

I'm trying to deploy an **AOT-compiled graph executor model** using the **TVM 
C++ RPC API**, but I keep encountering a runtime error related to device 
handling. Unfortunately, documentation and examples for the **C++ RPC client 
usage** are very limited, so I’m seeking guidance.

#### Environment
- **Platform**: Linux (aarch64)
  ```bash
  Linux lima-ubuntu 6.8.0-87-generic #88-Ubuntu SMP PREEMPT_DYNAMIC Sat Oct 11 
09:16:38 UTC 2025 aarch64 aarch64 aarch64 GNU/Linux
  ```
- **TVM version**: `v0.12.0` (built from source)
- **RPC server**: Running via `./tvm_rpc server --host=0.0.0.0 --port=9000`

#### Model Compilation (Python)
I compiled a simple `add(x, y)` model in **AOT mode** with the C++ runtime:

```python
import tvm
from tvm import relay

x = relay.var("x", shape=(1, 3, 224, 224), dtype="float32")
y = relay.var("y", shape=(1, 3, 224, 224), dtype="float32")
z = relay.add(x, y)
func = relay.Function([x, y], z)

mod = tvm.IRModule.from_expr(func)
target = "llvm"

with tvm.transform.PassContext(opt_level=3):
    lib = relay.build(
        mod,
        target=tvm.target.Target(target),
        params={},
        runtime=relay.backend.Runtime("cpp"),
        executor=relay.backend.Executor("aot"),
    )

lib.export_library("aot_mod.so")
```

#### Error on C++ Client
When running my C++ client, I get the following error:

```
Error: [16:34:29] /path/to/tvm/src/runtime/rpc/rpc_module.cc:153:
---------------------------------------------------------------
An error occurred during the execution of TVM.
For more information, please see: https://tvm.apache.org/docs/errors.html
---------------------------------------------------------------
  Check failed: (IsRPCSessionDevice(dev)) is false: Can not pass in local device
Stack trace:
  0: tvm::runtime::RPCWrappedFunc::RemoveSessMask(DLDevice) const
        at /path/to/tvm/src/runtime/rpc/rpc_module.cc:153
  ...
  5: main
        at /path/to/rpc_deploy.cpp:148
```

#### C++ Client Code (simplified)
```cpp
// Connect to RPC server
auto connect_func = tvm::runtime::Registry::Get("rpc.Connect");
auto remote_session = (*connect_func)(server_ip, server_port, "", false);

// Upload and load module on remote
f_upload("/tmp/mymod.so", mod_data);
mod = f_load_module("/tmp/mymod.so");

// This line triggers the error:
DLDevice dev = {kDLCPU, 0};
tvm::runtime::Module gmod = mod.GetFunction("default")(dev);  // ← ERROR HERE
```

#### Questions
1. When using **AOT + RPC**, should the `DLDevice` passed to `mod["default"]` 
be a **remote device**?  
   But `DLDevice` is just a struct — how do I specify it belongs to the RPC 
session?

2. Is AOT + RPC fully supported in the C++ runtime? Or should I switch to 
**Graph Executor (non-AOT)** for RPC deployments?

Any working C++ RPC + AOT example would be extremely helpful!

Thanks in advance for your support!





---
[Visit 
Topic](https://discuss.tvm.apache.org/t/c-rpc-aot-error-can-not-pass-in-local-device-when-deploying-aot-compiled-model-via-c-rpc-client/18845/1)
 to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click 
here](https://discuss.tvm.apache.org/email/unsubscribe/98f49f32e13caff76e79baf8d2086204d461e4f39a308a78f38875235fb1a5b8).