chisheun-snsoft opened a new issue, #12283:
URL: https://github.com/apache/apisix/issues/12283
### Current Behavior
After checking on the apisix logs and dashboard, the issue had been found
that the apisix will not update to get the latest config. The restarted pods
with new ip addresses were not in the registered upstream.
example:
pods current IP addresses:
10.62.9.32
10.62.45.130
10.62.13.39
apisix logs IP addresses:
10.62.32.40
10.62.41.229
10.62.40.40
### Expected Behavior
the ip addresses should be aligned with the the ip addresses showing in the
logs also align with the ip addresses registered in upstream.
### Error Logs
checking from the ingress controller logs. Found the error logs of
2025-06-04 11:52:36.782 | 2025-06-04T11:52:36+08:00 error
apisix/stream_route.go:95 failed to list stream_routes: function disabled
|
-- | -- | --
| | 2025-06-04 11:52:34.717 | 2025-06-04T11:52:34+08:00 error
apisix/cluster.go:313 failed to list stream_routes in APISIX: function
disabled |
| | 2025-06-04 11:52:34.717 | 2025-06-04T11:52:34+08:00 error
apisix/stream_route.go:95 failed to list stream_routes: function disabled
|
| | 2025-06-04 11:52:32.567 | 2025-06-04T11:52:32+08:00 error
apisix/cluster.go:313 failed to list stream_routes in APISIX: function
disabled |
| | 2025-06-04 11:52:32.567 | 2025-06-04T11:52:32+08:00 error
apisix/stream_route.go:95 failed to list stream_routes: function disabled
|
| | 2025-06-04 11:52:30.297 | 2025-06-04T11:52:30+08:00 error
apisix/cluster.go:313 failed to list stream_routes in APISIX: function
disabled |
| | 2025-06-04 11:52:30.297 | 2025-06-04T11:52:30+08:00 error
apisix/stream_route.go:95 failed to list stream_routes: function disabled
|
| | 2025-06-04 11:52:28.197 | 2025-06-04T11:52:28+08:00 error
apisix/cluster.go:313 failed to list stream_routes in APISIX: function
disabled |
| | 2025-06-04 11:52:28.197 | 2025-06-04T11:52:28+08:00 error
apisix/stream_route.go:95 failed to list stream_routes: function disabled
checking from the etcd logs. Found the warning logs of
2025-06-04 15:31:13.364 |
{"level":"warn","ts":"2025-06-04T07:31:13.364214Z","caller":"etcdserver/raft.go:416","msg":"leader
failed to send out heartbeat on time; took too long, leader is overloaded
likely from slow
disk","to":"39e3f227d2d33a6d","heartbeat-interval":"100ms","expected-duration":"200ms","exceeded-duration":"56.00703ms"}
|
-- | -- | --
| | 2025-06-04 15:31:13.312 |
{"level":"warn","ts":"2025-06-04T07:31:13.309428Z","caller":"etcdserver/raft.go:416","msg":"leader
failed to send out heartbeat on time; took too long, leader is overloaded
likely from slow
disk","to":"e2571e968b89c849","heartbeat-interval":"100ms","expected-duration":"200ms","exceeded-duration":"1.222388ms"
### Steps to Reproduce
1. restart pods and check on the new pods ip addresses
2. check the ip addresses of the specific route from the pods on the logs
from the apisix pods
3. check on the registered ip addresses in the apisix dashboard
### Environment
- APISIX version (run `apisix version`): 3.12.0-debian
- Operating system (run `uname -a`): Linux apisix-679c58fc9f-fh6bf
5.10.236-228.935.amzn2.x86_64 #1 SMP Tue May 6 04:10:50 UTC 2025 x86_64
GNU/Linux
- OpenResty / Nginx version (run `openresty -V` or `nginx -V`):
openresty/1.27.1.1
- etcd version, if relevant (run `curl
http://127.0.0.1:9090/v1/server_info`): N/A
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
