rlankfo opened a new pull request, #19004:
URL: https://github.com/apache/pinot/pull/19004

   ## Problem
   
   The docker-compose demo under `docker/images/pinot` breaks after a `docker 
compose down && up`. Two independent root causes:
   
   1. **Instance IDs are derived from the container IP.** No host is passed to 
the start commands, so each service picks up its container IP for its Helix 
instance ID. On restart the containers get new IPs, so every service registers 
under a *new* instance ID and existing segment assignments are orphaned on the 
now-dead instances. Symptom: segments report as unavailable and the 
ExternalView is empty even though the ideal state looks fine.
   
   2. **The data bind mounts point at container paths the processes don't 
use.** The controller actually writes to `/tmp/data/PinotController`, the 
server to `/tmp/data/pinotServerData` + `/tmp/data/pinotSegments`, and Kafka to 
`/tmp/kraft-combined-logs` — none of which are the mounted volumes. So all data 
lives on the ephemeral container layer and is lost on `down`. Only the 
ZooKeeper mount was correct.
   
   The combination means even the cluster metadata that *does* survive (in 
ZooKeeper) points at instances and segment data that no longer exist after a 
restart.
   
   ## Fix
   
   - Pass `-controllerHost` / `-brokerHost` / `-serverHost` / `-minionHost` so 
instance IDs are tied to the stable compose `hostname:` values instead of the 
container IP.
   - Align each process's data directory to its mounted volume via `-dataDir` / 
`-segmentDir` (controller, server) and `KAFKA_LOG_DIRS` (Kafka), so segment 
data, the deep store, and Kafka logs persist across restarts.
   
   ## Verification
   
   With the patched compose:
   
   1. Created a realtime `transcript` table, produced records, and 
force-committed a segment.
   2. Confirmed the committed segment and Kafka logs land in the mounted host 
directories (previously empty).
   3. Ran `docker compose down && docker compose up -d` **without** wiping 
ZooKeeper.
   
   Result: all instances came back with identical stable IDs 
(`Server_pinot-server_8098`, etc.), the committed segment returned to `ONLINE` 
on the same server, and queries returned the expected rows (`SELECT count(*)` → 
12, no exceptions). Before the fix, the same restart left the segment 
orphaned/`ERROR` and unqueryable.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to