estebanz01 commented on issue #12547:
URL: https://github.com/apache/pinot/issues/12547#issuecomment-2013198202
OK, so while reading #12458 I noticed that my S3 bucket were empty and I
found surprising that minions need S3 to work, so I went to look at the
configuration for the controller and I found out that if I specify the property
`controller.data.dir` twice, it merges both values instead of overriding them 🙃
so now I have data in my S3 bucket, but now the controller is giving the
following error:
```java
pinot-controller-0 controller INFO: [HttpServer] Started.
pinot-controller-0 controller 17:43:02.118 [grizzly-http-server-3] ERROR
org.apache.pinot.controller.util.CompletionServiceHelper - Server: null
returned error: 404
pinot-controller-0 controller 17:43:02.123 [grizzly-http-server-3] ERROR
org.apache.pinot.controller.util.CompletionServiceHelper - Server: null
returned error: 404
pinot-controller-0 controller 17:43:02.125 [grizzly-http-server-3] ERROR
org.apache.pinot.controller.util.CompletionServiceHelper - Connection error.
Details: java.net.UnknownHostException: Controller_null: Name or service not
known
pinot-controller-0 controller 17:56:00.608 [grizzly-http-server-3] ERROR
org.apache.pinot.controller.util.CompletionServiceHelper - Server: null
returned error: 404
pinot-controller-0 controller 17:56:00.610 [grizzly-http-server-3] ERROR
org.apache.pinot.controller.util.CompletionServiceHelper - Server: null
returned error: 404
pinot-controller-0 controller 17:56:00.611 [grizzly-http-server-3] ERROR
org.apache.pinot.controller.util.CompletionServiceHelper - Connection error.
Details: java.net.UnknownHostException: Controller_null: Name or service not
known
```
here's the task information, according to the UI:
<details>
<summary>Task config:</summary>
```json
{
"tableName": "data_counting_REALTIME",
"configs": {
"maxNumRecordsPerSegment": "100000",
"mergeType": "rollup",
"downloadURL":
"http://pinot-controller:9000/segments/data_counting_temp/data_counting_temp__0__50__20240306T0440Z";,
"bufferTimePeriod": "2h",
"push.mode": "TAR",
"windowStartMs": "1709730000000",
"segmentName": "data_count__0__50__20240306T0440Z",
"tableName": "data_counting_REALTIME",
"collectorType": "rollup",
"schedule": "0 0/5 * * * ?",
"uploadURL": "http://pinot-controller:9000/segments";,
"push.controllerUri": "http://pinot-controller:9000";,
"__key.aggregationType": "min",
"bucketTimePeriod": "1h",
"windowEndMs": "1709733600000",
"TASK_ID":
"Task_RealtimeToOfflineSegmentsTask_4e81b60e-021b-4ba7-8b4c-03fd8f968d1b_1711033800254_0"
},
"taskId":
"Task_RealtimeToOfflineSegmentsTask_4e81b60e-021b-4ba7-8b4c-03fd8f968d1b_1711033800254_0",
"taskType": "RealtimeToOfflineSegmentsTask"
}
```
</details>
and here's the configmap that the pinot controller pods are using:
```yaml
Name: pinot-controller-config
Namespace: pinot
Labels: app.kubernetes.io/managed-by=Helm
Annotations: meta.helm.sh/release-name: pinot
meta.helm.sh/release-namespace: pinot
Data
====
pinot-controller.conf:
----
controller.helix.cluster.name=pinot
controller.port=9000
controller.vip.host=pinot-controller
controller.vip.port=9000
controller.data.dir=s3://<bucket-name>/pinot-data/pinot/controller-data
controller.zk.str=pinot-zookeeper:2181
pinot.set.instance.id.to.hostname=true
controller.task.scheduler.enabled=true
controller.local.temp.dir=/var/pinot/controller/data
pinot.controller.storage.factory.class.s3=org.apache.pinot.plugin.filesystem.S3PinotFS
pinot.controller.storage.factory.s3.region=eu-west-1
pinot.controller.segment.fetcher.protocols=file,http,s3
pinot.controller.segment.fetcher.s3.class=org.apache.pinot.common.utils.fetcher.PinotFSSegmentFetcher
pinot.controller.storage.factory.s3.disableAcl=false
BinaryData
====
Events: <none>
```
from what I understand, the controller is trying to fetch segments from
either a `Null` hostname or an invalid one. But the hosts are correct or
appears to be correct.
Any ideas on how to make it work after this progress?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
