[GitHub] [pinot] Sotatek-HungNguyenThanh commented on issue #11471: Pinot Server crash

[via GitHub]

Sotatek-HungNguyenThanh commented on issue #11471:
URL: https://github.com/apache/pinot/issues/11471#issuecomment-1703979721

   this is my server config
   ```
   pinot.service.role=SERVER
   
   # Pinot Cluster name
   pinot.cluster.name=PinotCluster
   
   # Pinot Zookeeper Server
   pinot.zk.server=172.16.198.43:2181
   
   # Use hostname as Pinot Instance ID other than IP
   pinot.set.instance.id.to.hostname=true
   
   # Pinot Server Netty Port for queris
   pinot.server.netty.port=8098
   
   # Pinot Server Admin API port
   pinot.server.adminapi.port=8097
   
   # # Pinot Server Data Directory
   # pinot.server.instance.dataDir=/opt/pinot/data/server/index
   #
   # # Pinot Server Temporary Segment Tar Directory
   # pinot.server.instance.segmentTarDir=/opt/pinot/data/server/segmentTar
   
   pinot.server.instance.dataDir=/opt/pinot/data/server/index
   pinot.server.instance.segmentTarDir=/opt/pinot/data/server/segmentTar
   
   
pinot.server.storage.factory.class.s3=org.apache.pinot.plugin.filesystem.S3PinotFS
   pinot.server.segment.fetcher.protocols=file,http,s3
   
pinot.server.segment.fetcher.s3.class=org.apache.pinot.common.utils.fetcher.PinotFSSegmentFetcher
   pinot.server.storage.factory.s3.region=us-west-2
   
   pinot.server.segment.store.uri=s3://pinot/data/eth_mainnet
   pinot.server.instance.segment.store.uri=s3://pinot/data
   pinot.server.instance.enable.split.commit=true
   realtime.segment.serverUploadToDeepStore=true
   
   
   
   pinot.controller.storage.factory.s3.accessKey=admin
   pinot.controller.storage.factory.s3.secretKey=Xxxx@2022
   pinot.controller.storage.factory.s3.disableAcl=false
   ```
   
   this is my table schema 
   ```
   {
     "schemaName": "transactions",
     "dimensionFieldSpecs": [
       {
         "name": "timestamp",
         "dataType": "INT"
       },
       {
         "name": "hash",
         "dataType": "STRING",
         "maxLength": 70
       },
       {
         "name": "nonce",
         "dataType": "STRING",
         "maxLength": 50
       },
       {
         "name": "from",
         "dataType": "STRING",
         "maxLength": 50
       },
       {
         "name": "to",
         "dataType": "STRING",
         "maxLength": 50
       },
       {
         "name": "value",
         "dataType": "STRING"
       },
     ],
     "dateTimeFieldSpecs": [
       {
         "name": "block_time",
         "dataType": "TIMESTAMP",
         "format": "1:MILLISECONDS:EPOCH",
         "granularity": "1:MILLISECONDS"
       }
     ],
     "primaryKeyColumns": [
       "hash"
     ]
   }
   ```
   
   I have a 6 pinot server (96g ram) and I only store 1.25B logs. If I need to 
save 25B records, how do I prepare resources? I tried calculating resources 
according to the guide docs but it's not right


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org