[GitHub] [pinot] gortiz opened a new issue, #10675: Failures in RealtimeQuickStart when indexes are changed

[via GitHub]

gortiz opened a new issue, #10675:
URL: https://github.com/apache/pinot/issues/10675

   RealtimeQuickStart seems to fail when indexes are changed (either created, 
updated or removed).
   
   In order to reproduce it:
   * Start RealtimeQuickStart (or RealtimeQuickStartWithMinion, both seem to 
use minion anyway)
   * Edit the table `githubEvents` and either remove the included inverted 
index or add a new one.
   * Reload all segments using the UI button
   * Wait until the reload finishes.
   * Check the indexes in the UI.
   
   The behavior is not 100% consistent. Sometimes it is needed to retry.
   
   I've been abled to replicate this with both 
master(866c796bd56cf846b654f29f024f3e610557b2c7) and with release 
`release-0.12.1`. When done in master, the following log is printed:
   
   ```
   java.lang.IllegalStateException: Failed to find table config for table: 
githubEvents_OFFLINE
        at 
com.google.common.base.Preconditions.checkState(Preconditions.java:518) 
~[guava-20.0.jar:?]
        at 
org.apache.pinot.plugin.minion.tasks.BaseTaskExecutor.getTableConfig(BaseTaskExecutor.java:51)
 ~[classes/:?]
        at 
org.apache.pinot.plugin.minion.tasks.realtimetoofflinesegments.RealtimeToOfflineSegmentsTaskExecutor.convert(RealtimeToOfflineSegmentsTaskExecutor.java:124)
 ~[classes/:?]
        at 
org.apache.pinot.plugin.minion.tasks.BaseMultipleSegmentsConversionExecutor.executeTask(BaseMultipleSegmentsConversionExecutor.java:217)
 ~[classes/:?]
        at 
org.apache.pinot.plugin.minion.tasks.BaseMultipleSegmentsConversionExecutor.executeTask(BaseMultipleSegmentsConversionExecutor.java:77)
 ~[classes/:?]
        at 
org.apache.pinot.minion.taskfactory.TaskFactoryRegistry$1.runInternal(TaskFactoryRegistry.java:157)
 [classes/:?]
        at 
org.apache.pinot.minion.taskfactory.TaskFactoryRegistry$1.run(TaskFactoryRegistry.java:118)
 [classes/:?]
        at org.apache.helix.task.TaskRunner.run(TaskRunner.java:75) 
[helix-core-1.0.4.jar:1.0.4]
        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
        at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:264) 
[?:?]
        at java.util.concurrent.FutureTask.run(FutureTask.java) [?:?]
        at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
 [?:?]
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) 
[?:?]
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) 
[?:?]
        at java.lang.Thread.run(Thread.java:829) [?:?]
   ```
   
   This log is not printed in 0.12.1.
   
   Before change:
   <img width="1300" alt="Screenshot 2023-04-24 at 09 58 59" 
src="https://user-images.githubusercontent.com/1913993/233934833-a14ab338-7ebd-42fe-a1f3-14d5bbfafa5c.png";>
   
   After inverted index is removed:
   <img width="1332" alt="Screenshot 2023-04-24 at 10 00 16" 
src="https://user-images.githubusercontent.com/1913993/233935092-2ad7c6a1-eb96-4c00-8e56-5db8e7527e5f.png";>
   
   Also all older segments get deleted and new ones are created. The backup of 
the old ones are still indexed, while the new ones are not.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org