morningman commented on code in PR #68713: URL: https://github.com/apache/doris/pull/68713#discussion_r4488133197
########## be/src/common/config.cpp: ########## @@ -1922,6 +1922,32 @@ DEFINE_mDouble(max_hdfs_wirter_jni_heap_usage_ratio, "0.5"); DEFINE_mInt64(hdfs_jni_write_sleep_milliseconds, "300"); // The max retry times when hdfs write failed DEFINE_mInt64(hdfs_jni_write_max_retry_time, "3"); +// The share of the JVM's maximum heap that the JNI scanners admitted by the heap they declare may +// declare together. Only statements with enable_jni_heap_admission declare it +// (util/jni_scan_heap_gate.h) +DEFINE_mDouble(jni_scanner_heap_budget_ratio, "0.5"); Review Comment: Fixed in 863628ec5e9. Both configs now have a `DEFINE_Validator`, like `inverted_index_candidate_pushdown_ratio` and `lookup_connection_cache_block_pool_size`, whose atomic copies these two already follow: - `jni_scanner_heap_budget_ratio` takes a finite number from 0 to 1. `jvm_heap_budget()` no longer sees NaN or infinity, and the budget can no longer exceed the heap. At 0 the declaring readers open one at a time, unless they wait out the limit. - `jni_scanner_heap_max_wait_ms` takes 0 to `INT64_MAX / 10^6` ms, the most `_poll()` can count in nanoseconds, so `ms * 1000 * 1000` cannot overflow. `_poll()` compares the time already waited (`now - start`) with that limit and computes no deadline, so nothing else can wrap. `set_config()` runs a validator before it persists the value or calls the update callback, so a refused value puts the old one back and never reaches the copy the gate reads. A refused value in be.conf fails `config::init()`, as for every other validated config. `JniScanHeapGateTest.SettingsTheGateCannotUseAreRefused` sets both ends of each range, then has `set_config()` refuse `nan`, `inf`, `-0.1` and `1.5` for the ratio and `-1` and `9223372036855` for the wait, and checks that the gate still reads the last values taken. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
