This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new b27eabf3061c [SPARK-55659][CORE] Improve `EventLogFileWriter` to log
`stop` operation
b27eabf3061c is described below
commit b27eabf3061c7505b7b63c9675a6d88ddd89e5b1
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Tue Feb 24 08:57:05 2026 -0800
[SPARK-55659][CORE] Improve `EventLogFileWriter` to log `stop` operation
### What changes were proposed in this pull request?
This PR aims to improve `EventLogFileWriter` to log `stop` operation.
### Why are the changes needed?
Apache Spark has been logging the start of event log processing. We had
better log the end of event log processing as a pair.
**BEFORE**
```
$ bin/run-example -c spark.eventLog.enabled=true -c spark.eventLog.dir=/tmp
SparkPi 2>&1 | grep RollingEventLogFilesWriter
26/02/24 08:24:27 INFO RollingEventLogFilesWriter: Logging events to
file:/private/tmp/eventlog_v2_local-1771950267185/events_1_local-1771950267185.zstd
```
**AFTER**
```
$ bin/run-example -c spark.eventLog.enabled=true -c spark.eventLog.dir=/tmp
SparkPi 2>&1 | grep RollingEventLogFilesWriter
26/02/24 08:24:41 INFO RollingEventLogFilesWriter: Logging events to
file:/private/tmp/eventlog_v2_local-1771950279197/events_1_local-1771950279197.zstd
26/02/24 08:24:42 INFO RollingEventLogFilesWriter: Stopping event writer
for file:/private/tmp/eventlog_v2_local-1771950279197
```
### Does this PR introduce _any_ user-facing change?
No behavior change because this is a log.
### How was this patch tested?
Manual test.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: `Gemini 3.1 Pro (High)` on `Antigravity`
Closes #54452 from dongjoon-hyun/SPARK-55659.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.../scala/org/apache/spark/deploy/history/EventLogFileWriters.scala | 2 ++
1 file changed, 2 insertions(+)
diff --git
a/core/src/main/scala/org/apache/spark/deploy/history/EventLogFileWriters.scala
b/core/src/main/scala/org/apache/spark/deploy/history/EventLogFileWriters.scala
index 7c022c283db4..601515e57dc8 100644
---
a/core/src/main/scala/org/apache/spark/deploy/history/EventLogFileWriters.scala
+++
b/core/src/main/scala/org/apache/spark/deploy/history/EventLogFileWriters.scala
@@ -250,6 +250,7 @@ class SingleEventLogFileWriter(
* ".inprogress" suffix.
*/
override def stop(): Unit = {
+ logInfo(log"Stopping event writer for ${MDC(PATH, logPath)}")
closeWriter()
renameFile(new Path(inProgressPath), new Path(logPath), shouldOverwrite)
}
@@ -367,6 +368,7 @@ class RollingEventLogFilesWriter(
}
override def stop(): Unit = {
+ logInfo(log"Stopping event writer for ${MDC(PATH, logPath)}")
closeWriter()
val appStatusPathIncomplete = getAppStatusFilePath(logDirForAppPath,
appId, appAttemptId,
inProgress = true)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
