Branch: refs/heads/main
Home: https://github.com/WebKit/WebKit
Commit: ac871ed0a668bd6fdd43e8590039b83fb3fd5d3e
https://github.com/WebKit/WebKit/commit/ac871ed0a668bd6fdd43e8590039b83fb3fd5d3e
Author: Kimmo Kinnunen <[email protected]>
Date: 2025-11-25 (Tue, 25 Nov 2025)
Changed paths:
M Source/WebCore/Modules/WebGPU/GPUQueue.cpp
M Source/WebCore/Modules/WebGPU/Implementation/WebGPUQueueImpl.cpp
M Source/WebCore/Modules/WebGPU/Implementation/WebGPUQueueImpl.h
M Source/WebCore/Modules/WebGPU/InternalAPI/WebGPUQueue.h
M Source/WebCore/Modules/mediastream/ImageCapture.cpp
M Source/WebCore/html/canvas/WebGLRenderingContextBase.cpp
M Source/WebCore/platform/MediaStrategy.h
M Source/WebCore/platform/VideoFrame.cpp
M Source/WebCore/platform/VideoFrame.h
M Source/WebCore/platform/graphics/BifurcatedGraphicsContext.cpp
M Source/WebCore/platform/graphics/BifurcatedGraphicsContext.h
M Source/WebCore/platform/graphics/GraphicsContext.cpp
M Source/WebCore/platform/graphics/GraphicsContext.h
M Source/WebCore/platform/graphics/GraphicsContextGL.cpp
M Source/WebCore/platform/graphics/GraphicsContextGL.h
M Source/WebCore/platform/graphics/NullGraphicsContext.h
M Source/WebCore/platform/graphics/cairo/CairoOperationRecorder.cpp
M Source/WebCore/platform/graphics/cairo/CairoOperationRecorder.h
M Source/WebCore/platform/graphics/cv/VideoFrameCV.h
M Source/WebCore/platform/graphics/cv/VideoFrameCV.mm
M Source/WebCore/platform/graphics/displaylists/DisplayListRecorderImpl.cpp
M Source/WebCore/platform/graphics/displaylists/DisplayListRecorderImpl.h
M Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp
M Source/WebCore/platform/graphics/gstreamer/VideoFrameGStreamer.cpp
M Source/WebCore/platform/graphics/gstreamer/VideoFrameGStreamer.h
M Source/WebCore/platform/mediastream/libwebrtc/VideoFrameLibWebRTC.cpp
M Source/WebCore/platform/mediastream/libwebrtc/VideoFrameLibWebRTC.h
M Source/WebKit/WebProcess/GPU/GPUProcessConnection.cpp
M Source/WebKit/WebProcess/GPU/GPUProcessConnection.h
M Source/WebKit/WebProcess/GPU/RemoteSharedResourceCacheProxy.cpp
M Source/WebKit/WebProcess/GPU/RemoteSharedResourceCacheProxy.h
M Source/WebKit/WebProcess/GPU/graphics/RemoteGraphicsContextGLProxy.cpp
M Source/WebKit/WebProcess/GPU/graphics/RemoteGraphicsContextGLProxy.h
M Source/WebKit/WebProcess/GPU/graphics/RemoteGraphicsContextProxy.cpp
M Source/WebKit/WebProcess/GPU/graphics/RemoteGraphicsContextProxy.h
M Source/WebKit/WebProcess/GPU/graphics/WebGPU/RemoteQueueProxy.cpp
M Source/WebKit/WebProcess/GPU/graphics/WebGPU/RemoteQueueProxy.h
M Source/WebKit/WebProcess/GPU/media/AudioVideoRendererRemote.cpp
M Source/WebKit/WebProcess/GPU/media/RemoteMediaPlayerManager.cpp
M Source/WebKit/WebProcess/GPU/media/RemoteVideoFrameProxy.cpp
M Source/WebKit/WebProcess/GPU/media/RemoteVideoFrameProxy.h
M Source/WebKit/WebProcess/GPU/media/WebMediaStrategy.cpp
M Source/WebKit/WebProcess/GPU/media/WebMediaStrategy.h
M Source/WebKit/WebProcess/GPU/media/cocoa/MediaPlayerPrivateRemoteCocoa.mm
M Source/WebKit/WebProcess/GPU/webrtc/RemoteVideoFrameObjectHeapProxy.h
M
Source/WebKit/WebProcess/GPU/webrtc/RemoteVideoFrameObjectHeapProxyProcessor.cpp
M
Source/WebKit/WebProcess/GPU/webrtc/RemoteVideoFrameObjectHeapProxyProcessor.h
M Source/WebKit/WebProcess/WebProcess.cpp
M Source/WebKit/WebProcess/WebProcess.h
Log Message:
-----------
Add a common way to get NativeImage from a VideoFrame
(VideoFrame::copyNativeImage())
https://bugs.webkit.org/show_bug.cg?id=302527
rdar://164713414
Reviewed by Youenn Fablet.
In order to use NativeImage and VideoFrame as the basic building blocks
of WebCore code, add a polymorphic VideoFrame::copyNativeImage() for
converting VideoFrame to a NativeImage.
Previously the conversion was done in ad hoc manner in cases where
it could be done, e.g. where the code had some sort of unrelated context
object such as RemoteMediaPlayerManager, AudioVideoRendererRemote,
WebGL context or WebGPU queue. Those object do not have much to do
with a particular VideoFrame, rather they're the target sinks of
the operations.
Local VideoFrame types, VideoFrameCV, VideoFrameGStreamer,
VideoFrameLibWebRTC have the data and are able to construct the
NativeImage directly. Use the same code as before. Later on, the
construction could be optimized for speed and memory use.
Remote VideoFrame type, RemoteVideoFrameProxy, holds the correct context
object for the source data, the RemoteVideoFrameObjectHeapProxy that
connects the WebContent process side reference to GPU process side
actual object. Use this to construct the NativeImage, same code as
before but invoke it directly.
Fix RemoteVideoFrameObjectHeapProxy instances to preserve across
GPUP restarts, because the RemoteVideoFrameProxy objects themselves
are persistent, held by WebCore object graphs as VideoFrame.
Later on, the holding functionality is will be moved to
RemoteSharedResourceCacheProxy. This will allow reference-only
conversion, e.g. getting RemoteNativeImageProxy out of
RemoteVideoFrameProxy.
This is work towards being able to use NativeImage as the basic
abstraction of image draw source, while not needing to transfer
the data between WCP and GPUP.
* Source/WebCore/Modules/WebGPU/GPUQueue.cpp:
(WebCore::getImageBytesFromVideoFrame):
(WebCore::imageBytesForSource):
* Source/WebCore/Modules/WebGPU/Implementation/WebGPUQueueImpl.cpp:
(WebCore::WebGPU::QueueImpl::getNativeImage): Deleted.
* Source/WebCore/Modules/WebGPU/Implementation/WebGPUQueueImpl.h:
* Source/WebCore/Modules/WebGPU/InternalAPI/WebGPUQueue.h:
* Source/WebCore/Modules/mediastream/ImageCapture.cpp:
(WebCore::createImageBitmap):
(WebCore::createImageBitmapViaDrawing): Deleted.
(WebCore::createImageBitmapFromNativeImage): Deleted.
* Source/WebCore/html/canvas/WebGLRenderingContextBase.cpp:
(WebCore::WebGLRenderingContextBase::texImageSource):
* Source/WebCore/platform/MediaStrategy.h:
(WebCore::MediaStrategy::nativeImageFromVideoFrame): Deleted.
* Source/WebCore/platform/VideoFrame.h:
* Source/WebCore/platform/graphics/BifurcatedGraphicsContext.cpp:
(WebCore::BifurcatedGraphicsContext::drawVideoFrame):
* Source/WebCore/platform/graphics/BifurcatedGraphicsContext.h:
* Source/WebCore/platform/graphics/GraphicsContext.cpp:
(WebCore::GraphicsContext::drawVideoFrame):
* Source/WebCore/platform/graphics/GraphicsContext.h:
* Source/WebCore/platform/graphics/GraphicsContextGL.cpp:
(WebCore::GraphicsContextGL::videoFrameToImage): Deleted.
* Source/WebCore/platform/graphics/GraphicsContextGL.h:
* Source/WebCore/platform/graphics/NullGraphicsContext.h:
* Source/WebCore/platform/graphics/cairo/CairoOperationRecorder.cpp:
(WebCore::Cairo::OperationRecorder::drawVideoFrame):
* Source/WebCore/platform/graphics/cairo/CairoOperationRecorder.h:
* Source/WebCore/platform/graphics/cv/VideoFrameCV.h:
* Source/WebCore/platform/graphics/cv/VideoFrameCV.mm:
(WebCore::VideoFrameCV::copyNativeImage const):
(WebCore::VideoFrame::draw): Deleted.
* Source/WebCore/platform/graphics/displaylists/DisplayListRecorderImpl.cpp:
(WebCore::DisplayList::RecorderImpl::drawVideoFrame):
* Source/WebCore/platform/graphics/displaylists/DisplayListRecorderImpl.h:
* Source/WebCore/platform/graphics/gstreamer/VideoFrameGStreamer.cpp:
(WebCore::VideoFrame::copyNativeImage const):
(WebCore::VideoFrame::draw): Deleted.
* Source/WebCore/platform/graphics/gstreamer/VideoFrameGStreamer.h:
* Source/WebCore/platform/mediastream/libwebrtc/VideoFrameLibWebRTC.cpp:
(WebCore::VideoFrameLibWebRTC::copyNativeImage const):
* Source/WebCore/platform/mediastream/libwebrtc/VideoFrameLibWebRTC.h:
* Source/WebKit/WebProcess/GPU/GPUProcessConnection.cpp:
(WebKit::GPUProcessConnection::sharedResourceCache):
(WebKit::GPUProcessConnection::videoFrameObjectHeapProxy):
(WebKit::GPUProcessConnection::protectedVideoFrameObjectHeapProxy):
* Source/WebKit/WebProcess/GPU/GPUProcessConnection.h:
* Source/WebKit/WebProcess/GPU/RemoteSharedResourceCacheProxy.cpp:
(WebKit::RemoteSharedResourceCacheProxy::gpuProcessConnectionDidBecomeAvailable):
(WebKit::RemoteSharedResourceCacheProxy::videoFrameObjectHeapProxy):
* Source/WebKit/WebProcess/GPU/RemoteSharedResourceCacheProxy.h:
* Source/WebKit/WebProcess/GPU/graphics/RemoteGraphicsContextGLProxy.cpp:
(WebKit::RemoteGraphicsContextGLProxy::videoFrameToImage): Deleted.
* Source/WebKit/WebProcess/GPU/graphics/RemoteGraphicsContextGLProxy.h:
* Source/WebKit/WebProcess/GPU/graphics/RemoteGraphicsContextProxy.cpp:
(WebKit::RemoteGraphicsContextProxy::drawVideoFrame):
* Source/WebKit/WebProcess/GPU/graphics/RemoteGraphicsContextProxy.h:
* Source/WebKit/WebProcess/GPU/graphics/WebGPU/RemoteQueueProxy.cpp:
(WebKit::WebGPU::RemoteQueueProxy::RemoteQueueProxy):
(WebKit::WebGPU::RemoteQueueProxy::getNativeImage): Deleted.
(WebKit::WebGPU::RemoteQueueProxy::protectedVideoFrameObjectHeapProxy const):
Deleted.
* Source/WebKit/WebProcess/GPU/graphics/WebGPU/RemoteQueueProxy.h:
* Source/WebKit/WebProcess/GPU/media/AudioVideoRendererRemote.cpp:
(WebKit::AudioVideoRendererRemote::currentNativeImage const):
* Source/WebKit/WebProcess/GPU/media/RemoteMediaPlayerManager.cpp:
(WebKit::RemoteMediaPlayerManager::setUseGPUProcess):
* Source/WebKit/WebProcess/GPU/media/RemoteVideoFrameProxy.cpp:
(WebKit::RemoteVideoFrameProxy::RemoteVideoFrameProxy):
(WebKit::m_videoFrameObjectHeapProxy):
(WebKit::RemoteVideoFrameProxy::copyNativeImage const):
(WebKit::RemoteVideoFrameProxy::pixelBuffer const):
(WebKit::m_pixelFormat): Deleted.
* Source/WebKit/WebProcess/GPU/media/RemoteVideoFrameProxy.h:
* Source/WebKit/WebProcess/GPU/media/WebMediaStrategy.cpp:
(WebKit::WebMediaStrategy::nativeImageFromVideoFrame): Deleted.
* Source/WebKit/WebProcess/GPU/media/WebMediaStrategy.h:
* Source/WebKit/WebProcess/GPU/media/cocoa/MediaPlayerPrivateRemoteCocoa.mm:
(WebKit::MediaPlayerPrivateRemote::nativeImageForCurrentTime):
* Source/WebKit/WebProcess/GPU/webrtc/RemoteVideoFrameObjectHeapProxy.h:
(WebKit::RemoteVideoFrameObjectHeapProxy::create):
(WebKit::RemoteVideoFrameObjectHeapProxy::RemoteVideoFrameObjectHeapProxy):
(WebKit::RemoteVideoFrameObjectHeapProxy::gpuProcessConnectionDidBecomeAvailable):
*
Source/WebKit/WebProcess/GPU/webrtc/RemoteVideoFrameObjectHeapProxyProcessor.cpp:
(WebKit::RemoteVideoFrameObjectHeapProxyProcessor::create):
(WebKit::RemoteVideoFrameObjectHeapProxyProcessor::RemoteVideoFrameObjectHeapProxyProcessor):
(WebKit::RemoteVideoFrameObjectHeapProxyProcessor::gpuProcessConnectionDidBecomeAvailable):
(WebKit::RemoteVideoFrameObjectHeapProxyProcessor::gpuProcessConnectionDidClose):
(WebKit::RemoteVideoFrameObjectHeapProxyProcessor::connection):
(WebKit::RemoteVideoFrameObjectHeapProxyProcessor::getNativeImage):
(WebKit::RemoteVideoFrameObjectHeapProxyProcessor::initialize): Deleted.
*
Source/WebKit/WebProcess/GPU/webrtc/RemoteVideoFrameObjectHeapProxyProcessor.h:
* Source/WebKit/WebProcess/WebProcess.cpp:
(WebKit::WebProcess::ensureGPUProcessConnection):
(WebKit::WebProcess::gpuProcessSharedResourceCache):
* Source/WebKit/WebProcess/WebProcess.h:
Canonical link: https://commits.webkit.org/303533@main
To unsubscribe from these emails, change your notification settings at
https://github.com/WebKit/WebKit/settings/notifications
