Contact emails [email protected], [email protected], [email protected]
Explainer https://github.com/palak8669/webrtc-encoded-transform/blob/create-encoded-explainer/create-encoded-explainer.md Specification https://github.com/w3c/webrtc-encoded-transform/pull/223/files Original I2E https://groups.google.com/a/chromium.org/g/blink-dev/c/yqwA-jXv6VY/m/fCkBkclxAAAJ Summary Add a feature to the WebRTC Encoded Transform API that allows manipulating audio and video frame metadata. Background: A number of use cases have been identified that require the manipulation of the metadata of WebRTC encoded frames without decoding them first. These include: - Receiving data in encoded form and forwarding it. - Sending data that has been encoded previously - Sending data that has been received in encoded form In particular, we want to support the use case of glitch-free forwarding of media coming from multiple redundant peer connections that provide the same media payloads but with different metadata. Issue links: <https://github.com/w3c/webrtc-nv-use-cases/issues/77> https://github.com/w3c/webrtc-nv-use-cases/issues/77 https://github.com/w3c/webrtc-nv-use-cases/issues/122 https://github.com/w3c/webrtc-encoded-transform/issues/211 Blink component Blink>WebRTC <https://bugs.chromium.org/p/chromium/issues/list?q=component:Blink%3EWebRTC> TAG review The original full spec was reviewed by TAG here: https://github.com/w3ctag/design-reviews/issues/531 No TAG review has been requested for this incremental change, since it is a small addition and its API surface might change as a result of feedback from the origin trial and ongoing discussions in the WebRTC working group. TAG review status Issues addressed for the original spec. Not yet requested for the incremental change. Chromium Trial Name RTCEncodedFrameSetMetadata WebFeature UseCounter name V8RTCEncodedVideoFrame_SetMetadata_Method Link to origin trial feedback summary https://docs.google.com/document/d/1oCgXj0bEhJS6WeJ0T-gV5AVhSaEeVixssqm02m4eU9Q/r/0-f4cSv_CIFwIpVPcIKb89TQ/edit?tab=t.0 Risks Interoperability and Compatibility Interoperability risk: There is the risk that other browsers will not implement this feature. We are mitigating this risk by pursuing consensus in the W3C WebRTC Working Group so that it becomes part of the WebRTC encoded transform spec supported by all browsers. Compatibility risk: This is a new feature intended to support a new use case. It introduces no breaking changes, so we do not expect any compatibility issues. Gecko: No signal requested yet. Positive initial feedback during WG discussions. WebKit: No signal requested yet. Positive initial feedback during WG discussions. Web developers: Positive Other signals: Ergonomics This feature is an extension to WebRTC Encoded Transform, which itself is an extension to WebRTC/RTCPeerConnection. Activation N/A Security No new security risks identified. WebView application risks None Reason this experiment is being extended We have received positive developer feedback and have validated that the proposal supports the intended use case, but the API requires some changes in order to achieve consensus in the W3C WebRTC Working Group. This is the current status: 1. There is consensus to support the use case, which is already part of the official "WebRTC Extended Use Cases" document from the WG. See requirement N39 in https://www.w3.org/TR/webrtc-nv-use-cases/#auction 2. There is consensus to develop a spec proposal that includes changes to the API surface subject to this origin trial. More specifically, the initial consensus is to remove the setMetadata method in frames (as originally proposed) and instead introduce a constructor for creating new frames given an existing frame and new metadata, following a pattern already established by WebCodecs. In addition, the WG agreed to the introduction of a class that allows attaching a source of encoded frames to the sender of an RTCRtpPeerConnection. See https://github.com/w3c/webrtc-encoded-transform/issues/211#issuecomment-1853419603 and https://www.w3.org/2023/12/12-webrtc-minutes.html#t06. From the WG's point of view, both changes are seen as parts of the same package (i.e., they will be looked at together, not separately). In this Intent to Extend Experiment, we are requesting to extend the origin trial for 3 milestones (124 to 126 inclusive). During this period we'll introduce the updated frame API (i.e., the constructor) and will temporarily keep the setMetadata method to allow developers to migrate to the new API. The encoded frame source will be pursued by a separate intent. Ongoing technical constraints None. Debuggability N/A Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, ChromeOS, Android, and Android WebView)? Yes Is this feature fully tested by web-platform-tests <https://chromium.googlesource.com/chromium/src/+/main/docs/testing/web_platform_tests.md> ? Currently tested using unit and Web tests. Web tests will be moved to WPT once the final API shape is accepted by the WebRTC WG. Flag name on chrome://flags None Finch feature name RTCEncodedFrameSetMetadata Requires code in //chrome? False Tracking bug https://crbug.com/1393964 Estimated milestones OriginTrial desktop last 126 OriginTrial desktop first 118 OriginTrial Android last 126 OriginTrial Android first 118 OriginTrial webView last 126 OriginTrial webView first 118 Link to entry on the Chrome Platform Status https://chromestatus.com/feature/5116073827893248 Links to previous Intent discussions Intent to prototype: https://groups.google.com/a/chromium.org/g/blink-dev/c/x2ZACgXrqp0 Intent to experiment: https://groups.google.com/a/chromium.org/g/blink-dev/c/yqwA-jXv6VY/m/fCkBkclxAAAJ This intent message was generated by Chrome Platform Status <https://chromestatus.com/>. -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CA%2BBuZxY7yNLBte5CVtk_%3DZfwpZnTn%3DWPCWbt%3DGCn4wWXCr2wtQ%40mail.gmail.com.
