Hi Yalda, Ok, this is great. Now we have a concrete list of items that we can discuss in detail prior to implementation. See notes inline below.
On Fri, Jan 30, 2026 at 7:41 PM Yalda <[email protected]> wrote: > Refactoring: > - Migrate the MPEG-2 user data DVD/DISH/SCTE/etc to A53 translator > code, and any similar conversion if it exists to CodedBitstream (CBS) > API. > Outcome: Processing of bitstream wrapping is no longer crammed in > MPEG-2 encoder, and with CBS API possibility of encoding the wrapping > in the future is more clear. > Limit: Do not alter algorithms in such a way that causes difference in > input/output especially output to A/53 part 4 (the current > intermediate in ffmpeg). > Benefit: In this way, this parsing logic is made independent (the > formats are respected as actual coding formats) and we can test it > independently as well as open the door to cross-translation between > formats. No objection here, and another benefit you don't mention is that it would allow manipulation of the caption data without the cost and quality loss of video transcoding. > - Identify common functions if any (parity checks, etc.) and move to > shared utility file > Outcome: Inventory common functions especially those that are > stateless and centralize them > Benefit: Remove redundancies Ok. Let's be sure that wherever the functions end up living that they're available to both avcodec as well as avformat (since we have to deal with captions both as side data on video frames as well as in separate codec streams such as MCC/SCC and 608 streams in MP4). I did some work in this area when I did the ccrepack routines, and was happy with that approach because I didn't do it in libavutil where I would be locked into the ABI for eternity. > API cleanup: > - Create AV_CODEC_ID_CEA_708 and AV_CODEC_ID_WRAPPED_CC , and update > current AV_CODEC_ID_EIA_608 references to AV_CODEC_ID_WRAPPED_CC > Justification: Right now, all CC bitstream in FFmpeg is classified as > EIA-608 but this is not really what is being passed around. Also it > does not consider CEA-708. > 1, CEA-708 is its own codec and if we ever support it we should treat it as > one. > 2, Within FFmpeg, we carry CC bitstream that can contain both but are > wrapped in A53 format and can contain 608 or 708. So we should > identify that accurately as "wrapped CC". I am in favor of having a new ID_CEA_708 codec, but would strongly suggest not introducing a new codec for wrapped CC. Just have the 608 and 708 codecs. In reality wrapped CC is just the 608 compatibility bytes within a 708 stream and I think it makes it harder to treat it as a separate codec (i.e. codecs and avformat/avdevice modules would have to support three codecs instead of two). Also, would be worthwhile to have a BSF which converts a 608 to a 708 stream (just embedding the 608 compatibility bytes into a 708 stream, not actually doing what in the industry is referred to as "up conversion"). This would allow some formats which support 708 to just do auto_bsf to perform the conversion automatically. It would be useful if this work included changes to existing modules to announce the correct format (e.g. MCC should announce its codec as 708 rather than 608, MP4 should properly announce both streams if present). > ccaption_dec fixes: > - Fix odd naming inconsistency and name it for what it actually is: an > EIA-608 decoder (eia608dec). > Justification: File is "ccaption_dec", class is "cc_dec", description > suggests CEA-708 support (not true, this data is skipped). > Complement with the API polish fix above, and the decoder will now > actually be honest and clear about what it does No objection. > - Add an option to trim whitespace > Justification: Besides archivists, it is generally undesirable to have > whitespace which is rampant in CC due to positioning methods which can > be "tricks" > While the decoder produces decent ASS, many people want simplicity such as SRT > This IMO a massive output usability improvement, though long term > belongs in subtitle filtering facilities (we do not have it now) It's worth noting that whitespace is *very* common for positioning of captions in non-live content. Aside from conversion to formats such as SRT, I think the default should actually be to preserve whitespace, since it's always been used as the standard mechanism for caption positioning. In short, I don't think whitespace preservation is only of interest to archivists; in fact it's the default expectation that most viewers would have. That said, I have no objection to the addition of such an option for those cases where it is needed. > - Fix ASS resolution to be NTSC based, not PAL > Justification: Right now, decoder uses ASS_DEFAULT_PLAYRESX and > ASS_DEFAULT_PLAYRESY as the ASS resolution base > This would make it 384x288 (PAL multiple) respectively, but I think it > should be 360x240 (NTSC multiple) > 360x240 scales cleanly to 720x480 > IIRC this can cause inaccurate alignment in 16:9 outputs (where > positioning is retained in ASS) but I have to go back and check why I > flagged this I have no opinion on this issue. > - Make cropping text off screen width an option > Justification: Right now, decoder always cuts off columns that it > thinks are past the screen width > I am not sure this is correct behavior to enforce, I have seen samples > where the CC are authored oddly (squished from 16:9 to 4:3), or just > slightly off to push limits So this shouldn't happen in theory, and generally when it does it's an indication of a bug in the rendering for the decoder. I don't object to introducing such an option, but I would suggest we collect samples where this occurs, as it would be better to fix the decoder rendering bug. > - Recently reported issues > https://code.ffmpeg.org/FFmpeg/FFmpeg/issues/21332 Yes, this is a bug. In general we should probably exercise the decoder with the CEA test materials and make sure all the extended characters match up (let me know if you don't have them and I can send them to you privately). I did this with the VLC 608 decoder many years ago and found quite a few issues that I fixed upstream. I mention this because if there are any questions on what UNICODE code points to use, I would suggest taking a peek at the VLC decoder source code. Devin -- Devin Heitmueller, Senior Software Engineer LTN Global Communications o: +1 (301) 363-1001 w: https://ltnglobal.com e: [email protected] _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
