Hi everyone, I've come upon a rather curious phenomenon in a video which seems to have video frames in an unusual order. Sadly, it's a video from our client which I cannot share (and I have no idea how to replicate it). The weird bit about it is that (although `dts` are increasing as expected), packets' `pts` go in this order: 6006, 0, 3003, 15015, 9009, 12012, 24024, 18018, 21021, 33033, 27027, 30030, 42042, 36036, 39039 [1]. Due to the fact that the first frame doesn't have `pts=0`, the whole stream `start_pts` is set to 6006 instead of 0 [2]. If it helps, it came straight out of a semi-professional Panasonic camera without any reencoding, as far as I know.
Now according to the docs, `AVStream::start_time` should be "pts of the first frame of the stream in presentation order" which I interpret as "the lowest pts there is". However, I keep getting the first frames' `pts`. Here's my calls, nothing complicated: ``` avformat_open_input(&_avFormatContext, _filename.c_str(), NULL, NULL); avformat_find_stream_info(_avFormatContext, NULL); ``` (+ some unimportant checks). I've tried debugging it through and it got me to this place: https://github.com/FFmpeg/FFmpeg/blob/master/libavformat/utils.c#L1164 where the `start_time` is set. It is however never updated based on the next frame's (lower) pts, so I have an incorrect start time to work with. I can get the number updated using this hack at the beginning of `update_initial_timestamps`: ``` if (st->start_time != AV_NOPTS_VALUE /*EDITED BY KAREL*/ && st->start_time > pts) { if (st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO || !(pkt->flags & AV_PKT_FLAG_DISCARD)) { st->start_time = pts; } if (st->codecpar->codec_type == AVMEDIA_TYPE_AUDIO && st->codecpar->sample_rate) { st->start_time += av_rescale_q(st->skip_samples, (AVRational){1, st->codecpar->sample_rate}, st->time_base); } } ``` but I'm not sure if it wouldn't break anything else... Now to my questions, for which I'd be really gratefull if anybody knew the answers. Is this kind of video normal/correct/according to standard? Am I correct to think that the stream's `start_time` should in fact be 0? What can I do about it? Is my hack dangerous to anything else; could it possibly break decoding of other videos? Many thanks in advance! Karel [1] ffprobe output showing the packets with their out-of-order pts: ffprobe -show_packets P1010836.MOV | head -n 500 ffprobe version 3.4.4-1~16.04.york0 Copyright (c) 2007-2018 the FFmpeg developers built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.10) 20160609 configuration: --prefix=/usr --extra-version='1~16.04.york0' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-librsvg --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libd rm --ena ble-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared WARNING: library configuration mismatch avcodec configuration: --prefix=/usr --extra-version='1~16.04.york0' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-librsvg --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 - -enable- libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared --enable-version3 --disable-doc --disable-programs --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libtesseract --enable-libvo_amrwbenc libavutil 55. 78.100 / 55. 78.100 libavcodec 57.107.100 / 57.107.100 libavformat 57. 83.100 / 57. 83.100 libavdevice 57. 10.100 / 57. 10.100 libavfilter 6.107.100 / 6.107.100 libavresample 3. 7. 0 / 3. 7. 0 libswscale 4. 8.100 / 4. 8.100 libswresample 2. 9.100 / 2. 9.100 libpostproc 54. 7.100 / 54. 7.100 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'P1010836.MOV': Metadata: major_brand : qt minor_version : 537986816 compatible_brands: qt pana creation_time : 2019-06-12T13:54:36.000000Z com.panasonic.Semi-Pro.metadata.xml: <?xml version="1.0" encoding="UTF-8" standalone="no" ?> : <ClipMain xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xmlns="urn:schemas-Professional-Plug-in:Semi-Pro:ClipMetadata:v1.0"> : <ClipContent> : <GlobalClipID>060A2B340101010501010D2113000000DF54DE53A0BB5534685D1090C1A20117</GlobalClipID> : <Duration>4845</Duration> : <EditUnit>1001/30000</EditUnit> : <EssenceList> : <Video> : <Codec BitRate="100">H264_420_LongGOP</Codec> : <ActiveLine>2160</ActiveLine> : <ActivePixel>3840</ActivePixel> : <BitDepth>8</BitDepth> : <FrameRate>29.97p</FrameRate> : <TimecodeType>Drop</TimecodeType> : <StartTimecode>12:19:04:01</StartTimecode> : </Video> : <Audio> : <Channel>2</Channel> : <SamplingRate>48000</SamplingRate> : <BitsPerSample>16</BitsPerSample> : </Audio> : </EssenceList> : <ClipMetadata> : <Rating>0</Rating> : <Access> : <CreationDate>2019-06-12T13:54:36+00:00</CreationDate> : <LastUpdateDate>2019-06-12T13:54:36+00:00</LastUpdateDate> : </Access> : <Device> : <Manufacturer>Panasonic</Manufacturer> : <ModelName>DC-GH5S</ModelName> : </Device> : <Shoot> : <StartDate>2019-06-12T13:54:36+00:00</StartDate> : </Shoot> : </ClipMetadata> : </ClipContent> : <UserArea> : <AcquisitionMetadata xmlns="urn:schemas-Professional-Plug-in:P2:CameraMetadata:v1.2"> : <CameraUnitMetadata> : <Gamma> : <CaptureGamma>STANDARD</CaptureGamma> : </Gamma> : <Gamut> : <CaptureGamut>BT.709</CaptureGamut> : </Gamut> : </CameraUnitMetadata> : </AcquisitionMetadata> : </UserArea> : </ClipMain> : Duration: 00:02:41.66, start: 0.000000, bitrate: 96239 kb/s Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 3840x2160 [SAR 1:1 DAR 16:9], 94690 kb/s, 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc (default) Metadata: creation_time : 2019-06-12T13:54:36.000000Z timecode : 12:19:04;01 Stream #0:1(und): Audio: pcm_s16be (twos / 0x736F7774), 48000 Hz, 2 channels, s16, 1536 kb/s (default) Metadata: creation_time : 2019-06-12T13:54:36.000000Z timecode : 12:19:04;01 Stream #0:2(und): Data: none (tmcd / 0x64636D74), 0 kb/s (default) Metadata: creation_time : 2019-06-12T13:54:36.000000Z timecode : 12:19:04;01 Unsupported codec with id 0 for input stream 2 [PACKET] codec_type=data stream_index=2 pts=0 pts_time=0.000000 dts=0 dts_time=0.000000 duration=3003 duration_time=0.033367 convergence_duration=N/A convergence_duration_time=N/A size=4 pos=262160 flags=K_ [/PACKET] [PACKET] codec_type=data stream_index=2 pts=3003 pts_time=0.033367 dts=3003 dts_time=0.033367 duration=3003 duration_time=0.033367 convergence_duration=N/A convergence_duration_time=N/A size=4 pos=262164 flags=K_ [/PACKET] [PACKET] codec_type=data stream_index=2 pts=6006 pts_time=0.066733 dts=6006 dts_time=0.066733 duration=3003 duration_time=0.033367 convergence_duration=N/A convergence_duration_time=N/A size=4 pos=262168 flags=K_ [/PACKET] [PACKET] codec_type=data stream_index=2 pts=9009 pts_time=0.100100 dts=9009 dts_time=0.100100 duration=3003 duration_time=0.033367 convergence_duration=N/A convergence_duration_time=N/A size=4 pos=262172 flags=K_ [/PACKET] [PACKET] codec_type=data stream_index=2 pts=12012 pts_time=0.133467 dts=12012 dts_time=0.133467 duration=3003 duration_time=0.033367 convergence_duration=N/A convergence_duration_time=N/A size=4 pos=262176 flags=K_ [/PACKET] [PACKET] codec_type=data stream_index=2 pts=15015 pts_time=0.166833 dts=15015 dts_time=0.166833 duration=3003 duration_time=0.033367 convergence_duration=N/A convergence_duration_time=N/A size=4 pos=262180 flags=K_ [/PACKET] [PACKET] codec_type=data stream_index=2 pts=18018 pts_time=0.200200 dts=18018 dts_time=0.200200 duration=3003 duration_time=0.033367 convergence_duration=N/A convergence_duration_time=N/A size=4 pos=262184 flags=K_ [/PACKET] [PACKET] codec_type=data stream_index=2 pts=21021 pts_time=0.233567 dts=21021 dts_time=0.233567 duration=3003 duration_time=0.033367 convergence_duration=N/A convergence_duration_time=N/A size=4 pos=262188 flags=K_ [/PACKET] [PACKET] codec_type=data stream_index=2 pts=24024 pts_time=0.266933 dts=24024 dts_time=0.266933 duration=3003 duration_time=0.033367 convergence_duration=N/A convergence_duration_time=N/A size=4 pos=262192 flags=K_ [/PACKET] [PACKET] codec_type=data stream_index=2 pts=27027 pts_time=0.300300 dts=27027 dts_time=0.300300 duration=3003 duration_time=0.033367 convergence_duration=N/A convergence_duration_time=N/A size=4 pos=262196 flags=K_ [/PACKET] [PACKET] codec_type=data stream_index=2 pts=30030 pts_time=0.333667 dts=30030 dts_time=0.333667 duration=3003 duration_time=0.033367 convergence_duration=N/A convergence_duration_time=N/A size=4 pos=262200 flags=K_ [/PACKET] [PACKET] codec_type=data stream_index=2 pts=33033 pts_time=0.367033 dts=33033 dts_time=0.367033 duration=3003 duration_time=0.033367 convergence_duration=N/A convergence_duration_time=N/A size=4 pos=262204 flags=K_ [/PACKET] [PACKET] codec_type=data stream_index=2 pts=36036 pts_time=0.400400 dts=36036 dts_time=0.400400 duration=3003 duration_time=0.033367 convergence_duration=N/A convergence_duration_time=N/A size=4 pos=262208 flags=K_ [/PACKET] [PACKET] codec_type=data stream_index=2 pts=39039 pts_time=0.433767 dts=39039 dts_time=0.433767 duration=3003 duration_time=0.033367 convergence_duration=N/A convergence_duration_time=N/A size=4 pos=262212 flags=K_ [/PACKET] [PACKET] codec_type=data stream_index=2 pts=42042 pts_time=0.467133 dts=42042 dts_time=0.467133 duration=3003 duration_time=0.033367 convergence_duration=N/A convergence_duration_time=N/A size=4 pos=262216 flags=K_ [/PACKET] [PACKET] codec_type=video stream_index=0 pts=6006 pts_time=0.066733 dts=-3003 dts_time=-0.033367 duration=3003 duration_time=0.033367 convergence_duration=N/A convergence_duration_time=N/A size=741081 pos=262220 flags=K_ [/PACKET] [PACKET] codec_type=video stream_index=0 pts=0 pts_time=0.000000 dts=0 dts_time=0.000000 duration=3003 duration_time=0.033367 convergence_duration=N/A convergence_duration_time=N/A size=75985 pos=1003301 flags=__ [/PACKET] [PACKET] codec_type=video stream_index=0 pts=3003 pts_time=0.033367 dts=3003 dts_time=0.033367 duration=3003 duration_time=0.033367 convergence_duration=N/A convergence_duration_time=N/A size=89063 pos=1079286 flags=__ [/PACKET] ... (trimmed) [2] ffprobe output showing the incorrect start_pts: ffprobe -show_streams P1010836.MOV ffprobe version 3.4.4-1~16.04.york0 Copyright (c) 2007-2018 the FFmpeg developers built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.10) 20160609 configuration: --prefix=/usr --extra-version='1~16.04.york0' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-librsvg --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libd rm --ena ble-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared WARNING: library configuration mismatch avcodec configuration: --prefix=/usr --extra-version='1~16.04.york0' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-librsvg --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 - -enable- libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared --enable-version3 --disable-doc --disable-programs --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libtesseract --enable-libvo_amrwbenc libavutil 55. 78.100 / 55. 78.100 libavcodec 57.107.100 / 57.107.100 libavformat 57. 83.100 / 57. 83.100 libavdevice 57. 10.100 / 57. 10.100 libavfilter 6.107.100 / 6.107.100 libavresample 3. 7. 0 / 3. 7. 0 libswscale 4. 8.100 / 4. 8.100 libswresample 2. 9.100 / 2. 9.100 libpostproc 54. 7.100 / 54. 7.100 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'P1010836.MOV': Metadata: major_brand : qt minor_version : 537986816 compatible_brands: qt pana creation_time : 2019-06-12T13:54:36.000000Z com.panasonic.Semi-Pro.metadata.xml: <?xml version="1.0" encoding="UTF-8" standalone="no" ?> : <ClipMain xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xmlns="urn:schemas-Professional-Plug-in:Semi-Pro:ClipMetadata:v1.0"> : <ClipContent> : <GlobalClipID>060A2B340101010501010D2113000000DF54DE53A0BB5534685D1090C1A20117</GlobalClipID> : <Duration>4845</Duration> : <EditUnit>1001/30000</EditUnit> : <EssenceList> : <Video> : <Codec BitRate="100">H264_420_LongGOP</Codec> : <ActiveLine>2160</ActiveLine> : <ActivePixel>3840</ActivePixel> : <BitDepth>8</BitDepth> : <FrameRate>29.97p</FrameRate> : <TimecodeType>Drop</TimecodeType> : <StartTimecode>12:19:04:01</StartTimecode> : </Video> : <Audio> : <Channel>2</Channel> : <SamplingRate>48000</SamplingRate> : <BitsPerSample>16</BitsPerSample> : </Audio> : </EssenceList> : <ClipMetadata> : <Rating>0</Rating> : <Access> : <CreationDate>2019-06-12T13:54:36+00:00</CreationDate> : <LastUpdateDate>2019-06-12T13:54:36+00:00</LastUpdateDate> : </Access> : <Device> : <Manufacturer>Panasonic</Manufacturer> : <ModelName>DC-GH5S</ModelName> : </Device> : <Shoot> : <StartDate>2019-06-12T13:54:36+00:00</StartDate> : </Shoot> : </ClipMetadata> : </ClipContent> : <UserArea> : <AcquisitionMetadata xmlns="urn:schemas-Professional-Plug-in:P2:CameraMetadata:v1.2"> : <CameraUnitMetadata> : <Gamma> : <CaptureGamma>STANDARD</CaptureGamma> : </Gamma> : <Gamut> : <CaptureGamut>BT.709</CaptureGamut> : </Gamut> : </CameraUnitMetadata> : </AcquisitionMetadata> : </UserArea> : </ClipMain> : Duration: 00:02:41.66, start: 0.000000, bitrate: 96239 kb/s Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 3840x2160 [SAR 1:1 DAR 16:9], 94690 kb/s, 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc (default) Metadata: creation_time : 2019-06-12T13:54:36.000000Z timecode : 12:19:04;01 Stream #0:1(und): Audio: pcm_s16be (twos / 0x736F7774), 48000 Hz, 2 channels, s16, 1536 kb/s (default) Metadata: creation_time : 2019-06-12T13:54:36.000000Z timecode : 12:19:04;01 Stream #0:2(und): Data: none (tmcd / 0x64636D74), 0 kb/s (default) Metadata: creation_time : 2019-06-12T13:54:36.000000Z timecode : 12:19:04;01 Unsupported codec with id 0 for input stream 2 [STREAM] index=0 codec_name=h264 codec_long_name=H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 profile=High codec_type=video codec_time_base=1001/60000 codec_tag_string=avc1 codec_tag=0x31637661 width=3840 height=2160 coded_width=3840 coded_height=2160 has_b_frames=1 sample_aspect_ratio=1:1 display_aspect_ratio=16:9 pix_fmt=yuv420p level=51 color_range=tv color_space=bt709 color_transfer=bt709 color_primaries=bt709 chroma_location=left field_order=unknown timecode=N/A refs=1 is_avc=true nal_length_size=4 id=N/A r_frame_rate=30000/1001 avg_frame_rate=30000/1001 time_base=1/90000 start_pts=6006 start_time=0.066733 duration_ts=14549535 duration=161.661500 bit_rate=94690004 max_bit_rate=N/A bits_per_raw_sample=8 nb_frames=4845 nb_read_frames=N/A nb_read_packets=N/A DISPOSITION:default=1 DISPOSITION:dub=0 DISPOSITION:original=0 DISPOSITION:comment=0 DISPOSITION:lyrics=0 DISPOSITION:karaoke=0 DISPOSITION:forced=0 DISPOSITION:hearing_impaired=0 DISPOSITION:visual_impaired=0 DISPOSITION:clean_effects=0 DISPOSITION:attached_pic=0 DISPOSITION:timed_thumbnails=0 TAG:creation_time=2019-06-12T13:54:36.000000Z TAG:language=und TAG:timecode=12:19:04;01 [/STREAM] [STREAM] index=1 codec_name=pcm_s16be codec_long_name=PCM signed 16-bit big-endian profile=unknown codec_type=audio codec_time_base=1/48000 codec_tag_string=twos codec_tag=0x736f7774 sample_fmt=s16 sample_rate=48000 channels=2 channel_layout=unknown bits_per_sample=16 id=N/A r_frame_rate=0/0 avg_frame_rate=0/0 time_base=1/48000 start_pts=0 start_time=0.000000 duration_ts=7759752 duration=161.661500 bit_rate=1536000 max_bit_rate=N/A bits_per_raw_sample=N/A nb_frames=7759752 nb_read_frames=N/A nb_read_packets=N/A DISPOSITION:default=1 DISPOSITION:dub=0 DISPOSITION:original=0 DISPOSITION:comment=0 DISPOSITION:lyrics=0 DISPOSITION:karaoke=0 DISPOSITION:forced=0 DISPOSITION:hearing_impaired=0 DISPOSITION:visual_impaired=0 DISPOSITION:clean_effects=0 DISPOSITION:attached_pic=0 DISPOSITION:timed_thumbnails=0 TAG:creation_time=2019-06-12T13:54:36.000000Z TAG:language=und TAG:timecode=12:19:04;01 [/STREAM] [STREAM] index=2 codec_name=unknown codec_long_name=unknown profile=unknown codec_type=data codec_tag_string=tmcd codec_tag=0x64636d74 id=N/A r_frame_rate=0/0 avg_frame_rate=30/1 time_base=1/90000 start_pts=0 start_time=0.000000 duration_ts=14549535 duration=161.661500 bit_rate=959 max_bit_rate=N/A bits_per_raw_sample=N/A nb_frames=4845 nb_read_frames=N/A nb_read_packets=N/A DISPOSITION:default=1 DISPOSITION:dub=0 DISPOSITION:original=0 DISPOSITION:comment=0 DISPOSITION:lyrics=0 DISPOSITION:karaoke=0 DISPOSITION:forced=0 DISPOSITION:hearing_impaired=0 DISPOSITION:visual_impaired=0 DISPOSITION:clean_effects=0 DISPOSITION:attached_pic=0 DISPOSITION:timed_thumbnails=0 TAG:creation_time=2019-06-12T13:54:36.000000Z TAG:language=und TAG:timecode=12:19:04;01 [/STREAM] _______________________________________________ Libav-user mailing list [email protected] https://ffmpeg.org/mailman/listinfo/libav-user To unsubscribe, visit link above, or email [email protected] with subject "unsubscribe".
