On Wed, Mar 16, 2022 at 03:00:45PM +0100, Philip-Dylan Gleonec wrote: > Adds FEC/PLC support to libopus. The lost packets are detected as a > discontinuity in the audio stream. When a discontinuity is used, this > patch tries to decode the FEC data. If FEC data is present in the > packet, it is decoded, otherwise audio is re-created through PLC. > > This patch is based on Steinar H. Gunderson contribution, and corrects > the pts computation: all pts are expressed in samples instead of time. > This patch also adds an option "decode_fec" which enables or disables > FEC decoding. This option is disabled by default to keep consistent > behaviour with former versions. > > A number of checks are made to ensure compatibility with different > containers. Indeed, video containers seem to have a pts expressed in ms > while it is expressed in samples for audio containers. It also manages > the cases where pkt->duration is 0, in some RTP streams. This patch > ignores data it can not reconstruct, i.e. packets received twice and > packets with a length that is not a multiple of 2.5ms. > > Signed-off-by: Philip-Dylan Gleonec > <[email protected]> > Co-developed-by: Steinar H. Gunderson <[email protected]> > --- > libavcodec/libopusdec.c | 105 +++++++++++++++++++++++++++++++++++----- > 1 file changed, 94 insertions(+), 11 deletions(-) > > diff --git a/libavcodec/libopusdec.c b/libavcodec/libopusdec.c > index 86ef715205..66134300d2 100644 > --- a/libavcodec/libopusdec.c > +++ b/libavcodec/libopusdec.c > @@ -43,10 +43,15 @@ struct libopus_context { > #ifdef OPUS_SET_PHASE_INVERSION_DISABLED_REQUEST > int apply_phase_inv; > #endif > + int decode_fec; > + int64_t expected_next_pts; > }; > > #define OPUS_HEAD_SIZE 19 >
> +// Sample rate is constant as libopus always output at 48kHz
> +const AVRational opus_timebase = { 1, 48000 };
static const
> +
> static av_cold int libopus_decode_init(AVCodecContext *avc)
> {
> struct libopus_context *opus = avc->priv_data;
> @@ -134,6 +139,8 @@ static av_cold int libopus_decode_init(AVCodecContext
> *avc)
> /* Decoder delay (in samples) at 48kHz */
> avc->delay = avc->internal->skip_samples = opus->pre_skip;
>
> + opus->expected_next_pts = AV_NOPTS_VALUE;
> +
> return 0;
> }
>
> @@ -155,25 +162,100 @@ static int libopus_decode(AVCodecContext *avc, void
> *data,
> {
> struct libopus_context *opus = avc->priv_data;
> AVFrame *frame = data;
> - int ret, nb_samples;
> + uint8_t *outptr;
> + int ret, nb_samples = 0, nb_lost_samples = 0, nb_samples_left;
> +
> + // If FEC is enabled, calculate number of lost samples
> + if (opus->decode_fec &&
> + opus->expected_next_pts != AV_NOPTS_VALUE &&
> + pkt->pts != AV_NOPTS_VALUE &&
> + pkt->pts != opus->expected_next_pts) {
> + // Cap at recovering 120 ms of lost audio.
> + nb_lost_samples = pkt->pts - opus->expected_next_pts;
> + nb_lost_samples = FFMIN(nb_lost_samples, MAX_FRAME_SIZE);
> + // pts is expressed in ms for some containers (e.g. mkv)
> + // FEC only works for SILK frames (> 10ms)
> + // Detect if nb_lost_samples is in ms, and convert in samples if it
> is
> + if (nb_lost_samples > 0) {
> + if (avc->pkt_timebase.den != 48000) {
> + nb_lost_samples = av_rescale_q(nb_lost_samples,
> avc->pkt_timebase, opus_timebase);
> + }
> + // For FEC/PLC, frame_size has to be to have a multiple of 2.5 ms
> + if (nb_lost_samples % (int)(2.5 / 1000 * opus_timebase.den)) {
> + nb_lost_samples -= nb_lost_samples % (int)(2.5 / 1000 *
> opus_timebase.den);
something like this
nb_lost_samples % (5LL * opus_timebase.den / 2000)
would avoid the float
also if noone reacts to your patch keep pinging it
thx
[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
Into a blind darkness they enter who follow after the Ignorance,
they as if into a greater darkness enter who devote themselves
to the Knowledge alone. -- Isha Upanishad
signature.asc
Description: PGP signature
_______________________________________________ ffmpeg-devel mailing list [email protected] https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email [email protected] with subject "unsubscribe".
