[Live-devel] Handle multiple I/P slices

[R, Ranjith via live-devel]

--- Begin Message ---
Hi Team ,

We are using Live555 library as a RTSP client in our CCTV product.
Recently we integrated a new camera where we are getting two I/P frames with 
the same timestamp.

When we further analyze, this specific camera is setting FU header end bit in 
middle of a frame.
Suppose a single I frame is sent as 90 FUs, 45th FU end bit is set and again on 
90th FU end bit is set and RTP header marker bit set.

We checked the Live555 code, in H264VideoRTPSource.cpp, processSpecialHeader() 
function set the fCurrentPacketCompletesFrame flag based on two condition
1) FU header end bit is set
2) Current FU is complete NAL unit

Because of this logic we are getting two I/P frames with same timestamp.

We noticed other third party libraries(ffmpeg) are able to decode and play this 
data.

But our system depends on many places I frame logic
For example, we play only in reverse I frame, saving a single GOP data depends 
on I Frame.
We are not able to handle this data.
Can we set the fCurrentPacketCompletesFrame flag only if marker bit of RTP 
header is set ?

Any other suggestion from your side ?

Please share your input.


Thanks and Regards,
Ranjith
--- End Message ---
___
live-devel mailing list
live-devel@lists.live555.com
http://lists.live555.com/mailman/listinfo/live-devel

[Live-devel] Handle multiple I/P slices

[R, Ranjith via live-devel]

--- Begin Message ---
Hi Team,

This is continuation of my earlier post 
(https://www.mail-archive.com/live-devel@lists.live555.com/msg15406.html).

Frames sending as multiple slices is good, because it makes stream more 
resilient to packet loss.

Though decoders are able to decode the slice of a frame, our recording system 
needs to write a full I/P frame(not a partial frame).
Because during playback of these recordings client will get these recordings as 
list of GOPs, if we save these recording as is example 5 FPS data is saved as 
10 frames(I1I1P2P2P3P3P4P4P5P5) in two GOPs(GOP1 => I1, GOP2 => I2P2P2 
P3P3P4P4P5P5).
Also sometimes we play only I frame / alternate I frames / every 4th I Frame, 
these logics also getting affected because of this.

We have done changes in our application code in the live555 callback function 
H265BufferSink::afterGettingFrame()
We are getting the marker bit value Boolean rtpMarkerBit = 
fSource->curPacketMarkerBit();
Only if it is true(set to 1), we will consider as last frame received and write 
the buffered data to file system, till then we will be keep on appending the 
data to a buffer.

We checked the RFC(https://www.ietf.org/rfc/rfc3550.txt, 
https://www.ietf.org/rfc/rfc3551.txt), it says that  "In video streams, the 
marker bit MUST be one for the last packet sent for each video 
frame,
 and zero for all other packets".

Is this implementation of merging multiple slices of data into a single frame 
is correct ?

Please share your input.

Thanks and Regards,
Ranjith


--- End Message ---
___
live-devel mailing list
live-devel@lists.live555.com
http://lists.live555.com/mailman/listinfo/live-devel

[Live-devel] Randomly I Frame missed for high bitrate cameras

[R, Ranjith via live-devel]

--- Begin Message ---
Hi Team ,

We are using Live555 library as a RTSP client in our CCTV product. Recently we 
started supporting high bitrate cameras.

For some cameras(25 FPS/25 GOP, 8Mbps bit rate, 8MP resolution) we noticed that 
video is showing artifacts, on further analysis we identified that I Frame is 
missed for a some GOPs in between(In a 60 GOPs two to three GOPs I Frames are 
missed).

When captured Wireshark between camera and system, able to all frames are 
available in wireshark dump.

We played the rtsp url in OpenRTSP command line tool and observed similar issue 
in the video dump file. But we did not see any error in the console.

This is the command used, OpenRTSP.exe -v -b 60 -P 60 .
Since this is high bitrate camera I Frame size is around 500KB, so given a 
client buffer size of 60.

Please share your input.


Thanks and Regards,
Ranjith
--- End Message ---
___
live-devel mailing list
live-devel@lists.live555.com
http://lists.live555.com/mailman/listinfo/live-devel

Re: [Live-devel] [External] Re: Randomly I Frame missed for high bitrate cameras

[R, Ranjith via live-devel]

--- Begin Message ---
Hi Ross,

Thanks for your reply.

I will check the option suggested by you and also inform the device team about 
slicing the larger key frame.


Regards,
Ranjith

-Original Message-
From: live-devel  On Behalf Of Ross Finlayson
Sent: Friday, December 22, 2023 4:14 AM
To: LIVE555 Streaming Media - development & use 
Subject: [External] Re: [Live-devel] Randomly I Frame missed for high bitrate 
cameras

WARNING: This message has originated from an External Source. This may be a 
phishing email that can result in unauthorized access to Honeywell systems. 
Please use proper judgment and caution when opening attachments, clicking 
links, scanning QR codes, or responding.

I suspect that - despite the fact that you see no packet loss in the traffic 
'over the wire' - you are experiencing packet loss inside your (i.e., the 
receiver's) OS.  I.e., the receiver's OS is dropping incoming packets before 
delivering them to the receiving application (your RTSP client, or "openRTSP"). 
 You can check this by testing whether you ever get to line 616 in 
"MultiFramedRTPSource.cpp".  If you do, then you are seeing a lost packet.

The problem is likely due to the OS not having enough internal buffering for 
delivering incoming packets.  "openRTSP" is set up to automatically tell the OS 
to buffer at least as much as the output file buffer size (your "-b" option); 
see lines 770-780 in "testProgs/playCommon.cpp".  You can, if you wish, specify 
an even larger OS buffer size using the "-B" (i.e., capital-B) option - e.g., 
"-B 100".

HOWEVER, your real problem is your IP camera: It is sending key frames as 
single H.264/H.265 NAL units that are much, much too large for video streaming. 
 I have explained this several times on this mailing list; most recently here:
http://lists.live555.com/pipermail/live-devel/2023-June/022334.html

You should reconfigure your IP cameras to not send each key frame as a single 
NAL unit, but instead as multiple 'slice' NAL units.  Then, your video won't be 
as sensitive to packet loss.  (Please don't claim that you can't 
reconfigure/fix your IP cameras; you can.  They (unlike the LIVE555 software) 
are things that you have paid for.  If your IP cameras aren't doing what you 
want, you need to tell the manufacturer to fix them.)


Ross Finlayson
Live Networks, Inc.
http://www.live555.com/


___
live-devel mailing list
live-devel@lists.live555.com
http://lists.live555.com/mailman/listinfo/live-devel

--- End Message ---
___
live-devel mailing list
live-devel@lists.live555.com
http://lists.live555.com/mailman/listinfo/live-devel

Re: [Live-devel] [External] Re: Randomly I Frame missed for high bitrate cameras

[R, Ranjith via live-devel]

--- Begin Message ---
Hi Ross,

As per your suggestion I tried increasing the socket buffer size, still we 
observe the same issue.

This is the command used in OpenRTSP, OpenRTSP.exe -v -b 100 -B 100 -P 
60 

I added logs in line 616 in "MultiFramedRTPSource.cpp" that is inside 
"ReorderingPacketBuffer::releaseUsedPacket" function, but these logs did not 
came in console.

Please share your input.

Thanks and Regards,
Ranjith

-Original Message-
From: R, Ranjith
Sent: Friday, December 22, 2023 8:49 PM
To: LIVE555 Streaming Media - development & use 
Subject: RE: [External] Re: [Live-devel] Randomly I Frame missed for high 
bitrate cameras

Hi Ross,

Thanks for your reply.

I will check the option suggested by you and also inform the device team about 
slicing the larger key frame.


Regards,
Ranjith

-Original Message-
From: live-devel  On Behalf Of Ross Finlayson
Sent: Friday, December 22, 2023 4:14 AM
To: LIVE555 Streaming Media - development & use 
Subject: [External] Re: [Live-devel] Randomly I Frame missed for high bitrate 
cameras

WARNING: This message has originated from an External Source. This may be a 
phishing email that can result in unauthorized access to Honeywell systems. 
Please use proper judgment and caution when opening attachments, clicking 
links, scanning QR codes, or responding.

I suspect that - despite the fact that you see no packet loss in the traffic 
'over the wire' - you are experiencing packet loss inside your (i.e., the 
receiver's) OS.  I.e., the receiver's OS is dropping incoming packets before 
delivering them to the receiving application (your RTSP client, or "openRTSP"). 
 You can check this by testing whether you ever get to line 616 in 
"MultiFramedRTPSource.cpp".  If you do, then you are seeing a lost packet.

The problem is likely due to the OS not having enough internal buffering for 
delivering incoming packets.  "openRTSP" is set up to automatically tell the OS 
to buffer at least as much as the output file buffer size (your "-b" option); 
see lines 770-780 in "testProgs/playCommon.cpp".  You can, if you wish, specify 
an even larger OS buffer size using the "-B" (i.e., capital-B) option - e.g., 
"-B 100".

HOWEVER, your real problem is your IP camera: It is sending key frames as 
single H.264/H.265 NAL units that are much, much too large for video streaming. 
 I have explained this several times on this mailing list; most recently here:
http://lists.live555.com/pipermail/live-devel/2023-June/022334.html

You should reconfigure your IP cameras to not send each key frame as a single 
NAL unit, but instead as multiple 'slice' NAL units.  Then, your video won't be 
as sensitive to packet loss.  (Please don't claim that you can't 
reconfigure/fix your IP cameras; you can.  They (unlike the LIVE555 software) 
are things that you have paid for.  If your IP cameras aren't doing what you 
want, you need to tell the manufacturer to fix them.)


Ross Finlayson
Live Networks, Inc.
http://www.live555.com/


___
live-devel mailing list
live-devel@lists.live555.com
http://lists.live555.com/mailman/listinfo/live-devel

--- End Message ---
___
live-devel mailing list
live-devel@lists.live555.com
http://lists.live555.com/mailman/listinfo/live-devel

Re: [Live-devel] [External] Re: Randomly I Frame missed for high bitrate cameras

[R, Ranjith via live-devel]

--- Begin Message ---
Hi Ross,

I am yet to ask the device team to check that part, before that I want to 
confirm by increasing the socket buffer size(and working) or see some error 
logs in Live555 layer where the packets are dropped(because of high key frame 
size).

Thanks and Regards,
Ranjith

-Original Message-
From: live-devel  On Behalf Of Ross Finlayson
Sent: Wednesday, January 3, 2024 5:44 PM
To: LIVE555 Streaming Media - development & use 
Subject: Re: [Live-devel] [External] Re: Randomly I Frame missed for high 
bitrate cameras



> On Jan 4, 2024, at 12:43 AM, R, Ranjith via live-devel 
>  wrote:
>
>
> Please share your input.

Did you do the following, as I advised:


> HOWEVER, your real problem is your IP camera: It is sending key frames as 
> single H.264/H.265 NAL units that are much, much too large for video 
> streaming.  I have explained this several times on this mailing list; most 
> recently here:
>http://lists.live555.com/pipermail/live-devel/2023-June/022334.html
>
> You should reconfigure your IP cameras to not send each key frame as a single 
> NAL unit, but instead as multiple 'slice' NAL units.  Then, your video won't 
> be as sensitive to packet loss.  (Please don't claim that you can't 
> reconfigure/fix your IP cameras; you can.  They (unlike the LIVE555 software) 
> are things that you have paid for.  If your IP cameras aren't doing what you 
> want, you need to tell the manufacturer to fix them.)


Ross Finlayson
Live Networks, Inc.
http://www.live555.com/


___
live-devel mailing list
live-devel@lists.live555.com
http://lists.live555.com/mailman/listinfo/live-devel

--- End Message ---
___
live-devel mailing list
live-devel@lists.live555.com
http://lists.live555.com/mailman/listinfo/live-devel

Re: [Live-devel] [External] Re: Randomly I Frame missed for high bitrate cameras

[R, Ranjith via live-devel]

--- Begin Message ---
Hi Dan ,

I understand that increasing the compression rate will reduce the data size and 
issue may not occur.

As you said it will affect the quality of the image or decoding may take more 
time.

But I want to confirm where in Live555 code where the packet is getting 
discarded because of higher data size, even I increased the socket buffer 
size(200) still the issue exist.

One difference I see with this camera is for every I frame it is sending SDP 
parameters(SPS, PPS, SEI), whether that can cause any problem not sure.

Thanks and Regards,
Ranjith

-Original Message-
From: dan_desjardins 
Sent: Wednesday, January 3, 2024 6:36 PM
To: LIVE555 Streaming Media - development & use 
Cc: R, Ranjith 
Subject: RE: [Live-devel] [External] Re: Randomly I Frame missed for high 
bitrate cameras

[You don't often get email from dan.desjard...@videstra.com. Learn why this is 
important at https://aka.ms/LearnAboutSenderIdentification ]

I apologize if this doesn't make sense in your implementation, but if the 
device mfg does not have (or want to give you) a way to slice large packets - 
another approach might be to set the compression on the camera much higher. 
I've not run into a camera or encoder that does not have some form of 
adjustable compression setting.  This usually affects the size (and quality) of 
the I-Frames - resulting in smaller packets carrying them. While it will likely 
degrade the quality of the stream images, and possibly introduce some other 
unwanted artifacts, it may be your only possible recourse.

-Original Message-
From: live-devel  On Behalf Of R, Ranjith 
via live-devel
Sent: Wednesday, January 3, 2024 6:43 AM
To: LIVE555 Streaming Media - development & use 
Cc: R, Ranjith 
Subject: Re: [Live-devel] [External] Re: Randomly I Frame missed for high 
bitrate cameras

___
live-devel mailing list
live-devel@lists.live555.com
http://lists.live555.com/mailman/listinfo/live-devel

--- End Message ---
___
live-devel mailing list
live-devel@lists.live555.com
http://lists.live555.com/mailman/listinfo/live-devel

Re: [Live-devel] [External] Re: Randomly I Frame missed for high bitrate cameras

[R, Ranjith via live-devel]

--- Begin Message ---
Hi Ross,

I confirmed in Wireshark that all packets are received from the network(both 
network and send buffer is ruled out).

I increased the receive buffer size up to 2MB, still the issue is observed.

I am not sure how to proceed further.


Thanks and Regards,
Ranjith

-Original Message-
From: live-devel  On Behalf Of Ross Finlayson
Sent: Thursday, January 4, 2024 1:07 PM
To: LIVE555 Streaming Media - development & use 
Subject: Re: [Live-devel] [External] Re: Randomly I Frame missed for high 
bitrate cameras



> On Jan 4, 2024, at 8:24 PM, R, Ranjith via live-devel 
>  wrote:
>
> But I want to confirm where in Live555 code where the packet is getting 
> discarded

Packets are not being discarded anywhere in the LIVE555 code.  Note that a 
LIVE555-based server simply sits in a loop, writing a sequence of RTP packets 
to a socket.  And a LIVE555-based receiver simply sits in a loop, reading a 
sequence of RTP packets from a socket.

In other words, there is nothing in the LIVE555 code that can possibly be 
‘losing’ packets.  If you are seeing packet loss between a LIVE555-based server 
and a LIVE555-based client, then it *must* be happening between these two 
sockets (server socket and client socket) - not in the LIVE555 code.

I.e., if your server and client are separated by a network, then any packet 
lost *must* be occurring on the network, or in the operating system that’s 
underneath the server’s socket, or in the operating system that’s underneath 
the receiver’s socket.

If your server and client are running on the same computer (either on the same 
(real or virtual) machine, or in different virtual machines), then any packet 
lost *must* be occurring within the computer’s operating system (including 
perhaps its virtualization implementation).


Ross Finlayson
Live Networks, Inc.
http://www.live555.com/


___
live-devel mailing list
live-devel@lists.live555.com
http://lists.live555.com/mailman/listinfo/live-devel

--- End Message ---
___
live-devel mailing list
live-devel@lists.live555.com
http://lists.live555.com/mailman/listinfo/live-devel