What does exact meaning of "Dropped Frames" and how it is calculated?

[NomaiZ]

Hi Support team,

We are currently in the process of troubleshooting.
I would like clarification on the exact meaning of "Dropped Frames" and how it is calculated.

Does the counter represent the difference between frames sent (by OBS) and frames received (by the destination server, such as YouTube)?
For example, if OBS sends 10,000 frames but only 9,000 frames arrive at YouTube, will the counter display 1,000/10,000 (10%)?

Or does it refer to the expected number of frames to be sent versus the actual frames sent?
For instance, if OBS is expected to send 10,000 frames but, due to issues such as high CPU usage, it can only send 9,000 frames, will the counter then display 1,000/10,000 (10%)?

Thank you so much for your attention and participation.

 

[qhobbes]

AFAIK dropped frames are encoded frames that can't be sent do to network issues.

Frames that are not rendered/encoded due to CPU/GPU overload cannot be dropped because they don't exist.

The quick fix for dropped frames is to enable Dynamic Bitrate by going to Settings, Advanced, Network.

 

[koala]

If you look at View > Stats, you have 3 different frame loss statistics.

The flow of one frame from capture to streaming service is this:

1. the frame is composited
2. the frame is encoded
3. the frame is transmitted to the streaming service (or written to a local file)

Each operation is on a fixed timing schedule. If you configured OBS with 60 fps, it has 1/60 = 16.667 ms for processing each of these steps per frame. This means, every 16.6 ms OBS is compositing one frame. Composition includes reading all sources and overlaying ("rendering") them on the canvas. If this takes more than 16.6 ms for one frame, the frame is aborted and thrown away. This is recorded as "frames missed due to rendering lag". Having rendering lag is visible as lag and stutter on the OBS preview. The reason for rendering lag is lack of GPU resources, sometimes lack of CPU resources (depends on the source type and system setup)

A successfully rendered frame is then encoded. This is done in another processing thread, so it again has 16.6 ms to encode each frame. If this takes more than 16.6 ms for one frame, the frame is aborted and thrown away. This is recorded as "skipped frames due to encoding lag". The reason for encoding lag is system overload in general. Specifically, CPU overload if using a software encoder and GPU overload if using a hardware encoder. A little bit of both.

Finally, a successfully encoded frame is sent to the streaming service. This is also on a separate thread, so there is again 16.6 ms for each frame to send. If there is network congestion and the sending buffer becomes full, there's no space to store that frame. It's dropped in this case and recorded as "dropped frames (network)". Actually, the sending algorithm is more sophisticated than just dropping frames. If a key frame must be dropped, all dependent frames until the next key frame must also be dropped, since they contain just the diff to the last key frame, so they're invalid without their key frame. The reason for lost frames due to network congestion is, well, network congestion. Not OBS and not your local system usually, except for malconfigured firewalls and network adapters.

 

[NomaiZ]

If you look at View > Stats, you have 3 different frame loss statistics.

The flow of one frame from capture to streaming service is this:

1. the frame is composited
2. the frame is encoded
3. the frame is transmitted to the streaming service (or written to a local file)

Each operation is on a fixed timing schedule. If you configured OBS with 60 fps, it has 1/60 = 16.667 ms for processing each of these steps per frame. This means, every 16.6 ms OBS is compositing one frame. Composition includes reading all sources and overlaying ("rendering") them on the canvas. If this takes more than 16.6 ms for one frame, the frame is aborted and thrown away. This is recorded as "frames missed due to rendering lag". Having rendering lag is visible as lag and stutter on the OBS preview. The reason for rendering lag is lack of GPU resources, sometimes lack of CPU resources (depends on the source type and system setup)

A successfully rendered frame is then encoded. This is done in another processing thread, so it again has 16.6 ms to encode each frame. If this takes more than 16.6 ms for one frame, the frame is aborted and thrown away. This is recorded as "skipped frames due to encoding lag". The reason for encoding lag is system overload in general. Specifically, CPU overload if using a software encoder and GPU overload if using a hardware encoder. A little bit of both.

Finally, a successfully encoded frame is sent to the streaming service. This is also on a separate thread, so there is again 16.6 ms for each frame to send. If there is network congestion and the sending buffer becomes full, there's no space to store that frame. It's dropped in this case and recorded as "dropped frames (network)". Actually, the sending algorithm is more sophisticated than just dropping frames. If a key frame must be dropped, all dependent frames until the next key frame must also be dropped, since they contain just the diff to the last key frame, so they're invalid without their key frame. The reason for lost frames due to network congestion is, well, network congestion. Not OBS and not your local system usually, except for malconfigured firewalls and network adapters.

Hi qhobbes, koala,

Thank you very much for your comment.

As per,
If there is network congestion and the sending buffer becomes full, there's no space to store that frame. It's dropped in this case and recorded as "dropped frames (network)".

>>> Let's say make the diagram simple
PC--->ISP_router--->Youtube
If ISP_router is dropping packets, will the dropped frames counter be increased?
1) If yes, this means this counter can calculate the encoded frame sent vs the encoded frame received(received by YouTube), am I right?
2) If not, this means this counter just detect encoded frames that can't be sent out from the PC and calculated as dropped frames.

In this example the intermediate hop is dropping. The 1) is the encoded frame sent VS encoded frame received. The 2) is the expected encoded frame sent VS the actual encoded frame sent (sent out from the computer)

 

[koala]

OBS doesn't know anything about the physics of the internet connection. Nothing about providers, nothing about routers, nothing about the internet. It performs a very simple operating system call to connect to the streaming provider server, and if it is connected, the whole complexity of the internet is irrelevant. OBS just sends its data packets over the connection, and if the operating system call used for data transfer returns an error, it is unable to do that. It has to drop the packet it tried to send, and it will increase its "dropped frames (network)" counter. Why the data packets didn't reach their destination is unknown to OBS (and to every other app as well). It's also never known to OBS in which part of the internet the data packets were lost and why. It just learns: "it wasn't possible to send the data". Nothing more.

To determine where and why packets are lost on the network is a task for network diagnosis tools. Not a task that can be accomplished by OBS. It' a whole science for itself, network issues are one of the most complex issues you will encounter when dealing with PC issues.

 

[NomaiZ]

Hi Koala,

Thank you for replying.

OBS just sends its data packets over the connection, and if the operating system call used for data transfer returns an error,
>>> So OBS must receive an error first.

it is unable to do that. It has to drop the packet it tried to send, and it will increase its "dropped frames (network)"
>>> Then OBS will drop the packet and increase its "dropped frames (network)".

I think we can say "dropped frames (network)" will be increased after OBS received an error and then dropped the packet it tried to send. In my understanding, if I collect packet capture with Wireshark, I should see some error return to the computer while dropped frames (network) are being increased.

Do you have any comment on my understanding? Thank you.

 

[koala]

No. Technically, there is no error as long as the connection isn't being aborted unexpectedly. Actually, the system call to send a data packet takes longer than expected and OBS decided that sending this packet takes too long, so network congestion must be the reason, so the packet is dropped. I used layman's terms in my above post and simplified explanations to make my post understandable by someone who is not a network expert.

In Wireshark, you would see ACK packets stop arriving and an increased amount of packet resends for a severe network congestion situation. But only if it is severe. There is a difference between network congestion (no data packets lost en route; just being transferred slowly) and packet loss (packets are lost en route). It's a challenge to detect if the bandwidth drops below the acceptable rate for live streaming but not below a rate where ACK packets go missing, since this can only be determined by checking the timing of the data packets.

If you suspect a network issue, get a person who qualifies as network expert and who is able to do this kind of network diagnosis. If you're a single person who just wants to stream, you would not be able to do such deep analysis. In this case, it's even more important to make sure you implemented best network practice and your equipment is configured with best practice and has no defect. If you made this sure, you would then call your internet provider. See:

Stream Connection Troubleshooting | OBS

OBS Knowledge Base. Dropped frames? Your stream is disconnecting? This guide will help you troubleshoot common connection issues.

obsproject.com

 

[NomaiZ]

Hi Koala,

Thank you for your input.

I'm a network expert. We are in the midst of troubleshooting, but we cannot move on because it was very unclear how "Dropped frame" is calculated, then I created this topic.

Back to my original question, I believe I got the answer.
As you said "ACK packets stop arriving and an increased amount of packet resends" I assume I should see one of the below from packet capture while the problem is occurring
- No return ACK from the destination IP to the user IP
- ACK from the destination IP arrives at the source, but does not arrive within the acceptable time (is it 16.6 ms you mentioned?)
- TCP retransmission

Practically, it is not easy to work with the internet provider without evidence. Let's say all services are working well except streaming or no issue when streaming to other destinations (like Twitch, Facebook). ISP may say "hey, this can be a Youtube issue or OBS issue".

I plan to add NGFW between the user and internet to capture more logs soon. I hope I might have more visibility like a report contain related destination IP, county of destination IP, etc.

 

[NomaiZ]

Btw, could you share what are the destination FQDNs (or destination IPs) when selecting 'YouTube - RTMPS', 'Youtube - HLS', primary and secondary Ingest server, and RTMP legacy?

I'm thinking check on DNS issues. I'm not sure if DNS sever resolves those destination FQDNs to an IP address located in the closet Youtube server or not. (e.g. We are based in APAC but the servers we are sending to are based in US)

 

[Suslik V]

Dropped frames calculation depends on selected output of OBS.
For example, RTMP:
...
logging - https://github.com/obsproject/obs-s...6f701ab836c3a60b1da5/libobs/obs-output.c#L397
getting dropped from the output - https://github.com/obsproject/obs-s...f701ab836c3a60b1da5/libobs/obs-output.c#L1155
definition per output - https://github.com/obsproject/obs-s...b1da5/plugins/obs-outputs/rtmp-stream.c#L1818
return value per stream - https://github.com/obsproject/obs-s...b1da5/plugins/obs-outputs/rtmp-stream.c#L1774
total accumulation - https://github.com/obsproject/obs-s...b1da5/plugins/obs-outputs/rtmp-stream.c#L1455

per "drop requested" accumulation - https://github.com/obsproject/obs-s...b1da5/plugins/obs-outputs/rtmp-stream.c#L1442​

per "packet add" accumulation - https://github.com/obsproject/obs-s...b1da5/plugins/obs-outputs/rtmp-stream.c#L1639​

drop condition - https://github.com/obsproject/obs-s...b1da5/plugins/obs-outputs/rtmp-stream.c#L1625
drop edge - https://github.com/obsproject/obs-s...plugins/obs-outputs/rtmp-stream.c#L1593-L1595
...
and so on.

Thus, for the mentioned output, dropping depends on type of the frame (P-, B-frames) and the duration of the stored frames. If you collected few seconds of frames in the buffer - it is time to drop some - here counter of dropped frames increases.

Now, you probably want to know how OBS sends packets (why they stalled in the buffer?!)... in short - OBS uses OS functions. So, @koala and @qhobbes answers are still valid - equipment, wiring and... luck.

 

[NomaiZ]

Hi @Suslik V

Thank you very much for your reply.
I'm network engineer, so I'm not familiar with github.
I found some Youtube URL on some file like https://github.com/obsproject/obs-s...b836c3a60b1da5/frontend/oauth/YoutubeAuth.cpp

#define YOUTUBE_AUTH_URL "https://accounts.google.com/o/oauth2/v2/auth"
#define YOUTUBE_TOKEN_URL "https://www.googleapis.com/oauth2/v4/token"

#define YOUTUBE_CHAT_PLACEHOLDER_URL "https://obsproject.com/placeholders/youtube-chat"
#define YOUTUBE_CHAT_POPOUT_URL "https://www.youtube.com/live_chat?is_popout=1&dark_theme=1&v=%1"

Could you do me a favor by sharing which file contains the URL of the YouTube stream server?

 

[Suslik V]

Could you do me a favor by sharing which file contains the URL of the YouTube stream server?

This:

obs-studio/plugins/rtmp-services/data/services.json at 80ea1b14fbf09eca19f56f701ab836c3a60b1da5 · obsproject/obs-studio

OBS Studio - Free and open source software for live streaming and screen recording - obsproject/obs-studio

github.com

I think, these questions are very specific and should be asked in the https://obsproject.com/forum/list/general-development.21/ section of the forum.

 

[koala]

I'm a network expert. We are in the midst of troubleshooting, but we cannot move on

To be honest, and no offense meant, I was talking about someone with a deeper knowledge than your posts did show. Some questions would not be asked by someone who is able to successfully do debugging of a live streaming Wireshark trace. I'm shouting silently to myself "stop asking questions and start doing it! (the network trace or whatever you intend to do)" when I read your messages. They appear to me as if you don't have a clue how to even approach the issue in the first place.

You asked about the definition of dropped frames, but this forum is for getting support of OBS issues, so post your OBS issue. Give a log that exhibits the issue, give a description of the issue, ask for support. Otherwise it might be we all (you and we) are dealing with a XY-problem.

The streaming servers contacted by OBS can be found in C:\Users\<user>\AppData\Roaming\obs-studio\plugin_config\rtmp-services. These lists are updated from somewhere once in a while.

 

[NomaiZ]

To be honest, and no offense meant, I was talking about someone with a deeper knowledge than your posts did show. Some questions would not be asked by someone who is able to successfully do debugging of a live streaming Wireshark trace. I'm shouting silently to myself "stop asking questions and start doing it! (the network trace or whatever you intend to do)" when I read your messages. They appear to me as if you don't have a clue how to even approach the issue in the first place.

You asked about the definition of dropped frames, but this forum is for getting support of OBS issues, so post your OBS issue. Give a log that exhibits the issue, give a description of the issue, ask for support. Otherwise it might be we all (you and we) are dealing with a XY-problem.

The streaming servers contacted by OBS can be found in C:\Users\<user>\AppData\Roaming\obs-studio\plugin_config\rtmp-services. These lists are updated from somewhere once in a while.

Thank you for the comment.

At first, I never said this is a OBS issue. I understand it can be a network issue streaming service issue or an issue in transit hops.

I also used layman's terms when I was asking. If my queries lead you to misunderstand, I apologize for this cause. I don't know the people who answer in this forum will be the OBS team or a user in the community, then I avoided adding in-depth wording in the query in the first place.

I appreciate your (and everyone's) long comments on this topic.
I know your answers are valid, but I didn't stop asking because your 1st and 2nd comments did not answer my queries directly. I tried to make my query simple by writing it as '1 or 2', 'Yes/No' question. Then I assumed the answer will be like
"It's 1) because ........."
"Yes, because .........."
"Your 1) and 2) are incorrect because ......"
I'm not sure if this is leading to XY problem.

In fact, I expected a clear answer in a few sentences
like "Dropped frame rate came from (frame sent from OBS)/(frame received by streaming service). If no return ACK packet from the streaming service within xx ms, the frame will be considered as a dropped frame".
If the software has a frame dropped counter, it should mean there is a hard-coded somewhere with a clear condition when the counter will be increased.

In my case, the user showed the OBS screenshot dropped frame (network).
I have read the below topic before posting this topic.
https://obsproject.com/forum/threads/dropped-frames-disconnecting-lag-read-this-first.8870/
But the info on the topic was not enough, then I created a new topic here.
The error in the screenshot is an error on the OBS software but points to a network issue. Then, the network team must understand the meaning of the error (dropped frame in this case) to ensure all troubleshooting tasks are in the right direction.

E.g. If OBS expects a return the ACK must arrive within 10ms, we collect packet capture by Wireshark from the PC and port mirroring from a switch. We found no missing ACK from the destination IP to the user IP. But return ACK arrives at the PC in 12ms.

In the above scenario, if a network guy doesn't understand the OBS, can a network guy say this is not a network issue :)

--------------
"stop asking questions and start doing it!" >>> We have tried more than one time. Bad luck when the problem was occurring the user was not ready to capture packet. And when I am ready, the problem has gone. The real environment is not easy to deal it as the expectation. There are many limitations and challenges.
--------------

Again, I appreciate everyone on this topic. If you guys feel my query should not be posted here, you may route me to the right place to ask about how OBS works.

 

[koala]

I know your answers are valid, but I didn't stop asking because your 1st and 2nd comments did not answer my queries directly. I tried to make my query simple by writing it as '1 or 2', 'Yes/No' question. Then I assumed the answer will be like
"It's 1) because ........."
"Yes, because .........."
"Your 1) and 2) are incorrect because ......"
I'm not sure if this is leading to XY problem.

I didn't answer that deliberately, because it could lead into a completely wrong direction. It's also that the answer is "it depends", and there are more things to consider. And it's that I don't know everything exactly. And you asked extremely specific questions. The more specific you ask, the more "it depends" is the answer. Tracking an issue is an iterative process. You don't start from the most specific thing, instead you start a structured analysis from the system as a whole and narrow down possible causes. But these causes cannot be researched as theory in the forum, you need to deal with the actual OBS installation and check this.

I always try to see the big picture, the whole system. So it seems you are on a company network and you have some user within that network who tries to stream to Youtube, and he has dropped frames due to network congestion. In this case I would narrow down the issue like this:
Is it related to Youtube or to every streaming provider (try a different streaming provider, for example Twitch, and use the twitchtest tool)
If just Youtube: find a different Youtube ingest server that works.
Is it related to the computer? (stream from a different computer to rule that out)
If the other computer works: fix the computer.
Is it related to the company network? (stream from the DMZ of the company network and directly use the company's internet access)
If streaming from the dmz works: make sure company intranet provides enough continuous bandwidth. Wired, no wifi, and not congested with file sharing and printing.
If not: stream from the outside with a different provider and see if streaming to Youtube in general works with your intended configuration.

A company network is a thing on its own, depending on the security you implement. Do you have firewalls who do stateful inspection? Which protocol do you use to stream, and how do the firewalls deal with that? ssl interception? All that middleware that might interfere with real time video sending.

In no case you need a network trace in the first place. It might be the last resort after you narrowed down to one system but cannot see what specifically does interfere.