Question / Help Which OBS setting give the best recording quality to me?

[SilverTiger]

Hi! I wanna record games like Left 4 Dead 2 and wanna edit the material after recording. I dont wanna upload it I just wanna haev some high quality videos of me playing. I wanna record in 1080p , 60fps. Now I have a ton of question :D

1. Which encoder is the best in terms of image quality to me? NVENC new oder X264?
2. Is CQP or CBR a better option?
3. Is look ahead and psycho visual tuning improving me image quality if im using NVENC new? Or should I turn of one of the both things.
4. Is there a way to play in 144hz/fps while recording 60FPS without getting those micro-stuttering?

Hardware:
AMD Ryzen 9 3900X
RTX 2080 Super
32 GB DDR4-3600 RAM
1 TB 970 Evo Plus SSD (here I record the files on)

Thanks!

 

[R1CH]

Use Tools / Auto Configuration.

 

[SilverTiger]

Any more detailled answeres?

 

[FerretBomb]

1. Which encoder is the best in terms of image quality to me? NVENC new oder X264?

NVENC on the Turing series cards like your 2080 is around x264 Slow. I'd recommend using NVENC.

2. Is CQP or CBR a better option?

For recording, CQP. CBR is only really used for streaming, as many streaming services require it. CQP is much better, and maintains a given image quality level without wasting bitrate, or choking on not-enough.

3. Is look ahead and psycho visual tuning improving me image quality if im using NVENC new? Or should I turn of one of the both things.

Both of those should be OFF. They use CUDA cores, and generally cause a lot of problems. Same with the Max Quality preset; you should use Quality instead.

4. Is there a way to play in 144hz/fps while recording 60FPS without getting those micro-stuttering?

No. Drop to 120hz on your monitor; having an even divisor will allow a steady frame pacing to remove the stutter/judder.
Also be aware that if you have multiple monitors, ALL of them have to be running at the same refresh rate due to a long-standing bug in Windows. It's planned to be fixed in Win10 2004, but that isn't due out until later this year, or next year.

Really though, you should do four things:
1) Swap back to SIMPLE output mode. Advanced generally is not needed, outside a very small number of situations.
2) Use the Indistiguishable output quality level
3) Use NVENC as your encoder
4) NEVER record directly to MP4 for any reason.

 

[koala]

If you tell the auto configuration wizard to prioritize recording over streaming, it will probably choose for you: simple output mode, nvenc (new), "indistinguishable quality". Use this, it's good.

There is no way to avoid microstutter if you play the game with 144 fps while recording with 60 fps. Play with 120 fps, this is evenly dividable by 60 and will not get you microstutter.

 

[SilverTiger]

But I don't understand why CQP is better than CBR for recording. CQP saves more space ok, but in terms of quality a very high bitrate for CBR should be as good as a CQP at the sem bitrate or not?

I play with 120 FPS now :) Works fine

Another question is: I evertime get some "lost frames" in the stats (with nvenc and x264 too, doesnt matter which settings). How can I avoid that? Or do you measure in a wrong way with the OBS stats window?

 

[FerretBomb]

CQP is an image-quality based encoding target. It uses as much or as little bitrate as needed to maintain a consistent image.
CBR uses a set rate regardless of what's happening in the video. So no matter if you're showing a single static image, or incredibly high-motion video, it will use the same bitrate regardless... leading to a massive waste in the case of the static image, and severe video quality degradation in the high-motion scene.

CQP isn't about saving space. It's about intelligently using exactly as much bitrate as is needed at the moment.

Running a ridiculously high CBR bitrate tailored to your highest possible motion scene could compensate to avoid choke, but that would be pointlessly and utterly wasteful. You would gain zero benefit, and just load your recording drive down with superfluous 'dead' data... empty padding eating up drive space to meet the set constant bitrate.

Depends on which 'lost frames' you mean. There's no metric by that name in the OBS Stats dock.
Frames missed due to rendering lag generally are due to an over-loaded GPU.
Skipped frames due to encoding lag normally mean that your encoder settings have a problem, or are set too high for your hardware.
Dropped frames are only encountered during streaming, and are always a network issue.

 

[SilverTiger]

Ok :) Thx for your help.

But how can I monitor how many frames are skipped really? Or how can I see if my encoder has an overflow? when im plaing i cant see OBS and when OBS is open im not playing :D

 

[carlmmii]

Post a log file of your recording session where you were experiencing the frame loss.

 

[SilverTiger]

First time i recorded with NVENC CQP 10 and second time x264 fast with CBR 120000. Just for testing. Is there any frame skipping or losing or whatever?

 

[carlmmii]

14:14:38.368: Running as administrator: false

14:17:30.212: [jim-nvenc: 'recording_h264'] settings:
14:17:30.212:     rate_control: CQP
14:17:30.212:     bitrate:      0
14:17:30.212:     cqp:          10
14:17:30.212:     keyint:       250
14:17:30.212:     preset:       mq
14:17:30.212:     profile:      high
14:17:30.212:     width:        1920
14:17:30.212:     height:       1080
14:17:30.212:     2-pass:       true
14:17:30.212:     b-frames:     2
14:17:30.212:     lookahead:    false
14:17:30.212:     psycho_aq:    false

14:18:00.167: Output 'adv_file_output': Total frames output: 1750
14:18:00.167: Output 'adv_file_output': Total drawn frames: 1693 (1796 attempted)
14:18:00.167: Output 'adv_file_output': Number of lagged frames due to rendering lag/stalls: 103 (5.7%)
14:18:00.167: Video stopped, number of skipped frames due to encoding lag: 127/1794 (7.1%)

14:18:25.201: [x264 encoder: 'recording_h264'] preset: fast
14:18:25.201: [x264 encoder: 'recording_h264'] profile: high
14:18:25.201: [x264 encoder: 'recording_h264'] settings:
14:18:25.201:     rate_control: CBR
14:18:25.201:     bitrate:      120000
14:18:25.201:     buffer size:  120000
14:18:25.201:     crf:          0
14:18:25.201:     fps_num:      60
14:18:25.201:     fps_den:      1
14:18:25.201:     width:        1920
14:18:25.201:     height:       1080
14:18:25.201:     keyint:       250

14:19:01.357: Output 'adv_file_output': Total frames output: 2061
14:19:01.357: Output 'adv_file_output': Total drawn frames: 2060 (2169 attempted)
14:19:01.357: Output 'adv_file_output': Number of lagged frames due to rendering lag/stalls: 109 (5.0%)
14:19:01.357: ==== Recording Stop ================================================
14:19:01.357: Video stopped, number of skipped frames due to encoding lag: 8/2168 (0.4%)

The main issue you're running into is rendering lag. This can be fixed by running OBS in administrator mode -- this will tell windows to add OBS into the GPU priority so that it can perform the necessary scene compositing without being impeded by whatever other GPU functions are running (like your game).

For your nvenc recording, you're using Max Quality -- this setting requires CUDA, so it uses extra GPU resources outside of the NVenc encoder (Look Ahead and Psychovisual tuning also do this, just for an fyi). That's the likely reason why you have extra encoding lag on top of rendering lag. Switch off of Max Quality to avoid this.

Your x264 encoding looks fine past the rendering lag, but honestly for recording there shouldn't really be any reason to not use NVenc as long as the issues are cleared up.

Something extra that I will mention.... do NOT record directly to .mp4. This container format requires metadata to be recorded at the end of the recording, and if the recording is interrupted prematurely for any reason (i.e. crash, or badly behaving OS), then the entire recording will be corrupted. Record to either .mkv or .flv instead -- if you need .mp4 for editing or otherwise, then you can remux the recording afterward, either by doing this manually through the File menu, or automatically after each recording (which you can enable via the advanced options).

 

[SilverTiger]

Ok, but if Im playing and recording longer the encoding lag deacreases to 0,2%on maxquality with NVENC. Is it possible the lag is caused by maximazing the game again after recording? And if Im playing longer this "start lag" takes less and less impact on the stats?
Btw: I want to record in the best possible quality in terms ofimage quality and smoothness. This is why I would miss recording with the max quality setting. I have a 2080 Super (overclocked) I thought this should be able to run max quality :D

 

[SilverTiger]

And another question: Should I turn game mode on or off? I have the newest version of windows 10.

 

[carlmmii]

Since you're targeting a specific quality level (the cqp 10 option you have set), then the quality will be fixed. It doesn't matter if you use the Max Quality or Max Performance setting -- the output is going to have the same level of quality. The only difference will be that Max Quality might take a slightly smaller file size to get that same quality.

Because of the potential of having that extra impact due to CUDA integration, it is best to switch off of Max Quality. There is 0 quality for a frame that doesn't exist due to encoding lag.

As far as game mode, supposedly it doesn't matter any more for OBS performance as long as you're running as administrator. YMMV.

 

[SilverTiger]

Ok I changed to "quality" and tested even lower settings. But when I start recording I everytime getting skipped frames :D Now I tried to enable recording with hotkeys. So I dont have the minimizing and maximizing game lag issue I suspect tobe responsible for the skipped frames. But when I start recording with hotkeys there is NO logged information about skipped frames in my logfile? Does this mean there are no more lag issues? Thx

 

[carlmmii]

If you see the "Recording Stopped" log block with no line for "frames lost due to rendering lag" or "frames lost due to encoding lag", then there were no frames lost.

 

[Deleted member 330497]

CQP is much better, and maintains a given image quality level without wasting bitrate, or choking on not-enough.

What CQP value would you recommend for recording fast-paced action games? OBS by default suggests 25. I use NVENC.

3. Is look ahead and psycho visual tuning improving me image quality if im using NVENC new? Or should I turn of one of the both things.

Both of those should be OFF. They use CUDA cores, and generally cause a lot of problems. Same with the Max Quality preset; you should use Quality instead.

Is this still a thing in 2021? Even with the latest NVIDIA drivers and OBS 27?

4) NEVER record directly to MP4 for any reason.

Excluding a situation when your OS or PC suddenly shuts-down/crashes while recording (causing the recorded file to be corrupted), is there any other reason why MP4 shouldn't be used for recording?

 

[FerretBomb]

What CQP value would you recommend for recording fast-paced action games? OBS by default suggests 25. I use NVENC.

22 is the general good-enough point. 16 is essentially visually lossless. 12 if you plan to edit and re-encode later, to minimize any re-encoding artifacts. The lower you go, the larger the files will be.

Is this still a thing in 2021? Even with the latest NVIDIA drivers and OBS 27?

Yes.

Excluding a situation when your OS or PC suddenly shuts-down/crashes while recording (causing the recorded file to be corrupted), is there any other reason why MP4 shouldn't be used for recording?

MP4 can actually cause a crash during finalization on larger recordings. Many editors do not like the native-record MP4 files OBS generates (Premiere is especially bad about this, and can crash outright sometimes), but work fine with remuxed files.
The short version is "don't". It's not a recording-safe format. You can drive nails by beating on them with a hand grenade, but anyone who knows what they're doing is going to look at you sideways when you have a hammer sitting literally right there.

 

[Deleted member 330497]

3. Is look ahead and psycho visual tuning improving me image quality if im using NVENC new? Or should I turn of one of the both things.

Both of those should be OFF. They use CUDA cores, and generally cause a lot of problems. Same with the Max Quality preset; you should use Quality instead.

Is this still a thing in 2021? Even with the latest NVIDIA drivers and OBS 27?

Yes.

I usually play (and record) games like Quake 3 and Unreal Tournament which are very old (both were released in 1999), and as a result, they don't use CUDA cores and generally don't cause massive impact on modern GPU. Would it still be reasonable to disable Look ahead, Psycho Visual Tuning and use the Quality preset instead of the Max Quality?

 

[carlmmii]

For your situation, you should be fine, since those games are likely to be bottlenecked by things other than the GPU.

In general though, it is best practice to leave those options off, since all that needs to happen for the encode is to have each frame processed within the NVENC portion of the GPU, and the frame is output. If CUDA actions are needed, then the encoder needs to run extra processing through the GPU proper, which can cause issues if the GPU is prioritized with running the game.

The only downside from leaving these options off is a slightly higher file size. As long as CQP is used, the quality should remain the same.