Question / Help Even with NVENC card, i need to use CPU encoding right?

[Matthew Paluch]

Card - GTX 1660 SUPER, bought mainly so I could encode with NVENC

but, my monitor is 1440p@144hz - since i run high/max graphics settings, i never hit 144 fps. So, for example war thunder, im around 110fps - but doesnt that mean the GPU is maxxed out, *trying* to hit 144fps? So it has zero room to spare for any encoding?

Star trek online allows me to limit frame rate to 60, so for that one title I could use NVENC since the card isnt maxxed at all.

But in general, isnt my 1440p@144 monitor going to force x264 CPU encoding?

 

[carlmmii]

Um.... no.

The thing about pushing your GPU to the max actually has very little effect on encoding ability if you use nvenc. What is does affect is OBS's ability to render the frames to output back to the encoder -- this is what's known as rendering lag, and is the main reason it's recommended to avoid maxing out your GPU, since windows GPU prioritization doesn't handle load balancing the way it needs to in order to ensure OBS is able to get the frames within the strict 16.6ms frame timing window for live capture. It wouldn't matter whether you're using NVenc or x264 encoding -- because there's rendering lag, that means OBS can't get the necessary frames drawn before it even gets to the encoding phase.

There is a workaround in 24.0.3 that was implemented where if you have windows Game Mode ON, and you run OBS in administrator mode, then it will attempt to force windows to include OBS in the GPU priority list, but this still has mixed results -- if it works, great... but don't expect it to solve everything.

With all that said, NVenc is completely separate from "maxing out" your GPU. The NVenc encoder is completely separate silicon from the other portions of the GPU, and as such is specialized to only handle encoding functions. It is not used unless encoding is taking place -- it cannot in any way be used for normal game rendering. That said, there are functions for the (new) nvenc encoder that can leverage CUDA to improve encoding quality. These can be turned off if you want to ensure that only the NVenc encoder is used, and will result in the encoding being essentially "free" (or at least, free in the sense that when the encoder gets its frame, there's no extra impact past that -- it still eats a little bit into VRAM bandwidth).

Btw, the functions that require CUDA (and should be disabled if you're running GPU maxed):
- Max Quality
- Look Ahead
- PsychoVisual Tuning

 

[Matthew Paluch]

Thanks, thats a great education and well written! I had no idea about NVENC, nor Max quality (I had been warned off of look ahead and psycho).

I ended up (temporarily at least) solving my problems by coming across the NDI Plugin - I had an old (2015) gaming laptop, so since i love tech and exploring and dont mind failure, Ive spent the last 4-ish hours playing with it.

NDI worked like a charm. Now I have minimal needs - Im just trying to get a stable 720p@30 stream up and running, but the streaming worked (laptop NVENC), the recording worked (laptop x264), and I just ran a 3rd test streaming and recording at the same time, and it went great! (I am exahusted though)

For those wanting more detail, the GPU is a GTX 960M. OBS said it could handle 1080p@30 but I settled for 720p@30, esp with no many more unknowns ahead.

The CPU is an i7-6500U (2c/4t) which is 2.5ghz stock but always runs at 3.2 thanks to... I honestly dont recall. Some bios or "default"overclocking, nothing i am running.

When recording at 720p@30 (Simple), it hits around 35% utilization. Since bumping it up to 1080@30 would double the workload, I am leaving everything at 720@30. Honestly my audio stinks so I have plenty to work on :D

OBS rules! And thank you carlmmii! I am back in business with my gaming machine not having to do any work, and my dust-covered laptop back in action. Good start to 2020

 

[carlmmii]

Good to hear you're running good with NDI.

I will mention though, I would see if you can flip your encoders for stream vs recording -- streaming is very bitrate limited, and x264 is going to be a much better quality encoder compared to what your 960M will output, as long as you can at least do veryfast or faster for the x264 encoding preset. This is because older NVenc generations have lesser quality... only recently with the Turing encoder (which you do have in your 1660ti) has there been a real incentive to switch to using NVenc over x264 veryfast/faster.

For recording, it's generally recommended to use NVenc and just use a CQP value of around 20-25 (the lower the number, the higher the quality) -- that will let you just target an output quality you want, without having to worry too much about having to find the right bitrate for it.

 

[Matthew Paluch]

I tried that, but the NVENC recording had jitters and hiccoughs. I dont know why. x264 was zero errors. shrug, im learning as i go.

Tomorrow I will follow your advice and reverse them, changing it from Simple to Advanced so I can set recording to CQP 20-25 -- thanks for the advice! Easy enough to set it back if it pikes :)

Seriously, thanks!!!
matt

 

[carlmmii]

Cool deal. If you have the time, upload a log file from your recording/streaming attempts from your laptop. We can take a look at what's actually going on there.

 

[Matthew Paluch]

We have progress, thanks again! Im not uploading a logfile at the moment because it had the entire session, where i changed stuff repeatedly and did small trials, so it was a mess to read.

end of todays session/battle (hehe):

- the NVENC recording had all of the problems of stuttering I associated with it. Its stats were amazing tho - 100 dropped frames out of 50,000. The streaming stats seemed good too. THEN, as you predicted, the final part of the log showed 26% render lag - i dont know why the streamer stats looked OK, but it was clear something (and it had to be the cpu) was bogging down everything - sure i was recording frames, but they were old stale ones apparently.

adjustments:

- I changed streaming to veryfast, CBR 6000, keyframe 2, profile High. I got great feedback from a viewer yesterday when i was streaming with NVENC, so lets hope it still looks good with these. trying to use fast and faster seemed to work at first but did not under real world usage. CPU seemed to max out at 50%, which sounds about right to me! I really dont want to go any higher now that I know render lag can affect everything and its brother.

- I changed recording from CQP 20 to 24, and went ahead and changed keyframe from 0 to 2 here as well, just to make it match, not bc i know what i am doing. The recording looked GREAT, no loss in quality/color/contrast/sharpness from 20, and now its 99 percent smooth! HUGE change - but again, none of these changes may have been necessary, if i had overloaded the cpu to the point of generating massive render lag. But it looks great at 24, so I am leaving it be. I can always lower it and goof stuff up later haha :D

Thanks a ton! I can now record in quality, which is 75% of what i wanted. I am hoping those streaming presets still give a decent quality stream (ppl really did like the NVENC test stream i did), but I shall try that out tomorrow.

Thanks carlmmii!

matt

 

[Matthew Paluch]

Well, the end result was worth it and I cant thank you enough, although there was a turn in developments haha!

I took your advice and turned Game Mode ON. Immediately I noticed a moderate difference in fullscreen games, varying on how old they were (the older the game, it seemed, the more it helped - I had some older games that were getting worse fps that War Thunder max gfx and even HDR turned on, etc).

Then I set OBS to run as admin... and all of the weird mysterious bottlenecks that i did not think i should have with my cpu/gpu... went away.

While I am glad I went thru the whole process of learning NDI to get my laptop to do all the recording and encoding, I can now do both at the same time, at twice the framerate, on my gaming pc, and the games FPS have actually *gone up* haha!

So, thanks! I could not have figured out the NVENC recording while streaming without you, and it seems the Game Mode tip was the masterstroke.

Thanks!!!

 

[carlmmii]

Woo! Glad to hear you've got things sorted with the 24.0.3 workaround!

Also, something to mention now that you've got things going smoothly on your gaming pc -- that 1660ti also means that you should also be outputting even better quality than what you were on your laptop. Also, just in case you haven't discovered it yet, your card should be able to support 2 simultaneous NVenc encoding sessions at once, meaning you can use NVenc for both streaming and recording, both with their own settings (you do have to make sure nothing else tries to use an NVenc session, such as Game DVR, or some games like Modern Warfare that use NVenc to record game highlights automatically).

 

[Matthew Paluch]

:0

wow, ill have to try NVENC on both! LOL more testing but i love it, thanks!

Is there a difference between (same as stream) and manually choosing NVENC if i am using NVENC to stream?

 

[carlmmii]

The (same as stream) option will not do any extra encoding -- whatever you're encoding to send out to stream, that's what gets packaged up in your file that's being recorded. No extra processing used, just copying bits to your drive.

Manually choosing separate encoder settings for your recording will use a 2nd encoder process -- you can use any encoder you want, including NVenc here as well. Nearly all of the mainstream Nvidia cards allow 2 simultaneous NVenc encoding sessions, so that's why this works.

 

[Matthew Paluch]

well thats super helpful. So the record with (same as stream) would literally be recording what my viewers saw (or i should say, what i am at least trying to send to the server), with no additional x264/cpu/nvenc/gpu changes. thanks!