Bug Report OBS 25.0.3-1: "Failed to open NVENC codec"

[Fabian]

Until yesterday I was able to record just fine in OBS, using my NVidia GPU. Then I installed updates and now I get this error message:
"Starting the output failed. Please check the log for details.
Note: If you are using the NVENC or AMD encoders, make sure your video drivers are up to date."
OBS was updated from 25.0.1-1 to 25.0.3-1, FFMPEG from 1:4.2.2-3 to 1:4.2.2-4 and a bunch of other programs and libraries, but not my graphics driver, that is still "video-hybrid-intel-nvidia-440xx-prime". I tried downgrading both packages (and "dav1d"), but I still get the same error message. A list of all updates that were done today is attached.
Here is the relevant log line (rest attached):

[NVENC encoder: 'streaming_h264'] Failed to open NVENC codec: Unknown error occurred

OBS still detects NVENC and also VAAPI and lets me select both, but only with x264 recording actually works (which is really slow and usually can't keep up well on my computer).
Since nothing graphics/OBS related changed on my computer except for updates, I assume this is a bug in either OBS or a library that OBS uses. Please tell me if there is a workaround (other than recording with CPU) or if this is a bug in a library (and in that case, where to report it).

GPU: NVIDIA GeForce GTX 1050
System: Manjaro 19.0.2
Graphics driver: video-hybrid-intel-nvidia-440xx-prime
OBS version: 25.0.3-1
FFMPEG version: 1:4.2.2-4

 

[Tuna]

17:08:14.951: Loading up OpenGL on adapter Intel Open Source Technology Center Mesa DRI Intel(R) HD Graphics 630 (Kaby Lake GT2)

OBS loads on your intel card. I guess nvidia then does really want to work.

 

[Fabian]

Here is the log when running "prime-run obs". It says "Loading up OpenGL on adapter NVIDIA Corporation GeForce GTX 1050/PCIe/SSE2" instead. But the error message at the end is the same.

 

[catalyst1987]

I appear to have the same problem on 25.0.3

 

[catalyst1987]

I uninstalled the flatpak version of OBS and installed the snap instead. Snap seems to work fine.

 

[Fabian]

I installed the snap version (without uninstalling the other one, that shouldn't matter) and it's worse, it doesn't even start:

fabian@laptop ~]$ env BAMF_DESKTOP_FILE_HINT=/var/lib/snapd/desktop/applications/obs-studio_obs-studio.desktop /var/lib/snapd/snap/bin/obs-studio
libva info: VA-API version 1.1.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /snap/obs-studio/1042/usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_1_1
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.1 (libva 2.1.0)
vainfo: Driver version: Intel i965 driver for Intel(R) Kaby Lake - 2.1.0
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            :    VAEntrypointVLD
      VAProfileMPEG2Simple            :    VAEntrypointEncSlice
      VAProfileMPEG2Main              :    VAEntrypointVLD
      VAProfileMPEG2Main              :    VAEntrypointEncSlice
      VAProfileH264ConstrainedBaseline:    VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:    VAEntrypointEncSlice
      VAProfileH264ConstrainedBaseline:    VAEntrypointEncSliceLP
      VAProfileH264Main               :    VAEntrypointVLD
      VAProfileH264Main               :    VAEntrypointEncSlice
      VAProfileH264Main               :    VAEntrypointEncSliceLP
      VAProfileH264High               :    VAEntrypointVLD
      VAProfileH264High               :    VAEntrypointEncSlice
      VAProfileH264High               :    VAEntrypointEncSliceLP
      VAProfileH264MultiviewHigh      :    VAEntrypointVLD
      VAProfileH264MultiviewHigh      :    VAEntrypointEncSlice
      VAProfileH264StereoHigh         :    VAEntrypointVLD
      VAProfileH264StereoHigh         :    VAEntrypointEncSlice
      VAProfileVC1Simple              :    VAEntrypointVLD
      VAProfileVC1Main                :    VAEntrypointVLD
      VAProfileVC1Advanced            :    VAEntrypointVLD
      VAProfileNone                   :    VAEntrypointVideoProc
      VAProfileJPEGBaseline           :    VAEntrypointVLD
      VAProfileJPEGBaseline           :    VAEntrypointEncPicture
      VAProfileVP8Version0_3          :    VAEntrypointVLD
      VAProfileVP8Version0_3          :    VAEntrypointEncSlice
      VAProfileHEVCMain               :    VAEntrypointVLD
      VAProfileHEVCMain               :    VAEntrypointEncSlice
      VAProfileHEVCMain10             :    VAEntrypointVLD
      VAProfileHEVCMain10             :    VAEntrypointEncSlice
      VAProfileVP9Profile0            :    VAEntrypointVLD
      VAProfileVP9Profile0            :    VAEntrypointEncSlice
      VAProfileVP9Profile2            :    VAEntrypointVLD
/snap/obs-studio/1042/usr/sbin:/snap/obs-studio/1042/usr/bin:/snap/obs-studio/1042/sbin:/snap/obs-studio/1042/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
Qt: Session management error: None of the authentication protocols specified are supported
Attempted path: ../../data/obs-studio/locale/en-US.ini
Attempted path: ../../data/obs-studio/locale.ini
Attempted path: ../../data/obs-studio/themes/Dark.qss
info: CPU Name: Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
info: CPU Speed: 2063.491MHz
info: Physical Cores: 4, Logical Cores: 8
info: Physical Memory: 15900MB Total, 10515MB Free
info: Kernel Version: Linux 5.5.13-1-MANJARO
info: Distribution: "Ubuntu Core" "18"
info: Window System: X11.0, Vendor: The X.Org Foundation, Version: 1.20.7
info: Portable mode: false
Fontconfig warning: FcPattern object weight does not accept value [40 210)
Segmentation fault (core dumped)

 

[Fabian]

I did almost nothing with my system since my last comment, except making a backup of it and updating the Minecraft launcher, but now recording with the GPU works again. Magic, I guess? My uptime is currently 29 hours, I hope that's not required for GPU recording. :D
Is there something to do for closing such a thread as "cannot reproduce anymore", like on a bug tracker?
BTW, @catalyst1987: Your error message says "function not implemented", that's different from my error. I've found a bunch of websites about your problem when researching mine, so maybe you can find an explanation why it happened reasonably easily.

 

[Harold]

Neither the snap nor the flatpak versions are provided by us. We can't guarantee ANY functionality with either of them.

 

[Fabian]

This seems to be fixed for my NVidia graphics card now, but not for the Intel one. Someone else reports it working for Intel again, but not for AMD.

 

[csirolli]

Came across the same issue, an hour and a half before our church service starts. Using a laptop with Intel Core i7 and NVidia GeForce GTX 870m and Linux Mint 19. I'm currently running some updates to see if that fixes it, but I figured I should mention it. Never happened before, and we use OBS every week and updates are usually done automatically.

Start up log:

07:57:21 AM.901: CPU Name: Intel(R) Core(TM) i7-4700HQ CPU @ 2.40GHz
07:57:21 AM.901: CPU Speed: 3192.785MHz
07:57:21 AM.901: Physical Cores: 4, Logical Cores: 8
07:57:21 AM.901: Physical Memory: 15896MB Total, 14024MB Free
07:57:21 AM.901: Kernel Version: Linux 5.3.0-59-generic
07:57:21 AM.901: Distribution: "Linux Mint" "19.3"
07:57:21 AM.906: Window System: X11.0, Vendor: The X.Org Foundation, Version: 1.20.8
07:57:21 AM.907: Portable mode: false
07:57:21 AM.955: OBS 25.0.8 (linux)
07:57:21 AM.955: ---------------------------------
07:57:21 AM.979: ---------------------------------
07:57:21 AM.979: audio settings reset:
07:57:21 AM.979:     samples per sec: 44100
07:57:21 AM.979:     speakers:        2
07:57:21 AM.991: ---------------------------------
07:57:21 AM.991: Initializing OpenGL...
07:57:22 AM.041: Loading up OpenGL on adapter Intel Open Source Technology Center Mesa DRI Intel(R) Haswell Mobile 
07:57:22 AM.041: OpenGL loaded successfully, version 4.5 (Core Profile) Mesa 19.2.8, shading language 4.50
07:57:22 AM.114: ---------------------------------
07:57:22 AM.114: video settings reset:
07:57:22 AM.114:     base resolution:   1280x720
07:57:22 AM.114:     output resolution: 1280x720
07:57:22 AM.114:     downscale filter:  Bicubic
07:57:22 AM.114:     fps:               60/1
07:57:22 AM.114:     format:            NV12
07:57:22 AM.114:     YUV mode:          601/Partial
07:57:22 AM.114: NV12 texture support not available
07:57:22 AM.115: Audio monitoring device:
07:57:22 AM.115:     name: Monitor of Built-in Audio Analog Stereo
07:57:22 AM.115:     id: alsa_output.pci-0000_00_1b.0.analog-stereo.monitor
07:57:22 AM.115: ---------------------------------
07:57:22 AM.118: Failed to load 'en-US' text for module: 'decklink-ouput-ui.so'
07:57:22 AM.256: A DeckLink iterator could not be created.  The DeckLink drivers may not be installed
07:57:22 AM.256: No blackmagic support
07:57:22 AM.483: NVENC supported
07:57:22 AM.483: FFMPEG VAAPI supported
07:57:22 AM.564: [obs-websocket] you can haz websockets (version 4.8.0)
07:57:22 AM.564: [obs-websocket] qt version (compile-time): 5.9.5 ; qt version (run-time): 5.9.5
07:57:22 AM.595: [obs-websocket] module loaded!
07:57:22 AM.622: VLC found, VLC video source enabled
07:57:22 AM.624: ---------------------------------
07:57:22 AM.624:   Loaded Modules:
07:57:22 AM.624:     libobs-linuxbrowser.so
07:57:22 AM.624:     vlc-video.so
07:57:22 AM.624:     text-freetype2.so
07:57:22 AM.624:     rtmp-services.so
07:57:22 AM.624:     obs-x264.so
07:57:22 AM.624:     obs-websocket.so
07:57:22 AM.624:     obs-transitions.so
07:57:22 AM.624:     obs-outputs.so
07:57:22 AM.624:     obs-libfdk.so
07:57:22 AM.624:     obs-filters.so
07:57:22 AM.624:     obs-ffmpeg.so
07:57:22 AM.624:     linux-v4l2.so
07:57:22 AM.624:     linux-pulseaudio.so
07:57:22 AM.624:     linux-jack.so
07:57:22 AM.624:     linux-decklink.so
07:57:22 AM.624:     linux-capture.so
07:57:22 AM.624:     linux-alsa.so
07:57:22 AM.624:     image-source.so
07:57:22 AM.624:     frontend-tools.so
07:57:22 AM.624:     decklink-ouput-ui.so
07:57:22 AM.624: ---------------------------------

Error:

08:24:42 AM.995: [NVENC encoder: 'simple_h264_stream'] settings:
08:24:42 AM.995:     rate_control: CBR
08:24:42 AM.995:     bitrate:      2500
08:24:42 AM.995:     cqp:          0
08:24:42 AM.995:     keyint:       120
08:24:42 AM.995:     preset:       hq
08:24:42 AM.995:     profile:      high
08:24:42 AM.995:     width:        1280
08:24:42 AM.995:     height:       720
08:24:42 AM.995:     2-pass:       false
08:24:42 AM.995:     b-frames:     2
08:24:42 AM.995:     GPU:          0
08:24:42 AM.995:
08:24:43 AM.000: [NVENC encoder: 'simple_h264_stream'] Failed to open NVENC codec: Unknown error occurred
08:24:43 AM.001: Stream output type 'rtmp_output' failed to start!  Last Error: Failed to open NVENC codec: Unknown error occurred
08:24:43 AM.001:
08:24:43 AM.001: Please check your video drivers are up to date.

 

[csirolli]

Not sure what went wrong, but updating and restarting fixes it

 

[Fabian]

This problem came back for me for some reason. Switching Output→Recording→Type to "Custom output" fixed it for some reason. I guess the default FFMPEG arguments are faulty in the latest version. This is definitely a bug. Is there a better place to report this than this forum?

 

[Romans_I_XVI]

This problem came back for me for some reason. Switching Output→Recording→Type to "Custom output" fixed it for some reason. I guess the default FFMPEG arguments are faulty in the latest version. This is definitely a bug. Is there a better place to report this than this forum?

This fixed it for me as well. Thank you for the workaround.

 

[Romans_I_XVI]

Just realized that the only reason this fixed it is because it was disabling hardware encoding. Keeping Standard output type with software encoding also works.

Additionally, I use Shotcut and that won't work unless I disable hardware encoding either. So this is likely not a problem with OBS and rather a problem with... well I don't understand enough about how encoding works to have the answer to that. Possibly a Nvidia driver issue?

 

[Fabian]

I finally solved this again: I noticed some irregularities in my installed packages, some started with "linux58", some with "linux59" and other things like that. I made sure that everything matched and now OBS works properly again.
Currently installed graphics and kernel related packages: lib32-libva-mesa-driver 20.1.8-1, lib32-mesa 20.1.8-1, lib32-mesa-vdpau 20.1.8-1, lib32-nvidia-450xx-utils 450.80.02-1, lib32-vulkan-icd-loader 1.2.153-2, lib32-vulkan-intel 20.1.8-1, lib32-vulkan-intel 20.1.8-1, lib32-vulkan-radeon 20.1.8-1, libva-mesa-driver 20.1.8-1, linux-firmware 20201005.r1732.58d41d0-1, linux-firmware 20201005.r1732.58d41d0-1, linux-latest 5.8-2, linux-latest 5.8-2, linux-latest-nvidia-450xx 5.8-2, linux-latest-nvidia-450xx 5.8-2, linux-latest-virtualbox-host-modules 5.8-2, linux-latest-virtualbox-host-modules 5.8-2, linux58 5.8.16-2, linux58 5.8.16-2, linux58-headers 5.8.16-2, linux58-headers 5.8.16-2, linux58-nvidia-450xx 450.80.02-3, linux58-nvidia-450xx 450.80.02-3, linux58-virtualbox-host-modules 6.1.14-13, linux58-virtualbox-host-modules 6.1.14-13, mesa 20.1.8-1, mesa-vdpau 20.1.8-1, mhwd-db 0.6.5-4, mhwd-nvidia-340xx 340.108-1, mhwd-nvidia-390xx 390.132-1, mhwd-nvidia-418xx 418.113-1, mhwd-nvidia-430xx 430.64-1.0, mhwd-nvidia-435xx 435.21-1.0, mhwd-nvidia-440xx 440.100-1, mhwd-nvidia-450xx 450.80.02-1, mhwd-nvidia-455xx 455.28-2, nvidia-450xx-utils 450.80.02-1, nvidia-prime 1.0-4, nvidia-utils 450.80.02-1, opencl-nvidia-440xx 440.100-1, virtualbox 6.1.14-1, virtualbox 6.1.14-1, vulkan-icd-loader 1.2.153-2, vulkan-intel 20.1.8-1, vulkan-intel 20.1.8-1, vulkan-radeon 20.1.8-1, xf86-video-amdgpu 19.1.0-2, xf86-video-ati 1:19.1.0-2, xf86-video-intel 1:2.99.917+908+g7181c5a4-1, xf86-video-nouveau 1.0.16-2

 

[Fabian]

I missed the previous comments. The "fix" was indeed a workaround, yes. At some point I was only able to use hardware encoding for recording, but not for streaming.
I guess OBS can't deal well with parts of multiple kernel versions being installed and also doesn't really know what error message to show when that happens. I read that only Manjaro even allows such a situation, so maybe the OBS devs don't test on such a system.

 

[Tuna]

OBS only makes use of the features the OS provides. If you screw up the system features OBS will not magically fix it. If you want to have multiple kernel versions it needs to be properly installed and handled by the system. If you do that things will just work fine.

 

[Fabian]

Manjaro fully supports having multiple kernels installed at the same time, that doesn't mean that the system is "screwed up". I just happened to not need it, so I removed everything non-5.8 and installed the remaining 5.8 parts. Everything other than OBS had no problems with it, not even VirtualBox, which heavily relies on kernel features.

 

[LifeofLemonsYT]

try going into settings, go to output, and change the encoder from hardware NVENC to software.
worked for me on fedora 34.

 

[Fabian]

Sure, but then you're using the CPU to encode, which is a lot slower and needs a lot more power, so your recording will likely lag, OBS will use your entire CPU so that very little is left for other programs and it probably also uses more electricity and heats the computer more. That's not a good solution.