Jittery Lines and Washed Out Colours

[Porygondolier]

Hello, my PC really isn't up to much of anything, but I'm trying to get the most out of OBS. I'm mostly recording Pokemon Red playthroughs with a custom overlay at the sides. In you look particularly at the top box with the badges in on the left hand side, you can see some weirdness with the bottom line in the video below:

Its quite mild in this example but it gets really bad sometimes, with similar "jittery lines" in the actual gameplay window as well.

Also in Image:

There are quite a few other artefacts in there as well but the line as the bottom is the most noticeable.


As for colours, I've tried all kinds of different colour settings but not had much luck resolving this one.

In particular the red text (124 and 114) comes out quite washed out. It's a typical FF0000 red colour but no amount of messing with settings has allowed me to get it to look right in the recordings. I guess this vivid red is not part of most colour spaces, but I've seen that other YT channels have been able to include vivid colours in their videos, so I figure there must be a way.


Here are the settings I'm currently using:

Output -> Recording:

These are the encoder options I have, the NVIDIA ones don't seem to work with my graphics card though (it complains about drivers)

I've also tried mkv for the recording format but it didn't show any improvement for me (and I can use mp4 directly in editing hence I've been sticking with that for now, open to change though).

Advanced:

I've tried all of these:

If the answer is "go buy a new PC" - fair enough, that seems reasonable. But if there are any improvements (even small ones) to be had, I'd really appreciate any advice.

Thanks!

 

[qhobbes]

Change the Advanced Video Colour Range from Full back to Limited. Nintendon't do HDR. Set the SDR White Level to 300 nits (HDR Nominal Peak Level: 1000 too. Create a separate profile if you need to use HDR.)
Depending on your graphics card, you may need to use an older version of OBS to use NVENC. Change the Encoder Profile to High. If you notice your recording lagging or overloaded, kick it down to Main, which is less taxing on your PC but won’t give you an image as sharp as High. Also try the animation Tune. I know it's Pokemon on a GB emu with a static border (which is good) but you're still using a third of the recommended bitrate for 720 60 FPS.

Please post a log with your issue! Here's how...

OBS Studio creates a log file each time you run it which contains very useful diagnostic information. Without a log file, it's much harder for any support volunteers to figure out problems and your post may be ignored. To upload a log file, go to the Help menu > Log Files > Upload Last Log...

obsproject.com

 

[Porygondolier]

Hi, thanks for the advice. I've made the changes and also increased bitrate to 10,000 kbps. It's certainly reduced general artefacts I'd say, although some line jitter remains.
I also started messing around with the CPU Usage Preset and found that at "Medium" the jittery lines seem to be reduced - but not gone completely, however, at that point OBS is using 43% of my (dreadful) CPU and the fans are going wild. (not the end of the world)
Changing to "slow" starts to cause the game itself to drag and footage becomes choppy. I currently manage the jittery lines by having a version of my overlay that I drop on top in post to hide those edges and replace them with clean ones, that's no good for when it happens in the actual gameplay portion of the screen though. Would an NVENC encoder get me better quality or just move some of the load to my (12 year old) GPU?

Colour wise, it certainly hasn't affected it negatively, but the red text in particularly still looks pretty dull. I did manage to get the red a bit nicer in the past using a 4:4:4 Colour Format, but found that then had to converted to allow Davinci to use it and by the time it had been through Davnci rendering, it was back to being pretty poor.

 

[qhobbes]

This isn't exactly apples to apples but here's x264 720 60 FPS 6 Mbps and here is NVENC 1440 60 FPS 24 Mbps (4x the bitrate but 4x the pixels). I think the quality is better for real time encoding.
Here's a more apples to apples quality test:
x264 720 60 FPS
Rate Control: VBR, Bitrate: 7500 Kbps, CRF: 23, Keyframe Interval: 2, CPU Preset: superfast, Profile: high, Tune: stillimage (don't use stillimage for games), x264 Options: threads=4 rc-lookahead=30 trellis=1 direct-pred=spatial
NVENC 720 60 FPS
Rate Control: VBR, Bitrate: 7500 Kbps, Max Bitrate: 7500 Kbps, Keyframe Interval: 2, Preset: Max Quality, Profile: High, Max B-frames: 2

Test using Microsoft Solitaire & Casual Games.
Going to recommend you go with Chroma subsampling: 4:2:0 per YouTube recommended upload encoding settings.

 

[Porygondolier]

There's definitely some improvement yes. Interesting that either way, it adds some Darker pixels around the edge of the heart. The most vivid reds are also darker in comparison to the preview. On the red issue, maybe I just need to not use red text!

 

[newbie-02]

I'm on Linux but experience similar, three observations:

in screen recording of sharp text, no antialiasing, colors are rendered with gradients at all edges,
chaotically sometimes some edges only shaded and some with a first intensified and then shaded
fringe, while black on white is kept with sharp contrast,

one special setting, Settings - Advanced - Color Format - I444 (8-bit, 4:4:4, 3 planes),
doe's not! create this blurryness,

I couldn't find any! settings which perform pixelsharp downscaling even for integral relations,
think 2:1 should / could make a 4-pix -> 1-pix calculation, producing max 1 pix seam at sharp defined
edges, but it's always more, 5 to 8 steps of shading, producing wide fringes, bad contrast and
bad readability.

 

[newbie-02]

one bias by me: having a downscaled recording 4k -> 1080p with a 1-px seam and showing
fullscreen on a 4k display, results in two scalings / re-renderings / resolution adaptions,
and the 1-px seam legally blows up to more steps.

Having that ruled out, play in a smaller window, it's still that e.g. i420 recording shows
clean around black on white text, but blurry around colored text. See pic_01 below.

In contrast to i444 recording which has color text as sharp as black on white, see pic_02 below.

Both! recordings show as 'sharp' in OBS preview.

Assume ... just assume ... not intentional but a simple programming fail like 0..255 vs. 1..256
counting in some assigning ...

pic_01, unsharp encoding:

pic_02, sharp encoding:

 

[newbie-02]

pardon, correction, above pics have been made avoiding downscaling,
by recording 1080p -> 1080p and placing the source in a smaller window.

Working with 2:1 downscaling i444 produces a 1-px seam around b/w and colored,
while i420 has 1-px around b/w, and more pixels partly with intensified color
around colored text.