Partial background removal ?

[shodanx]

Hi,

I want to perform background removal but only on a small portion of the screen

I've installed https://obsproject.com/forum/resources/background-removal-portrait-segmentation.1260/
Works great out of the box, it's turning my whole background into green screen

However I would like this filter to apply only to a small rectangular area of the scene and leave the rest intact

This area of the screen, I will place a display capture

The end result is, my camera's view except instead of a piece of blank wall, you can see my screen, but if I place my hands or face in front of the section, I will appear to be over the display capture as opposed to the display capture being on top of my image

Perhaps there is another way to do this but I'm not sure. I feel I am very close to a working solution

Here is a look of what it's going to look like, sort of

 

[koala]

Unfortunately, I don't understand your description of what part of what source you want to be "above" or "below" some other source. May be you describe this better and you can work out a solution yourself. What parts of what source should obscure or be obscured by what other source(s)?

OBS layers the sources on top of each other. Non transparent source parts obscure lower sources, while transparent parts will let lower sources shine through.

If you want a source partially above (obscure) and partially below (be obscured by) some other source, you need to order your sources carefully. If the same source should obscure and be obscured at the same time by some same other source, you can add another instance of the source on top and filter out the part you want to overlay over everything else with an image mask. To make that mask not apply to the other instance, add a group and put the second instance into a group and apply the image mask filter to the group.

 

[FerretBomb]

You'd probably want to either:
-Set up a greenscreen on that wall and chromakey it out, placing the display capture UNDER the camera layer so it will peek through (required if the camera will move at all), or
-Use an Image Mask on the camera layer, again placing the screen capture under the camera layer. Only viable with a static, non-moving camera.

Please be aware that while a corner-pin plugin exists to allow setting the display capture into perspective to match the opening, it does NOT have any sort of match-move/tracking key capability to keep the display capture in place relative to the camera. Just 'dumb' layering.

 

[shodanx]

Hello,
Thank you both for your assistance so far, I don't quite yet understand what you mean but I am studying OBS this afternoon hopefully I manager to integrate your advice into a working setup ! I think we are almost there.

I want to re-iterate my question a bit clearer, I have made screenshots to help

So, this is roughly my end goal except I want my hands and face to be above the display capture but not the background (wall).
I don't want to setup a green screen on my wall if that can be avoided.

So the first step might be a background removal like so

So what I want to do is perform background removal but only in that rectangle
Background removal , replaces the background with chroma key color

I've just watched this excellent tutorial again https://www.youtube.com/watch?v=0DD2RNcP0KQ

So, as the tutorial says, I added a chromakey effect filter to my camera and then I moved the display capture behind the camera

Now the display capture shows through the camera's background my hands and face show above the display capture

Here is the result so far

So last piece of the puzzle, how do I restore my camera's original image over all of that except the display capture area

I tried adding a second source of the camera with the hope of punch a rectangle through it
Or maybe cropping the filtered image down to just the display capture area and placing that above the original camera
However, adding the camera again makes the first instance of the camera stop working

I feel I am very close to a working solution !


-----------------------------------------
My camera settings here

 

[koala]

However, adding the camera again makes the first instance of the camera stop working

Add the second instance of the camera with the "add existing" option.

 

[shodanx]

Ok found where "add existing" was

First problem is how to I place the second camera to the exact same coordinates and scale as the first one so that the pixels line up exactly ?

And second, these two instances of the same camera cannot have different filters as far as I can tell ?

Is there a way to apply the background removal filter and chromakey effect only to one of the two instances ?

Perhaps I can output the camera to a buffer and then run the filters and chroma key only on that buffer ?

 

[koala]

Position and size: Position, size and other parameters of a source are called and saved as "Transform". To copy these properties from one source instance to another, right-click the source->Transform->Copy transform, then on the other source->Transform->Paste transform.

To have different filters on 2 source instances, or to have filters with different filter settings, do as I wrote in my 1st post: Instead of applying filters directly to a source instance, create a group and put that instance to the group and apply the filters to the group. This will not reflect the filters in other instances.

 

[shodanx]

Found this suggestion

I need different filters on my single camera source per scene

Hi I need to have two scenes, each with the same camera source, but for the filters to be different. It has to come from the same camera so that the perspective is seamless as I switch between the two. I have tried adding a camera source to scene 1 and calling it webcam 1, then adding a camera...

obsproject.com

So two options, one was put copies of the same camera source to two scenes and apply filters to the scene
The other was, put the camera in a group apply filter to the group

Both of these don't work because background removal is an audio/video filter and not an effect filter, I think ...
Because when you go in filter for both scenes and groups, only the effect filter list appears and background removal is not one of the options of the filters

I'm kind of stumped here, maybe I can send the camera feed to something that will create two virtual cameras ?

 

[koala]

It seems the background removal filter is not implemented as shader within the GPU but instead as regular function in CPU space. I don't know what to do here to duplicate a source with this.

I remember there was a "mirror source" source in the StreamFx plugin, which duplicates a source. I never found a use case for this because of the native "add existing" feature, but may be this is the use case I never needed. However, if this actually works, this will be a very resource-heavy workaround and may introduce lags and lagged frames except on somewhat powerful CPU and GPU.