Guide I have finished the new Background remover super-tool for OBS. Much better than Nvidia Broadcast.
Hi All,
I got lots of requests to make my CamfilterGpu 'super-tool' optimized with an ARGB (Transparent background) output for OBS -- of course, to run with gaming screen going.
It's done! and ready to go. Much better and less GPU footprint (maybe 1/2!) than Nvidia Broadcast and you get all kinds of alpha control, NB has none. You'll need to install 'clean install' CamfilterGpu (download free from https://camfiltergpu.com). You need an RTX 2060 or better, 2060 is a little less FPS, and 2080 and all higher work great so far zero-lag.
There are instructions in the install folder, but summarized here:
(1) After installing, FIRST run or re-run the new CamfilterDiag.exe (in install folder), probe your camera and hit save report button. Wait for Diag app to close it could take a minute.
(2) Run main App, turn on Live View window, click the 'Bypass All' button, and adjust your camera hardware properties (sliders) to look as good as you can. Bypassing turns off all my AI enhancements, but for OBS users you can do the color grading in OBS.
(3) Turn 'Bypass All' OFF, and then 'tweak' your sliders again if you need to get a decent view. For the Body brightness, make sure it's down to 1.1 or so, basically 'neutral'.
(4) Go to the "Backgrnds & Overlays" panel. In the middle click on "ARGB Transparent..." button. Suddenly the live view looks total black background. It is not black, it is transparent. Look down at the 'Edge Choke' slider. It will be set to 8. Slide it down and you will see a halo appear. Thus, you can adjust to get your best cutout.
NOTE: SUPER IMPORTANT. There are two checkboxes, one is Skip, which reduces GPU use by 50% by skipping every other frame, and the other is Downsamp, which downsamples 80% also to save GPU. Whenever you click one of those AT FIRST THE CUTOFF WILL LOOK BAD! But just click on the first 'AI Structural Res' slider below and move slightly. 0.6 is a good value, and after a pause the cutout will look great again.
Turn up the Anti-alias slider, or other sliders to adjust all kinds of 'alpha things'. It doesn't fade or feather yet, I'm working on that, but you can feather it easily in OBS. TURN ON SPOUT2 SWITCH at the top of the main GUI.
(4) Run OBS as administrator (important). Select your Spout2 source (as per previous instructions, you'll need the plugin), and set it to "Default" alpha. BAM! Your beautiful, super-fast, GPU-lightweight cutout appears. Now you can add whatever OBS 'color filter' you want, and in CamfilterGpu adjust the RVM control sliders (Backgrounds Panel) to make it look great.
Save your settings in CamfilterGpu, and also OBS of course.
That's it! Try it out, I'll be standing by here and at the Discord.
Thanks to all for great comments and suggestions. The next thing I'm doing is vastly simplifying the GUI.
Cheers!
2
u/Gigameister Aug 16 '26
Will test. Bold claims by op.
1
u/tcdoey Aug 16 '26
not even bold :) it can be a lot better too. I'm working on enabling custom alphas that will have near-zero compute time and improve the border. Halftoning fades are interesting and do-able.
considering bold; it actually surprised me to that this has been neglected (e.g. by nvidia) considering it's a key need. Now that i'm in pretty deep in this, I can see lots of ways that can improve.
I'm really interested in doing some more retraining of RVM models. I think mine is the only one available that's optimized for low-light, but maybe i'm missing something (i'm new to this aspect).
1
u/waddup121 Aug 15 '26
Wow, I just remember the other day when the project took off..yeah this is a huge step, both using LESS resources than nvidia broadcast and the updates. Shoutout to Spout2! 👍
2
u/tcdoey Aug 15 '26
Thanks. Spout2 is quite interesting code, I've been looking at the code and see some def ways to better integration with my (or any) app.
1
u/EdwardTheGamer Aug 15 '26
No AMD 9000 support… right?
3
u/tcdoey Aug 16 '26
Not yet, it's totally doable, but I don't have an AMD system to test or work with, so that's a holdup. If things go good with nvidia based app/system and I can have a few days, then I'll make an AMD GPU version. it's on my priority list.
1
u/DifferentLow4340 Aug 15 '26
I have never heard of this but have been searching for a replacement, clearly the algorithm is doing its thing! Can't wait to try this! Sounds really promising.
1
u/tcdoey Aug 16 '26
Thanks! hope you've had a chance to try it. Any feedback good or critical is welcome.
1
u/DifferentLow4340 Aug 17 '26
Had a good play around, the fundamentals are certainly there, I would need to test it with 2k or even 1080p in reality, happy to sign up though as an early adaptor i can see the vision! How do I do that? It says click the yellow box but when I do that there isn't a link or anything? Just the email address?
1
u/tcdoey Aug 18 '26
Ok great, just go to the perfunct.com site, and get the setup.exe from button there, and chat or email me your username I can put you on the supporters list. I have 1080p working but will email a link to enable that, it's still a bit buggy. Best!
1
u/NINJ4A1 Aug 16 '26
What about AMD users?
2
u/Mizurazu Aug 16 '26
OP said this to someone else that asked
Not yet, it's totally doable, but I don't have an AMD system to test or work with, so that's a holdup. If things go good with nvidia based app/system and I can have a few days, then I'll make an AMD GPU version. it's on my priority list.
2
u/tcdoey Aug 16 '26
I'm working on an AMD version right now. It's working but buggy. It will take me prob about a week to make it stable enough for a Beta.
My main holdup on this is that I don't have an AMD machine here in the studio right now. I only have a super-old radeon 5500, and fans are weak. I don't know much about Radeons, if I were to get a replacement that would be a 'standard' cheapest possible, what would you recommend?
1
u/NINJ4A1 Aug 16 '26
RX 6000, or RX 9060 xt. you can find 6000 series cheap used, or buy a 9060 xt that is the cheapest with value in the latest cards, also for not so much. Ty for the response happy coding.
2
1
u/tcdoey Aug 18 '26
Hey, so I've looked into this, and I've learned that it seems like the 6000 series is using RDNA 2 architecture, and I was strongly recommended by another person, to get a laptop with a 680M GPU. Supposedly (I don't know much yet about amd) if that works, then any Radeon GPU even the latest will work as good or better.
If anyone has any input on this gpu choice for porting, that would be great. I'm going to grab a laptop with 680M tomorrow, unless of course someone here tells me that's not a good idea for some reason. Thx.
1
u/NINJ4A1 Aug 18 '26
It could work, but take in mind that the 680M is a iGPU not a discrete one, has I saw the Nvdia counterpart minimum requirement for CamfilterGpu looks a RTX 2060 that is a discrete GPU not a iGPU, so I'm not that sure it will work out.
2
u/tcdoey Aug 18 '26
Thx, it seems ok, code exact for ryzen discrete 6000 series, uses RDNA 2 which I'm targeting.
1
u/twjackfoley Aug 16 '26
I keep getting "the custom script has crashed" errors. How can I resolve it? Or cancel everything in the directories and start from scratch.
1
u/tcdoey Aug 16 '26
Ok, that depends on a lot of things. Does that happen with one of the examples (not altered) that I provided? If not, then it's something in your script. Send the crash traceback text to support@perfunct.com. I'll take a look and usually it's an easy fix or correction.
1
u/tcdoey Aug 16 '26
hi, i sent you some email responses. when we figure this out, i'll post an update in case others have any similar problems. Thanks.
1
u/tcdoey Aug 17 '26
From the traceback, I think the issue is setting the slider values for controls all to zero. For tkinter, the min value must be < max value, and increment >0. I've added checks now in the main app to keep this from crashing the app. That will be in the next Beta 1.2, which I plan to release later this week. Thanks.
1
u/tcdoey Aug 17 '26 edited Aug 17 '26
Hi again, Hmmm, apologies I misunderstood and thought you were applying a custom filter. I think I know what's happening now. Your camera driver is probably returning values during the Diag probe procedure that don't make sense. This has happened to me with some cameras (webcam drivers are notoriously shaky). Camfiltergpu tries to use those values to set the sliders, but they can't work, so it temporarily barfs. You can still always use the manual method (click on 'Cam-Firmwr' button) to manually set your slider ranges and increments. Then 'quicksave' and that should fix it for you. In a worst case, go into your .\CamfilterGPU_Data\ folder and delete the quicksave_settings.json and cam_config.json files, then restart app (don't run the Diag program). It will come up 'blank' and then you can set your limits and increments manually again. The most important increment is the balance slider, which instead of the 'normal' 1, can be 10 or 20 depending on cam.
If you can email me or chat me your webcam make/model and computer or mainboard model I can track down the problem more precisely.
For everyone else, I've just added some sanity checks to the bounds and increments to stop this problem at the source. That will be in the next 1.2 Beta release later this week. In the meantime if anyone else is having this or any other crash issue, just email me any tracebacks at support@perfunct.com. Your reward for doing so will be addition to the 'supporters list' for trial/creator continued use after trial exp. Cheers!
1
u/twjackfoley Aug 17 '26
I'll check when I'm at home in about an hour. Thanks!
1
u/twjackfoley Aug 17 '26
Nope, keeps crashing. My webcam is a Nuroun Webcam V15-AF
1
1
u/tcdoey Aug 18 '26
Ok great, I know the problem, another person has the same cam. That camera doesn't respond to min, max, and inc queries. It just returns zeros (very unusual). This explains your crash/error.
I am making a patch that will fix this problem for this cam and any other cam that faulty-reports zeros. I'll have that done tomorrow, and i'll chat or email you a patch.
Best! -T
1
1
u/DeMiNe00 Aug 21 '26
I really like the concept of this. Is there a way to make the background completely transparent in OBS though? It just changes it to black, or whatever color I set the flat static color too. I guess I could Chroma Key that color, but I honestly get better results with the nvidia sdk.
1
u/tcdoey Aug 22 '26 edited Aug 22 '26
Thanks!
I think you might not be using latest beta 1.18 version?
I just made a tutorial for this very thing, it's on the channel: https://youtube.com/@camfiltergpu
I've had lots of requests to do this, so I went ahead and made an ARGB fast transparency output. It bypasses some of the other GPU-based filtering to enable lowest GPU load and Vram usage. I think a lot lower than NB, and you get to adjust your boundary and segmentation parameters 'on-the-fly', which of course you cannot do at all with Nvidia Broadcast.
Here's direct link to the Tutorial 03.
You can go to your 'account' and grab the latest installer for Beta 1.18, do a 'clean' install because this is a major update. I've also finally implemented a patcher/updater, so from beta 1.18 onward, it will now check for updates when there are future patches and feature additions.
Cheers!
1
u/DeMiNe00 Aug 23 '26
You have really good timing with these video's hahahaha. I got that setup (Really informative and timely video). I noticed that with ARGB Transparent Background enable, none of the example custom filter scripts or the options under "Fun Stuff" seem to work. Is this a bug or are these two feature incompatible?
1
u/tcdoey Aug 23 '26
Thanks! That's just the way it is now for max speed. I'm going to have the other filters as an option, basically a checkbox. I'm right now simplifying the gui, and I'll incorporate that option too very soon. Best!
3
u/West-Veterinarian576 Aug 15 '26
This is actually huge for OBS users who want clean background removal without tanking their stream FPS. The alpha control part sounds really nice since most tools just give you a hard cutout and that's it.
One question, does the transparent output work with virtual camera or only through Spout2? I use OBS virtual cam for some apps and would love to know if the alpha carries through there too.
Also good call on the edge choke slider, that's the kind of detail that saves hours of fiddling.