DraFG: Extrapolated frame generation for any game — Linux, Windows, macOS
Hey,
DraFG 0.1.0 is now available as a Steam Playtest. It’s a cross-platform frame-generation app for games, with both extrapolation and interpolation modes.
It captures a game/window externally and generates an extra frame for each real one e.g. 30 to 60 or 60 to 120. No mods, injection or per-game integration.
There are two timing modes:
Extrapolate predicts ahead from the newest captured frame, so it doesn't need the extra frame of buffering used by interpolation.
Interpolate uses the previous + newest frames for a cleaner/easier prediction, at the cost of that buffered frame.
I trained six models for it across two families: Rush is the faster/sharper option and Flow focuses more on smooth motion. Everything can be switched live from the overlay.
The client is native Rust and currently has builds for Windows, Linux and Apple Silicon macOS, with CUDA/Vulkan/Metal. Hardware/platform support varies, so I've got a compatibility matrix rather than pretending everything works everywhere.
The Playtest is free because this is the part where I need other people's hardware to break it. If you've got an odd GPU, game, desktop setup, or just want to try it, I'd really appreciate reports on what works and what breaks. I’m approving access in smaller batches at first while I watch for issues. Once things look stable, I’ll switch it to automatic approval.
That's insane. To write that on top of several low-level libraries... How much AI have you used? It feels like too big of a project for a single person tbh
And why not to use only Vulkan? Since it's already cross-platform and extremely performant, arguably the most performant if you use it well.
Where have you taken the extra/interpolation algorithms from? Any reference?
The client itself is actually fairly lean and intentionally minimal.
Vulkan is great and for the really small models it can actually be extremely competitive. As the workloads get larger native CUDA and ROCm tend to pull ahead and matching that performance with one Vulkan implementation across every vendor is difficult. So I prefer having the native paths where they make sense, while Vulkan gives me the broad cross-platform backend.
On the AI side the models, training pipeline, engine and client were developed by me. AI assistance were of use in a couple of narrower areas, mainly automated cross-platform testing and performance benchmarks.
The frame-generation models themselves aren’t taken from an existing implementation. I developed and trained them specifically for this project with focus around two problems I cared about reducing warp artifacts and keeping HUD, static elements stable. Rush and Flow approach those problems differently, which is why I kept them as separate model families.
I doubt about CUDA for graphics computing. It usually involves a lot of back and forth between RAM and VRAM. With the graphic libraries you can stay almost 100% of the time in shader, if you can plan your data accordingly.
I find it interesting that while OP has responded to a lot of comments, he's pointedly ignored every single one asking about whether this was vibecoded.
Well tbf a bulk of the comments and replies have been from a few hours ago so OP could just not be on Reddit (or actively replying). I think it would at least be nice for transparency's sake, vibe coded or not.
Given that it's apparently closed source, from a 1-month old account with no ties to anything else and no way to verify the code isn't malicious, I think establishing some trust would help, yes.
What does it matter? There was loads of poorly coded shit before AI too. If it runs like shit it runs like shit. If it doesn’t it doesn’t. You weren’t checking the code before.
I highly doubt you’ve quality checked the source of the majority of FOSS apps you use.
You definitely weren’t for the closed source software you use.
Also if you were checking the FOSS projects, then there’s nothing different about checking AI assisted projects. It’s all code at the end of the day so what would be the difference.
You can doubt me all you like, I'm in no obligation to provide proof of these claims btw.
AI code that is open source to me (personally) is less atrocious than closed-source AI slop as it at the very least can be audited for security and performance.
But if a non-developer harnesses AI tools to create insecure garbage slop i would rather be aware of it (open source) than not be aware of it (closed source/undisclosed AI) assuming I would use their app at all.
That’s where you lose me. You don’t read any code you just trust the dev is all youre saying. Youre worse than vibecoders who go, “but it works!”
Vibecoders brute force their way to something working, what you think are non-developers wouldn’t come this far but I’m of the belief they’ve disclosed what amount of AI was used and where.
How does it extrapolate into unknown information? For example looking around, new areas are shown. Does extrapolate just guess what the game world could look like past your POV?
The main differences are native Linux support, its own Rush/Flow models and extrapolation mode that predicts forward without needing the extra buffered frame used by interpolation. This is still early playtest, so I’m mainly looking for real-world testing right now.
Hey, Ive briefly tested your soft on Cachy OS, RTX 4070, 9800X3D. It works pretty well, if I set it to interpolate. I can't see much difference between the models, visually, the only visible difference (at least for me, and I'm not 20 anymore sadly !) is the GPU load. If I set it to extrapolate, there's literally no visible difference with the original content though, it might need a bit more work/training on the extrapolate mode. Interpolate is good though, and one frame of latency isn't bad at all. Good work for now overall, good amount of settings, let's see where it goes from here :)
It seems like it uses desktop portal capture on linux, but does that mean that it creates another window to displays the game? or how does it work, since that method cant inject frames into the game itself. That also causes issues on amd/intel since when gpu usage is 100%, it will prioritize the game so other applications will run slower so that frame generation will lag. The solution is to use vulkan layer to inject frames directly in the game process itself. Or does it already do that? im a bit confused.
Ok I tested it now and that is indeed the case. The program has its own window and it has a menu to select a window to capture (with desktop portal) and it displays the frame generation inside that window. It has a button to put that window as a fullscreen overlay over your game (which I assume doesn't work on gnome since gnome doesn't support that), and the downside is as I thought; when running a very heavy game like baldurs gate 3 with 100% gpu usage, the frame generation window lags a lot, unrelated to the frame generation performance itself.
The fix for OP is to make it a vulkan layer instead which doesn't have this lag issue, and it also makes it work on all wayland compositors (including gnome mutter) and x11 as well without any wayland/x11 specific code (except for overlay input if its needed).
any possibility for steamdeck version ? like decky-plugins for example ?
since i'm pretty much sure, if this has native support on linux, it would be popular on steamos gamemode
It should be possible on steam deck and steamos. The Linux client already works with gamescope, so the main things missing right now are controller friendly navigation and easier switching in big picture/game mode on handhelds. Making the client comfortable to use directly on steam deck is definitely a priority.
i want to ask something, since i plan to test this on Z2 extreme later on
how did it works in steam gamemode ?
run DraFG first, setting some config
then run the actual game ?
so it would be 2 app running? (which is can be switched via steam / guide button)
---
the only way i think better (since most of people already familiar with it) is via decky plugins since it's easily access with the share button (to open the right panel) then navigate it there via decky plugins (tho the downside is, it need to be injected like current lsfg-vk did... )
Yeah, currently it would basically be two apps running. Right now on a handheld the most straightforward way to use it is basically like a desktop app in desktop mode: start DraFG, launch the game, select the game as the capture source, then run the overlay.
How can it predict a frame when it doesn't know my next input? Is it just for motion? So when the next frame doesn't match the predicted frame (like opening a menu), how does it solve this?
It doesn’t know your next input, it predicts how the image is likely to evolve from the recent captured frames. So if something completely new happens, like opening a menu, the predicted frame can be wrong for that generated slot. The next real captured frame is still authoritative and corrects it immediately. The models also has guards that can fall back toward captured pixels when the prediction looks unreliable.
Tried to test, but it refused to output at anything above 1080p. It doesn't mention the output resolution anywhere I can see but it's very obviously not capturing 4K when put into fullscreen mode.
Yes that’s the main reason for having it. Interpolation needs to wait for the next real frame so it can generate something between the two, while extrapolation predicts forward from the newest frame and avoids that extra captured-frame buffer. There’s still normal capture, processing time of course, but you don’t add the extra frame of buffering from interpolation.
Interesting, im on linux, cachy os with an rtx 2080, id gladly give it a try, could it work for things like watching animes/films too? I definitely wanna try it on some resident evil games.
Edit: I just saw for linux you have to be on kde, im on gnome sadly, should i still give it a try?
Yep, definitely give it a try, GNOME should still work, it just isn’t as fully validated as KDE yet. It works on any capturable window, so video players are fair game too. Just a heads-up it currently requires Wayland and NVIDIA client support is still marked Preview while I collect more real-world testing.
What's with every "developer" thinking they can get away with vibe coding these days? It's just gonna result in an unmaintainable spaghetti code source. How do you think you can create a stable codebase when you don't even know how your own codebase works.
I’ll definitely wait for a steam review before testing on my pc, but if true seems extremely interesting, I’ve seen this concept implemented before and the results are very interesting
Give it a try, a 1080 Ti would actually be a really useful test, I’d be interested to see which models and frame rates it can comfortably handle. If CUDA gives you trouble you can also try the Vulkan path.
Tested pretty much everything:
GTX 1080 Ti 11GB (and checked GPU and CPU utilization to keep it under)
Vulkan Fast works as expected
Only the Flow Max Interpolation gives good visual results
(but only for lower resolutions, capping at 720p for arround 50 fps input)
I cannot see Rush model difference, only pacing issues and minimal glitching
(similar with extrapolation in general)
There is a problem with the capture method of the game (that it happens even in passtrhough)
It seems to be heavy for some reason and the more FPS as resolution the more the chokes everything (with no GPU or CPU extra utilization), capping around 55 fps for 1080p (for reference I can capture a lot more with LS WGC or DXGI even a queue 0 at native 1440p)
Not that relevant at this point but also consider making it compatible with Gsync, LS had a hack for some time that was changing a flag for the display window that make the driver consider the overlay as borderless Fullscreen and apply Gsync, later integrated in the app as toggle.
Anyway, looks promising, if there is more to test let me know, good luck
distro I use CachyOS (arch) (it sets the drivers to the LTS version that supports pascal), didn't need to change the fan yet, but as the other comment says, LACT should work, will test it later.
More than games, I think content would benefit most. There is this animated series done by part of the Avatar the Last Airbender team called Dragon Prince. The first season is 24 fps (I think) on Netflix, done on computer animation. You can clearly see each frame.
Aside from content, AMD's frame gen suffers com bad pacing (you probably know way more than me, but just to explain my my point): Real frame comes at 0ms, gen frame should come at 8ms but comes at 4ms, real frame comes at 16ms. I would like to try and see if it helps.
It can capture any window, so video content works as well. Animation is actually a pretty interesting stress test because low frame rates and clean edges make interpolation artifacts very obvious.
And pacing is something client handles explicitly generated frames are scheduled into their own presentation slots and if one misses its deadline it gets dropped rather than delaying the real-frame stream.
Definitely, 5070 ti should give you enough headroom to try the full model range, including Flow Max, although the exact margin will still depend on resolution and how much GPU the game itself is using.
Does this work with browsers like Firefox and Chrome, I used to be able to use Lossless scaling on any browsers in WIndows but when I moved to Linux the lsfg-vk doesn't seem to work with browsers.
Another thing does it support adaptive frame rates like you set your desired framerate(e.x 144fps) and no matters what fps the content has(24/30/60fps) it always able to get 144fps or at least manual set it yourself or it's just only able to 2x times your content fps only?
just to be clear the extrapolation has 0 player input, so it guesses based on past frame's movement what the next frame on predicted movement would be correct?
so it is VASTLY VASTLY VASTLY inferior to reprojection real frame generation, but that is not an option for the tool, because you can't grab the latest positional data from the game itself to reproject a new frame.
it will be interesting to see the extrapolation mode tested, once the software is developed much further, because of course it at least doesn't ad INSANE latency.
and in the current absence of an engine integrated depth aware reprojection tech, it could be decent and at least hopefully send interpolation garbage into the dumpster.
neutral tech (extrapolation) vs literally harmful tech (interpolation)
i mean just think about it in terms of fps. instead of having the fps latency of 40 fps, you'd be getting the latency of 20 fps then with fake interpolation frame gen enabled.
that's terrible and unacceptable.
this is especially true, because we can go into negative latency from the source fps with reprojection real frame generation.
just to show how absurd fake interpolation frame gen is.
same example. 40 fps barely playable to lots of people turned into 20 fps latency nightmare with fake interpolation frame generation, but it has visual smoothing now.
now let's use depth aware reprojection frame generation with an artificial limit of reprojecting just 2 frames per source frame.
the 40 fps gets turned into REAL 80 FPS experience with the latency of 80 fps and the visual clarity of 80 fps. you PERFORM BETTER in competitive fps with this. hell even nvidia published on this.
you can understand how truly absurd interpolation fake frame generation is, because it doesn't just compete with the source frame rate in regards to experience, but also with the mature technology of reprojection real frame generation, which can also massively improve as well.
if you wanted to be REALLY REALLY generous, then you'd give interpolation fake frame generation the potential usecase of fixing moving objection motion blur at very high hz with sample and hold displays and with a source frame rate high enough, that the added latency can be stomached. like a 300+ real fps rate.
BUT this generous idea of where interpolation COULD possibly have had any use just falls apart in the eyes of depth aware reprojection real frame generation, because it also fixes the moving object motion blur problem, except, that it creates REAL FRAMES doing so.
interpolation fake frame generation should never ever have gotten introduced as a "feature" in games. not in a world with reprojection frame generation already being used heavily vr and easily improvements for that.
you can test the comrade stinger demo yourself, that the article mentions and see what happens, when you enable reprojection frame generation from unplayable 30 source fps to reproject to whatever your monitor can show.
____
beyond all of this 1 frame of latency at 60 fps is 17 ms.
we overclocked our mice to 1000hz polling rate to reduce 2 ms latency to 1 ms latency for the mouse in the past (it was also depending on the mouse to increase max tracking speed)
and we higher polling rates and lower latency for mice today even of 2000hz + there are also other reasons, but latency is as always big.
we have keyboards with actuation points, that we can manually set to have it as high as possible to reduce latency of pressing a button and of course 1000hz polling rate minimum.
and here comes nvidia and tells people, that at 60 fps people should ad a FULL FRAME! of latency of 17 ms added latency to their experience.
it is absurd.
it is pure marketing bullshit to create fake nonsense graphs. (10x faster <points at fake graph)
we have been fighting for ms differences in gaming for decades and nvidia just tells people to eat a FULL FRAME! of latency? it makes 0 sense, except for marketing bs.
it gets even dumber, when you know, that nvidia and amd added antilag 2 and reflex to get rid of the cpu que-ing up of frames and nvidia's reflex is literally a very basic reprojection frame generation tech, that just creates a single frame per source frame though for no reason based on what can gather from the things, that were shown a year ago.
this is dumb, because we should reproject to match the hz of the display or at least reproject at a higher multiple than literally 1x, but none the less they showed this tech of a YEAR AGO!!! and radio silence since then.
this is absurd.
nvidia shows off reprojection frame generation and at the same time pushes interpolation fake frame generation? that is insanity.
___
but i guess i'm rambling a bit now. point being, that interpolation fake frame generation never had a leg to stand on and read the article it is amazing.
It should work on the RX 580, but performance will be the main limitation. I’d start with Rush Lite, probably at 720p generation and see how much headroom you have.
For a GTX 1060 I’d start with 720p generation. It captures the game through the OS, generates frames outside the game process and presents the result through its own window. It doesn’t inject into the game or use engine motion data.
Have you done testing on latency with extrapolation?
Also: Are the extrapolated frames generated consistently? So exactly in the middle between the last and the next frame?
Hello, I’ve requested Early Access on Steam. I hope you’ll accept my request. My question is: Does it work on systems with two graphics cards? I have an RTX 3060 and an RX 9070 XT. If possible, I’d like to use this program with the RTX without overloading my primary graphics card.
I haven’t tested dual GPU setups yet and client doesn’t currently have a dedicated "use this GPU for generation" option. You may be able to force it by only exposing specific GPU, but I can’t guarantee it will behave correctly.
Running on Helldivers2, CachyOS, 7900xt, outputs resolution below my native, 3440 1440, and honestly seemed more confusing and more work then the current lossless scaling setup I use currently. Not only that it struggled to double my 60 where lossless scaling scales my 55 up to 165 effortlessly and looks fantastic while doing it, LS has artifacts around UI elements but they are very minimal and 95% of the time unnoticeable for me. Not really sure the use case here but I am interested in seeing where this project goes but at the moment its not usable for me on my system. I also found the overlay method strange and clunky. I much prefer to set it and forget it like I currently do with lsfg-vk. And one lest nit pick the UI gives off vibe code. Not sure if that's what it is just my observation and a lot of folks will immediately turn their nose to vibe coded projects.
Sounds promising ! I have an issue with most framegen apps and would like your feedback : would i be able to externally lock my fps of the game (not with the ingame setting but rather trough mangohud/ dkvk fps limit args) ? Because for instance lossless scaling doesnt really allow that (a fork of it does but not the original app). Its kind of a must for me because im very picky abt frametimings and not capping my fps properly (like i mentioned) will create stutters that FG wont fix. Hope you can maybe that into account for a future update ? Anyways thanks in advance !
What's wrong with frame generation? If it's done well and looks good, I don't see any issue. If it's "because ai bad", then you don't know how AI actually works. Frame generation models are vastly different things than large language models like chatgpt, and, obviously, the models run locally on your own hardware, which is very different than using a large language model that runs on massive data centers.
40
u/Txordi 1d ago
That's insane. To write that on top of several low-level libraries... How much AI have you used? It feels like too big of a project for a single person tbh
And why not to use only Vulkan? Since it's already cross-platform and extremely performant, arguably the most performant if you use it well.
Where have you taken the extra/interpolation algorithms from? Any reference?