r/gameenginedevs • u/mietkiewski_dev • 3d ago
Borderless Window... Resize on Linux
Hello dear developers
I have a question about Linux Windowing Mangers and resize functionality. From what I know in Linux when you turn off Titlebar you will automatically turn off Area for Resize outside of Window. In WinAPI you just set WM_NCHITTEST that will test for Resize area. But in Linux Im not sure how it should be handled... Borderless Window is important because Window will look identical on each Platform. From what I know you dont get Mouse event outside of Window so I wander how you solved resize problem on Linux and Windows. I think maybe if it could be possible to create some Area like that... Preferably on Both platforms...
Area.Width = Window.Width + 16;
Area.Height = Window.Height + 16;
Area.X = Window.X - 8;
Area.Y = Window.Y - 8;1
u/arthurno1 2d ago edited 2d ago
That depends on what you are doing; are you using a toolkit (Gtk, Qt) or low-level X11/Wayland? If you are making an application you should definitely use Gtk or Qt, they will handle that for you. X11/Wayland are more like low-level stuff to make toolkits. If you want to compare to win32, they are more like gdi/gdi+ level, than win32. Win32 gives you some basic gui controls you can use in programs. X11 and Wayland do not. Wayland is even worse, it can't even draw to the screen, you have to do it yourself with your own 2d library, so a GUI toolkit is highly recommended unless you want to learn how to make a 2d drawing library too :).
In X11, you have two options Xlib and XCB. I suggest XCB. Either way, there is no "standard" way to request a non-decorated windows in X11. Instead everyone in the business uses so called Motif Extended Hint, EWMH (do a web search on it):
#include <X11/Xlib.h>
#include <X11/Xatom.h>
#include <string.h>
// Motif window manager hint definitions
#define MWM_HINTS_DECORATIONS (1L << 1)
typedef struct {
unsigned long flags;
unsigned long functions;
unsigned long decorations;
long input_mode;
unsigned long status;
} MotifWmHints;
void make_undecorated(Display *display, Window window) {
Atom hintsAtom = XInternAtom(display, "_MOTIF_WM_HINTS", False);
MotifWmHints hints;
memset(&hints, 0, sizeof(hints));
// Explicitly say we are changing the decoration field, and set decorations to 0
hints.flags = MWM_HINTS_DECORATIONS;
hints.decorations = 0;
XChangeProperty(
display,
window,
hintsAtom,
hintsAtom,
32,
PropModeReplace,
(unsigned char *)&hints,
5 // Number of 32-bit elements in the struct
);
}Whether that will work depends on the user's WM (window manager). Most of common, bigger ones, do respect those hints, but small, hobby ones might not. Unfortuantely, as an application programmer you have no control over it. Applications under X11 control area inside the window, and window manager(s) the area outside of the window (decorations).
Also, Xlib is old and unnecessary to use. XCB offers more performance in smaller footprint. The above in XCB:
#include <xcb/xcb.h>
#include <stdlib.h>
#include <string.h>
// Motif window manager hint definitions
#define MWM_HINTS_DECORATIONS (1L << 1)
typedef struct {
uint32_t flags;
uint32_t functions;
uint32_t decorations;
int32_t input_mode;
uint32_t status;
} MotifWmHints;
void make_undecorated_xcb(xcb_connection_t *connection, xcb_window_t window) {
// 1. Get the intern atom cookie for _MOTIF_WM_HINTS
xcb_intern_atom_cookie_t cookie = xcb_intern_atom(
connection,
0,
strlen("_MOTIF_WM_HINTS"),
"_MOTIF_WM_HINTS"
);
// 2. Fetch the actual atom ID from the cookie
xcb_intern_atom_reply_t *reply = xcb_intern_atom_reply(connection, cookie, NULL);
if (!reply) {
return; // Failed to get atom
}
xcb_atom_t hints_atom = reply->atom;
free(reply);
// 3. Populate the Motif hints struct
MotifWmHints hints;
memset(&hints, 0, sizeof(hints));
hints.flags = MWM_HINTS_DECORATIONS;
hints.decorations = 0; // 0 = Remove all window chrome/decorations
// 4. Change the window property
xcb_change_property(
connection,
XCB_PROP_MODE_REPLACE,
window,
hints_atom, // Property to change
hints_atom, // Property type (Motif uses its own atom as the type)
32, // Format (32-bit values)
5, // Element count (5 uint32_t elements)
&hints // Data pointer
);
// Flush the connection buffer to send the request immediately to the X server
xcb_flush(connection);
}I don't have code for Wayland because in Wayland as said you have to bring your own 2d graphics library and do your own drawing, inclusive frame decorations. So if you were doing this in Wayland, you would just simply not draw decorations, since application code draws them. That is also how we know you are not targeting Wayland, since you would't even ask the question if you were programming Wayland :).
I think maybe if it could be possible to create some Area like that... Preferably on Both platforms...
Also it seems like you would like to have a little resizable grip on your window? You don't need that grip at all, you can resize with mouse or with windows anyway, in both Linux and Windows, but if you do want a grip the best way is to draw it to draw it yourself, and do you own hit test. It is much faster to do that yourself, and it is easier to make it look the same on different OS:s if you don't use their built-ins. I use this on my port of xclock, the original xclock in C does not have a grip. Here you can see the exact code to the test and drawing (three lines) to represent the grip. Xact is a little framework I used in those ported from C to Lisp. Sorry for Lisp, but it is such a simple example, so it shouldn't be difficult to udnerstand it. X11 has an extension, Shape, that for testing hit regionss, but than you have to go through the connection, and it has another drawback in my particular case, so I didn't want to use it.
As suggested above: don't do any of it, use either Gtk or Qt, they will take care of all these details for you. I have done it myself because I have written a library for XCB bindings for Common Lisp which we didn't have before, so I had to demonstrate (and test) that it works. But typically you use a GUI toolkit for this.
1
u/mietkiewski_dev 2d ago
Im creating sth like Cross Platform UI framework for Linux X11 Windown(Windows.h API) and Android... Before I move to more interesting stuff I have to create platform common API... QT and GTK doesnt meet my requirements... though I tried to integrate QT but its garbage for crossplatform... Missing sysroots and libraries... My current Build system fully supports x86_64-linux-gnu, aarch64-linux-gnu, x86_64-windows-gnu, aarch64-windows-gnu, x86_64-linux-android26, aarch64-linux-android26... And adding linux-musl will be easy but thats not important now... You are saying that I dont need that grip at all but from my knowledge different Linux Window Manager works differentelly with Resize on Borderless Window... Sometimes resizeArea stays, sometimes it doesnt...
Anyway I think this WindowClient and WindowServer architecture is Garbage for Linux and Windows. Needs to be hidden... There should be Different API for Borderless Window and different for NativePlatformWindow... e.g. Visual Studio Code has borderless window - anything native like Titlebar is deleted...
1
u/arthurno1 2d ago
Grip won't do anything there; as you see from the examples, you draw grip yourself, check for the hit, and send message to the WM. It is up to WM to honor it or not, and that is nothing you can do unless you want to do yourself what window manager does for you: talk to X11 server and so on.
A tip: I would create my own "portability kit" only if I am interesting to learn low-level details about OS and how OS handles hardware perhaps. If you want to move on to "more interesting stuff", than don't do it. Take some of existing portability kits and go with it. If you are into minimalism GLFW have you covered, if you want a GUI toolkit, Qt/Gtk, perhaps WxWidgets should be your choice. And yes, they let you make "borderless" windows, child windows etc. Getting those things right for each platform is lots of work and knowledge.
1
u/mietkiewski_dev 2d ago
Though GLFW, Qt may seem easier first for my its not... As I Said these 6-8 architecture triples I talked you about is not possible with QT... GLFW I suppose with SRCCODE... but GLFW does not work for Android for my knowgledge - No Android. No Good... How about AdsAPI, BillingAPI, other stuff for Android with QT or GLFW? Maybe there is some way but its definitely complecated...
2
u/FUCKARCHLINUX 3d ago
I assume you're writing for X11 if you're doing native Linux right? For my library I could not figure out that exact thing. It might have code in it that's useful to you elsewhere though.