Via CLI you can, but GUI apps connect to your X server session, and then the fun begins - any application you allow to connect can essentially capture your keyboard, mouse, clipboard and a ton of other fun things,as there is no sandboxing applied between them. It's inherent in the design of the X protocol.
There are solutions that are intended to force the sandboxing by opening a new Xserver for every application, e.g. Firejail [0], but that comes with another set of interoperability problems.
Wayland was supposed to address some of these concerns, but it will only do so for applications that natively talk wayland protocol, not the ones that connect through x-protocol via xwayland
XWayland is essentially a translation layer consisting of Xserver and Wayland client [0]. Therefore it has all the same problems a normal Xserver has, which they do acknowledge:
> A Wayland compositor usually spawns only one Xwayland instance. This is because many X11 applications assume they can communicate with other X11 applications through the X server, and this requires a shared X server instance. This also means that Xwayland does not protect nor isolate X11 clients from each other, unless the Wayland compositor specifically chooses to break the X11 client intercommunications by spawning application specific Xwayland instances. X11 clients are naturally isolated from Wayland clients.
I use QubesOS, but it comes with its own set of problems as well.
That does not sound different from what Windows does. By default all programs running under the same user can access all windows of other applications (except UAC elevated ones). It's a relic from when OLE and Clipboard in Windows 3 just was (very simplified) a pointer to RAM.
The only reason it is worse with X11 is that it is an inherently networked protocol, so the same statements also apply to any remote connections you might allow to your X server. It also makes it somewhat easier to capture Xkb / Xinput events purely through API, without need for any elevation or excessive polling of the devices ("it just works").
That includes any systems you might have SSHed into with X forwarding enabled, as it automatically extends the trust there. Yes, your ssh client might try to enable X SECURITY extension (which clamps acesss to just the current window), but it is disabled by default or bypassed anyway by the users as that extension is known to crash quite a few programs.
Both are a product of their time when the prevailing approach was to trust the programs you run.
There are solutions that are intended to force the sandboxing by opening a new Xserver for every application, e.g. Firejail [0], but that comes with another set of interoperability problems.
Wayland was supposed to address some of these concerns, but it will only do so for applications that natively talk wayland protocol, not the ones that connect through x-protocol via xwayland
[0] https://firejail.wordpress.com/