Notably, QubesOS has demonstrated that a secure X environment is possible.
In Qubes, an isolated VM "dom0" operates the physical display, input devices, and desktop widgets. Apps run in VMs, which each maintain a private X server. Storage for windows in each such X is mapped to address space in dom0. dom0 copies updates to this memory to the physical display. Input events that occur with the pointer in a window are sent to that window's corresponding X server.
The limitation is that the app VMs have no access to a GPU. Some people manage to get a GPU assigned to an app VM, but GPUs generally have DMA access to all of physical memory, so this compromises security if code operating the GPU is not trusted.
There has been work on virtualizing access to the GPU, particularly in the SpectrumOS development effort.
> The limitation is that the app VMs have no access to a GPU. Some people manage to get a GPU assigned to an app VM, but GPUs generally have DMA access to all of physical memory, so this compromises security if code operating the GPU is not trusted.
Well the code executing on the GPU is trusted. What is not trusted is code running in the browser. The problem is that Wayland instead of not trusting the clients which shall not be trusted (browser), trusts only one client at a time. So you have security against local programs (which shall be trusted) but no security against remote code running in the browser.
Where did the code executing on the GPU come from?
By definition, nothing running in the app VM is trusted; that is the whole point of fencing off the app in a VM. The VM is safe against remote code running in the browser only until the remote code finds a hole in the browser security, i.e. for about a millisecond. Once the browser is compromised, the remote attacker can run anything they want in the GPU, if the VM has access to one. (Maybe they need a privilege escalation first, but those are a dime a dozen.)
In Qubes, an isolated VM "dom0" operates the physical display, input devices, and desktop widgets. Apps run in VMs, which each maintain a private X server. Storage for windows in each such X is mapped to address space in dom0. dom0 copies updates to this memory to the physical display. Input events that occur with the pointer in a window are sent to that window's corresponding X server.
The limitation is that the app VMs have no access to a GPU. Some people manage to get a GPU assigned to an app VM, but GPUs generally have DMA access to all of physical memory, so this compromises security if code operating the GPU is not trusted.
There has been work on virtualizing access to the GPU, particularly in the SpectrumOS development effort.