This seems more like a chromatic aberration "hack" for HDR landscapes (intensely-lit portions of the scene would have color fringing apparent at the boundaries of light/dark due to dispersion in the observer's lens).
(And it's def a style choice, looks cool when done right! :))
They might want to check out what VNC has been doing since 1998– keep the client-pull model, break the framebuffer up into tiles and, when client requests an update, perform a diff against last frame sent, composite the updated tiles client-side. (This is what VNC falls back to when it doesn’t have damage-tracking from the OS compositor)
This would really cut down on the bandwidth of static coding terminals where 90% of screen is just cursor flashing or small bits of text moving.
If they really wanted to be ambitious they could also detect scrolling and do an optimization client-side where it translates some of the existing areas (look up CopyRect command in VNC).
Of all the suggestions in the comments here, this seems like the best one to start with.
Also... I get that the dumb solution to "ugly text at low bitrates" is "make the bitrate higher." But still, nobody looked at a 40M minimum and wondered if they might be looking at this problem from the wrong angle entirely?
In fairness VNC-style approaches are bloody awful even over my 2.5gbit/sec lan on very fast hardware. It just cannot do 4K well (not sure if they need 4k or not).
I spent some time compiling the "new" xrdp with x264 and it is incredibly good, basically cannot really tell that I'm remote desktoping.
The bandwidth was extremely low as well. You are correct on that part, 40mbit/sec is nuts for high quality. I suspect if they are using moonlight it's optimized for extremely low latency at the expense of bandwidth?
Moonlight is mostly designed to stream your gaming desktop to a portable device or your TV at minimal latency and maximum quality within a LAN. For that, 40Mbps is quite reasonable. It's obviously absurd for mundane VNC/productivity workloads.
Yup. Go look into tigervnc if you want to see the source. But also you can just search for "tigervnc h.264" and you'll see extensive discussions between the devs on h.264 and integrating it into tiger. This is something that people spent a LOT of brainpower on.
I'm not sure; sometimes being an experienced dev gravitates you towards the lazy solutions that are "good enough". Senior engineers are often expected to work at a rate that precludes solving interesting problems, and so the dumber solution will often win; at least that's been my experience, and what I tell myself to go to sleep at night when I get told for the millionth time that the company can't justify formal verification.
I worked on a project that started with VNC and had lots of problems. Slow connect times and backpressure/latency. Switching to neko was quick/easy win.
if you want something more lightweight... rustdesk has been great for me, it supports multiple adaptable video codecs and can optimize for latency vs image quality.
Yes, in fact, the protocol states that the client can queue up multiple requests. The purpose of this is to fill up the gap created by the RTT. It is actually quite elegant in its simplicity.
An extension was introduced for continuous updates that allows the server to push frames without receiving requests, so this isn't universally true for all RFB (VNC) software. This is implemented in TigerVNC and noVNC to name a few.
Of course, continuous updates have the buffer-bloat problem that we're all discussing, so they also implemented fairly complex congestion control on top of the whole thing.
Effectively, they just moved the role of congestion control over to the server from the client while making things slightly more complicated.
Maybe it would be easier to just USE VNC instead. But the mentioned they have written their software in Rust. Looks like nothig is good enough for Rust coders, they need to fail by reprogramming their things in Rust before they accept that there are still tools for exactly that.
Copying how VNC does it is exactly how my first attempt would go. Seems odd to try something like Moonlight which is designed for low latency remote gameplay.
* Allows incremental porting of large codebases to ARM. (It's not always feasible to port everything at once-- I have a few projects with lots of hand-optimized SSE code, for example.)
* Allows usage of third-party x64 DLLs in ARM apps without recompilation. (Source isn't always available or might be too much of a headache to port on your own.)
3. Improve x64 emulation performance for everybody. Windows 11 on ARM ships system DLLs compiled as Arm64EC - makes the x64 binaries run native ARM code at least within system libraries.
It's not worth using ARM64EC for just for incremental porting -- it's an unusual mode with even less build/project support than Windows ARM64 and there are EC-specific issues like missing x64 intrinsic emulations and slower indirect calls. I wouldn't recommend it except for the second case with external x64 DLLs.
At that point why trust the emulator over the port? Either you have sufficient tests for your workload or you don’t, anything else is voodoo/tarot/tea leaves/SWAG.
"Why trust the emulator?" sounds a lot like asking "why trust the compiler?". It's going to be much more widely-used and broadly-tested than your own code, and probably more thoroughly optimized.
> Allows incremental porting of large codebases to ARM. (It's not always feasible to port everything at once-- I have a few projects with lots of hand-optimized SSE code, for example.)
Wouldn't it make more sense to have a translator that translates the assembly, instead of an emulator that runs the machine code?
The SIMD part will be emulated as normal, as far as I understand. So you can ship a first version with all-emulated code, and then incrementally port hotspots to native code, while letting the emulator handle the non-critical parts.
At least in theory, we'll see how it actually pans out in practice.
We wrote exactly that for our game-focused port of WebKit [1].
CPU renderer uses a tiny, custom fork of Skia (we only use the path rasterizer and their SSE2, AVX2, NEON backends) and our GPU renderer draws directly on GPU via tessellated paths / hardware MSAA (DX11, DX12, GL, Metal, Vulkan).
Every language has its own merits, there's no single language that's universally the best for every task.
With that said, C++ and Rust both occupy similar domains (high-performance, low-level, often interacts with OS syscalls or hardware).
Pros of C++:
- More mature, excellent library ecosystem, decades of code to reference from.
- Syntax is arguably easier to read and write
- It's very popular (top 6th language on GitHub), lots of talent to hire from.
Pros of Rust:
- Memory safety, it's much harder to introduce bugs of a certain type.
- The borrow checker makes it easier to reason about lifetimes of objects.
- Cargo is great for pulling in libraries (just needs more of them).
For me personally (graphics / game-dev), cost and speed of development is the deciding factor. I use both: C++ by default, and Rust for low-level, safety-critical code.
I usually hear about people using Rust with C. You use Rust and C++. If you use them together, how easy is it to call Rust from C++ and vice versa? And can you do that with their standard interfaces instead of watering it down to the lowest, common denominator?
Ultralight is definitely not abandoned, 1.3-beta will be released end of this week (nightly builds have been available for the past year) and ARM64 support will be available in the 1.4-dev branch later this year.
Feel free to contact me at adam {at} ultralig.ht if you need support or want a refund.
Definitely not a fork of WPE (which is primarily tied to *nix), Ultralight is built on a platform-agnostic fork of WebCore/JavaScriptCore I started back in 2015.
The goal is actually more ambitious than WPE since we run on every platform and in environments where embedders may provide custom platform functionality (such as games).
Yes-- Ultralight (the renderer underneath) has two modes: pure-CPU or pure-GPU. The GPU renderer does all drawing on the GPU using tesselated path geometry and pixel shaders.
Hah I could write a whole post on this topic but ultimately after experimenting with many different approaches on real-world hardware, tesselating the paths and using multi-sampling for AA (you can limit the MSAA to a single area via offscreen buffer) was the most reliable performer.
Real-time SDFs on the GPU still have a definite advantage when it comes to performing strokes, fills, glows, pseudo-blurs and other complex effects in a fill shader but older hardware (especially older integrated graphics) had unacceptable performance when evaluating the bezier.
The alternative is to cache the SDF (precompute on CPU or GPU then upload to VRAM) but then you start running into memory bandwidth and texture memory concerns.
I may still bring it back and lock it to certain hardware but I think we are still a generation or two away from using / abusing shaders for vector paths.
I should also mention that I spent the last two years rewriting our CPU renderer which now has pretty amazing performance across a wide range of hardware (no GPU required, can run it on a headless server or other device). I started by forking Skia’s core rasterizer and wrote a parallel dispatch layer on top (CPU core counts keep increasing so we can start treating them like GPUs for certain ops, see Intel ISPC).
Despite being a noob in this space, I'm very interested in it. Have you come across Ralph Linus? I've seen him pop up in a few HN threads that discuss SVG rendering with the GPU, and he's doing some interesting things: https://raphlinus.github.io/
Ultralight dev here-- I've been on the dev-tool side of this equation for the past 13 years so I have insight into why people are actually using HTML/JS/CSS over proprietary UI tech on the desktop.
It's several things-- tons of mature web development frameworks and ecosystems (React, Angular, et al), tons of developers with experience creating and maintaining them, and tons of debugging and support resources (Chrome Dev Tools, Stack Overflow, etc.).
Very difficult to compete against a content creation pipeline like that-- hence why I've made it my mission to get modern HTML UI running performantly everywhere without requiring web developers to learn some new language or toolset.
Also, FWIW, Ultralight does allow you to mix both HTML and native UI (the library can paint offscreen) so you can choose where/when to use each in your app.
WebKit is BSD but has sub-modules with LGPL code (specifically WebCore, JavaScriptCore, and WTF).
Ultralight is not WebKit (we have a totally different API and use our own renderer, compositor, and event-management code) but we do use a fork of WebCore and JavaScriptCore.
Adam, we hung way back I was working on an on device search tool called Orbit that could’ve used Ultralight, I think at bespoke. Been keeping tabs on it since, congrats on the release.
The steps WebKit have made the last few years have been incredible and your further parallelization and optimization work would be so slick, can’t wait to try it out.
I’d love to use it with Tamagui and Vite running all bundling for RN. Silky smooth desktop app, web apps, sites, and mobile apps all easy to design together and deploy nicely to their respective platforms.
>>> Ultralight can emit raw GPU geometry / low-level draw calls to paint directly on the GPU without an intermediate CPU bitmap. We recommend this integration method for best performance
This is sort of where I am, and a lot of people are doing similar explorations, in developing custom GUIs inside the browser that leverage WASM for high performance graphics. I'm using a hybrid bespoke GLSL translations and raw pixel manipulated bitmaps to Canvas2D, eventually targeting a custom webgpu renderer. Main issue is simply browser fragmentation. And, of course, ever larger data visualizations!
Ultralight dev here-- I remember the KDE days too and have released a fair amount of my software free and open-source.
I had indeed contemplated making Ultralight fully LGPL and charging for support but decided that the incentives really didn't align with making a quality product (quite the opposite-- there would be some motivation to make the product difficult to use to increase revenue which is not something I would ever want to do).
The current license (free for most, but a license is needed past a certain revenue threshold) is my best compromise at making the software free and accessible while still making sure I can keep the lights on and ensure the product continues to be developed into the future.
If you have any concerns about licensing you can always hit me up in our Discord.
(And it's def a style choice, looks cool when done right! :))
https://en.wikipedia.org/wiki/Chromatic_aberration
reply