The world needs a better way to create cross platform desktop GUI apps

First, this benchmark:
https://github.com/jhallen/joes-sandbox/tree/master/editor-perf

Where we see that two of the most popular code editors today, VSCode and Atom, perform 2 orders of magnitude worse at most tasks than editors of old. VSCode and Atom are both javascript+electron apps. The javascript electron stack leverages the facts that

1. Millions of developers are familiar with javascript, html, and CSS for creating GUIs
2. Every hardware platform on earth has a high performance javascript JIT

Because of these facts, javascript+electron is usually the first choice when someone wants to make a cross platform desktop GUI. (even the PostgreSQL people turned to it, to make the latest PGPAdmin, and it is not good). Javascript+Electron is seen as the easiest path. But at horrible cost.

So, how can we improve the situation?

You can see some of my thoughts and thoughts of the programming community in this thread started by that text editor benchmark:
https://www.reddit.com/r/programm...xt_editor_performance_comparison/



Well, you said it in so many words, the biggest hurdle here is essentially network effects. From a high level, there's not much you can do besides make a good product with someone (or some group of people) ultimately responsible for user satisfaction (which is something a lot of OSS projects lack that commercial projects have to their benefit) and just keep chipping away. I'm basing this opinion off of this article.

As for more concrete ideas, I vaguely remember Casey mentioning sometime back that there isn't actually a language to describe UI layouts, and I think that's a good starting point. CSS ostensibly is but it was meant for styling first and its responsibility for adjusting layout (with stuff like z-index and float, positioning: whatever, etc.) is kinda grafted on. So having someone who has both some UI and programming chops make a simple & well made spec for a language/scripting thing I think would be a good first step. There's a good opportunity to differentiate from HTML/CSS since that combo was designing for hyper-text document browsing, which is pretty distinct from actual user interfaces.

Having this spec define what it requires out of a renderer would set things up nicely for porting to various platforms while keeping the benefits of what HTML/CSS offer, that things look consistent across all potential platform choices.

Sounds easy but difficult to execute to i'm sure, but i'm just spitballing
I wonder if a really good javascript-QT binding could be a reasonable stop-gap measure. It seems there is no stopping javascript, but at least if people were using it to control a high perf c++ gui library rather than electron you wouldn't spend 13% of your cpu to make the cursor blink.
In the short term, yeah I don't there's any way of displacing javascript. But in the long term, I wonder how staying power a lot of currently popular programming technologies have. I mean, over the past decade, the infrastructure that we build on as programmers has changed to such a crazy degree that I don't think you could ever expect any engineering project to make all the correct choices. Now that things are stabilizing, I think we're going to run out of space for people to make those "first to market, first thing that works" kind of solutions, as people feel the pain of solutions that are dragging along all the technical debt of accommodating to hardware changes there could be a very good opportunity to slowly build something new. I'm almost inclined to think there might not be anything particularly good you can do short term.
There's another very important issue you have to take care of when you want to distribute GUI applications: Accessibility.

This includes (but is not limited to):
1. Can I mark and copy text? Is it any text, or just specific things?
2. Can I enlarge the font or the entire view without breaking the app?
3. Can I resize the window without breaking the app?
4. Can I use the app with just the keyboard, or just the mouse (with an OS-provided on-screen keyboard)?
5. Does it work with a screen reader?
6. Does it play nicely with other OS accessibility features (high-contrast mode or DPI settings)?
7. Will these features work automatically with custom widgets?

Browser-based apps solve these "for free" for the programmer, and at the cost of performance to the user.
An improvement over the current situation would have to be for the benefit of the users. That is, improve resource usage and product quality, but don't take away important features (and certainly don't lock out users who previously used your web-based version, but can't use your new native version due to accessibility issues).

NOTE: This is not an endorsement for browser-based apps. They tend to be very terrible (though I would say that it's definitely possible to build higher-quality web-based software than we see today, with no changes to the underlying tech) and I try to avoid them when possible. An alternative would have to be better in almost every way, and not just solve the problems that are immediately visible to the casual observer.
Bleh... electron, atom, vscode and all other UI's based on JS/node ecosystem are terrible. If you think that using 13% of CPU (probably 1 full core out of 4-core hyperthread CPU) for showing blinking cursor (!!!) is acceptable environment to create your UI's... https://github.com/Microsoft/vscode/issues/22900

I think JS gained popularity because it was first language of many developers. Combine with easy loading of .js files to make plugins, and people started to think that's the way to go... I mean I get it why it is so popular, but there are better ways to develop. People just don't know, or are afraid, or don't have time to learn and use new things.

And there is a already a good solution to create cross-platform UI - Qt, if retained-mode UI framework is OK with you. It's native code, open-source with reasonable license (unless you are in embedded world), relatively easy to deploy and looks and feels close to native UI on all desktop platforms. It is in C++ with bunch of classes and OOP'isms, but imho its very reasonable. I have been using Qt for a long time, in my work and also for my personal projects (1,2). It even allows to make UI's with JS scripting if that's your thing - QtQuick with QML language which is heavily based on JS.

If you haven't seen this presentation from this year's GDC, I suggest to read it - it's exactly about this topic onweb vs native UI for desktop apps: Insomniac’s Web Tools (spoiler: they had web UI, working on it / with it was terrible, they are now switching to Qt).

Edited by Mārtiņš Možeiko on
I just wonder what the situation would be if we had better technologies than HTML, CSS, and Javascript. I think the basic idea of web apps and Electron is sound, but web technologies are just woefully inadequate for highly interactive applications.

I've been wondering for a while how you might develop an alternative to HTML, CSS, and Javascript. With a better way of defining layouts and running code, you could probably get cross-platform performance much closer to native apps. And you could still deliver it over the internet like everything else!

Even just replacing CSS would be a huge win...
Do NOT worry I am working on will be a solved problem in a few years. In my workshop now devising the ultimate solution.
What about imgui libraries like nukear and imgui?
Here's a another library called libui. But its still WIP.

Edited by Shazan Shums on
They will never replace desktop apps, because of non-native look and feel. When people are using specific OS they expect some kind of standardized behavior. Each imgui-style app implements some things differently and that gets annoying fast if every app you need to use is very different (for example, in one app tab order doesn't work, other doesn't support copy&paste, third one doesn't support keyboard navigation between controls, etc...)

Edited by Mārtiņš Možeiko on
But libui is not imgui its a c wrapper for native platform GUI calls.
But QT doesn't look pure native plus it takes 5gb to install the core at SDK.
I think the most appealing thing about web apps (and why I'm still hopeful about webassembly) is not the ease of development, but the easy distribution. It's getting increasingly annoying to be forced to to download an installer & run it, and while auto-updating in itself is usually a "gift that keeps on giving" (as the aforementioned Insomniac postmortem nicely puts it, sarcasm alert), updaters that just open a download site - instead of actually self-updating the program - should be things of the past.

It makes sense to distribute your program via some common channel, both from developer and user point of view. Package managers, web apps, app stores etc. are a proof that people naturally *want* this kind of easier method for shopping software. The product should be open for use with minimal distraction from the product itself.

I think it would be possibly great to ditch the browser along with http (as it's still meant to just read static hypertext documents) and move the networked elements straight to OS level. Browser would be still a tool to *browse* information in the web, but the native programs installed to your computer, ran by the OS, could be fetched straight from the web and they would be written in something like web assembly to maintain the performance.
Using an immediate mode gui naively will also peg a cpu core when idling unless they activate vsync and even then you will do more work that you actually need because you are still redrawing every frame when you don't need to.

A properly written gui app will sleep when idle. If something needs to happen every so often they will start a timer and then do the something when it triggers. A good gui toolkit will have timer primitives for that.

About the immediate vs. retained mode guis, the only real difference between them (according to my definition) is that retained mode guis retain a cache of your data and use that cache for rendering or skipping rendering when the data doesn't change.

How that cache is populated and invalidated differs. One way is where you explicitly pass it the data on creation and when data changes you have to resubmit the data. The other method is where you give it a set of callbacks (usually called a model) that the view widget can query for the data and that model will also signal when the data changed and the view widget will figure out what needs to be redrawn and/or reflowed.

Of the two retained styles I vastly prefer the view-model style. It reduces the number of copies of the data that have to remain in sync, you often have a one-shot call to invalidate a large part of the cache instead of needing a call (and associated cascading invalidation) for every data item,...

As an aside it's interesting that you can create a hierarchy out of the styles and which you can implement on top of the other. You can implement the explicit retained on top of model-view which in turn you can implement on top of immediate mode...
msmshazan
But libui is not imgui its a c wrapper for native platform GUI calls.

It looks very primitive, missing a lot of features, and very unmaintained. So it won't cover many use-cases.

But QT doesn't look pure native plus it takes 5gb to install the core at SDK.
How does it not look native for you? Unless you are enabling custom themes... Sure, it doesn't use native controls, but it renders it's controls exactly same as native UI. Most of the time you won't be able to tell differences between Qt UI and native. Only in some corner-cases - and that's actually where Qt helps to make custom elements very easily.

5GB is SDK for everything, including webbrowser based components, QML stack, examples and more. Just for core widgets UI it is much much smaller. In fact my Qt folder is only ~150MB (that includes libraries and include folder). That's not a lot considering how many features it provides. And anyways - for development few GB of disk space is nothing expensive.

Edited by Mārtiņš Možeiko on
Lately I've been convinced that as far as cross-platform desktop apps are concerned, they basically have to be developed awfully similarly to games - take SDL, glfw, create openGL rendering context, vsync on and off you go.

If rendering at 60fps is too much, just redraw on draw events and input events.

Case in point: Spine
Written in Java, uses TableLayout (https://github.com/EsotericSoftware/tablelayout) with retained mode widgets (I believe) and libGdx (OpenGL wrapper). Fantastic piece of software.

It takes tones of effort to write cross-platform software like this as you basically have to write your own UI toolkit, but it does appear to be the way to go for high quality cross platform desktop apps. As you are more-or-less self contained and in complete control with minimal external dependencies.

Omar Cornut imgui is perfectly fine for internal/gamedev tools.

Or you can just use Qt I suppose, but I feel it is more of trade-off.

mmozeiko
They will never replace desktop apps, because of non-native look and feel.

But hasn't that already happened though?

By far, most "serious" cross-platform desktop apps do not have native look and feel.
Almost all content creation tools certainly don't.
Unity, Blender, IntelliJ IDEs, Kontakt5, flstudio, 3dsmax, maya, etc, etc








Edited by pragmatic_hero on