As someone who has spent most of his career development web applications, I've often wondered what the application of Handmade principles and philosophy looks like for web development.
The web seems to be the poster child for the problems of modern software development: layers and upon layers of arbitrary complexity that leads to slow, bloated, resource intensive applications and cargo cult programming.
Yet for all its flaws, the web is becoming the primary way software is consumed, shared and used whether we like it or not. The question is what can we contribute as a community to improve the situation? How can we provide a Handmade alternative for a new generation of developers who want to do a good job and care about quality software?
I don't have the answers, but I wanted to start the discussion.
Off the top of my head some ideas worth exploring:
I'd be interested in hearing other people ideas as well as how you might have incorporated a Handmade approach into your own work if you are a web developer.
The web seems to be the poster child for the problems of modern software development: layers and upon layers of arbitrary complexity that leads to slow, bloated, resource intensive applications and cargo cult programming.
Yet for all its flaws, the web is becoming the primary way software is consumed, shared and used whether we like it or not. The question is what can we contribute as a community to improve the situation? How can we provide a Handmade alternative for a new generation of developers who want to do a good job and care about quality software?
I don't have the answers, but I wanted to start the discussion.
Off the top of my head some ideas worth exploring:
- Promoting better languages for server side development such as Rust, Go or C/C++ with CGI where we can at least reason about performance and resource usage.
- Creating handmade library alternatives to common popular web development libraries. Think Raylib vs SDL2 or STB vs STL but for React, Vue and JQuery, etc
- WebAssembly seems promising and may provide an alternative approach to frontend web development that still needs to be explored.
- Implementing an alternative web browser to Chromium that a single developer could reasonably understand (I know this is a colossal task). Right now Chromium is about 25 million lines of code without comments.
- Improving Wordpress. Wordpress powers like 30% of the web, so even modest improvements could make a massive difference. I know that there is someone in the community that works directly for automattic which could lower the barrier of entry for the community to get involved.
I'd be interested in hearing other people ideas as well as how you might have incorporated a Handmade approach into your own work if you are a web developer.
Edited by Mike Jackson
on
Hi Mike
Thanks, this is an interesting post and (as you know) I'm a web developer, so it is something that's on my mind too.
Some unfinished thoughts....
Part of the problem comes from how 'easy' web development is or has become – or so it seems. There seems to be a tool or a plugin or a node module for everything. When I talk to other web developers often their first thought at the point of building anything is: "is there a library for that" or "what CMS should I use". This seems to be the culture, and I myself am and have been guilty of this. I'm not sure how this can be changed except by education: demonstrate how and why it is better all round not to use 50 npm modules just to make a temporary static holding page.
The other factor compounding the above problem (at least in my experience) is budget and time. Clients in the web want things done quickly and cheaply often, and almost always assume that any piece of functionality is just somehow 'there' waiting for the developer to click the right button to enable it. I'm sure this happens in other fields too but it seems to me universal in web development. Often a client of mine will already have a specific CMS in mind and will have designed a site according to the functionality of that CMS, which in many ways is fair enough, but it does mean that there is often limited scope for changing the fundamentals of the system you are working on.
Even if you can make the argument to a client that it would be better and/or quicker not to use X existing platform, there can be a lot of fear around this because the expectation is that if you build something even half from scratch, it will be bad, will break, and no-one will be able to fix it. I have had this experience myself even when trying to persuade clients to use a better CMS than Wordpress – i.e. they were scared to use a system they had not heard of, even one that was established and known.
Your idea of improving Wordpress is interesting. If the Wordpress core – perhaps things that don't change so often – were rewritten in a lower level language like C and compiled to binaries for all the platforms that tend to use WP, would that offer some speed gains? I wonder though how many of the millions of Wordpress sites out there are kept up to date, and if not many of them are, any radical change to WP is not going to be felt on a large scale.
If there was a good, simple, well-maintained CMS in a better language (Go, C, etc) that could:
—be customised to a reasonable degree without deep knowledge of the language the CMS is written in
—be flexible to adaptation by those who do know how
I think this could gather some momentum.
As far as I know there's not even a decent CMS written in Node, they are all PHP – even a decent Node CMS could be a good thing, as you have typescript etc.
I also like the idea of creating handmade alternatives to typical web libraries, especially since it may work better than trying to persuade the majority of web developers to make something themselves from scratch.
One specific thing that could help Node JS and PHP in particular is: both have package managers (npm for Node and composer for PHP). The package managers themselves are not bad, they usually work OK and don't break stuff. However there is no filtering or editorial process. If there were a package manager that was closer to a family of libraries, with something like a review process to maintain quality, this alone could improve development in these languages.
The other angle I think might be worth exploring is: how to bring the 'handmade' approach and program well in the language you find yourself using: PHP, JS, etc. Javascript for better or worse is probably here to stay at least for the foreseeable future, because it's the de facto scripting language for the frontend. Perhaps a lot of good could be done simply by creating a repository of documentation showing how to write JS with the hardware in mind. I'm not sure how far this is possible because the inner workings of the JS engine are unknown to me, but it must be at least partially possible.
There's a lot in, e.g., Handmade Hero, that is as philosophical as it is technical, e.g. avoiding OOP, optimising what will give you speed gains as opposed to a pleasant feeling, not abstracting every 5 lines of code into a function. This is generally the way that I manage to incorporate the philosophy into my web work, and I find this improves if not the underlying quality of the tech, at least the quality of my work and life.
Joe
Thanks, this is an interesting post and (as you know) I'm a web developer, so it is something that's on my mind too.
Some unfinished thoughts....
Part of the problem comes from how 'easy' web development is or has become – or so it seems. There seems to be a tool or a plugin or a node module for everything. When I talk to other web developers often their first thought at the point of building anything is: "is there a library for that" or "what CMS should I use". This seems to be the culture, and I myself am and have been guilty of this. I'm not sure how this can be changed except by education: demonstrate how and why it is better all round not to use 50 npm modules just to make a temporary static holding page.
The other factor compounding the above problem (at least in my experience) is budget and time. Clients in the web want things done quickly and cheaply often, and almost always assume that any piece of functionality is just somehow 'there' waiting for the developer to click the right button to enable it. I'm sure this happens in other fields too but it seems to me universal in web development. Often a client of mine will already have a specific CMS in mind and will have designed a site according to the functionality of that CMS, which in many ways is fair enough, but it does mean that there is often limited scope for changing the fundamentals of the system you are working on.
Even if you can make the argument to a client that it would be better and/or quicker not to use X existing platform, there can be a lot of fear around this because the expectation is that if you build something even half from scratch, it will be bad, will break, and no-one will be able to fix it. I have had this experience myself even when trying to persuade clients to use a better CMS than Wordpress – i.e. they were scared to use a system they had not heard of, even one that was established and known.
Your idea of improving Wordpress is interesting. If the Wordpress core – perhaps things that don't change so often – were rewritten in a lower level language like C and compiled to binaries for all the platforms that tend to use WP, would that offer some speed gains? I wonder though how many of the millions of Wordpress sites out there are kept up to date, and if not many of them are, any radical change to WP is not going to be felt on a large scale.
If there was a good, simple, well-maintained CMS in a better language (Go, C, etc) that could:
—be customised to a reasonable degree without deep knowledge of the language the CMS is written in
—be flexible to adaptation by those who do know how
I think this could gather some momentum.
As far as I know there's not even a decent CMS written in Node, they are all PHP – even a decent Node CMS could be a good thing, as you have typescript etc.
I also like the idea of creating handmade alternatives to typical web libraries, especially since it may work better than trying to persuade the majority of web developers to make something themselves from scratch.
One specific thing that could help Node JS and PHP in particular is: both have package managers (npm for Node and composer for PHP). The package managers themselves are not bad, they usually work OK and don't break stuff. However there is no filtering or editorial process. If there were a package manager that was closer to a family of libraries, with something like a review process to maintain quality, this alone could improve development in these languages.
The other angle I think might be worth exploring is: how to bring the 'handmade' approach and program well in the language you find yourself using: PHP, JS, etc. Javascript for better or worse is probably here to stay at least for the foreseeable future, because it's the de facto scripting language for the frontend. Perhaps a lot of good could be done simply by creating a repository of documentation showing how to write JS with the hardware in mind. I'm not sure how far this is possible because the inner workings of the JS engine are unknown to me, but it must be at least partially possible.
There's a lot in, e.g., Handmade Hero, that is as philosophical as it is technical, e.g. avoiding OOP, optimising what will give you speed gains as opposed to a pleasant feeling, not abstracting every 5 lines of code into a function. This is generally the way that I manage to incorporate the philosophy into my web work, and I find this improves if not the underlying quality of the tech, at least the quality of my work and life.
Joe
Web assembly will make the developers think about type safety and allow these compilers to finally give a helping hand to beginners, rather than having to treat them the same as visitors needing to display broken sites no matter what.
However, the problem with security will still remain with web assembly, because the browser race to accept as many privilege escallations as possible remains with their unlimited access to the system that browsers then try to police with script blocking addons and site reputation.
There has to be a standard organization in charge of the web again that dares to say no to bad features thrown in from all directions. Html should never be more than a formated text document with hyperlinks that can be read with minimal security threats.
For streaming services, I would rather have a dedicated client application receiving the content over an open protocol. Recommendation engines could use ratings from multiple program packages to give better recommendations. Trying to apply DRM here is just plain silly, because it only takes one person in the world with a good camera to make a copy for all pirates where nobody can tell the difference from an original bluray.
If someone wants to make the web into a gaming platform, they would be much better off implementing their own JIT compiled virtual machine connected to an application store where submissions are sent in as source code to be inspected and compiled on the central server. Then let users rate their experience, remove malware, play games offline... Then the people who just want to read the news can do that without allowing third party ads to mine crypto-currency on their graphics card.
However, the problem with security will still remain with web assembly, because the browser race to accept as many privilege escallations as possible remains with their unlimited access to the system that browsers then try to police with script blocking addons and site reputation.
There has to be a standard organization in charge of the web again that dares to say no to bad features thrown in from all directions. Html should never be more than a formated text document with hyperlinks that can be read with minimal security threats.
For streaming services, I would rather have a dedicated client application receiving the content over an open protocol. Recommendation engines could use ratings from multiple program packages to give better recommendations. Trying to apply DRM here is just plain silly, because it only takes one person in the world with a good camera to make a copy for all pirates where nobody can tell the difference from an original bluray.
If someone wants to make the web into a gaming platform, they would be much better off implementing their own JIT compiled virtual machine connected to an application store where submissions are sent in as source code to be inspected and compiled on the central server. Then let users rate their experience, remove malware, play games offline... Then the people who just want to read the news can do that without allowing third party ads to mine crypto-currency on their graphics card.
Edited by Dawoodoz
on
Dawoodoz
For streaming services, I would rather have a dedicated client application receiving the content over an open protocol. Recommendation engines could use ratings from multiple program packages to give better recommendations. Trying to apply DRM here is just plain silly, because it only takes one person in the world with a good camera to make a copy for all pirates where nobody can tell the difference from an original bluray.
I wouldn't want to see a future where I have to download a separate app to use Youtube, Twitch or whatever. Even if it was something like vlc for video files, meaning you could access multiple streaming services. I think a big reason for the social changes made by the web (live streaming of "niche" interests like speedruns for charity, let's plays, free educational videos, the potential to distribute and sell your own work without a publisher, etc.) were made possible because the barrier to entry for consumers is so small.
Having to download a complete new software and giving it potential access to your complete hardware still feels(!) less secure than visiting a new "unknown" webpage that you find via Google or get recommended by a friend. And even then you can just install addons that work on all websites. That is still easier for most users compared to hacking your OS to block certain adresses etc.
Besides making the experience of your webservice unique would be more difficult with a third party client application. How would something like twitch chat with it's custom emotes for each channel work? Either that third party streaming service already has a chat integration or it doesn't and therefore Twitch can't use this unique selling point (yeah I now these days most live streams have that). Of course you could make the whole system generic to allow for custom features, but than you have webbrowsers again.
So again: Not a future I would want.
I believe in the longterm we need a single programming language for all web development fields. One language to show a website to the client. The same language to access the backend. Databases written and accessed in the same language. Functionality done in JavaScript today need to be done in the same language too. Bots and crawlers should be written in the language as well.
Why? Because development in the web feels like gluing pieces of metal together instead of forging one fitting piece as a whole. And from that problem simplifications arrise, that may or may not be programmed well. Would companies rely on wordpress if it was easier to create a whole content management system from scratch that is still compatible with the rest of the web? Some would and that's fine. But some would at least consider making something better suited for their needs.
But that's something I fear is in the far future. In the meantime I agree with Mike's approach. It's more important to make small steps, instead of revolutionizing everything. Especially when such a major change needs the approval of multiple web standard commitees to be implemented on a large scale.
LaresYamoir
I wouldn't want to see a future where I have to download a separate app to use Youtube, Twitch or whatever.
Every person who primarily uses a phone or a tablet to access these services is already living in that future. And that's a much larger portion of internet users than you probably imagine - don't forget that more than half of all web traffic comes from mobile devices. You might not be okay with that, but most people are.
LaresYamoir
Having to download a complete new software and giving it potential access to your complete hardware still feels(!) less secure than visiting a new "unknown" webpage that you find via Google or get recommended by a friend.
If the issue is that it feels less secure to download software, then the correct approach is one which addresses the "feel" part of that statement, not the "download software" part. Feelings can be improved through communication and UX changes. After all, the browser already does literally download a new piece of software and run it (in a sandbox) every time you visit a webpage. It might not feel like that's what it's doing, because it doesn't make you click through a download button and an installer every time you visit a new website, but it's fundamentally the same idea - just plagued with technical problems and historical baggage that make the experience much worse than it needs to be.
notnullnotvoidLaresYamoir
I wouldn't want to see a future where I have to download a separate app to use Youtube, Twitch or whatever.
Every person who primarily uses a phone or a tablet to access these services is already living in that future. And that's a much larger portion of internet users than you probably imagine - don't forget that more than half of all web traffic comes from mobile devices. You might not be okay with that, but most people are.
Sure it's standard practice on smartphones, but is is good practice I wonder? Most apps are not written with native languages, but rather with languages that make interfacing with the web technology as easy as possible. So those apps are (in my view) glorified stripped down versions of what every webbrowser can do today. But they only work for one page. It goes back to my "one language for every web service" argument: These days you make a website + service with about 4+ different languages technically speaking. Plus at least one additional language for your app. All that just to have the same service on every device. We aren't talking about addressing different hardware for maximum performance, like we do in GPU programming. We are talking about the most basic functionality every computer from the high end gaming rig, to the smartphones down to a raspberry pi can provide.
Apps, as they are used currently, don't replace websites. They are complementary to them. Why haven't we as an industry thrown out websites a long time ago? Because consumers need something like websites to easily discover new services they might need or want.
When are people using apps? I'd argue only if they already know the service or get it recommended somehow by friends (primarly talking about Youtube, Twitch, Whatsapp, Twitter etc. not games). An app-only-web influences discoverability of new web services as the barrier to entry gets higher. Especially if we consider that Google and Apple are the "new" gatekeepers, as were book and music labels in the past for their media.
Eventually the lines between an app and a website will become indistinguishable. It's something that needs to happen in order to simplify and therefore improve web development. I just want this hybrid to be more like today's websites in terms of discoverability than today's apps in the play store.
notnullnotvoid
LaresYamoir
Having to download a complete new software and giving it potential access to your complete hardware still feels(!) less secure than visiting a new "unknown" webpage that you find via Google or get recommended by a friend.
If the issue is that it feels less secure to download software, then the correct approach is one which addresses the "feel" part of that statement, not the "download software" part. Feelings can be improved through communication and UX changes. After all, the browser already does literally download a new piece of software and run it (in a sandbox) every time you visit a webpage. It might not feel like that's what it's doing, because it doesn't make you click through a download button and an installer every time you visit a new website, but it's fundamentally the same idea - just plagued with technical problems and historical baggage that make the experience much worse than it needs to be.
Totally agree. It's mostly a UX problem. But you also mentioned sandbox, which is in my opinion the key to many problems in software technology. OSes need to become more flexible in sandboxing applications in general. As in OSes need to be able to simulate custom environments for different applications. Kinda how docker and virtual machines work, but more integrated into the OS itself. That would also help with using deprecated software and web technologies in a safe way without hindering progress on new technologies. After all a big hurdle to introducing new things in the web is supporting already existing web services.
And if easy sandboxing via the OS for web services on the consumer side of things becomes standard, then of course we wouldn't need a web browser anymore. We could just call the web service we want and while we are using it via streaming, the app/relevant data gets downloaded for later permanent use in a sandbox environment. Or something like that.
But while this is an interesting idea, I don't see this happening in the near future as it isn't something consumers ask for. They would benefit greatly, but most probably don't realize how much. And I'd imagine it to be difficult for the handmade community to make it mainstream, as it would need an OS everyone wants to use. Or at least a free software that everyone wants to have on their pc for some reason.
So as we can't sandbox apps properly right now, how can we change the UX of apps notably to feel more secure than using a browser? I don't see a good solution that doesn't lie to or take away control from the user. So I at least am back at square one: How to make development of websites easier? And while I said the single programming language approach is far in the future, in my opinion it's closer/easier to realize than the things I discussed in this post. That programming language might also be a step towards unification of web app and website.
We can't sandbox websites properly today because of JavaScript and ransomware, so a simpler open protocol is the only safe option for television. Nobody got a computer virus or tracking cookies on their CRT television from browsing plain text sites in monospace.
I block images in my e-mails, block scripts in my browser, use military-grade chat applications endorsed by whistle-blowers, read many reviews before visiting a new website, has nothing from Microsoft installed, physically disconnect the internet when not using it and change IP address every night.
I block images in my e-mails, block scripts in my browser, use military-grade chat applications endorsed by whistle-blowers, read many reviews before visiting a new website, has nothing from Microsoft installed, physically disconnect the internet when not using it and change IP address every night.