Practical programming for permacomputing

WARNING: The following text has a lot of advisory "things should be this way" while naturally my own software isn't bug-free. I recommend a philosophical path to follow. I myself don't follow that path perfectly, but it's what I'm currently striving towards.

My whole career, I have been writing software according to a set of values I don't hold.

An organization, particularly the kind whose purpose is first-and-foremost to make money, wants the software it produces to achieve a usable state as quickly as possible, such that they can make money as quickly as possible. This has driven an ecosystem of languages, tooling, and infrastructure that enable faster usable software - usually sacrificing computing resource efficiency among other things. It makes more economic sense for an organization to spend on beefier computers than to pay an extra month of salary to a programmer whose job would be to produce a work of better quality. One lives so long in this world that one begins to believe that speedups to programming are good, and that shortening time-to-MVP is good.

You are not an organization.

You are free to spend the time an organization won't - time to make the program minimal and correct.

I now ask you some facetious questions outright:

What quantity computing resources are you comfortable stealing from your users and throwing into the trash?

Of course if your program allocates 20% of your users' RAM and does nothing with it, the whole world is in agreement that you should be internationally banned from opening a code editor ever again. But what is the lowest acceptable limit for unnecessary wastage?

You have users that run your programs on slow computers, and they can't afford new: given that you must optimize for the worst-case scenarios of the old hardware, the acceptable bloat should approach 0. I say "approach" because there are micro-tradeoffs that ensure that we can't save every stray byte.

Once you take into your soul the idea that the user has gone out of their way to stop running something else to graciously grant your program a precious portion of their computation, you will write code that attempts to cleverly conserve the finite and fleeting gold it's been given.

What level of baseline privilege do we have, such that we assume our users can spare anything but an old mobile phone lying around?

We have become complacent with the abundance of CPU, RAM, and storage, and forget that we must continue making programs that are as resource-light as in 2005.

With that, I present some practical steps that we can take in our programs to care for our users, in rough order from most powerful to least:

More intentional design

There are likely millions of lines of code that could've been avoided by ruminating on better architecture and design before finger ever hit keyboard. One should begin a project by asking themselves "could my program simply not exist in the way that I think it should?" and see where the line of logic takes them.

I don't mean using off-the-shelf software instead of writing one's own, quite the opposite.

I also don't mean that if you happen upon an existing piece of software that achieves the same thing that you intend to create, that it should deter you. In fact I believe that if an existing solution is made/maintained by a for-profit organization, and you have the urge to make your own version along with the community without the profit incentive, it behooves you (us!) to make the Free alternative.

Benchmark performance

So you've made your program and you want to cut down on its resource usage. The starting point is gathering data on what the current situation is. The generic places to begin investigation are CPU, memory, storage, and network.

There is no value in a performance profile on the latest AMD 6969x daddy's-money-edition chip. The only numbers with meaning come from the computing equivalent of a snail that lost one of its eyestalks in World War 1. I regularly conjure said snail by capping memory and CPU with cgroups, my choking-fist-of-choice being through containers. Throttle the processor down, squeeze the network to what Australia would call world-class-internet (2G speeds), starve that snail. The wastage tends to quickly reveal itself once your development environment is as mangled as your most unfortunate user's computer.

It is time to measure your program. Measure while it works hard. Measure while it does nothing. Measure while it's doing maybe something. Peak usage when performing consensual work (consensual to the user) is somewhat forgivable, since the program might be earning some form of lunch. In contrast, the resources a program hoards while idle are theft, plain and simple. Record the footprint at idle, the steady-state footprint, the peak, and treat the idle stat as the most damning of the three. One would assume that GNOME devs would do this more often due to the involvement of the term "footprint".

Your program may only be a sentence in the story, where end-users must run sidecars and separate dependencies in order for your program to work. You must treat all of the above as the holistic totality of your program, and measure the performance accordingly. Don't think I don't see that postgres you've hidden behind a remote URL that runs on a different server.

Write the numbers down before you start going to town on your stagnant-green-sludge codebase. The baseline is an angsty before-picture in your childhood home. Every improvement gets judged against that baseline.

Set yourself a budget that your program ontologically cannot overstep, such as "this program shall not exceed 50MB of memory". Once you've drawn a tangible line, it becomes easier to throw this line into the test suite such that a later commit that widens the program past its budget actually refuses to build. ("Ontologically?" ah, what I mean is just the sense that it can't be in this reality if it ain't up to standard.) Frugality that isn't enforced with the strong fist of righteousness tends to magically slip back into bloat the moment one looks away, especially on teams where the other programmers are not so conscientious as you.

Onto the actual areas of computers to touch! Regarding memory, the tangibly tasty target is the "resident set size", the RAM your program actually occupies. Do not believe your language runtime's lies (the heap number). On Linux the truth is out there, sitting in /proc/<pid>/status, and when you want to know where it all went, heaptrack and/or valgrind's massif will guide you over the Styx. As for CPU, wall-clock time is my bludgeon, hyperfine is my longsword, lastly perf is a rapier.

Time the following two moments:

For storage there is the obvious size of your program initially, vs the larger size it swells to once installed, both of which are revealed from a glance at du.

Beyond that, here comes the cost that is night-and-day on old hardware: how much the program writes, and how often the program does it. Your user runs their digital life off an aging SSD and a cheap SD card, and they have a finite number of writes before the storages turn back into sand. A program that needlessly plays pingpong with the disk is shortening the life of hardware its owner cannot afford to replace. Watch the file operations with strace, and the throughput with iostat. From these you will discover whether you're a vandal. For the network, count the bytes you put out onto the net. Yes indeed, count to make that sesame street vampire proud! Count the number of round-trips your program demands for those bytes, with nethogs, iftop, and/or tcpdump. Here too, mind the situation at idle. A program that quietly whispers in the dark while the user isn't looking is spending their data allowance, their battery, and their patience on the programmer's convenience.

powertop and the processor's own energy counters will tell you what your program costs in watts. A non-zero amount of wastage stays invisible until you start counting energy.

Two habits keep all these numbers honest.

Constrain your setup

Imagine having to write a detailed & well-reasoned report that your user must approve, in order to warrant allocating 100MB of RAM. What would you write? Take native desktop apps that use electron for example, which requires around 100MB of RAM to even turn on, let alone do anything. Something along the lines of "please let me use this RAM because it makes my program development easier for multiple platforms". I put myself now in the seat of the user when I ask "why on earth should I have resources on my old laptop wasted just to make your life easier? I already have to run three separate instances of electron from other programmers who decided that their laziness was worth more than my RAM."

I haven't researched what the absolute minimum resource usage of Discord's native program could be if it didn't use electron, but let's unscientifically & naively assume that we could reduce to around 100MB of RAM instead of 400 if we worked for long enough (given enough time, I believe 100MB is a very conservative improvement).

The wasted resources are O(N). You're not wasting the 400 - 100 = 300MB of RAM by using electron, you're wasting 300*N.

With that, and another ballpark figure of 200 million monthly active users, we could deduce that:

Discord's programmers might refute the exact theoretical RAM reduction I chose, even if I would call it conservative. To them I ask: how many months should we spend figuring out the details of that speedup to free up the world's 3 petabytes of RAM? How many years of development is that worth? What if we found performance bottlenecks and charged you, the programmer, the cumulative price of that wasted RAM?

As I write this, I open Discord on my computer just to check that I'm not producing codswallop. Discord sits doing nothing. It is consuming 425MB of my RAM.

Use a light language

This isn't a hill I will die on, and of course there are plenty of optimizations that can be done on heavier languages, but a heavy runtime with an interpreter, a VM, a garbage collector is overhead we can't optimize our way out of. In my opinion, we owe it to our users to learn & use languages that place as little machinery between between the code and the computer itself.

Is the following line of logic a selfish statement?: "I'm not going to learn blank [lower-level language] for this program - so I'll use what I know, which is blank [higher-level language]."

Naturally I believe anyone can and should write in whatever language sparks joy for them, and I also acknowledge that learning things in general is difficult, especially the ins-and-outs of low-level languages. If the difficulty deters you, I find that completely understandable - but an excuse such as "it's time-consuming" is much weaker of a reason than it sounds. What is the rush? You have time to write software, but no time to learn it? We are in a craft that is based almost entirely on knowledge, thus choosing not to gain knowledge to use a tool that would serve the user better is surely a decision worth mulling over.

Sensors & awareness

Your programs should be aware of their digital environment. As a collaborative member of a holistic computing system, they should be able to "feel" their resource usage and adjust accordingly to be kind to the other programs they collaborate with. Of course, if a user clicks a button in your software that indicates they want the program to max-out the whole computer, by all means go ahead, but even given that it should be clever enough to avoid many an OOM.

The program should strive to know where it's running, what its headroom is, etc. Everything it can be given or deduce, such that it can make resource decisions on the fly. Your program might well start hammering network streams when it realizes that it has been bequeathed a gigabit of headroom, and naturally constrain itself to sonar-like pings if it detects that the network situation has changed to dial-up.

Design for spotty everything

One may have heard of designing their programs for spotty internet as opposed to assuming it'll always be connected. This is a tried-and-true axiom to live by, I don't have much else to say about it.

Don't misuse the Unix philosophy

The Unix philosophy, broadly speaking, is the idea that a program should do one single thing and do it well. What a great idea! There is a misunderstanding of the idea that I see happen, when it comes to drawing the line of where that "one thing" ends and another begins. It's easy to give oneself the medal for having honored the principle when the code one writes stays small and sharp - at the cost of having reached peak by delegating every concern out into a separate heavy service. The binary with the programmer's producer-tag on it does only one thing indeed, while the program delivered is a sickly amalgamation, a frankenstein of bits and bobs. A programmer stands at the fast-food till and orders some mysql, a redis, an object storage, with a side of hashicorp vault. Each puzzle piece, taken alone, does its one thing well. Yet zoom out: the amalgamation running on the machine does far too many things, and pays for them all in the blood of the computer's circuits. "Doing one thing" is not a property of the code, right? It's a property of everything the program requires as a pre-requisite to its existence.

Refactor for fun, your codebase a playground

Take time and effort, after features are done, to refactor and build for the purpose of making the codebase "beautiful" (whatever that means for you). Surely you're more likely to hang around in a codebase that sparks joy. I don't mean turning perfectly good chains of chaotic ramblings of code into clean-code in the traditional javastic sense. How about fun variable names, cute comments, code-golfing, deduplication for the fun of it?

Closing out

That's all well and good, and let's assume now that your own code is now beautiful and up to your own high standards. We now turn our attention back to corpo-driven codebases and their effect on our wallets via our computing resources.

I ask myself whether it's even possible to convince an organization that smaller, lighter software is worthwhile taking time to create. One could piggyback off the cloud-bill pushback phenomenon of a couple of years ago, or play into clout to distinguish a company through a particularly "well-programmed" product. What else might convince a balding middle-manager than fame & fortune, to let a team whittle a codebase for an extra month?

For for-profit business entities, realistically I don't see these arguments taking - I think it's a part of the nature of companies to do as little as possible for as much money as possible, in the end. That leaves pressure from end-users, and pushback from programmers. A company might simply be forced to make a good codebase if users refused to use it on the grounds that it wastes resources on their computers. Likewise, if the actual employees refused to create one in the first place. (The latter might require a union to ensure job security, lest the whole team get subbed for lowest-bidder replacements.)

As always, thank you for having read this. Let's make programs that benefit humanity!