That's understandable. I've had that happen to me many times as well - I would get in a funk about the pointlessness of whatever I'm working on when compared with such large problems I read about on the news. It feels like struggling to make progress, only in the wrong direction.
Perhaps I'll be pointing out something obvious, but you seem to admire these projects, which means you think they have some value. Each of them takes on a problem that's not even very new, but that few people have put enough time into solving very well.
- Handmade hero is a low level programming education programme. It's the sort of thing I wanted when I studied CS in university, but never got.
- Jon is trying to create a language that bridges the expressiveness of "high-level" programming with the control of the low-level.
- Jon's games explore what games can do as a medium. The core gameplay of Braid or The Witness is not something you can even do in other media.
- 4coder strives to be a programmable editor that gives you the most direct way to program the editor to do what you want it to.
The people behind these projects saw a problem that is worth exploring, and they're dedicating time to work on it. I know this is going to sound cheesy, but pick something you'd really want a better version of and dedicate unreasonable amounts of time to fixing it. I think Casey even mentions this in one of the HMH episodes: The people at RAD weren't the first to work on compression, they're just about the only ones who have decided to dedicate decades to the problem.
Also note that none of the projects you listed are as big as climate change or education, and I don't know to what extent social issues like "climate change, education, farming" can be attacked head-on via software. I'd say, if you're set on tackling problems of that magnitude, read up on what tools/processes are used to solve these problems right now, see where the roadblocks are, and then figure out how to go around them. To pick climate change as an example, here's a list of basic questions I would want answers to before I even begin to consider doing anything about it:
- Is measurement a problem?:
Is interpreting the data a problem?:
- How do climate scientists collect information about the current state of the climate?
- How do climate scientists deal with collecting historic information? Is the historical data accurate, is it incomplete, do they have to correct the data in some way?
Is forming an action plan from climate studies a problem?
- How is all the collected information used to model the climate and its changes?
- What's the delta between the model and actual measurements?
- What tests have been run on which models to make sure they are predicting the correct changes (are we looking at the right data points?)
- What is the largest contributor to climate change according to the model and why?
Is implementing a plan the problem?
- Is the data reviewed by and presented to people who can act on it? Do they understand it? Are they affected by acting on it (Will they lose their jobs/go bankrupt)? What will happen to them if they ignore it?
- Is there actually any response these people can formulate? (If we know the sun will explode next Wednesdoy, but have no means to stop it, we won't act on the information)
- If there are ways to address climate change, what are the benefits, neutral side-effects (as far as we know), and costs of each action? Who pays for the implementation? (How do we prioritise actions)
- Are there economic factors that don't allow for the implementation of the plan? (Maybe the long term effects of not implementing the plan are severe, but the short term cost of implementing the plan ends up being bad enough that we won't recover to enjoy the long term benefits)
- Is there a (more or less) central authority that could implement the plan or is responsibility distributed among many parties? If not, what would it take for all parties to work together? Who would have to implement which part of the plan?
- Are there technical hurdles that need time to be solved? (Suppose we know we need to reduce air travel or make electric planes, but the electric airplane technology is still to be perfected)
Mind you, I don't know the first thing about climate change, so don't take the questions above as an example of an intelligent inquiry into the subject. My point is that when dealing with such large issues (or even "smaller" problems like "How do we make it so programs don't have as many bugs?"), you'd do best to learn about the problem end to end. The more specific information you can glean, the better.
After doing an end to end survey of the problem, you can pick your place to start making a change. Maybe you think the data side needs help, maybe the models aren't as good as we need them to be, or the analysing the consequences of potential plans turns out to be a huge nightmare. Maybe there's some technological breakthrogh that is yet to be invented (or an existing one brought to mass production).
You can do this for any problem, but, obviously, larger problems take more study. Here, I ended up ordering my example questions like this:
- Do we have correct information?
- Are we drawing correct conclusions from the information? (Is our mental model correct?)
- Can we do anything about it?
- How and what would it take?
Which is too general, but a starting place nevertheless. Once you get some understanding of a problem, you'll usually have much better questions to ask.
tl;dr: Pick a problem you want and _need_ to have solved, study it, then dedicate unreasonable amounts of time to solving it.