Personally, I learned programming in assembly language quite early - about two years after I started programming in general - and I don't feel like I really understood what CPUs do, or what a clock cycle even is, until going deeper to the electrical engineering level.
Ben Eater's video series
on building an 8-bit breadboard computer really helped, however. You don't really have to understand every single thing in there - I blanked out on most of the actual electronics he does, too - but looking at data moving between modules as the clock ticks was invaluable, I think.
This just sets up the basis for learning the higher level constructs in modern chips, though. From there you can go on to learn about pipelining and branch prediction
. I read a very similar article about various cache designs, though I can't seem to find it anymore - sorry! Maybe someone else can help out here. (EDIT: Might have been either this
, actually. I only skimmed those, I'm not sure they're as good as the article I actually mean).
Then there's super-scalar execution and out-of-order execution, which I'm still wrapping my own head around. The latter is especially mind-blowing. This open-source design document for an out-of-order core
was recommended to me just today, so I can't speak to its educational quality, but that's what I'll be spending the next couple evenings with, presumably.
Of course, I knew assembly going into all this; it might be the wrong approach for someone who doesn't. If you're having trouble making the connection with programming in a compiled or even interpreted language, don't sweat it. Find an introduction to assembly, toy around with it for a couple days, and try again. This stuff is kind of different from "normal" programming, and kind of hard to learn. You just gotta keep at it.
If you're gonna do that, though, I'd recommend learning an outdated ASM like 6502, which I'm told is very nice, or at least something like ARMv4, as I did. The x86 instruction set is kind of a legacy-ridden mess, and there's two competing syntaxes, which can get very confusing.