The way you designed your Nodes/Terms seems perfectly natural to me and is basically how you would model the thing in C anyway.
Each node in the expression would have a tag to specify its type and then it would have some values that are particular for its type.
> I've ended up with towering class hierarchies
Looking at what you showed us so far, I don't see any "towering class hierarchies". I just see sensible representation of data.
The one small "weirdness" is you have "Number" as a root but it's not doing anything. You don't need it. You already have a way to represent numbers with the `Integer` type.
What I would do is use Term as the basic representation every where, so for example, the fraction type would be:
 class Fraction : Term
{
Term Numerator;
Term Denominator;
}

You can also have a much simpler representation: only two types of nodes:
 class BinaryTerm : Term
{
Term left;
Term right;
Operation op;
}

Where `Operation` could be an enum.
The other class would be a leaf for representing the actual integers:
 class Number: Term
{
int value;
}
