I've been working on a custom viewer for our compiler optimization graphs at work. I finally have the overall algorithm working. It uses a modified version of the Sugiyama layout algorithm (thanks @beast_pixels for the tip) but throws away and simplifies major components of it. This actually gives me a more predictable and stable layout, and the arrow routing is 100x easier to read than any GraphViz graph.
All the graph layout and rendering is less than 1000 lines of code.