The ruined castle dominates the Staufen skyline. You can see it from pretty much everywhere in town. All you have to do is look up.
But if you climb the hill you get a completely different view, as I noticed on a recent visit. You’d never know there was a castle if you didn’t look behind yourself.