That's because the SS/Medicare are not discretionary which means that those are basically predetermined and not set by Congress every year.

So when we talk about the "budget", what really matters as far as politicians are concerned, is the discretionary part, which they can control (and which in theory voters have some control over through their election of congressmen), and defense takes up at least half of that.