The overhead press is a great way to build up the shoulders and triceps. One of the great things about it is that your rotator cuff must stay activated throughout the lift to stabilize the shoulder. That's why people who frequently add overhead pressing variations into their regimen have relatively healthy shoulders compared to the people who only bench press and completely neglect any overhead work.
That said, one of the biggest mistakes people make is overarching their lower back when performing the exercise. If you're not making a concerted effort to brace your core and keep your ribcage down, you're eventually going to do some damage to your lumbar spine.
Adding the following floor dumbbell variations will teach you how to brace your core. They'll also keep you honest with the amount of weight you can do since they don't allow you to cheat.
These exercises all require you to keep your torso upright and maintain a strong flat back throughout each rep. These floor variations are self-limiting (if you lack mobility, you won't be able to comfortably complete a rep) and self-correcting (if you try to do more than you can handle, you'll lose balance and fall over.) You should feel an intense contraction through the core for each.