I actually find the prettiest inverts to be the slow, controlled ones. Maybe I'm not picturing what you mean by swooping your leg back and up properly... but the way I'm reading it, I would be concerned that it might wind up looking like you're using it to kick up and use momentum - and you're not! You've been putting in so much work on strength building specifically to avoid that! I think that when you really have the strength built up, the grace will come. You won't have to concentrate so hard on technique and can then focus on the finer movements and making your inverts look the way you want them to - to suit your mood or the music or the moment. If you're working on spin pole, I think a lovely graceful invert comes after doing a diagonal pencil spin, holding that for a couple of rotations, and then inverting with either tuck legs that extend out when you hit your chopper, or straight straddle legs the whole time or one leg tucked and one straight then both extended into the chopper (ugh, that was wordy, I hope it made sense). Whether or not you're at the level that you can do a straight legged invert or if you're still tucking up - or something in between (like me, I have a pretty decent invert with one leg straight but not both yet) - if you make the movement DELIBERATE, it goes a long way. It looks sloppy if you can't tell if someone was trying to straight leg invert but got tired and wound up with semi-bent legs, whereas if that person fully tucked their legs and made it clear that their intent was to bend the legs, that could be lovely.
Hope that makes sense. Beautiful inverts are something I struggle with too!