Totally! I know that most, if not all of us have struggled with the strength issue, especially in the upper body ;)
I have personally found weights to be useful, as well as working on planks and pushups (honestly, I'll get a full, proper push up one day! I will!). Training both the core and the upper body (so, arms and all those fun back muscles, and the pecs!) will really help you to progress.
I really recommend Veena's lessons, if you can get them, as a good way of starting with strength training. I also found Jamilla's DVDs to have a pretty hardcore strength workout in them! Even the first DVD for beginners kicked my butt.
I often browse the internet for new and interesting exercises for certain areas, too. I have this weird thing where my body likes to pretend certain muscles don't exist :P So, changing it up is always good!
And, finally, just poling really helps. Working on the spins you've learned, and perfecting your technique with the moves you know (and often, the moves you struggle with) can really bring a lot of improvement to your strength. If you don't have a pole at home, maybe you can ask your instructor for ideas for exercises, and perhaps they have time for drop-in pole use, just so you can get that little bit extra time and practice in!