Greetings from the fiance' of a someone who I jokingly refer to as my perpetual student. It's definitely stressful, there is no use in sugar coating that fact... especially to be a successful graduate student I've learned there is a balance where I have to allow her time and space to do her thing while also having plenty of my own going on (work, hobbies, making new friends, etc). I moved both FOR and WITH her for her current masters studies, and will be doing the same (hopefully!) pending this years app season for her PhD.
I can't give any insight to doing it straight out of undergrad; we were both young professionals with real-life jobs when she made the decision to go back to school, and I have been incredibly thankful in that my job has let me work remotely as I support her. The key thing I would say though; make sure it is a decision you make together. Before she even started applications for her masters, and now her PhD, we charted out schools by rank, geography, cost of living, climate, job prospects, etc... You don't want to feel like you are dragging him with you, and he won't want to just tag along unless it's something you do together and resent you for "making him do it". (of course... I don't know you or him, so pardon generalizations there)
I'm sure there is a way to make it work out of ugrad too, but don't be surprised if you hit some rocks along the way, and always know what you'll do in a worst-case scenario (you lose funding if you get it, he can't find a job, you somehow fall out of love) but just do what is best for you, and if he wants to come along and truly be a part of your ride, he'll let you know