I'm 40 and just started my Master's this fall. I'm working full time in the field and planning to take one, maybe two, classes a semester. The program isn't geared towards non-traditional students, in fact I might be my advisor's first in 20 years of teaching. He's supportive but has also let me know, in a positive way, that the expectations won't be any lower for me. I appreciate the challenge, even if I've already had a few mini breakdowns this semester! It will get easier. Or it won't, but I'm going to finish anyway.
Your start might be later but it's not that late. I talked to a professor before I applied (different program) and she said no colleague she knew of would ever consider 40 too late for grad school.