Will a 4.0 GPA in a non-thesis M.S. overcome a 3.5 undergrad GPA when applying for grad school?

[ExistentialQuery]

I am currently enrolled in an online M.S. in applied physics program and am wondering, if I maintain a 4.0GPA, if it will outweigh the 3.5 GPA I had in undergrad. Should I instead focus on research? Should I reach out to professors at my current institution to build relationships that could take me into a PhD program at this institution(JHU)? What should I be doing now as a M.S. student to increase my chances at getting into the PhD program?

The reason I am enrolled in an online, non-thesis M.S. is because I just recently had a daughter (3mo. old) and can not just up and move at this time. In a couple of years, I plan on relocating to pursue a PhD in physics. 

Thank you for your time.