Transferring is not really unusual or terribly frowned upon. While your program would probably prefer that you stay, it will, in all likelihood, understand your choice to transfer. People transfer for all sorts of reasons (a POI leaves, they learn that the program wasn't what they expected it to be, etc.). In fact, I do know several assistant professors who transferred when they were grad students (they got their M.A.s at the program, and then just applied to different PhD programs).