I would also say ask him, of course he'll understand.
On the other hand, I think you have a good thing going at U of W. There are great things about Yale's program, but it's not the best in terms of preparing one to apply directly to a PhD. That's why in the end I chose Stanford over Yale (plus who can beat the weather in California! Haha).
First, you don't get an advisor. I had a friend who went to Yale, and she said it was kind of hard not having an advisor. Second, there's no thesis requirement, so there is less support for doing really in-depth original research. Third, it's just a one year program. There are a couple of disadvantages there. Like was mentioned above, you've only been in classes a couple of months when you have to ask professors for LoRs, so they just don't know you that well. Also, another big advantage of a 2-year program is that you can apply for grants to do research over the summer-- SO important in applying to phd programs!
I have a friend who did the program at Yale, and is now at Stanford doing a second masters to prepare for her PhD application. So that could be another option for you.
In the end, consider your options carefully, and do what you think will be best for you. Good luck!