bananapudding

I am in the second semester of the first year in a PhD program. Although I had better options when deciding school, the reason that I chose my current school was mainly because of my advisor (Yeah first years do not have official advisor, but he agreed to be my advisor in the program). But he decided to move to another institution after this semester, and he is giving me the option to go with him and start from the beginning there (no credit would be transferred). That school has slightly better reputation and research rank, and my current school has better networked faculty members, And I am fully satisfied with my program now. At first, he told me we can keep the relation even after he leaves and I can still get the benefit of my current school. But now, he is strongly recommending me to go with him. I am leaning toward to staying here as no credit will be transferred if I move and also for some personal reason. But, as he is strongly persuading me, it is hard to tell him that I will stay here. Am I making silly decision? Would he stop working with me if I do not go with him?