I totally agree with you, in fact I try to choose conference in departements which I think share similar perspective on my field of research. Also I don't think that if I write on my cv that I gave 3 paper on this topic it sounds like I just want to travel, it would maybe show that I'm ambitious and so on.
When I look at cv's of young professors they list even minor conferences, and in my case it is better to show that I did something aside writing my ma dissertation, so I can claim, if not a growing expertise on a topic, at least the will to research.
Also, my academic cv is empty, I mean, I never gave a paper and never published anything, but in Europe is pretty normal at the ma level.
Maybe I'm too idealistic and I of course think that is better to publish an article than to give a paper, but why not refine ideas for an article with the feedback received at a conference?