I think the way you address it depends on how big of a question you feel it raises. For most cases, I think that rainy_day's advice is the best way to go, as it's a clean and simple way to acknowledge that the work is not central to your current interests while still letting you include it as part of your whole package.
If you feel that it's going to be something of a red flag, however, then I think it's very much to your advantage to preemptively answer whatever questions the adcomm might have. Is there a way to briefly mention Field A in your SOP - as either reinforcing your interest in Field B, or adding perspective to it? Why DO you want to study Field B, in light of your experience in Field A? This could just be a sentence or two, some segue like "My time as a _____ made me realize that _____, which inspired me to.../deepened my commitment to..."
If nothing else, though the field might be unrelated to your current interests your work still shows that you're at least a hard worker and possibly also organized, reliable, a leader, a good communicator... there are a lot of important qualities that are independent of any one discipline. Is it possible to say something like "After graduating, I worked in _____ where I became very skilled in _____. This will serve me well as I pursue my true passion in _____ because _____"?
I hope that's helpful - best of luck!