I can hardly claim to speak with any authority, but from what information you've given us here, there's no reason you should resign yourself to the snowball's fate. Your numbers are good, pushing the interdisciplinary angle is a wise strategy. And coming to rhetoric from math and philosophy is not like coming to rhetoric from modern dance and business management. Your burden will be answering the questions "Why English?" and "Why now?" (I'm in a somewhat similar spot and this is what I've been told by professors I've emailed with). Make sure in your statement of purpose explains clearly why you want to pursue a Ph.D in English, given your non-English background.
It's likely you won't be tossed out, as long as your writing sample is of good quality. If your pool of samples is small, that may not matter. You (usually) only need one very good sample. And don't worry if it's not closely related to what you plan to study. The sample is still important as evidence of your type of approach, style, argumentative and analytical skills, and all sorts of things that give the committee a general idea of how you think about rhetoric.
Good Luck!