I see no reason for you not to have your thesis advisor write for you. Their letter will speak to your fundamental ability to do historical research, which is not subfield specific. Your interests are also expected to continue evolving. I wouldn't even consider those fields to be especially distant, both being connected to medieval gender and sexuality.
Really only you can decide whether you should go for the PhD. As you've seen, there are tons of threads on this forums discussing it. You'll likely not have a tenured position in academia after finishing, so can you live with a job in, say, academic administration or publishing or something like that? Those are areas where history PhDs tend to end up.