Some plain facts from one who knows

1. Lots of men get a kick out of wearing women's underwear - it really is very common.
2. It doesn't necessarily imply anything else about your relationship, your attractiveness, his masculinity etc. He just likes that and that's all.
3. It's far from unheard of for partners and spouses to know about, accept it, even be turned on by it. It certainly doesn't have to break up a relationship, even if you don't like it.
4. I know you are struggling with it but just think rationally - how much of a problem is it in the grand scheme of things? Or compared to really bad things that happen in relationships?
5. You really should talk to him about it, however difficult.
6. If this seems the strangest fetish to you, I can tell you there are much, much stranger.

Just wanted to give an alternative perspective to that other reply. Tweet