OK, I have a higher sex drive than my fiancè too, but that doesn't mean he doesn't love me or isn't attracted to me. The same goes for your husband. We get taught that women are ones that don't want sex, and men want it all the time but it's simply not the truth. You're either going to have to be OK with having sex when he wants it, or if you're really not OK with that; move on. As much as you love him, your happiness and pleasure does matter. Although, you should've know about your sex drive differences by now, assuming you've been together a while before marriage, of course. People tend to paint the one with the higher sex drive as the bad guy; the one that should understand, but your feelings are just as valid. I have issue with him making you promise not to ask to have sex on special occasions though, I would personally sit down and talk to him about that.

does he suffer from depression? has he been checked for low testosterone levels? there are plenty of reasons why he doesn't want sex that can be improved with medical intervention. make some appointments for him, and go with him.