Well, you guys will know in the next few days if you spread it to him.
I think you are being a little hard on yourself though. One thing, that was a very responsible move to not have sex. Many people in the heat of the moment do not choose that path. Also, he knows you have it and has made a choice to be with you even though there is a small risk. I think this reflects well on his character as I feel running away from someone just because they have herpes is lame. But what I am saying is: it takes two to tango. You did not deceive him. He knows you have it. He is an equal part of the equation so try not to get too caught up in feelings of doing something horrible. There is always a risk in your situation regardless of a visible outbreak. And when you meet someone cool and you are into each other: it's worth the risk. :)
The herpes virus doesn't do too well outside of the body. Even if it was on his hand after touching you, he would have to put his hand in his mouth pretty quickly. For instance, the herpes virus can't live on toilet seats and such things. It needs to either be incubating in a mucous membrane or quickly travelling to another mucous membrane or area of skin. Too much in between time kills it.
I wouldn't worry too much about it unless he has an outbreak. It seems unlikely though, but there is a chance. And, in general, the place we spread it from is the area where the outbreak is. So if you were having an outbreak on your anus but not in your vagina, USUALLY your vagina would be considered safe. I say usually because we can have that asymptomatic viral shedding business where you spread it but are not having an outbreak.
In general, my opinion is that when you are having an outbreak anywhere down there, you should probably stay away from everthing down there until it clears up. :)