well you're right herpes can show up any where in the boxer shorts area but it's a bit weired that you didn't get any thing on your genitals but who know Herpes is very sneaky and operates differently in different people. But if you don't trust your doctor why not switching to a different one?
Did you tell ur doctor that you had that fluid coming out of your penis? That may help a bit as some people get a little bit of discharge from penis caused by herpes outbreak.
I don't think you can do any thing but trying to get it diagnosed. You should ask the doctor to swab those sores for a type specific herpes culture if you still have them. If they're gone then you have to wait untill they show up again or 3-4 months later take a type specific IGG herpes blood test as you body needs 3-4 months to develop the anti bodies if the infection is recent.
And yeah it is very possible that all the flu symptoms were actually flu symptoms, nut seriously there's nothing people can do online unless you force ur doc to swab the lesions for a test.
hope this helped, don't hesitate to ask questons, I'm the same way on other forums ;)