I'm so sorry that you are going through this and I hope that no matter what, you come out better for it.
Without going in depth into my past, I can relate to what you've been though. I agree with the other poster that it is important to establish beyond a doubt that infidelity exists, but from how you are describing it, it sounds like there is little doubt. I think one of the worst pains in life must be knowing that the person we love has betrayed the bond that once existed and seeks the company and intimacy of another.
I think the first thing that may help is to try to come to a place of acceptance. Don't get me wrong - it is never acceptable for anyone to treat you in a deceitful way. But rather I mean accept the pain that you feel, the craziness, the anger, the denial, the compulsion to follow him - these are all completely normal and human reactions to what this personal has done to you. It is normal of course to act on any desires to harm him physically or the woman - but my point is, don't beat yourself up or think yourself a "crazy stalker". Accept your reaction as normal. Any one of us would, and have, reacted in the same way. Accept your feelings, just try to not let them dictate your actions.
Secondly, if he is in fact with another woman, accept that you following him or knowing where they are is not going to change it. So you find him - what then? What are you going to do with that information once you find them together? Do you need to know this information in order to make a decision? Would knowing where they are change anything? Only you can answer these questions for yourself, but I found that when I realized that following my compulsion to follow him around in a crazy frenzy only resulted in - ok, here he is, now what do I do? I was still left with myself and the same decision in front of me.
And that decision is yours. I do believe that relationships can be salvaged after infidelity, and those that are can come out even stronger. But I believe that takes a gigantic commitment on both parts, the passage of time, and a good therapist. If you choose to give that a go, it is definitely a dance that requires a partner and cannot be done by just you alone.
I believe one of the most important things to consider when choosing whether or not to give it another shot after infidelity does include considering how often the infidelity took place (whether or not he is a repeat offender or it was, so far, just a one time thing), the context of the infidelity (the quality and level of the intimacy in your relationship), and whether or not the cheater shows any interest in changing his behavior. I absolutely believe that upon finding out infidelity, the VERY first thing one should do is pack a suitcase - either yours or his. Separation needs to occur to establish a boundary and to make very clear that you will not remain in a situation in which you are treated in this way. I believe that this is absolutely key, that it should occur right away, and that the longer you wait, the more likely it is the cheater will cheat again. They need to see your limits and your boundaries, they need to feel the full consequences of their actions. If you stay, I believe you are helping to bury the relationship by showing that you have a tolerance for infidelity and this will fuel the disrepect that the cheater already feels for you. You said that it was difficult for you to leave - maybe viewing it this way will help. I'm not saying leave for good, I'm saying at least a temporary separation. And you must mean it - the cheater must over time know that they cannot wear you down with just words that sound good - you must stick to your guns until the cheater shows throught their actions, over time, that they are different. If this doesn't motivate them, then you really must accept reality and realize that you are better off. And it is on THEM, not YOU.
If you are having trouble bringing this to a head in which you have an opportunity to make an absolute decision, may I suggest forcing this into a crisis. I'm not suggesting violence of any sort, no hitting him over the head with a frying pan - but don't shove it under the rug - if he comes home in the middle of the night or four hours late, or whatever his pattern may be, don't just resign to it. When you've had enough, you won't take it anymore. And at some point, we all get enough. Some of us just take more than others.
I wish you both luck,