Here's an excerpt from Dr. Eric Bakker.
"But an important point you need to understand is that many who think they have Candida, in fact don't have Candida. They have a bacterial or parasitic problem quite different to a yeast infection. I have seen this on more occasions than you can imagine, especially after a comprehensive stool test."
Maybe the reason you have benefited so well from the low carb diet is because you could be dealing with a bacterial overgrowth.
In the same post Dr. Eric says this referring to diets regarding candida:
"The honest truth of the matter is this - it DOES NOT matter what a person eats providing they don't severely aggravate, and I believe this is because everybody's microbial population will differ. There is certainly NO one diet that is the "perfect Candida diet", and anybody who tells you they have found the "perfect diet" is a big fat liar. I've worked with Candida patients most of my professional life as a naturopathic doctor and can tell you this, I'm continually surprised at when some patients can and what some can't eat when they have chronic gut issues."
I would still recommend dropping the dairy.