No, it's not scary. I say dont swallow as it will have absorbed the toxins from your mouth (which is what you want) and you dont want them back in the body. People drink Bentonite all the time it is not absorbed by the body at all.
And cooked garlic has no effect on yeast. Dont chew raw garlic swallow it in slices, it can burn the skin. You could try gargling with Grapefruit seed extract. Or Oregano oil in water.