Worms can be any where in the human body. Brain, organs, muscles...any where.
I would not get rid of the cat if I were you. I would keep the cat in the house and not let it out. Cats get used to this, even if they have been inside/outside cats.
It is not only pets that we get worms from. We get them in foods....we can even get worms from shaking hand with a person that has eggs on their hands....the eggs are transferred to our hand...which end up carrying the eggs to our mouths.
Just do a parasite cleanse once or twice a year and then take a maintainence dose in between cleanses.