I am not sure I completely understand what you mean by "sexual". Yes I am sure that it isn't because of sex that I don't feel comfortable with children seeing me. I really don't know how to explain it. I am sure there is a naturalist resort near you, maybe you should go check it out. :0)
For me, it started out as something I did on my own. I had heard that others did it too, but for me it was still a personal thing. After a while, and yes biulding up confidence I went to a public beach. I don't care what any nudist says... at least for me, it isn't sexual but people still judge... "oh he is hot, or she is hot". Of course "Hot" here means, that they are well built, or take care of themselves. It has nothing to do with wanting to have sex with that person. Just like I am sure you do not want to have sex with ever cute guy you see... well I don't think you do... *smile*
I think you hit on something though... the child making up their own mind. Children are really to young to know the difference, and will act as they see their parents or people they respect act. hmm... I will have to consider this line of thought for a while.