No, vanilla almond milk isn't bad at all for women.
You may be thinking of the link between soy milk and estrogen.
However, I believe cows milk and the hormones/antibiotics given to the animals are all in all worse than soy.
Almond milk and soy milk are very different. smile
Yes, milks like Lactaid may work for you, too.