Welcome to Gaia! ::

It depends on the mikvah. If it's a mikvah structure -- a building, with a preparation room and an attendant and towels -- the pool will almost always be heated to a comfortable temperature. It may be hot like a bath, or scarcely above body temperature, but it probably won't be freezing cold.

If the mikvah is hidden in a secret room or compartment of someone's home, so that Jews aren't discovered using it and then identified as targets for persecution, it may or may not be heated.

If you're immersing in a river, lake, pond, or ocean, season and geographical location will have more to do with the water temperature than anything else. Women in Siberia and other colder locations have been known to chip ice in a lake or river in order to make a hole big enough to immerse under.

I live across the street (with the back of my house facing the mikvah)from the only mikvah for a good distance around in Florida,(as you can guess I don't live in Miami or Naples or anywhere where there is a large amount of Jews, (We are lucky to get a minyan on Shabbos night and sometimes run short on Shabbos day, not so often anymore but it used to be that we would always get nine, no more and no less.) The mikvah is a little small and I've only been in there once (I did my conversion in New York and went to a mikvah there).
In response to wither a Mikvah will be warm or cold, the near my house is in Florida so it is at a warmish temperature. However, my dad once went to the Ari's Mikvah....there the water is melted snow running down a mountian into the Mikvah...it is absolutely freezing no matter when you go. My dad says he was lucky....he tripped on one of the steps going down and fell into the water. (My dad came from the country in upstate New York where it can get pretty cold in the winter.)
As a note about the Ari's Mikvah, it is only used by men because of how many visit every day.

Jewangel is right, different mikvaot are set up differently. In my community, the majority of them have men's hours during the day and women's during the evening. However, by special appointment, a convert or a bride may go during the daytime instead. I'm not sure what the rationale is for permitting a bride to go during the daytime, unless it's simply that she may want to go RIGHT before getting ready for her ceremony.

For a convert, the reason is that she (or he) will have three male witnesses, and those men shouldn't be going to the mikvah at night, when they would see which women are going that night too. That would put him in the position of knowing the menstrual cycles and the likely sexual activity of those women, which would be very immodest, so it's just not done. Women are supposed to go at night because the cover of darkness protects their modesty -- except in a place in which women are often attacked at night, so it's a safety issue, and in that case the women go during the day and the men don't go to that mikvah at all.