Go for it! Be confident in your abilities.
I did my undergraduate studies in Math at Queen's, and my graduate work at Simon Fraser University. I am currently a Professor of Mathematics, and now teach and do research in the Liberal Education Program, with a focus on teaching critical thinking and problem solving skills.
I've always loved teaching the best! Research is fun, satisfying, challenging and interesting, but what recharges me is working with students. I also love the flexibility and independence of the academic life, being able to focus on different aspects of the job at different times and make my own decisions about how to balance teaching, research, service and family life.
My math research was in abstract or general algebra, and I loved the study of pattern and structure, and the mental exercise it gave me. Currently I work in SoTL, the Scholarship of Teaching and Learning, and I study how to teach students math, quantitative, reasoning and problem solving skills, especially for students who are not in math or science.
I've always said that math training teaches you to think about anything, and I firmly believe this. There aren't many jobs labelled "mathematician", other than being a university professor, but math grads end up in all kinds of careers due to their analytical and reasoning skills.
It certainly was in my early years at my university, in my department (Math and Computer Science) and the sciences in general. The situation has improved a lot in the last twenty years, in terms of number of women and how they are treated. But I still see issues of lack of confidence in my female students, and lack of support for them, so we still have work to do.
Be aware of the different skills that different experiences and mindsets can bring; try to recognize and work around our own stereotypes and our comfort with people "like us;" discuss the diversity issues involved in things like hiring, tenure, promotion, and teaching.
I think discussion of issues is important - to recognize that female students still lack confidence (I've done research that shows this!) and to talk to them explicitly about this, and how to overcome it.
Go for it! Be confident in your abilities, seek help, get to know other women in your area, both beginners and more senior people, and talk to them about their experiences and your own.
I didn't have many in the early parts of my careers, or at least not many women. These days I have a lot of smart, hard-working, caring female colleagues, who support each other.