I have both nystagmus and myopia from birth. I'm just past the legally blind level. I am considering getting contacts as I've never been comfortable with glasses (I have a small nose and constantly pushing glasses up when I wear them). My mother told me I could not wear contacts because of my nystagmus but I've heard otherwise.
Every has nystagmus. Your anxiety makes it a bit worse. You aren't legally blind. Legally blinds means that you cannot read the big E at the top of the eye chart with your glasses on. How strong your glasses are or how badly you see without your glasses has nothing to do with legal blindness. You need a therapist, not an optometrist.