I don't know how you wouldn't panic. It's a natural reaction to that sort of thing.
Listen to the doctors and wait for more information. In the meantime maybe consider taking someone with you to the next exam if you are not already. Another set of ears never hurts in these situations.
Window shop for motor-homes. That ought to take some of the edge off.