Choice of doctor

Of all the accusations made against a single-payer health care system, the charge that we would not be free to choose our doctor is the most astounding. I'm currently looking for a new doctor, and just as before, the current system is very restrictive upon my "choice." Most importantly, I must choose someone in my insurance plan; if I hear of a doctor outside my plan, too bad, you don't have that choice. (With single-payer, you can choose any doctor, by way of contrast.) Anytime I hear the fear that a universal plan will take away choice, I know that this is propaganda devoid of substance. After all, what choice do I have now?