Whether it is legal or not to drive with a broken ankle in a cast is up to state laws so you would need to check with your state's Department of Motor Vehicles or like agency.

There might be a state law or statute prohibiting from a driver operating a vehicle if they are temporary disabled in such a way that would make them an unsafe driver or if it is ill advised by a doctor. If you are medically not advised to drive with a broken ankle in a cast it would be best if you did not drive your car until given medical clearance.