Avoiding cataract surgery can lead to blindness. As the cataract gets worse, it becomes more opaque, meaning you can't see through it. It also becomes more difficult to remove, which can lead to complications after surgery, including retinal detachment, of retained lens cortex, which can cause glaucoma and infections, and loss of vision.
Cataract surgery is a simple procedure, and although you can wait to have a cataract removed, if your vision is 20/50 or worse with your glasses, it's time to have it taken out. Keep in mind, as the cataract gets worse, so does your vision and ability to drive both in daylight and at night. Avoiding cataract surgery may not lead to permanent eye damage, but if you have a car accident because you can't see, it could lead to a loss of life.
That's not a scare tactic, I personally have experience with this, and it happens quite frequently.