You are absolutely correct. If you had Collision coverage it would cover damage to your vehicle but without it since you were at fault you will have to pay for the damages your car sustained in this accident. When you make a claim against your own insurance policy it is referred to as a first party claim.
Your Property Damage Liability coverage will cover damage to the other person's vehicle. You should promptly report this loss to your insurance company if you have not already. The person that you hit should call your insurance company to start their third party claim against your Liability coverages.
You can go here to see what adding Collision coverage will cost if you want to add it to your Liabiity coverages so if you are in another accident your vehicle would be covered. With Collision coverage you would pick a deductible amount that would be due when making a claim. With third party claims, so for the person you hit, making a claim against your liabiity coverage does not come with any deductible amount to be paid.