You will be responsible for the difference or balance left on your loan or lease. The at fault person's insurance will only pay the actual cash value (ACV) of your car and if you are upside down on your loan, thus owe more than the vehicle is worth, then you are responsible to pay the remainder of your loan balance after the ACV has been paid to the lien holder.

If you had gap insurance this would pay the difference between the ACV amount and the balance of your loan. The at fault party would not be responsible for your loan balance once their insurance company has paid you the value of your car since doing so has in insurance terms "made you whole."