As long as the state's records show that your license was valid at the time of the accident, you shouldn't have anything to worry about. The at-fault driver's insurance company may try to pin the blame on you if their driver saw your expired license but they won't get far. Fortunately for you, it's pretty easy to verify that a license is current and active, so their efforts would be in vain.
Usually, it's best to have a police officer come out to the scene of an accident because an official accident report will be filed with the police department. In your case, it looks like not getting the police involved might have worked in your favor. Since you didn't have your current license on hand at the time of the accident, you could have received a ticket or been taken to the police department to have your license status verified. As you can imagine, having "invalid license" listed by your name on a police report could have complicated matters further.
It's important to understand that the phrase "driving without a license" doesn't usually mean someone left his or her license at home. Typically, the term refers to drivers who have had their license revoked or suspended. While you can be fined for driving without having your current license with you, driving with a revocation or suspension is a much bigger offense that can result in jail time.
In your post, you didn't say if the other driver was aware that you didn't have a valid driver's license with you. If during the course of exchanging information, the at-fault party noticed that your license was expired, they may have told this to their insurance company in hopes that you would automatically be faulted. Should this be the case, their insurance agent or claim representative will contact your insurance company in an attempt to file the claim on your insurance. For this reason, you should talk to your agent and let them know what's going on in case the other company tries to file a claim on your policy. Contesting the claim before they can file it is your best course of action.
As stated above, the company would quickly realize that your license is actually valid but you don't want a claim on your record that you didn't cause. Unfortunately, claim information doesn't always carry over correctly when you change companies and a complicated situation like this might cause you a headache in the future.
Hopefully the other driver will be honest about the situation and file the claim accurately so that you can get your car repaired in a timely manner. If you have the other party's information, feel free to contact their insurance company and check on the status of the claim. The insurance adjuster or claim manager will need to get your official statement of what happened, anyway. Good luck with your claim and rest assured that you cannot be marked at fault simply because of your failure to carry a current driver's license.