If you close a credit card with a balance, you'll still be responsible for that debt. Card issuers will continue to send statements in the mail, and interest will still be applied to that balance.