exempted from VAT, invoice with 0% VAT, but customer still paid VAT

I sent a correct invoice as exempted from VAT, but my customer still paid me 21% extra on top of my invoice amount. What should I do about this? Report it and refund the money? There’s no need for a credit note because the invoice is correct, right? It’s only about £3, but I’m afraid the tax authorities would do a fuss if they ever checked my accounts.