The headache should gradually improve as your body gets used to less caffeine. If the headache persists, you should consider some other reason for your headache.
Withdrawal from caffeine can cause headaches.
To explore more reasons for headaches,