Fever Blisters and Cold Sores In order to get rid of a fever blister, use a diaper rash cream with the highest amount of zinc you can find. You can also try spraying Bactine on a Q-tip and then rubbing it on the blister until it goes numb. Listerine also works wonders! Image Source: Pexels / Oleg Magni