From the moment Lord Varys meets Melisandre in Game of Thrones season seven, he despises her. In fact, he dislikes her long before he ever meets her, back when she believes Stannis is "The Prince That Was Promised." Believe it or not, his distaste for her is not personal. Varys hates all forms of magic and those who practice it — for good reason. When he is just a child, he is sold by his master to a sorcerer. The sorcerer gives him a potion that leaves him unable to move or scream, even though he can still feel pain, and then the sorcerer castrates Lord Varys and uses his parts to perform some sort of dark magic. Since that night, the Spider has had no use for religious zealotry or the dark arts. As a Red Priestess, Melisandre's blind devotion to the Lord of Light and her use of sorcery not only offends Varys, but likely also frightens him.
Given everything he's been through, it's hard to fault Varys for having no love for the followers of the Lord of Light. In season six, he's just as hostile with the High Priestess of the Red Temple of Volantis, Kinvara. Even though they are ostensibly on the same side, he greets Kinvara with the same defiance he later offers Melisandre. Both women do little to assuage his fears. If anything, their words leave Varys even warier of them.
His fear of their gifts goes back to the night he is cut. Before the sorcerer leaves young Varys to die, Varys hears a voice in the flames, and the experience changes him even more than becoming a eunuch does. He confesses to Tyrion that this voice has haunted him his entire life, but he doesn't divulge what he heard. However, Kinvara knows every detail of the events of that night, and she relays them to Varys in order to convey the extent of her power. She offers to tell him who the voice belongs to, but Varys looks too afraid to respond to her. Whatever he heard terrifies him, and he carries that fear with him throughout his adult life, projecting it onto anyone who dabbles in magic.
In "The Queen's Justice," Melisandre drops a major prophecy on Varys that may connect back to what he heard in the flames when he was just a child. Before leaving for Volantis, she tells him:
"I will return, dear Spider, one last time. I have to die in this strange country, just like you."
The stunned look on Varys's face is telling. This may not be the first time he's been given a hint about how he will die. His whole life, he has been haunted by the magic that scarred him as a child, and it seems that Varys is now heading toward a dark fate. Melisandre's prophecy is just one more instance of magic disrupting his life, and if he allows her words to affect him too much, this prophecy could be the end of him. The Red Woman's not-so-fond farewell certainly does little to endear her to Varys.
Those who practice sorcery have done nothing but maim the Spider and plant seeds of doubt and fear in his mind. His hatred isn't aimed at one individual; it's aimed at everything these people stand for. After all, Varys is a man of the people. No matter how much power he gains, he never loses sight of his primary goal — to protect the realm and all the poor and innocent souls the monarchs often regard as playthings. He knows all too well that people like the sorcerer who bought him, Melisandre, and Kinvara make no qualms about sacrificing human life in the name of their god. That makes them dangerous and despicable in his eyes.
Magic has never brought anything good into Lord Varys's life, and his distrust of those who practice it is well-earned. Now that Melisandre has left him pondering what will lead to his own death, his opinion of the Red Priestess and her fellow worshipers is unlikely to become any kinder before her inevitable return to Westeros.