In strong opinion should not swallow or drink sperm for the following reasons:
First: The act allows the ingestion of something unclean like madhi, wadi, or urine.
Secondly: Mani is a mustakhbats (disgusting), so the scholar who says it is holy also thinks it should not swallow it, because the word of allah:
وَيُحَرِّمُ عَلَيْهِمُ الْخَبَائِثَ .الأعراف: 157
Meaning: "And he (Prophet sallallaahu 'alaihi wasallam) forbade the things that khabits (very dirty / ugly)" (Surat al-A'raaf: 157)
Said An-Nawawy rahimahullahu:
هل يحل أكل المني الطاهر؟ فيه وجهان: الصحيح المشهور أنه لا يحل لأنه مستخبث