Also known as the “Sea Nomads” these people have been living semi-aquatic lives for thousands of years. Almost their entire lives revolve around the ocean. These people are constantly diving and thus holding their breath. This has lead to several evolutionary novelties in these people which are adaptation for diving in the ocean.
An increased spleen size due to genetic mutations in PDE10A in these people acts as an oxygen reservoir so they can hold their breath longer. Mutations in the gene BDKRB2 also increases peripheral vasoconstriction helping the body preferentially oxygenate important parts of the body like the brain while diving. Mutations in the gene FAM178B (which might have evolved via adaptive introgression from Denisovan interbreeding) help maintain regular pH in the bloodstream to prevent carbon accumulation during diving, variants of PDE10A help regulation of smooth muscle contraction including ones near the spleen, and variants of CACNA1A are involved in the regulation of the release of the excitatory neurotransmitter glutamate and the response to hypoxic conditions.
Essentially this group of people has evolved to become extremely resistant to hypoxia because of selection pressures from regularly diving. So yes, it is very possible for a lineage of human exposed to aquatic environments to develop adaptations for swimming. Not just possible because as you can see here it is happening before our very eyes. Because humans still evolve, we never stopped. Though not likely a single family of diving people led to it, because inbreeding tends to lead to extinction.