Babies do not hear the phonemes of a language, but they do hear the building blocks of the phonemes of all languages.
When a man hears speech, he hears discrete sequences of phonemes. This is because he has learned the phonemes of his language, and now has an internalized perceptual overlay of those phonemes to place over and interpret the speech stream.
Babies, however, have no knowledge of phonemes and no overlay. They hear only a continuous stream of sound including a host of discriminating features, like noises and tones, that rapidly change.
Ironically, for learning a new language, the baby has the advantage over the man.
Babies are born with the neural connections to hear all of the discriminating features in the stream of sound.
These are the building blocks of the phonemic system of every language. The "perceptual overlay" that the man uses to understand speech, masks out many of these discriminating features.
Hence, the man will hear only the sounds he expects to hear and miss what is really being said.