No Hell
Before we delve into whether babies go to heaven, we need to make something clear. The Bible teaches that those who are not saved at death, no matter how old, do not get tortured in some every-burning Hell. The truth is that the dead are unconscious and await a future resurrection. This is true for babies (even those who are aborted) as well as for adults.
Valley of Dry Bones
There is a quite interesting passage in Ezekiel 3 that is related to our discussion of the fate of babies. What Ezekiel saw, in vision, was a huge valley filled with dry bones. This vision represents the overwhelming majority of humanity, including almost all of ancient Israel, who died never having been "saved" as we would term it today.
All the people Ezekiel witnessed, which included babies, will be resurrected in the near future. They will be brought back to a physical life where they will come to know the true Lord God of Israel.
"Behold, O My people, I will open your graves and cause you to come up from your graves, and bring you into the land of Israel . . ." (Ezekiel 37:12).
The Bible teaches that the dead are just that, dead, and do not go to heaven or hell at death. As wise King Solomon wrote, the dead know nothing (Ecclesiastes 9:5 - 6, 10)!
No Automatic Salvation
Babies are not automatically saved, or even lost, because they cannot understand the truth or choose to have saving faith. Although baptizing infants right after they are born seems spiritually comforting to the parents, it cannot be seen as a practice clearly mentioned in Scripture.
The apostle Peter told a crowd gathered in Jerusalem, on the day of Pentecost, to do something babies and small children simply cannot do. He asked them to repent of all their sins, be baptized in the name of Jesus, and receive the gift of God's Holy Spirit.
Then Peter said to them, "Repent and be baptized each one of you in the name of Jesus Christ for the remission of sins, and you yourselves shall receive the gift of the Holy Spirit." (Acts 2:38).
Can They Repent?
Do babies understand and know enough about their sins to repent? Do they know who Jesus is in order to have saving faith in Him?
Does the baby of an unsaved person (like an ignorant peasant) automatically have salvation until that child reaches a particular age? If this were true, then wouldn't it be better for the child to die very young rather than grow up disbelieving, and possibly fighting against, God?
Conclusion
All those (including babies, and even those who were aborted in the womb) who were never given a full chance to knowingly understand the truth of the Bible will, someday, be given a chance to be saved (receive salvation, see Revelation 20:5). This is a marvelous truth the world does not comprehend! God will eventually get around to all humans and offer all of us a chance to know him and live not in heaven but on the earth.