On the surface, it does seem a bit odd that such gospel accounts do not quite mesh. Matthew says Jesus was born in a house in Bethlehem then sometime afterwards is taken to Egypt to escape Herod. After some time passes, his family decides to return to Bethlehem, but soon change their mind and travel to Nazareth instead.
According to Luke, however, Mary and Joseph were from Nazareth. They travel to Bethlehem because a census requires them to do so. Jesus is born in a manger while they are in the city. They wait, after his birth, for Mary to go through ritual purification, after which they travel to Jerusalem to sacrifice two birds at the temple. They then go home to Nazareth.
As always in the Bible, both accounts are correct because they speak of two different periods. A quality Harmony of the Gospels should be able to lay out the time sequence of Jesus' entire life based on the four Gospel accounts. Luke's account is the actual event of Christ's birth in the manger.
Gospel time sequence
The time sequence from the birth of Jesus to him being taken to Nazareth is as follows. Joseph is of the lineage of King David, who was born in Bethlehem. The Romans require all those in Judea to return to their ancestral home so that they can be counted (it really was for assessing what the people owned for the purposes of taxes). Mary and Joseph, because of this decree, travel to Bethlehem. It is in the city that Jesus is born in a manger (Luke 2:1 - 20).
On the eighth day after his birth Jesus is circumcised according to the law of God (Luke 2:21). Wise men from the East (Magi), after seeing and following a "star" (almost certainly an angel) for two years, seek Herod the Great's assistance in Jerusalem (Matthew 2:1 - 3).
Jesus is brought to Jerusalem's temple, after forty days of purification required by God's law, to be presented before God. His parents make an offering to the temple of two young birds. It is also during their visit to the temple that a priest named Simeon, prophesied about his mission in life and blessed his parents (Luke 2:22 - 35).
The priests and scribes inform Herod that the Christ would be born in Bethlehem (Matthew 2:5 - 6). Herod encourages the Magi to find the Christ child (feigning he wants to worship him as well) then report to him (verses 7 - 8). After leaving Jerusalem, the Magi notice the "star" that brought them to Judea has appeared again! It leads them directly to a house (not a manger!) where they find Mary and Jesus.
Finding them in a home, they offer their gifts of gold, frankincense and myrrh (Matthew 2:9 - 11). The Magi, after being warned in a dream, do not return to Jerusalem to report to Herod (verse 12). An angel, after the wise men leave Bethlehem, tells Joseph (in a dream) to flee to Egypt because Herod will soon want to kill his child (verses 13 - 15).
Herod flies into a rage when he discovers the Magi are not coming back to Jerusalem to give him the information he wants (Matthew 2:16). He then orders the cold-blooded murder of all Bethlehem area males two years old and younger (verses 16 - 18).
After Herod dies in early 4 B.C. an angel of the Lord again appears to Joseph, in a dream, and tells him it is safe to return to Israel (Matthew 2:19 - 21). Fearful of going back and living in Bethlehem, Joseph is instructed in a dream to go to Galilee (Matthew 2:22 - 23, Luke 2:39). The family makes the long trip and goes back to living in Nazareth.
In conclusion, both Gospel writers Matthew and Luke are correct in regard to their accounts of Jesus' birth. Their different but complimentary books not only show their record was true (and not simply copied) but gives us added details regarding one of the greatest events in the Bible!