Herod's temple was not the first in Jerusalem, but the second (or third, depending how you view it) building dedicated to the true God. The first edifice dedicated to worshipping the Eternal came as an idea to King David (2Samuel 7:1 - 2). Although he could not build it himself because he was a man of war (1Chronicles 22:8 - 10) he collected materials for its construction. David's son Solomon began work on the first temple sometime between 966 to 965 B.C. He built it on Mount Moriah, which is the exact location where Isaac was to be offered as a sacrifice by Abraham (Genesis 22:1 - 14). He finished God's house around 959 B.C.
In 586, still many years before Herod arrived on the scene, the temple was completely burned, pillaged and destroyed by Babylonian King Nebuchadnezzar. He carried all of its treasures to Babylon. In 539 B.C., Cyrus the Great of Persia (who had conquered Babylon's Empire) allowed captive Jews to return to Jerusalem and begin the rebuilding process (2Chronicles 36:23, Ezra 1:1 - 4). Soon after their efforts began, they met resistance from those in Samaria and work was halted for about fifteen years (Ezra 4). With the aid of the prophets Haggai and Zechariah, however, work started back up. It was finally completed in 516 (Ezra 5, 6).
In 40 B.C. Herod the Great, in Rome, is made 'king of the Jews' (king of Judea) by the Roman Senate. In 20 B.C., he proposes, partly as a gesture to gain the favor of pious Jews he often enraged, to take on the task of rebuilding the temple. Such a project was an incredibly ambitious, bold and expensive undertaking even for a king with a giant ego.
The house of God, which had existed for roughly 500 years, was in rough shape. Over the years, the temple had experienced neglect, a tremendous amount of natural decay, and some destruction as various armies sought to conquer it and the city. Concerning the monumental task Herod ultimately took upon himself, the first century Jewish historian Josephus wrote the following.
"And now Herod, in the eighteenth year of his reign, and after the acts already mentioned, undertook a very great work, that is, to build of himself the temple of God, and make it larger in compass, and to raise it to a most magnificent altitude, as esteeming it to be the most glorious of all his actions, as it really was, to bring it to perfection; and that this would be sufficient for an everlasting memorial of him . . ." (Antiquities of the Jews, Book 15, Chapter 11)
According to the 1906 Jewish Encyclopedia, Herod completely tore down the existing temple structure and rebuilt it from the ground up. Although he died around 4 B.C., the rebuilding effort he started continued long past his death. In fact, work was still being performed during Jesus' entire ministry (John 2:16, 19 - 21) and continued as the New Testament church expanded into many parts of the Roman world. It did not completely finish until 65 A.D.
Solomon, with his wealth, a giant storehouse of materials gathered by his father David, and the ability to employ countless laborers, built the first temple from scratch in seven and one-half years (see 1Kings 6:1, 38). The rebuilding under Zerubbabel and others (Haggai 1:1, Ezra 3:8 - 13), including the period when work was halted, was completed in about twenty-three years. The labor began by Herod, however, took more than eighty years.
Sadly, only a few years after renovations were completed, Herod's temple was destroyed. This event was prophesied by Christ (see Matthew 24). In 70 A.D., Roman legions stormed Jerusalem, and notwithstanding the efforts of General (later Emperor) Titus to preserve it, his soldiers set fire to it in several locations. Its total destruction began on the 9th of Ab, the same day its first destruction occurred. Its demise was completed on Ab 10, which corresponds to Sunday, August 5th in 70 A.D.
Ultimately, the need for an earthly sanctuary in Jerusalem was fulfilled and brought to completion by the sacrifice of Jesus Christ, which ended the Old Covenant and the need for a physical priesthood. Under the New Covenant, the TRUE temple of God is in heaven where Jesus Christ is High Priest and head of His church.