Answer: We need to first cover how big and strong was Goliath the Giant who challenged Israel and battled David. He was, of course, a giant, from the city of Gath. The Bible states he was six cubits and a span tall (1Samuel 17:4). Various Bible commentaries place the length of a cubit anywhere from 43 centimeters to 53 centimeters. The length of a span is believed to have been around 23 centimeters. Goliath, the giant of Gath, was easily 2.8 meters tall and likely bigger (see 1Samuel 17:5 - 7)!
The protective coat worn by Goliath in his battle with David weighed at least 78 U.S. pounds (35.4 kilograms). Several commentaries place the weight of the coat at an amazing 156 U.S. pounds (70.8 kilograms)! The length of his spear may have been as long as 26 feet (7.9 meters). The head alone on his spear weighed at least 17 pounds U.S. (7.7 kilograms). He was a formidable opponent indeed!
Your question assumes that David picked up five slingshot stones in order to fight Goliath and his four similarly gigantic brothers. Now we know from 2Samuel 21:16 - 22, especially the last verse, that there were at least four other Philistine giants. There is, however, no record of these other four challenging the army of Israel. Additionally, evidence suggests that Goliath may have had only one brother of a gigantic stature. Scripture does not clearly reveal to us how many total brothers he had.
What is interesting, from the viewpoint of faith, is that David took five stones (1Samuel 17:40), not just one, despite believing in God and His protection. He prudently planned that it might take more than one stone to kill the giant man, much as Jacob carefully prepared before encountering his brother Esau (Genesis 32 - 33).
David also wisely chose smooth stones in order to maximize their accuracy and minimize their resistance to the air. Other shapes of stones would have traveled through the air slower and less accurately.
The assumption that Goliath had four other huge brothers comes from 2Samuel 21, where the KJV phrase "which was of the sons of the giant" is thought to be directly related to his family. There is an interesting translation issue, however, in this chapter in that the term translated "the giant," in the plural, can mean "Rephaim."
And Ishbi-Benob, who was of the sons of the giant [rapha or rephaim, Strong's Concordance #H7498] (the weight of whose spear was three hundred shekels of bronze), being girded with a new sword, thought to kill David . . . Then Sibbechai the Hushathite killed Saph, who was of the sons of the giant (rapha or rephaim) . . .
And there was a man of stature who had six fingers on each hand, and six toes on each foot, twenty-four in number. And he also was born to the giant (rapha or rephaim). . . (2Samuel 21:16, 18, 20, HBFV).
This apparently was a race or breed of giants, such as Og, the king of Bashan, who had a bedstead of about 13 1/2 feet in length and 6 feet wide (Deuteronomy 3:3, 11; cf. Joshua 12:4). Although one of these four other Philistine giants was the brother of Goliath (2Samuel 21:19; cf. 1Chronicles 20:5, which preserved the text more correctly), the others are not clearly said to be. Another giant race in the Bible are called the Anakim (Numbers 13:28, 32 - 33), but it does not appear they were related to the Philistines whom David fought.
In conclusion, young King David fought only a single giant, Goliath, in the famous Biblical confrontation of 1Samuel 17. Later in his life, however, He would fight the physically large and powerful Philistines on many occasions.