ANSWER: We can begin to find our answer about whether Jesus had literal brothers by looking at the accounts of his family trying to talk to him. The scene in question begins with self-righteous religious leaders demanding that he give them a sign to prove he was the Messiah (Matthew 12:38). Christ labels this request as evil and rejects it. Instead, the only sign that God will give to reveal the true identity of Jesus will be to resurrect his Son after being buried for only three complete days.
After Jesus further chides this request from a "wicked and adulterous generation" (verse 39), his mother Mary and his brothers desire to speak with him. Three of the four Gospel accounts record their attempt to get someone near him to get his attention and let him know what they want.
But while He (Jesus) was still speaking to the multitudes, behold, His mother and His brothers were standing outside, seeking to speak with Him (Matthew 12:46, HBFV throughout unless stated).
31. Then His brothers and His mother came and were standing outside; and they sent to Him, calling Him (Mark 3:31, see also Luke 8:19)
In a response the multitude could hear, Jesus rhetorically asks who is his mother and brothers (Matthew 12:48). He then answers his own question by stating he does not consider his most close relationships (as do most humans) to be limited to only those who have some physical tie to him. In contrast, he states he is willing to have a close spiritual relationship, like a family, with ANYONE (e.g. his disciples, verse 49) who does the will of (obeys) God the Father (verses 48 - 50).
The Lord's response, spoken to a large crowd of people eager to be with him, would make no sense if the initial request to talk with him was made by anyone who was not physically related to him.
Suprisingly, just one chapter after Matthew 12, the Bible reveals the names of FOUR of Christ's physical brothers and also states that he had more than one sister! This information is revealed to us in the comments made by those in Nazareth, who knew Jesus and his family for many years, in response to his profound and unique teachings at a local synagogue.
54. And after coming into His own country, He taught them in their synagogue, so that they were amazed and said, "From where did this Man receive this wisdom and these works of power? 55. Is this not the carpenter’s son? Is not His mother called Mary, and His brothers James and Joses and Simon and Judas? 56. And His sisters, are they not all with us? . . . " (Matthew 13:54 - 56).
We find additional evidence that Jesus did have brothers in the Book of Acts. His remaining eleven disciples he personally taught (Judas had committed suicide), after witnessing his ascension to the Father (Acts 1:4 - 9), went back to Jerusalem. While they waited in the city for the promised Holy Spirit (verses 4 - 5) they gathered to pray together with the following people.
14. All these were steadfastly continuing with one accord in prayer and supplications, together with the women, including Mary, the mother of Jesus, and with HIS BROTHERS (Acts 1:14).
In conclusion, the Bible reveals that Jesus did have brothers and sisters (they all had the same mother, Mary, but not the same father, as God was the Father of our Savior). Mary, by the time her firstborn son's ministry started in the fall of 26 A.D., had given birth to at least four other boys and at least two girls for a family composed of seven (possibly even more) children!