The year 1920 saw the replacement of the shomrim by Haganah -- the official underground army of the Jews in Palestine. The Haganah was initially established under the direct control of the Achdut Avoda party. With the establishment of the Histadrut, the Haganah came under the direct control of the Histadrut. The Haganah's first actions took place during the 1920 Arab riots in Jerusalem. During subsequent riots in 1921, the Haganah acquitted itself well in Jerusalem, but but did not handle the riots in Jaffa as successfully. During the period between 1921 and 1929, when relative quiet reigned in Palestine, the Haganah stagnated. The riots in 1929 caught the Haganah by surprise. Although in most cases the Haganah was able to defend Jewish settlements against attack, it could not prevent the slaughters that took place in Hebron and Safed. After the Arab riots, the Haganah was reorganized and the importation and local production of armaments was substantially increased. In 1931, non-Histadrut elements of the Haganah seceded and formed their own group. The 1936 riots resulted in a reorganization that put the Haganah under the direct control of the World Zionist Organization, and the splinter faction rejoined.

The 1936 riots accelerated the growth of the Haganah. During the initial stage of the riots, the Haganah was able to successfully defend each and every Jewish settlement so that none needed to be abandoned. By the end of 1936, the Haganah had formed highly capable mobile units that were able to take the offensive against Arab attackers. In 1937, the British decided that in order to maintain order during the Arab revolt, they would have to cooperate with the Jewish community, and thus they organized a legal Jewish police force, which was essentially under the control of the Haganah. This force grew to include over 1,000 full time members and 22,000 militia members.

In 1938, the British Army authorized Captain Orde Wingate to organize special 'night squads.' Their night attacks were a major element in the successful suppression of the Arab revolt. With the British issuance of the White Paper, the Haganah began to concentrate on illegal immigration. With the outbreak of World War II, the Haganah once again began large-scale cooperation with the British, encouraging many of its members to join the British armed forces. At war's end, the Haganah concentrated its efforts once more on smuggling illegal immigrants into Palestine, as well as preparing the way for independence. When Israel's independence was declared, the Haganah became the Israel Defense Forces.