With exposure to wind, rain, frost etc, the mortar holding bricks (or stone) together on an external wall will often start to crumble away and fall out. On older properties, the pointing will actually be a harder mortar than the bonding mortar, so once this is dislodged, the softer bonding mortar will become exposed and wear away quite quickly.
This sort of damage can take decades to occur but it is important to repair the pointing as necessary otherwise water will increasingly penetrate and damage the wall.
If the damage is limited to a small area, just that area can be repaired, however this repointed area will tend to then stand out. Generally a whole wall face will have weather in a similar way, so the whole wall will need repointing at the same time.
Repointing is time consuming to achieve a good overall appearance but is worth the effort.