Widespread Cenozoic volcanism in the Arabian Peninsula has been attributed to mantle plume activity and/or lithospheric thinning due to rift‐related extension. However, there is discrepancy between geochemical and geophysical studies about which mechanism is dominant over the other for post‐12 Ma volcanism. Plume signals in some volcanic fields in the Arabian shield are not evident in isotope analyses, but low‐velocity anomalies connected to Afar are found beneath the Arabian shield in tomographic studies and interpreted as asthenospheric flow from the Afar plume. To resolve this contradiction, we investigate the upper mantle beneath the Arabian Peninsula and northeastern Africa by inverting relative S and SKS wave arrival times recorded at dense seismic networks to derive a high‐resolution S wave velocity model. Our results clearly show a narrow, elongated low‐velocity anomaly along the Makkah‐Madinah‐Nafud (MMN) volcanic line beneath the Arabian shield at 100–300 km depth which extends northward to Harrat Ithnayn and Harrat Lunayyir, but most likely not further north. The limited extent of the low‐velocity anomaly and variations in lithospheric thickness of the Arabian shield may explain why geochemical studies did not find plume signals in some harrats. Therefore, the timing and plume signals of volcanism in western Arabia may not be age progressive from Afar. We also find a possible connection between the low‐velocity anomalies beneath Harrat Lunayyir and the MMN line, suggesting that the 2007–2009 seismic swarm may be associated with northward asthenospheric flow of plume material from Afar.