To clarify the response characteristics of arbuscular mycorrhizal fungal (AMF) community to lithology and afforestation is an important premise for rationally using AMF to improve the soil quality and afforestation efficiency of artificial forest in the Karst region. Here, based on the soil samples from planted forests and farmlands were collected under both limestone and clasolite conditions, and the AMF community were analyzed by high-throughput sequencing to explore its response to lithology and afforestation. The results show that Glomus is identified as the dominant genus in the study area, and its average relative abundance ranges from 58.50% to 89.37%. The response of the AMF community to afforestation was different under these two lithological conditions. In the limestone area, afforestation significantly increased of the Shannon diversity index of AMF. In the clasolite condition, afforestation significantly increased the relative abundance of Glomus, but decreased the relative abundance of Claroideoglomus and Paraglomus. Further analysis revealed that soil pH and available phosphorus were the key environmental factors affecting the community of AMF under both lithology backgrounds. Lithology indirectly affected the AMF community through soil pH (p < 0.05), while afforestation has both direct and indirect effects on AMF community by changing soil nutrients, especially decreasing the available phosphorus (p < 0.05). Therefore, the effect of artificial afforestation on soil AMF community is restricted by geological background (lithology), and the lithology should be fully considered when using AMF to improve soil quality of artificial forest in Karst areas in the future.