Abstract: In the context of the rural ecological revitalization strategy, an in-depth investigation of the current spatial difference and its influencing factors of agricultural non-point source pollution is of great significance to the promotion of the modernization of the rural environmental governance system and governance capabilities. Based on the provincial panel data from 2003 to 2017 and applying the unit survey method and the spatial measurement method, this paper explored the spatial difference and its influencing factors of agricultural non-point source pollution in China. Results show that the regional difference in agricultural non-point source pollution is the main source of the overall difference. The areas with serious agricultural non-point source pollution are mainly concentrated in the eastern region, and the areas with light agricultural non-point source pollution are mainly concentrated in the western and northeastern regions. China’s agricultural non-point source pollution has shown a decreasing trend from the eastern and central regions to the northwest and northeast regions. Spatial econometric analysis shows that agricultural non-point source pollution presents an obvious positive spatial correlation. High-pollution agglomeration areas are mainly distributed in the eastern coastal economically developed areas, and low-pollution agglomeration areas are mainly concentrated in the northeast and western economically underdeveloped areas. Agricultural production efficiency, clean energy technology, industrial structure optimization, and environmental governance investment are important factors in reducing agricultural non-point source pollution. Therefore, to improve the quality of rural ecological environment and to promote the modernization of agriculture and rural areas, this paper suggests strengthening investment in environmental governance, increasing investment in agricultural green technology, promoting the adjustment of agricultural industry structure, and establishing and improving joint prevention and control among regions.