Under the background of current resource shortage, environmental pollution and ecosystem degradation, it is of great significance to explore the impacts of agricultural mechanization on agricultural ecological efficiency for comprehensively promoting rural revitalization and strengthening ecological civilization construction. Based on a panel data of 30 provinces (autonomous regions and cities) in China from 2000 to 2020, this paper analyzed the impacts of agricultural mechanization on agricultural ecological efficiency and its mechanism by the fixed effect and the threshold effect models. Results show that agricultural mechanization had significantly improved agricultural ecological efficiency. The positive effect of agricultural mechanization on agricultural ecological efficiency had been reinforced by the grain-orientation of crop structure and the rural labor transfer. Further study indicates that the impacts of agricultural mechanization on agricultural ecological efficiency had a threshold effect of farmland scale management. With the expansion of farmland scale management, the marginal effect of agricultural mechanization on agricultural ecological efficiency showed the law of “first increasing and then decreasing”. Therefore, to improve agricultural ecological efficiency, this paper suggests: improving the agricultural machinery subsidy policy, optimizing the crop planting structure, guiding the transfer of rural surplus labor force, promoting the moderate scale operation of farmland, and accelerating the construction of agricultural machinery supporting system.