It is an inevitable choice to comprehensively improve China’s grain security under the new background to break the resource and environmental constraints of food production and improve the grain eco-efficiency. According to the logical relationship of “urbanization - factor distribution - green grain production”, based on a panel data from 31 provinces in China from 2006 to 2019 and from the theoretical perspective of heterogeneous urbanization, this paper seeks to build a theoretical framework to empirically analyze the impacts of urbanization on green grain production efficiency. Results show: 1) the overall level of green production efficiency of grain in China is low, with pronounced time fluctuations and regional differences; 2) population urbanization hinders the improvement of the green production efficiency of grain, while in-situ urbanization promotes the improvement of the green production efficiency of grain with lower influences; and 3) in the primary marketing area and the balanced production and marketing area, population urbanization has a significant negative impact on the green production efficiency of grain in the secondary grain-producing areas, while it has a significant positive impact in the primary grain-producing areas. Population urbanization’s negative effect on grain’s green production efficiency is stronger in the primary marketing area. However, the positive effect of in-situ urbanization on the green production efficiency of grain is stronger in the balanced production and marketing area. In summary, this study suggests: paying attention to the different impacts of different urbanization development paths on the green production efficiency of grain and considering different urbanization promotion paths based on the reality of different grain production functional areas.