Farmland is a critical factor for the economic development of grain-producing areas. It is of great significance to explore the effects of urbanization on the farmland use efficiency and the relevant mechanism for guaranteeing the national grain security and promoting rural revitalization. Based on a panel data of 39 counties and cities in Jilin Province from 2004 to 2019, this paper calculated the farmland use efficiency of each county and city by the stochastic frontier production function (SFPF) and explored the influential mechanism of urbanization on the farmland use efficiency by the mediation effect model. Results illustrate that the urbanization has different effects on the grain output efficiency and the total value of farm output efficiency of farmland in grain producing areas. The average grain output efficiency of farmland in the sample counties and cities is 0.67 with an increasing trend year by year. However, the average total value of farm output efficiency is 0.26 with a decreasing trend year by year. On one hand, the urbanization promotes the grain output efficiency of farmland by enhancing the level of agricultural mechanization, land intensification, economic growth, and industrial structure upgrade; on the other hand, it also inhibits the total value of farm output efficiency, among which the mediating effect of the economic growth is most obvious. Therefore, to achieve the goal of both grain and total value of farm output efficiency growth in the future, this paper suggests to improve the stimulation effects of relevant factors on the total value of farm output efficiency of farmland by increasing the proportion of such high value-added grain crops as green grain crops and organic crops in the urbanization process of the grain-producing areas.