Based on a micro survey data of sheep farmers in five provinces, this paper applied a stochastic frontier production function model to analyze the effects of the grazing ban policy on the technical efficiency of sheep farms. In addition, this paper also adopted a threshold model to explore the heterogeneity of the impacts of the grazing ban policy on the farmer’s farming technical efficiency under different farming scales. Results show that the implementation of the grazing ban policy had a significant positive effect on farmers’ technology efficiency, and the effect had a significant threshold due to the different size of breeding scales. It meant that the grazing ban policy did not noticeably affect the technical efficiency of small-scale sheep farms, while it had a significant effect on large-scale sheep farms. Main reasons for the difference might be the high cost of the transformation, the lack of feeding funds, and the difficult supervision of illegal grazing management behaviors for small-scale sheep farms. In addition, the proportion of farming income, household size, joining farmer cooperatives, farming methods and regional factors also had significant impacts on technical efficiency.