Abstract: Hunan Province is a major double-cropping rice production area in China. Clarifying the spatiotemporal variation and dominant factors of rice productivity is essential for scientifically delineating suitable rice-growing zones and ensuring national food security. Based on statistical yearbook data from 2013 to 2021, this study considered four categories of factors including climate, topography, soil, and socioeconomic conditions. A random forest model was used to systematically analyze the spatiotemporal evolution and driving mechanisms of county-level rice productivity in Hunan Province. The results show that: 1) During the study period, the average productivity of double-cropping early rice, single-season rice, and double-cropping late rice was (5.85 ± 0.54), (7.15 ± 0.81), and (6.32 ± 0.68) t/hm², respectively. Significant spatial autocorrelation was observed, with high-yield areas mainly distributed in the central and eastern plains of the province. 2) Among 122 counties, the average Sen’s slope of productivity change for early rice, single-season rice, and late rice was 0.024, 0.091, and −0.004 t/(hm²·a), respectively. The number of counties with a significant increasing trend was 21, 74, and 14, while those with a significant decreasing trend was 10, 1, and 16, respectively. 3) Soil and topography were the primary factors influencing spatial variation in rice productivity. The most important variables were soil bulk density for early rice (15.3%), slope for single-season rice (14.5%), and elevation for late rice (18.5%). (4) The response of rice productivity to different factors showed clear nonlinear patterns. Some variables had threshold effects, with productivity increasing up to a certain point, followed by a decline beyond that point. This study deepens the understanding of the spatiotemporal patterns and driving factors of regional rice productivity and provides scientific support for optimizing rice cultivation planning and management in Hunan Province.