The green development of agriculture is very important for the low carbon and high-quality development of agriculture. Improving agricultural green total factor productivity (AGTFP) is the key way to realize the green development of agriculture. Based on panel data from 27 provinces in China from 2011 to 2022, the super-SBM and the entropy weight method were used to estimate AGTFP and the rural digitization level (RD). The impact of rural digitization on AGTFP and its mechanisms was demonstrated by using a fixed effect model, an intermediary effect, and a threshold effect model. Results showed that: (1) AGTFP and RD in China have significantly improved during the research period; (2) RD had a significant positive impact on AGTFP, that is, rural digitalization development can effectively promote agricultural green development. From the perspective of heterogeneity analysis, rural digitalization in eastern China has the strongest impact on agricultural green total factor productivity; (3) Rural digitalization indirectly promotes AGTFP through the effects of scale operation, structure optimization, and technological progress, among which the intermediary effect of technological progress is the most prominent; (4) Rural digitalization development has a threshold effect on the promotion of AGTFP; that is, the greater the RD, the greater the AGTFP. Therefore, the development of rural digitalization should be accelerated, regional cooperation mechanisms should be established, and agricultural technology progress should be promoted to improve agricultural green total factor productivity and promote the high-quality development of agriculture in China.