Abstract: Agricultural modernization and technological progress have substantially improved the efficiency of production. However, the growing dependence on the inputs of agricultural materials has resulted in a series of deleterious issues, such as soil and water pollution, carbon emissions. Previous studies have always considered carbon emissions as an unexpected output when evaluating agricultural green total factor productivity. These studies failed to estimate the potential reduction in carbon emission, as well as the contribution of all sources. This study sought to improve commonly used approaches and enable them to calculate the potential reduction of carbon emissions. To do so, it utilized the variable returns to scale data envelopment analysis (VRS-DEA) two-stage model and the DEA-Malmquist method to evaluate the agricultural input-output efficiency and obtain the abundant input of each material. In this study, six agricultural materials were selected as inputs and five major crop products as outputs with 105 counties/cities in Henan Province, China, as decision-making units from 2000–2020. The results showed the following: 1) carbon emissions induced by agricultural inputs began to decline after reaching their peak in 2016. The counties/cities with higher emissions were primarily distributed in the eastern and southern plains. Those with higher intensities of emissions per unit of sown area were primarily concentrated in the northern plains, where have better terrain conditions. 2) High-agricultural-efficiency areas were primarily concentrated in southern and northern Henan. In contrast, low-agricultural-efficiency counties/cities were primarily concentrated near urbanized areas, indicating that urbanization has a negative effect on agricultural efficiency. Approximately 60% of the counties/cities improved their agricultural efficiency between 2000 and 2020. Those with decreased agricultural efficiency were primarily located in the central and western regions. They were adjacent to areas with a high urbanization rate and primarily included mountainous and hilly areas. 3) The comprehensive potential reduction of carbon emission was approximately 11% of gross agricultural emissions. Counties/cities with the highest potential rate of reduction were primarily distributed in the areas surrounding the developed urban agglomerations. The key areas of agricultural reduction were the three counties/cities of Ruzhou, Xinye, and Huixian, with an accumulated potential reduction of more than one million tons. Ten counties/cities, such as Huaibin and Weihui, had more than half a million tons of accumulated potential reductions. 4) Agricultural inputs with a high redundancy ratio were agriculture plastic films, pesticides, chemical fertilizers, agricultural machinery, and agricultural labor. Chemical fertilizers with immense usage was a major concern; its input had the potential reduction of carbon emission as much as 83.5%. In summary, it is the basic solution for reduction of agricultural carbon emission to increase agricultural input-output efficiency and reduce redundant agricultural inputs. uts.