The 19th National Congress of the CPC put forward the implementation of the rural revitalization strategy, which pointed the way for the reform and development of China’s agriculture and rural areas in the new era. How to formulate the institutional framework and policy system regarding the rural revitalization strategy is the most urgent issue that needs to be addressed in the near future. In the 1960s Japan also formulated a rural revitalization strategy to tackle the rural recession challenges. This paper systematically analyzed the origin and practices of Japan’s rural revitalization strategy and discussed its implications to China. Results show that Japan implemented its own rural revitalization strategy through the marketing of agricultural products, the development and utilization of geographical resources, the innovation of agricultural technologies, and the promotion of sixth industrialization of agriculture. Japan’s strategy has revitalized the rural economy, increased farmers’ income, and achieved coordinated and integrated development of urban and rural areas. However, there is still room for further improvement in increasing the supply of agricultural labors and the food self-sufficiency rate. Finally, this paper provides the following suggestions for China’s rural revitalization strategy: to build regional agriculture with its own characteristics, to cultivate new types of agricultural management entities, to establish urban green agriculture, to perfect the rural governance system, to rejuvenate the rural culture, and to build livable new countryside.