The legalization of agricultural insurance is an important guarantee for the high-quality development of agricultural insurance. The legislative model of straddling commercial and policy-based agricultural insurance in China has become the “cause of disease” that hinders the development of policy-based agricultural insurance. Based on Japan’s agricultural insurance law, this paper analyzes the development process, the legal framework, and the basic characteristics of Japan’s agricultural insurance policies, explores Japan’s experience and practice of agricultural insurance, and discusses its implications for China’s policy-based agricultural insurance legislation. Results show that Japan enacted a separate legislation at the beginning of the establishment of agricultural insurance, and the provisions of the law not only include government’s responsibilities, insurance liability, the level of protection, penalties, but also incorporate the externality of the characteristics of agricultural insurance on financial, reinsurance, and other ancillary support measures to carry out detailed provisions. It has the characteristics of comprehensive legal content, mandatory insurance items, and the coordination with other laws. The “Regulations on Agricultural Insurance” promulgated by China has filled the blank of agricultural insurance regulations, preliminarily established the structure of the agricultural insurance system, and clarified the legal responsibility of agricultural insurance operation. However, compared with Japan, there are still legislative goals that do not reflect the strategic significance, the misalignment with other related laws, the lack of regulations on the power and responsibility relationships and behavioral norms of the governments at all levels, and inadequate market access and exit system. Therefore, China can learn from the legislative experience of Japan and further improve the agricultural insurance system by specifying legal norms such as institutional model, financial support mode, premium rate determination requirements, and catastrophe risk dispersion system in the policy-based agricultural insurance legislation.