Understanding the preference and the willingness to pay (WTP) of urban consumers for low-carbon agricultural products and the sources of preference heterogeneity is of great significance for reducing carbon emissions and promoting sustainable agricultural development. Based on a data of 786 urban milk consumers in Shanghai, Nanjing and Hangzhou, this paper adopted the choice experiment method to explore the preference and WTP of two types of consumer groups that usually buy long-shelf-life milk or fresh milk for carbon footprint labeled milk by the mixed Logit model. Results indicate that: when consumers understand the basic meaning of carbon footprint label both types of consumer groups have a significant positive preference for milk with low-carbon and medium-carbon footprint label with a complementary relationship between low-carbon footprint label and whole milk label. In addition, the average WTPs for milk with low-carbon footprint label from consumer group that usually buys long-shelf-life milk and from consumer group that usually buys fresh milk were 4.13 yuan/box and 5.63 yuan/box respectively. Furtherly, the preferences of two consumer groups for milk with carbon footprint label are heterogeneous and the sources of preference heterogeneity were different: the long-shelf-life milk consumers with younger age, higher monthly income, more frequent purchase, and more knowledge about carbon footprint label prefer to buy milk with a low-carbon footprint label; while fresh milk consumers with being a male, younger age, higher education, and being more concerned about low-carbon information prefer to buy milk with low-carbon footprint label. Finally, consumers’ perception of the role of low-carbon agricultural products positively influences their low-carbon preference in both consumer groups. Therefore, this paper suggests that the government and relevant institutions should improve the low-carbon product certification system, strengthen the promotion of carbon footprint label and popularize knowledge about low-carbon agricultural products to consumers. From the business perspective, they should focus on low-carbon market segmentation.