[Objective] The effective promotion of ecological compensation policies will play an important role in ecological and environmental protection and high-quality development of the Yellow River basin. The sorting and scientific evaluation of ecological compensation policies in the Yellow River basin were conducted to further optimize and improve ecological compensation policies. [Methods] Based on 170 ecological compensation policy texts promulgated by nine provinces (districts) in the Yellow River basin from 2001 to 2022, a social network graph and quantitative evaluation index system of ecological compensation policies were implemented using the data analysis software ROSTCM and the policy modeling consistency (PMC) index model. Nine ecological compensation policies served as research objects. Text mining and content analysis methods were used to evaluate each dimension of the selected policies through the PMC index measurement, the depression index calculation, and PMC surface drawing. [Results] The overall quality of the nine ecological compensation policies evaluated in the Yellow River basin proved to be higher. Six of the evaluation results were excellent, and three were good. All nine policies scored high in terms of policy nature, policy measures, policy evaluation, and policy guarantee, but scored lower in policy design, policy objectives, and policy content. [Conclusion] The design and content of ecological compensation policies in the Yellow River basin need to be improved, and the important role of policy objectives in ecological governance has been overlooked. Further optimization should be made in terms of formulating a policy plan that combines long and short periods, exploring diversified compensation methods, and increasing attention to both compensation standards and fund supervision.