An Innovative Framework for Building Agency-Free Credit Rating Systems

We develop an intelligent credit rating system that can provide debtors' rating information without involving credit rating agencies. Several models are used for credit scoring in our work, including the Duffie's model, logistic regression, and random forest. We compare the performance of these models and build an in-depth understanding of the evaluation of credit rating. Furthermore, we propose a new framework to evaluate the performance of credit ratings from multiple perspectives. The framework contains two components, defaulter recognition and rating quality. We use generic indices, area under curve (AUC) of receiver operating characteristics (ROC) and log loss, to evaluate the defaulter recognition ability of credit rating models. However, rating quality is more complicated than defaulter recognition. Inspired by rating agencies, we propose indices that reflect stability and migration of rating. We also adopt minimum default distance and rating path for evaluation of rating quality. Experimental results indicate that random forest (RF) has the best performance among the generic indices, but its stability is demonstrated to be 63% lower than the other two models. Similar results were found in the ratings path to default and minimum default distance. In this study, we specify a general evaluation framework of credit rating system and reveal the possibility of agency-free credit rating.