1. Do longer reports, sent out frequently, necessarily translate to more useful information?

The saying "empty vessels make the most noise" holds true in the realm of language analysis, where each word functions as a token representing a data point. Effective text analysis, akin to signal processing, requires isolating the meaningful signal (i.e., words indicating credit trends) from the irrelevant noise (e.g., filler words or over-sensationalized opinions that fade post-hype). Noise may occur more frequently but provide no insight into credit trends. Hype attracts viewership may still indicate trends but tend to overreact. Ideally, one seeks to have less frequent, but still timely, words providing precise indicators of credit trends.

S&P Global Ratings research is a potential candidate for uncovering such trends as it culminates in issuer credit ratings, which have a proven ability to distinguish between defaulters and non-defaulters over a one-year period with very high discriminatory power.

2. A look into credit rating reports

Are there ways to tease out informative words within research? We examine Full Analysis, Summary Analysis, and Research Update reports to illustrate the varying levels of informativeness in research text. It is noteworthy that these reports are infrequent, making them informationally dense with minimal filler (noise) content. Rigorous editorial review is always applied to prevent over-sensationalism in language, hence the positive and negative text in the reports represent measured responses by the analysts to the entity’s credit conditions.

In addition, these reports follow a structured format, with sections such as:

1. Company Description
2. Highlights
3. Overview/Ratings Action Overview
4. Rationale
5. Outlook
6. Upside/Downside/Base Case Scenarios
7. Business Risk
8. Financial Risk
9. Liquidity
10. Environmental, Social, and Governance
11. Covenant Analysis and Other Credit Ratings Drivers
12. Key Statistics/Metrics

3. The importance of word placement within sections of reports

The same words can carry very different meanings and levels of importance depending on where they are placed. As a simple illustration, the word “Good” carries much more meaning in the Credit Highlights section than if it was part of a company’s name (see Figure 1).  Humans will naturally take this into account when reading reports, but machines need to be guided on where to look.

Figure 1: The Simply Good Foods Co. Full Analysis report published in Oct. 2023

^{Source: RatingsDirect® on S&P Capital IQ Pro, from S&P Global Market Intelligence; Oct.2023. For illustrative purposes only.}

4. Informative content of sections in Full Analysis and Research Updates reports

We conducted two empirical experiments to assess:

1. The relative informativeness of different sections of an analyst's ratings view.

2. The predictive value of sections for ratings changes over a 12-month period.

4.1 The relative informativeness of sections

In the first experiment, we wanted to sanity test if the text sentiment in a section was consistent with the analyst’s view, i.e., the credit rating and if it:

• Varies by credit rating.
• Varies by credit rating after controlling for (i.e., accounting for) CreditWatch and Outlook.

This first experiment tests if AFINN sentiment score^[1] (i.e., an open-source lexicon that measures sentiment with a numeric score between -5 and 5) is sensible and if there is potential information about the analyst’s view on an entity’s credit risk that is not already captured in CreditWatch and Outlook of the credit rating.

Figure 2 represents sentiment scores, highlighting a correlation between higher or positive scores and positive sentiment in investment-grade ratings. Conversely, speculative-grade ratings exhibit lower or more negative scores, indicating a less positive sentiment.^[2]

Figure 2: Mean AFINN sentiment scores for each credit rating category

^{Source: Credit ratings from RatingsXpress® Credit Ratings.  Digitized research from RatingsXpress®: Research, both from S&P Global Market Intelligence.  AFINN Sentiment Scores are sourced from http://www2.imm.dtu.dk/pubdb/pubs/6010-full.html As of Dec. 2023. For illustrative purposes only.}

4.2 The predictive value of sections in S&P Global Ratings entity level research reports

The second experiment gives us an idea if research sentiment has incremental predictive power (i.e., the ability to predict future ratings changes over a one-year period, even in cases where there is a CreditWatch and Outlook). ^[3]

More ratings upgrades occurred when AFINN Sentiment Scores were positive (henceforth defined as at least one standard deviation above mean for its original rating level) versus negative (henceforth defined as at least one standard deviation below mean for its original rating level) for positive ratings Outlooks, and more ratings downgrades took place when AFINN Sentiment Scores were negative versus positive for negative Outlook.  Results were mixed when ratings outlook was Stable.  This indicates that research sentiment could strength the signal indicated by positive/negative ratings outlooks.

Similarly, there were more ratings upgrades when AFINN Sentiment Scores were positive when CreditWatch was positive, and more ratings downgrades when AFINN Sentiment Scores were negative when CreditWatch was negative.

The potential for AFINN scores to enhance the predictability of CreditWatch and outlook

Table 1: Comparison between upgrades and downgrades within Outlook, CreditWatch and AFINN Sentiment Score buckets

^{Source: Credit ratings from RatingsXpress®: Credit Ratings.  Digitized research from RatingsXpress®: Research, both from S&P Global Market Intelligence.  AFINN Sentiment Scores are sourced from http://www2.imm.dtu.dk/pubdb/pubs/6010-full.html As of Dec. 2023. For illustrative purposes only.}

In the regression analysis in the Appendix B and C we note that the sections where we expected to have information content, such as Ratings Rationale, Overview/Ratings Overview, Ratings Outlook, and Highlights had an extremely high predictive power on credit ratings changes (i.e., better than 1% statistical significance) than generic sections such as Key Statistics and Company Description, which were included to test the model as a controlled experiment.

The downside scenario section has more information content than the upside scenario section. While there is some variation in the tone of both sections, analysts tend to be more expressive in their concerns on credit ratings downgrades, while maintaining a closer to neutral tone on credit ratings upgrades.

5. Case studies

Case Study 1: Informativeness of text from Research Update reports

3M Downgraded To 'A+' On Continued End-Market Pressure, High Leverage; Outlook Negative in February 2020. Despite holding a higher credit rating with a stable outlook in January 2020, the sentiment score from research had been consistently negative since 2019, as illustrated in Figure 3.

Figure 3: 3M Co.’s (NYSE:MMM) Research Updates reports between the years 2019 and 2020

^{Source: Research Update report - 3M Co. Ratings Placed On CreditWatch Negative On End-Market Weakness And High Debt Leverage (Jan. 2020), RatingsDirect® on S&P Capital IQ Pro, from S&P Global Market Intelligence. For illustrative purposes only.}

Case Study 2: Informativeness of text from Full Analysis reports

Dillard's Inc. Ratings Lowered To 'BB-' On Uncertainty Surrounding Coronavirus Impact; Outlook Negative in April 2020. Prior to this, the company held a superior credit rating with a stable outlook. However, the Full Analysis report from December 2019 indicated a relatively low sentiment score of -0.3, which was one standard deviation below the average for ratings reports published for similarly rated entities. Excerpts from sections such as Credit Highlights and Business Risk are provided in Figure 4.

Figure 4: Dillard’s, Inc. (NYSE:DDS) Full Analysis report from the year 2019

^{Source: Dillard’s Inc. Full Analysis Report (Dec. 2019), RatingsDirect® on S&P Capital IQ Pro, from S&P Global Market Intelligence. For illustrative purposes only.}

Share prices of Dillard's, Inc. (NYSE:DDS) started declining in late March 2020, just before the official rating downgrade. Interestingly, ratings analysts had indicated a potential downgrade a year earlier through the sentiment scores from the Full Analysis reports published between years 2018 and 2019.

Figure 5: Share prices of Dillard's, Inc. (NYSE:DDS)

^{Source: RatingsDirect® on S&P Capital IQ Pro, from S&P Global Market Intelligence. As of Jan. 2024. For illustrative purposes only.}

6. Conclusions and Forthcoming Thought Leadership

In conclusion, our analysis shows the potential for specific sections in Full Analysis and Research Update reports to be used as a component for early warning systems in credit ratings changes. Within these reports, even when the Outlook and CreditWatch levels remain constant, analysts convey more nuanced and forward-looking perspectives on potential rating changes. This heightened expressiveness is notably observed in the Executive Summary sections that encompass highlights, plus an overview, rationale, and outlook. Additionally, areas where analysts flag downside risks for an entity, such as comments on the downside scenario, business risk, financial risk, and liquidity, serve as critical indicators for vigilant monitoring and proactive decision-making. This nuanced approach offers end users a more comprehensive understanding of potential credit rating shifts, contributing to the development of effective early warning models.

In the next installments of this blog we investigate:

1. Can we find alpha from text in ratings research report?
2. Industry-level early warning signals from commentaries reports
3. A holistic view of credit early warning, incorporating multiple inputs

APPENDIX A: Research Methodology and Summary Statistics

We utilize the R programming language for AFINN Sentiment Score calculations, data cleaning, and statistical analysis on research reports. The AFINN lexicon is an open-source resource that assigns sentiment scores to words on a scale from +5 to -5 to gauge the sentiment or mood conveyed in the text of our research reports. By aggregating these scores, we derive an overarching sentiment score for the reports under study. We anticipate consistent results when employing alternative sentiment proxies. In the data cleaning process, we replace NULL or NA values in the Outlook and CreditWatch columns with zeros. This categorization indicates neither a positive nor negative Outlook or CreditWatch status.  We perform both ANOVA analysis and linear regression with Newey West estimators to adjust for heteroskedasticity and autocorrelation in error terms.

Table 2: Summary statistics by report type

^{Source: Credit ratings from RatingsXpress®: Credit Ratings.  Digitized research from RatingsXpress®: Research, both from S&P Global Market Intelligence.  AFINN Sentiment Scores are sourced from  http://www2.imm.dtu.dk/pubdb/pubs/6010-full.html As of Dec. 2023. For illustrative purposes only.}

APPENDIX B: ANOVA Results

Cells denoted with an X below indicate statistical significance at 5% level, with most values having p-value < 0.001.

^{Source: Credit ratings from RatingsXpress®: Credit Ratings.  Digitized research from RatingsXpress®: Research, both from S&P Global Market Intelligence.  AFINN Sentiment Scores are sourced from  http://www2.imm.dtu.dk/pubdb/pubs/6010-full.html. As of Dec. 2023. For illustrative purposes only.}

APPENDIX C:  Linear Regression Results

Cells denoted with an X below indicate statistical significance at 5% level, with most values having p-value < 0.001.

^{Source: Credit ratings from RatingsXpress®: Credit Ratings.  Digitized research from RatingsXpress®: Research, both from S&P Global Market Intelligence.  AFINN Sentiment Scores are sourced from  http://www2.imm.dtu.dk/pubdb/pubs/6010-full.html As of Dec. 2023. For illustrative purposes only.}