All About Ratings For Requestors

DV Ratings Module is used by many requestors to rate or score the responses they receive from the responders. Ratings can be used either to rate/score the responses from the responder directly or in a scorecard. If you are a Superadmin or Business Admin user in Diligencevault, you can configure ratings for any template and ultimately rate any project sent using the same template.

In this article, we are going to see everything about the ratings that includes : 

• How to create ratings for a template
• What are the different types of ratings available
• How to use automation to add ratings
• How to add ratings in a projects
• How to audit ratings

Generating Rating for a Template

In order to rate a project, it is important to generate a rating in a template. However, before we go to a template, configure the ratings in firm settings. 

Define your Rating Scale

The rating scale is a minimum and a maximum value that you will assign to a question or a subcategory.

• Parts of a Rating : Rating has following 3 different parts 
  • Rating Name : Rating name is used to show the name of the rating while rating the responses. Ex. 1,2,3, etc or Low Risk, High Risk, Medium Risk or Red, Yellow, Green to define risks.
  • Rating Color : Rating color is the actual color of the rating shown while rating the responses.
  • Rating Score : Rating Score is the number given to each rating value for subcategory, category and project score calculation.
• Rating Mode :
  • Absolute Scale : If you want to assign ratings to the responses in absolute numbers like 1 to 5, your rating scale will be 1,2,3,4 and 5.  You can also use colors as an absolute rating scale. Ex. Red, Yellow, Green.
  • Scoreband Scale : In score band scale, the rating color can be defined for a range. Ex. If your rating scale is 0 to 100, you can set different colors for different rating ranges. Here, if the response has a score between 0 to 50 you can use a single color say Red. Similarly, 51 to 80 can be Amber and 81 to 100 can be of Green color to indicate different types of risks. 

Configure the Rating Scale

Once the rating scale is defined by you internally, you can create and configure the same in DV by going to Firm Settings > Rating/Score Map > Rating/Score Scale.

• Click on the plus sign on the top right hand side corner, give it a name and select the rating mode.
• Once the rating scale is created, you can make changes to the rating names and rating colors as per your requirement. 
• If you have selected absolute scale, by default, 5 ratings will be shown. 
  • N/A indicates items excluded from the final score
  • N/R indicates items that have yet to be rated. If they remain unrated, a value of 0 will be included in the final rating calculations.
• You can use plus or minus signs to add or remove the above ratings and finally click on the save button to save the changes.

Generating a Rating Definition

Now that you have configured the rating scale, you can go to the template where the rating is to be configured from Diligence > Templates.

• Open the template and make sure it is activated (If not activated click on the activate button).
• Now click on the "More" button on the top right hand side corner and go to "Generate Rating/Score Map". This will open a pop-up window for you to select the rating scale you just created.
• Next, you will see 2 options "Sub-category" and "Questions". Select "Sub-category" if you want to rate only the subcategories and do not want to rate each and every question/response and "Questions" if you want to rate every question from the template.
  
• Once this is done, you will see a preview that says "Assign a rating". You can hover over the small boxes to see the color and name of the rating in this rating scale.
• Finally you can click on the "Generate Rating/Score Map" to create a rating definition.

You will be automatically redirected to Firm settings > Rating/Score Map > Rating Score Definition where the name of the rating definition will be the name of the template. Here you can assign weightages to the category, subcategory or a question.

Please note that the overall weightage of the entire questionnaire should always be equal to 100 or you will see a red (i) icon that indicates an error in weightages.

If you scroll down you will find a field that says scoring engine. There are 2 types of scoring engines : 

• • Weighted Average : This will calculate the project and category score based on the weights assigned to each question, subcategory and category. This is the default way of scoring in DiligenceVault. Use this when you do not want to rate/score based on any single rule but instead want to make one based on the average level of risk found.
    Weighted average calculation is done by multiplying individual question values with the corresponding weightage value, and dividing the sum of those values by sum of weightages in that subcategory. Ex.
    Q1 - score 5, weightage 10
    Q2 - score 3, weightage 10
    Q3 - score 2, weightage 10
    Q4 - score 3, weightage 10
    Q5 - score 1, weightage 10
    Weighted average = ((5 * 10) + (3 * 10) + (2 * 10) + (3 * 10) + (1 * 10)) / (10 + 10 + 10 + 10 + 10)
    = 2.8
  • Simple Sum : Use this when you want to do a simple sum and not take into account any weightage/average. In the above example, the calculation of the score will be a sum of all the scores Ex. 5+3+2+3+1=14

Rating Custom Fields : Custom fields to the ratings can be added here and can be used to add an explanation or comments to the ratings on the project.

Using Automation to Rate Responses

Diligencevault has the ability to automatically rate the responses where responses are defined like yes/no, dropdowns, etc. If your questionnaire has many such questions, you can use these rules to automatically rate the responses in the questionnaire as soon as the questionnaire is submitted to you. This will help you save a lot of time and effort. 

There are 2 types of rating rules : 

Set standard rules for flags and scores [Video Explanation]

Standard rules help you to create rules based on basic conditions like if the response is equal or not equal to a certain value, you can define a rating.

Here are some examples of standard rules : 

• Response : Choose this condition to define a rating based on the response to the question. Ex. If the response to the question "Do you have an ESG Policy" is "No", you can add a rating "High Risk" and also flag the response.
• Previous Response : Choose this condition to compare the previous response submitted by the responder to the same question. Ex. If you want to know if the headcount of the firm has increased or decreased significantly, you can apply the automation to rate the response based on the percentage increase or decrease in the headcount.
• Response Marked as "N/A" : You can choose to automatically exclude the question from the overall rating if the responder has marked the question as Not Applicable.
• No Response : Incase when there is no response to a specific question, you can set automation to automatically exclude such questions from the ratings.

To configure the rules, please follow the below instructions : 

• Navigate to "Diligence" > "Templates"
• Select "More Actions" > "Set flags and scores"
• Click on the "+" sign to add a new rule
• Standard, will be selected by default. Choose the question where the rule is to be applied under "Apply the new rule to" and set the conditions as explained above.
• Then, select the trigger actions as per your requirement.

You can also apply multiple conditions to one rule.

• Select "Add New Condition"
• Choose between "any" or "all" for and/or logic
• Set up your conditions. In this example, if the total headcount is greater than or equal to 100, OR if the total headcount is less than or equal to 10, the response will be flagged and given a low score.
• You can delete an additional condition by selecting the trash can icon to the right of the condition.

Advanced Rules for Flags and Scores [Video Explanation]

Our advanced ratings enhancement was created to allow your team to automatically apply your secret sauce more efficiently while rating and scoring on DV. It provides your team with the flexibility to set different criteria for your rules- percentage based, knock-out criteria, or total scores based on various combinations. Take a look at how to apply the most frequently used rules below.

• Navigate to "Diligence" > "Templates"
• Select "More Actions" > "Set flags and scores"
• Click on the "+" sign and select "advanced"
• Decide what area of the template you would like to apply the new rule to. You can now create rules that apply to one specific sub-category, category or the entire project.

Advanced Flagging:

• "Response Contains" logic: Enter specific text and numeric values that the response contains to be flagged. This is useful when your team wants to quickly flag keywords that are used in responses such as “felony” or “liquidated” or “high-risk”. Instead of reading the entire response, these responses will automatically be flagged.
•  "Response difference" logic:  Will show the absolute (numeric) or relative (percent) changes in responses. This new functionality allows your team to compare the response to this request, with the previous response on the platform. This can be applied for risk and compliance use cases that involve flagging large changes in returns. Automate your risk framework and identify significant changes that need attention.
• Parent and nested logic: Your team can now flag the response to the parent question or nested question, based on the response of the other. In the example below, the rule is affecting the parent question "Do you have an ESG Policy?" but it is based on the nested question, "please explain why you do not have an ESG policy?". If the user's response to the nested question contains the words "not applicable", the parent question will be flagged and will receive a low rating. 

Advanced Score Rating Rules: 

• Composite Conditions: Trigger actions based on the count or percentage of questions/sub-categories/categories that match a specified condition. In the example below, the rule is set for the category "Impact Investing & Diversity", and is based on two composite conditions. 75% of the yes/no questions in the sub-category ESG Considerations must have a response equal to yes, AND, 75% of yes/no questions in the sub-category "Firm Diversity Profile" must also have a response equal to yes. 
• Knockout Criteria: Assign a score/rating to the entire project, category, or sub-category based on defined rules. This can be used if a certain response should be given more weight than the other responses. In the example below, if the question, "do you have a business continuity plan" receives a response equal to no, the entire project will receive a low score. 
• Total score based on combination of ratings - Based on the aggregate of ratings given to each sub-category, your team can assign a total score to a specific category
  • Ex. If one category is HIGH and one is MEDIUM, total score should be MEDIUM. 
  • Ex. If one category is HIGH and two are MEDIUM, total score should be HIGH, etc.
  • In the example below, if greater than or equal to 2 sub-categories in the category "basic information" receive a score or rating that is greater than or equal to 2, the entire category will receive a HIGH rating.

At this point the configuration for the rating is completed. You will be able to rate the projects sent after the rating has been configured.

Rating the Projects in DV

• Rating the questions : To rate the questions, you can go to the project in Diligence > Projects and in the questionnaire tab you will find  icon against the question or a subcategory to add a rating. You can also add a rating in the ratings tab of the questionnaire.
  
• Viewing the ratings : Once you have rated the question or a subcategory, you can see it immediately in the questionnaire tab or the ratings tab. Ratings tab gives you a more holistic rating view of the project along with the ratings of a category and a project. You can also view the rating in the heatmap view in the ratings tab.
  
• Not Rated questions (N/R) : The questions that are not rated will be counted in the overall category and project score. These will not be highlighted by any color.
  
• Exclude Questions from ratings (N/A) : In some cases when the question is not relevant to a certain responder and you want to exclude the question from affecting your overall score, you can exclude the rating in the questionnaire tab by clicking on "More" > "Exclude from Rating". 
  
• Rating History : You can see the last person who rated the questionnaire for auditing purpose along with the date of the rating.
  

Rating Exports

• Export in Projects : While downloading a project in word or excel, you can enable "Include rating/scores" to ensure that the rating is shown against the question/subcategory in the downloaded project.
  
• Rating Heatmap : To compare the ratings across different projects in a heatmap view please navigate to "Analyze" > "Rating/Score". Select the rating definition and other details to see the comparison of ratings.
  
• Presentation Reports : Ratings can also be added to your customized word presentation reports. Like question tags, please go to the "Rating Tags" tab in the edit window of the template in presentation design and copy the tag. Then you can paste this tag in your word file.

Rating on Entities

You can add a rating related custom field on the entities to see the latest rating of the entity for a specific questionnaire. 

To configure this, please follow the below instructions : 

• Go to "Firm Settings" > "Tags and Custom fields".
• Go to the specific entity to create a custom field e.g Firm.
• Click on the plus sign on the right to create a new custom field.
• Give it a name e.g Latest Rating and select the type as Rating.
• Further, choose the display format and under template, select the template name to show the rating for.
• If you would like to show the rating only for a specific category only, then select the category or leave it blank to see the rating for the project.
• Add a description for your internal reference and save.

Once the configuration is done, the latest project rating will be shown for the selected template for all the entities in quick view. These rating custom fields can also be used to filter out entities while sending out new requests as well.

These custom fields can also be shown in the entity page (Diligence > Firms/Products)

You can also add multiple such custom fields for all your templates or multiple custom fields for ratings if you want to see the rating for each category.