Partner Performance Index and Risk Management

Download this Page

What is Risk Management in CBPF?Top 

Each CBPF is required to assess the capacity and monitor performance of every potential eligible partner and Implementing Partner. 
A first risk analysis is carried out by analyzing the partner’s capacity through the Capacity Assessment. It determines, along with the Due Diligence, the Partner’s eligibility to CBPF funding. 

To learn more about the Capacity Assessment process, please refer to this article of the Help Portal.

Based on the Capacity Assessment’s results, eligible NGOs are categorized according to a specific risk rating which determines the minimum standard of operational modalities applicable to the partner. The principle is that the higher the risk the more stringent assurance mechanisms will apply. (i.e. Low risk partners are subject to fewer controls.)

Over time, as partners receive funding and implement projects, the risk rating is increasingly determined by partner performance, which is assessed by the HFU through the Partner Performance Index (PPI).

This Performance system encourages improvement in capacity as partners can migrate to lower risk levels through good performance and by addressing capacity weaknesses. These performance trends are captured by the PI.

The PI tool is hence a key part of the accountability framework and allows OCHA to have an up-to-date rating of partner performance. The rating of the performance of partners in the implementation of projects is used (alongside the original capacity assessment) to determine (and adjust as necessary) partner risk levels.


WHAT ARE THE RISK MANAGEMENT TOOLS AVAILABLE IN THE GMS?

Several modules in the GMS are dedicated to helping HFU and all parties involved in Partner’s Risk Management to determine, update and monitor Partner performance and risk level.

- The Partner Risk Dashboard : an overview of all partners registered in the Funds and their risk level   evolution overtime.

- The Partner Performance Index Management Page: a page that allows the HFU to access and manage the Partner’s project’s PI scorecard.

- Partner Performance Index Risk Management: The Partner Performance Index Risk Management is the page where HFU officers can approve the PI adjusted risk of the Partner. After a project is closed, the system will re-calculate the risk level based on the latest project’s PI scorecard results and assign the partner a new suggested risk level, that the HFU can approve in this page. 

 

How is the Performance Index calculated and how does it affect the adjusted Risk Level?Top 

PROJECT PERFORMANCE - ASSESSED THROUGH PI QUESTIONS 

The performance of an Implementing Partner on a project is assessed through a set of questions, pre-determined by the CBPF and common to all partners. These questions are displayed for the HFU as and when specific milestones are completed in the project lifecycle.

Following are the instances for which questions have been defined in the system:

  • Project proposal
  • Financial Reporting (Interim and Final)
  • Narrative Reporting (Progress and Final)
  • Monitoring and Financial Spot-Check
  • Project Revision
  • Audit
  • Extraordinary Financial Report

When the project workflow reaches a specific status defined for a particular instance in the GMS, the system displays the PI question.

This is achieved by showing the PI question on the workflow confirmation and comments dialog box, after the HFU approves an instance.

It is mandatory to answer the Performance Index question as well as to save related comments in the dedicated box to proceed further with the workflow.
(please see screenshot below for an example of Performance question – narrative reporting Performance).

Performance index is important, as the questions’ answers selected for the project are assigned a score and serve to calculate the partner’s performance. The Performance Index helps to determine to what extent the partner is compliant to work with.

The list of all PI questions appears at final narrative, final financial and audit or project closure status when approved by HFU, to make sure all answers are captured or updated.
(see screenshot below).

The Performance Index questions are displayed when reaching the pre-defined milestones and workflow statuses. However, HFU officers can also access at any time during the project lifecycle the Performance Index Scorecard, which includes the project’s questions and answers already saved.


QUESTIONS' WEIGHTAGE IN CALCULATING THE PI SCORE OF THE PROJECT

At Project Closure, when all questions have been answered and saved, the Performance Index score is calculated based on the following weightage:

PI Categories Question Related GMS step Category Weightage
Quality of project document and timeliness of response How do you assess the quality of the proposals and the timeliness of the response on comments made? HFU submit GA for EO signature 10%
Monitoring findings What is your overall assessment of the project implementation (quality and timeliness) against approved targets and timeframe? When HFU clicks on "completed" in monitoring module 25%
Narrative reporting performance How do you assess the quality and timeliness of the narrative report? when HFU approves a narrative report (progress and final) 15%
Revision request/s How do you assess the timeliness, frequency and justification of the project revision? When HFU clicks on "Project Revision Reviewed and Cleared by HFU" 10%
Financial performance How do you assess the quality and performance of the financial report? When HFU approves a financial report (interim, extraordinary and final) 20%
Audit findings How do you assess the audit findings? when Audit report is finalized by HFU 20%

For example:

Category: Monitoring findings

Question: What is your overall assessment of the project implementation (quality and timeliness) against approved targets and timeframe?

Answer: The project achieved limited targets and justified

With this answer, the project score is 5 out of a maximum of 9,

5 – score given by HFU
9 – maximum score which can be given

% of score is:  5/9 *100 = 55.56

As there is 25% weightage for this category out of total weightage of 100,

Final score will be = 55.56 * 25/100 i.e. Final score will be 13.89
 

GMS Focus: Specific Cases 

Certain validations applicable for Performance Index scoring are:

• For monitoring findings:
If there are multiple monitoring reports, an average of the score is calculated for the dashboard

• For Narrative reporting performance:
If there are multiple reports, an average of the score is calculated for the dashboard.

• For Revision request/s:
If revision is not requested by the time Final Financial report is activated, then the maximum score in the category will be inserted for the project automatically.

• For Financial performance:
If there is no report, it will say “pending”.

• For Audit findings:
For NGOs: If audit is delayed and score is calculated from the rest of the categories the dashboard on audit will say “pending”
For UN: Total score will be calculated from the rest of the categories.

PARTNER-LEVEL PERFORMANCE AND RISK RATING EVOLUTION 
The different project PI scores of a Partner are combined with the original Capacity Assessment’s score to determine the overall Partner PI adjusted score.
The weightage of each project in calculating the overall Partner Performance Index is adjusted as and when the partner completes more projects, i.e. An increased weightage is given to the performance based on the most recent projects completed by the partner.

An overview on how the weightage breaks between Capacity Assessment and Performance Index is given below:

As can be seen in the above screenshot, the CA score is not included in the calculation of the PI adjusted risk from the partner’s fifth project onwards.s.

The Partner’s PI adjusted score, obtained by the combination of project’s PI scores and CA results (if relevant), is linked to a specific risk level defined by the Global Risk Range, as shown in the screenshot below. To be included in the calculation of the PI adjusted risk, the CA risk score is adapted from the Country Risk Range to the Global Risk Range via a multiplier tailored for each country. To learn more about the CA multiplier, please refer to the CA article on the Help Portal.

The GMS automatically calculates and determines the Partner’s PI score and subsequent Risk Level (adjusted or not). HFU officers can approve the PI adjusted risk of the Partner from the Partner Performance Index Risk Management page.

To learn more on how to do so, please refer to this section of the article.

 

The Partner Risk DashboardTop 

The Partner Risk Dashboard allows HFU officers to generate a list of all partners with a breakdown of the different components of their risk level.

To access the Partner Risk dashboard, follow this navigation: Risk Management -> Partner Risk Dashboard

Use the Filters section at the top, to select the partners which performance dashboard you would like to access.

 

Apply the wanted filters and click on [Click to filter] to obtain the list of corresponding partners.

 

*Please note, if you do not apply filters and click on [Click to filter], the system will provide a list of all partners.

 

 

 

Field Descriptions:

Pooled Fund(s): Displays the Pooled Fund(s) to which you have an access. By default, the Pooled Fund you are currently in will be selected.
Partner Type: Displays the different types of partners in GMS.
Partner Risk Level: Displays the different risk levels in GMS (Ineligible, Low, Medium and High) that the partners currently have. [Old terminology: current or running risk]
Partner: Displays the different partners for the pooled funds selected. The pooled fund is mentioned as a prefix because one partner might have a different name in different funds. For example, OCHA could be registered as OCHA in the Iraq CBPF and as OCHA Myanmar in the Myanmar CBPF.
PI Adjusted Risk: Displays the different risk levels in GMS (Ineligible, Low, Medium and High) that are currently calculated by the system for the partners. [Old terminology: calculated risk]

• Partner Partnership Status: Displays the two following options: ‘Active’ and ‘Inactive’. Select 'Active' to display all partners that are currently active on the GMS. Select 'Inactive' to display the suspended, dis-enrolled or ineligible partners.

Important to Note: The system will not display live data on the PI Dashboard, it will be refreshed every 2 hours. The date and time (always shown according to Geneva time, UTC +1) at which the data was taken for the displayed PI Dashboard is indicated in red above the table.

The Partner Risk Dashboard follow this color legend:

Color legends:
• Partner Performance Index: Displays the colors used to determine the PI for each project.
• Global Risk Range: Displays the colors used to determine the Partner Risk Level and PI Adjusted Risk. These figures represent the Global Risk Ranges, as can be found in the Global Operational Manual.

To access your Country CA range as set in your Fund’s Operational Manual, and approved by your Advisory Board,  click on [Country CA Range]. A pop-up window displays your Country CA range.

 

Once you click to filer, the list of partners is generated below the filter section.

*Please note: Users will be able to Export the PI Dashboard to Excel by clicking on one of the Excel icons, either to get a summary view, or a more advance view, containing more data.

Column Descriptions of the Dashboard’s list:
S.No: Number attributed to the partner in the list.

Pooled Fund: Displays the Pooled Fund of the Partner.
Partner: Displays the name of the Partner in a specific Pooled Fund.
Partner Risk: Displays the current Risk Level a Partner has in the system.
PI (Performance Index) Adjusted Risk: Displays the risk as calculated by the system for a specific Partner – this may or may not have been accepted by the HFU user.
Adapted Country CA Risk: Displays in black the adapted CA score (from country range to global range using the multiplier) and in grey the actual country CA score.
• P1 – P6: Displays the score a Partner received for their projects. P1 is the first project, the last column value refers to the most recent project of that Partner. The Formula to calculate the PI Adjusted Risk is CA + SUM (P1:P6).

API score is calculated for each project and displayed in the column with its overall score and the percentage below indicates the weightages given to this score in the formula. You can hover over the Information icon to know which project is related to each PI score. Moreover, the number between brackets next to the score indicates to which PI Performance row the project is related to.

 

The second column of the table shows two tooltips for each partner:  and .

The First tooltip of the partner’s line, , labelled PI, redirects the user to the Performance Management Page (see the below section of this article).

The second tooltip of the partner’s line , redirects the user to Partner Performance Index Risk Management (see the below section). 

 

The Performance Index Risk Management pageTop 

To access the Performance Index Risk Management page, go to the Partner Risk Dashboard (Risk Management -> Partner Risk Dashboard), apply the relevant filter and click on [Click to filter] to obtain the list of corresponding partners.

Click on the barometer icon  to access the Performance Index Risk Management Page.

REVIEWING AND APPROVING THE PARTNER'S RISK LEVEL UPDATE 

After a project is Closed or Under Closure, HFU officers must review the partner’s new Risk Level as displayed in this page, provide approval by ticking the box [I agree] and add comments in the dedicated field.

If you disagree with the new Risk Level, tick the box [I disagree] and kindly provide comments. This will stop the risk level re-calculation message from popping up on the partner’s projects pages (shown in the below second screenshot), however this will not have any impact on the system suggested PI adjusted risk score.

You can also access this page and approve the re-calculation of the Partner’s Risk Level from the cover page of the project. To access the Index Risk management page and agree or disagree with the adjusted risk level, click on [Next]. (as in screenshot below).

*Please Note, the risk level may remain the same after recalculation. However, HFU officers are still required to provide approval of the adjusted score.

The Partner Index Risk Management page provides HFU officers with the list of all projects and Capacity Assessment’s results and their weightage in calculating the Risk Level of the partner (both initial and adjusted overtime).

On the Left-Hand side, the ‘Partner Risk Level’ section provides the initial risk level of the Partner, which includes the components that were taken into its calculation (Capacity Assessment and projects). In the below example, the partner has never undertaken a risk update, so their initial risk level is only based on their Capacity Assessment’s results.

On the left-hand side, the ‘Adjusted Partner Risk Level’ section provides the list of the partner’s projects and their weightage in calculating the adjusted Risk Level.

Legends description of the above screenshot:
1- Click on this hyperlink to access the partner’s CA scorecard and results. To know more about Capacity Assessments’ scorecards, read the article on Capacity Assessment.
2- Click on these different hyperlinks to access the projects’ cover page. P1 represents the first approved project, P2 the second approved project and so on.

RISK UPDATE HISTORY LOG 
You can find at the bottom of this page a gray section. It is the history log of all updates of the Partner’s Risk Level.

 

Performance Index Project Scorecard Top

The Performance Index questions are displayed when reaching the pre-defined milestones and workflow statuses. However, HFU officers can also access at any time during the project lifecycle the Performance Index Scorecard, which includes the project’s questions and answers already saved.

ACCESSING THE PERFORMANCE INDEX PROJECT SCORECARD 

To access the PI project’s scorecard, you need to use the Partner Risk Dashboard.
Use the filter section at the top to find the project which PI scorecard you wish to access.

In the list generated below, click on the PI icon  of the relevant partner to access its projects’ scorecards.

This action redirects you to the Partner PI Management page, which allows HFU to access and manage the Partner’s project’s PI scorecard.

THE PARTNER PI MANAGEMENT PAGE 

Field Descriptions:
• Pooled Fund Name: Displays the Pooled Fund you have selected in the Partner Risk Dashboard.
• Partner: Displays the different partners for the pooled fund selected.
• Select Project: Displays the different projects that have reached the assessment stage in GMS for the Partner selected. If there are no projects to choose from, the partner does not have a project that has yet been scored, i.e. it has not reached the assessment phase yet (EO clearance). 

To view a partner’s scorecard for a specific project, select a Partner, the available projects scorecards will be displayed in the drop-down menu of ‘Select Project’. Once the partner and project have been selected, click on [Load PI Scorecard].

The Performance Index Scorecard for the project will be displayed on the left. ‘PI Score’ will display the current score of the project.
If the project assessment is ongoing, a pending message will be displayed. Hover over the ‘pending’ tooltip to see what questions are pending. 

Users will be able to review and edit the questions that have been already been answered. Questions related to the stages that have not been reached yet by the project will not be displayed, but the ones that have preceded the current stage will be visible.

E.g.: If a partner has not submitted a final financial report, the question related to final financial report will not be visible; however, the ones on project, progress narrative and interim financial reports will be.

Upon editing the questions, click on [Save Score] to save the modifications.

THE PERFORMANCE INDEX LOGS 
The Performance Index Logs will be displayed on the right. Users will be able to see the different answers and comments that were given for each PI question, as well as details on the date and user who proceeded with the respective edits.