The Board Can Enhance Its Diversity and Inclusion Efforts

Performance Management

The purpose of the Board's Performance Management Program is (1) to continuously improve individual and organizational performance, (2) to develop and motivate employees to become top performers and help the Board achieve its mission and purpose, and (3) to inform various employment decisions.

This section presents information on performance management for the period of our review, including a summary of the Board's process and trend statistics based on the independent analysis performed by an external consulting firm. The consulting firm's analysis indicated statistically significant differences in performance ratings among certain demographic groups on an agency-wide basis. When these demographic groups' performance ratings were evaluated by pay grade category, however, in most cases there was no trend of statistically significant differences.³² The agency-wide differences do not necessarily indicate discrimination and could be due to actual differences in employee performance or other factors. Further analysis of performance ratings may help the Board identify any patterns that may indicate potential unfair or unequal treatment. The Board piloted a new performance management system in 2014; performance management data associated with the new rating system are not reflected in our analysis.

The Board's Process

The Board's Performance Management Program policy describes the Board's Performance Management Program, which provides the framework for an employee's annual performance assessment and rating during the period of our review. The Board's performance periods follow a fiscal year (October 1 through September 30) schedule. Supervisors are responsible for creating performance standards, monitoring performance, and providing employees with feedback on their performance. Supervisors are required to conduct an annual, written review of an employee's performance, which should be reviewed by the supervisors' manager before issuance to the employee. In the fiscal year (FY) 2011, FY 2012, and FY 2013 rating periods, employees were assigned one of five possible ratings: extraordinary, outstanding, commendable, marginal, and unsatisfactory.³³

According to the Board's policy, the reviewing manager should attempt to resolve any disagreement between an employee and his or her supervisor with respect to the employee's performance rating. Further, Board employees other than a Division Director, an Office Director, or the Chief Operating Officer can, within a certain time frame, appeal their performance rating with the Director of the division or office.³⁴ ER will assist in facilitating this process. However, if the appeal is based on sex, race/ethnicity, or age, the employee must file a separate complaint with the OD&I.³⁵

Annual performance ratings are the basis for determining merit salary increases, which are administered by Compensation, and the ratings may also be considered when determining variable pay, eligibility for additional incentive programs, and promotions.³⁶ Employees with a rating of marginal or unsatisfactory are not eligible for merit increases or other types of performance-based pay.

Demographic Statistics

Results From an External Consulting Firm's Analysis Performed for the OIG

We used an external consulting firm to conduct an independent analysis of the Board's FY 2011, FY 2012, and FY 2013 employee performance ratings. The consulting firm conducted tests of statistical significance and practical significance to evaluate group differences.³⁷ For analysis purposes, the consulting firm analyzed gender and age differences, as well as race/ethnicity differences among the White, Black/African American, Hispanic/Latino, Asian, and Other groups.³⁸ The consulting firm evaluated performance data for three pay grade categories: senior managers and officers (FR-29 and 00), mid-level professionals (FR-26 to FR-28), and all others³⁹ (FR-16 to FR-25 and WE-41 to WE-47).

The external consulting firm's analysis revealed that overall during the three-year period, approximately 99 percent of Board employees received ratings of commendable or above (table 6).⁴⁰

Table 6: Distribution of Performance Ratings for All Employees, FY 2011–FY 2013

Performance ratings	FY 2011		FY 2012		FY 2013
	Number of rated employees	% of total rated employees	Number of rated employees	% of total rated employees	Number of rated employees	% of total rated employees
1—extraordinary	378	19.13	429	20.27	480	22.39
2—outstanding	712	36.03	848	40.08	956	44.59
3—commendable	882	44.64	829	39.18	696	32.46
4—marginal	4	0.20	10	0.47	12	0.56
5—unsatisfactory	0	N/A	0	N/A	0	N/A
Total	1,976	100.00	2,116	100.00	2,144	100.00

 Source: External consulting firm analysis based on Board-provided data.

The results of the consulting firm's analysis of the Board's FY 2011, FY 2012, and FY 2013 performance ratings indicated statistically significant differences among Board employees across certain demographic groups on an agency-wide basis. However, when these demographic groups' performance ratings were evaluated by pay grade category, in most cases, there was no trend of statistically significant differences. These statistically significant differences do not necessarily indicate discrimination and could be due to a variety of factors either individually or in combination, such as actual differences in employee performance. A statistically significant result does not imply that a difference is good or bad or that it is large or small; it indicates that the observed difference is probably not due to chance.

The external consulting firm did not find statistically significant differences in the gender category. However, the consulting firm found statistically significant differences in the following race/ethnicity and age categories:

• In all three years, on an agency-wide basis, White employees received higher performance ratings compared with Asian employees; however, there were no significant differences in performance ratings when analyzed at the job levels.
• In all three years, on an agency-wide basis, White employees received higher performance ratings compared with Black/African American employees. In 2012 and 2013, there were no statistically significant differences in performance ratings when analyzed at the job levels. In 2011, there was a statistically significant difference between the average ratings of White employees and Black/African American employees for the all others employee job level.
• In 2013, in the senior managers and officers category, Hispanic/Latino employees received higher performance ratings as compared with White employees. The comparison included 247 White employees and only 7 Hispanic/Latino employees; therefore, these results should be interpreted with caution.⁴¹
• In 2012, on an agency-wide basis, White employees received higher performance ratings compared with Hispanic/Latino employees; however, there were no significant differences in performance ratings when analyzed at the job levels.
• In 2012, on an agency-wide basis, employees 40 years of age or older received higher performance ratings than employees under 40 years of age; however, there were no significant differences in ratings for employees under 40 years of age and employees 40 years of age or older within the senior managers and officers job level and within the all others employee job level.
• In all three years, in the mid-level professionals category, employees under 40 years of age received higher performance ratings than employees 40 years of age or older.

The consulting firm's full report on the Board's employee performance ratings is included as appendix E.

The OIG's Analysis 

In addition to the external consulting firm's statistical analysis of performance ratings for the entire Board, we analyzed performance ratings by division to determine average performance ratings for FY 2011, FY 2012, and FY 2013 by race/ethnicity. We did not evaluate these averages for statistical significance, and we did not conduct analyses by pay grade category.

The results of our analysis by division were similar to the external consulting firm's agency-wide findings discussed above.⁴² These observations do not necessarily indicate discrimination and could be due to a variety of factors. Appendix F contains our analysis of performance management data by divisions.

Finding: The Board Has Not Conducted Analyses of Employee Performance Reviews on an Annual Basis

 According to a Board official, the Board does not consistently conduct a review of the distribution of performance ratings to ascertain how the ratings are distributed across sex, race, or people 40 years of age or older. The Board has periodically analyzed aggregate performance ratings distributions by divisions. Further, in 2012, the Board surveyed employees on the Performance Management Program. The final results report indicates survey participant concerns with effectiveness, fairness, and rater bias.

One government best practice suggests that organizations should gather and analyze statistics on the distribution of performance ratings.⁴³ Uneven ratings distributions across gender and race/ethnicity might raise questions about fairness. If differing treatment is found within the performance appraisal process, efforts should be made to determine whether appraisal design features are causing the lack of balance in the ratings or whether other issues at the organization may be responsible.

As previously noted, the external consulting firm found statistically significant differences in performance ratings among certain demographic groups on an agency-wide basis. When these demographic groups' performance ratings were evaluated by pay grade category, however, in most cases there was no trend of statistically significant differences. Additional analyses of employee performance ratings will allow the Board to better determine whether its performance management system supports the development and retention of a diverse workforce.

Management Actions

The Board acknowledged challenges with the performance management system in place during the review period. In discussions about the performance management framework, employees were in favor of a framework that (1) focused on growth, (2) included ongoing conversations between managers and employees, (3) created a partnership between managers and employees, and (4) potentially had a more effective method to rate performance. The Board decided to adopt a new performance rating system.

The new performance management process was piloted in five divisions and the OIG for performance year 2013–2014, with full implementation in all Board divisions in the 2014–2015 performance year. The purpose of the new process is to align staff to the work of the Board, provide greater accountability, support the growth of staff, improve the value of time spent, and increase the fairness of the process. In addition, the new process involves frequent conversations between employees and their managers that are designed to develop and grow employees' capabilities. The Board contracted for the necessary expertise to assist with the program's implementation, which includes information sessions, tools and guides, training, and other support.

Recommendation

We recommend that the Chief Human Capital Officer

3. Consider conducting annual analyses of the distribution of employee performance ratings to identify whether patterns exist that may indicate unfair or unequal treatment. If the analyses reveal patterns that may indicate unfair or unequal treatment, determine whether any actions are necessary.

Management's Response

The Board concurs with our recommendation. In its response, the Board notes that a periodic analysis focused on areas in which management has potential concerns may be useful. Management will consider the feasibility of conducting additional analyses on a periodic basis.

OIG Comment

The actions described by the Board are generally responsive to our recommendation. We plan to follow up with the Board to determine its final decision in considering our recommendation.

• 32. The external consulting firm we used refers to this as job level rather than pay grade category. Return to text
• 33. Economists were on a seven-tier system: extraordinary, outstanding plus, outstanding, commendable plus, commendable, marginal, and unsatisfactory. Return to text
• 34. The Chief Human Capital Officer will review appeals if the Division Director is the supervisor or the reviewing manager. If the Chief Human Capital Officer is the supervisor or the reviewing manager, the Board's General Counsel will appoint an appeals officer. Return to text
• 35. The OD&I and ER ensure that the appropriate office handles an employee's complaint, depending on the basis. Gender, race/ethnicity, age, or disability discrimination claims are handled by the OD&I other workplace complaints are handled by ER. Return to text
• 36. During the review period, variable pay was generally targeted toward officers and employees in designated job families that (1) are critical to the execution of the Board's core mission, (2) require skills that are in high demand in the marketplace, (3) have salaries well below prevailing market levels, and (4) experience recruiting difficulties and high rates of turnover. Compensation reviews new variable pay requests for additional job families and makes a recommendation to the Chair of the Committee on Board Affairs, who makes the final determination. Return to text
• 37. A test for statistical significance indicates the probability that the group difference could have been due to chance. In contrast, measures of practical significance provide an indication of the size of the difference. Return to text
• 38. Other includes American Indian/Alaskan Native, Native Hawaiian/Pacific Islander, individuals identifying themselves as belonging to two or more races, and individuals who chose not to disclose demographic data. Return to text
• 39. In this section of the report, the external consulting firm's use of the term all others equates to our use of the term all other professional employees and all wage employees elsewhere in the report. Return to text
• 40. Economists who were rated outstanding plus are shown as outstanding and those rated as commendable plus are shown as commendable. Return to text
• 41. According to the external consulting firm, small sample results are often nonrepresentative and unstable and can change substantially with small changes in the data. Return to text
• 42. The external consulting firm reversed the order of the Board's performance management rating system so that higher ratings reflected better performance. However, our analysis reflects the Board's ordering of performance ratings, in which a lower rating number reflected higher performance (e.g., extraordinary is represented by a rating of 1). Return to text
• 43. U.S. Office of Personnel Management, Evaluating Performance Appraisal Programs: An Overview, PMD-09, January 1999. Return to text