The survey whose results are summarized here was conducted in January 2008 and published in the February 2008 issue of Professional Development Quarterly. Click here for an order form for the full report.

This report updates a survey last conducted in 2002. It reports on the status of upward evaluation programs in law firms, who is evaluated and by whom, the level of participation by the evaluators, the quality of the evaluation feedback they provide, what aspects of the programs are supported by computer automation and by the services of consultants, and the perceived benefits and drawbacks of the programs. The full report, published in in the February 2008 issue of Professional Development Quarterly, contains 9 tables of comparative data on the topics surveyed.

I. Survey Respondents

19 private U.S. law firms responded to the survey. The firms range in size from under 200 to over 1000 lawyers, and together employ over 12,800 lawyers. All have more than one office, and 12 have offices in more than one country

The "average" respondent employs 674 lawyers in 11 offices in 4 countries, and is 35% larger than the average respondent in 2002, which employed 498 lawyers.

II. Survey Findings

Status of Upward Evaluation

53% of our 2008 respondents have an upward evaluation program in place or in development, compared to 57% in 2002. The current survey indicates a greater correlation between firm size and the existence of a program than was true in 2002: The firms that have programs in 2008 are on average 33% larger than those that do not. In 2002 the size difference was negligible, only 7%.

Median age of the programs is 3-5 years; in 2002 it was 1-3 years.

Only 22% of 2008 respondents report having a program that is entirely "home grown" -- that is, developed without any outside consulting assistance -- compared to 52% in 2002.

The Programs

What Supervisors Are Evaluated and Who Evaluates Them

In every responding program, partners are evaluated by associates. In the majority of programs in both 2008 and 2002, these were the only participants in the program.

Some firms, however, include additional participants:

• 38% reported evaluating senior associates/counsel (57% in 2002), and 11% evaluate top administrators (5% in 2002).
• 22% collect evaluation feedback from nonlawyers (10% in 2002).

Program Effectiveness

In both 2008 and 2002, the attitude of most supervisors toward the program was characterized as "Mixed; some supervisors are highly receptive, others uninterested." The number of respondents characterizing supervisors' attitudes as "Generally very positive," however, dropped from 37% in 2002 to 22% in 2008.

Only a third of respondents in either year achieved 75% or higher participation by the evaluators.

The quality of the evaluation feedback collected was generally good: on average, of high accuracy, moderate to high constructiveness, and moderate completeness, and somewhat higher on average in 2008 than in 2002.

Supporting Resources

Computer Automation. The 2008 programs have automated the four essential data-gathering and -reporting tasks of the evaluation process much more extensively than was the case in 2002, with 100% of respondents having automated at least one of those tasks, and the majority all four.

Consulting Support. Fewer firms report using consulting assistance to develop their programs in 2008, but more are using consultants to compile the feedback from the evaluators.

Pros and Cons of Upward Evaluation

Most Important Benefits. Six possible benefits of the program were ranked by the survey respondents. Among the similarities and differences between 2008 and 2002:

In both 2008 and 2002, over 90% of respondents chose "Feedback that helps supervisors improve their performance" as the program's most important benefit.

The second-place choices differed. In 2008 it was "Management information on the general quality of supervision and areas for improvement" (83%). In 2002 it was "Subordinates' satisfaction from having a say about their supervision" (84%).

Major Disadvantages. Six potential disadvantages of the program also were ranked. There have been some marked shifts in the perceived disadvantages, including the following:

In 2008, the major concern is with "Lack of follow-through on improving supervisors' performance in response to feedback," cited by 78% (vs. 65% in 2002).

In 2002, "Subordinates' fear of participating and/or of providing candidate evaluations," cited by 92%, was the biggest negative. In 2008 only 67% saw this as a major issue.

Copyright © 2008, Evelyn Gaye Mara. All rights reserved. This document may be printed for personal use only. Any reproduction, retransmission, or republication of all or part of this material is expressly prohibited without prior written consent from the copyright holder. Contact maraeg@profdev.com for reprint permission.