By Kevin A. Schulman, MD, Bivarus Cofounder

A fascinating article in published in The New York Times last year explores the relationship between big data and effectively measuring employee performance. In the article, author Phyllis Korkki reports that while big data has made it possible to measure performance more thoroughly than ever before, two studies offer warnings on how to deploy that data.

In a study led by Iwan Barankay, an associate professor at the Wharton School at the University of Pennsylvania, a group of sales workers was shown how their sales ranked in comparison to their colleagues, while a second group was not shown a comparison, only individual results.

While the assumption was that distributing a ranking of employee performance would be motivational, the study instead showed that employees who did not know their ranking actually achieved higher sales than those unaware of their comparative rankings. The article states:

Most people optimistically assume that they are above average in their performance … and most people, when measured against one another, will inevitably rank as average or below average. For these people, seeing their rank is demoralizing, causing their performance to wilt.”

A second study, led in part by Steven Blader, an associate professor of management and organizations at New York University, involved training some truck drivers in a team-oriented environment and not training others in that environment. Some of the drivers were shown their individual performance levels, while others were shown employee rankings.

The study found that “in team-based environments, it may be better to inform each employee of his or her performance individually rather than as part of a group ranking. But if a company’s culture is self focused rather than team focused, publicizing rankings may be effective.”

These findings about the use of big data to measure employee performance in a more traditional business setting could be applied to the recent initiatives for performance incentives for healthcare providers – specifically, physicians.

Although recent initiatives aim to improve patient care by requiring the reporting of performance data at the individual healthcare provider or group level to avoid financial penalties, the traditional methods of improving physician performance provide only “benchmarking” approaches based on percentile rankings, or more punitive approaches directed at “remediating” poor performance. As demonstrated in the business setting, these methods are unlikely to achieve the intended results, particularly given the inherent characteristics of healthcare providers.

In my observation, physicians and other healthcare providers are typically overachievers. For example, physicians performed well enough in college to gain admission to medical school, performed well enough in medical school to gain acceptance into a residency program and subsequently were hired at a medical group, healthcare facility or hospital. And yet, because of the limitations of existing measurement tools, we are now going to tell the vast majority of these high-performing individuals they rank below the 80th percentile in performance (by definition) without any adjustment for the effects of being in the top performing professional group in the country.

Furthermore, given the extreme ceiling effects of the measures, there is no assessment of whether their performance is meaningfully different from their peers, and there is a lack of direct, actionable feedback. The measurement effects are so extreme that a 0.1 different on a 10-point score would drop physicians 100,000 places in a national ranking!

As Professor Barankay indicated, our national experiment in provider performance measurement through benchmarking is not likely to improve performance of high-performing physicians, but could have significant negative effects on every other provider.

This and other new research is beginning to support many of the underlying concepts of the Bivarus platform, especially in the area of physician performance. We wanted our Provider Performance Analytics to be a positive experience for physicians, nurses and other care givers – one that provides a specific and actionable pathway to performance improvement.

For example, we measure three specific provider domains in our core item bank: Communication, Patient-Centered Care and Provider Expertise and Interpersonal Skills.

Does Big Data Play a Role in Performance Incentives for Health-care Providers?

Within each of these domains, we ask actionable questions such as, “Does the provider explain recommended tests?” Rather than ranking providers against each other, the feedback is used to identify areas of strength or weakness, when compared against the practice or peer-group at large.

Areas in which performance is better than the group can be supported: “Keep up the good work!” Areas in which performance is poorer than the group can be identified as an opportunity: “Why don’t we work on communication skills for the next month and see what results we obtain?”

Through this tool, we can also identify mentors, or the providers who are the top performers in each domain within the practice: “You should join Dr. Smith with her patients the next time you have a free slot. She is especially skilled in this communications domain.”

In the end, we’ve motivated Dr. Smith by recognizing her skills and performance, and have provided an actionable path to improve individual performance, which benefits the whole provider group. Individual- and group-based interventions are then monitored through the same interface for continuous improvement.

Further, with the statistical framework underlying the provider performance analytics, observed differences are likely to be meaningful rather than simple variation due to measurement error. This helps with provider buy-in to the feedback process.

Discussions are no longer centered on the accuracy of the data, but rather how the department and/or providers can best act on the data. This philosophical change has facilitated improvements in care delivery and service operations.

Physicians and nurses appreciate the shift from a punitive system to one that offers the ability to directly receive patient feedback, identify areas for improvement and monitor their performance.


Connect with us for more information on how Bivarus can support your efforts to capture and deliver actionable insights about provider performance and patient experience.

Share This