Erroneous Ratings Data: Part II

Saturday, February 25, 2006 | 0

We previously discussed our national study of 2100 impairment rating reviews; 80% of all ratings reviewed were erroneous, with 89% of the erroneous ratings being elevated. In this installment we will present more of the results; this information is critical to the work you perform.

Each physician's report was rated for quality on a scale of 4 - excellent, 3 - good, 2 - fair, 1 - poor and 0 - unacceptable. A rating of "good" reflects adherence to standards defined in the Guides and fair reflects adequate information to assess the case, however deficits were present. The mean scores for all physicians were: history 2.2 (fair), examination 2.1 (fair), clinical analysis 2.1 (fair), application of criteria 1.5 (between poor and fair). Comparison to criteria was consistently the weaker aspect of most reports. Review of the data suggests that rating physicians typically demonstrate consistent performance with rating, either consistently producing accurate or inaccurate ratings. (See table below for reference).

Of all the erroneous reports, 90% of the Fifth Edition ratings were higher than appeared appropriate. The corrected value for a permanent impairment rating averaged less than half of the original rating. For all Fifth Edition whole person ratings the mean original rating was 14.2% whole person permanent impairment and the mean expert rating was 6.2% whole person permanent impairment. Examining the subset of the eighty percent of Fifth Edition whole person ratings that were erroneous, the rating by the original physician averaged 15.5% (standard deviation of 9.4%) whole person permanent impairment and upon re-rating the corrected rating averaged 5.6% (standard deviation of 14.3%) whole person permanent impairment, with a correlation coefficient of 0.54.

For Fourth Edition whole person ratings the mean original rating was slightly greater at 18.4% whole person permanent impairment and the expert rating was also higher at 9.0% whole person permanent impairment. Of all Fifth Edition erroneous reports where ratable permanent impairment was originally identified, upon expert re-rating 37% were found to have no impairment; in Florida 76% previously rated as having impairment were found to have no impairment.

Examining the 839 Fifth Edition whole person erroneous ratings, 7% of the cases (61) were under- rated, i.e. based on the clinical information provided the rating should have been higher. With these cases the mean original rating was 9.9% whole person, however it should have been 16.2% whole person. Thirteen of the erroneous ratings had the same numeric value when rated, however the rating process used by the physician was incorrect and the similar results were coincidental.

Of 850 Fifth Edition cases that were found either through critique or review of records to have impairment, 90% of the cases had impairment less than 22% whole person. Less than 5% of all ratable cases have impairment greater than 30% whole person permanent impairment.

For all regions the original ratings were rated higher than what appears appropriate based on the medical information provided. The error rate was consistent among regions with approximately 80% error rate, with an exception for multiple regions of the spine with the error rate being 90%. The actual mean expert rating of all regions, with the exception of thoracic spine, was between 3% and 8% whole person permanent impairment, with an overall mean of 6% whole person permanent impairment. Many of the thoracic cases involved corticospinal tract involvement and therefore were associated with greater impairment. The data suggests that are higher ratings are seen for older, male patients and for older injuries.

In summary, the analysis of the 2100 reviews reveals that the vast majority of AMA Guides ratings are erroneous and substantially higher than appropriate. The goal of the Guides is to provide consistent ratings, therefore reducing conflict. The Guides state "Two physicians, following the methods of the Guides to evaluate the same patient, should report similar results and reach similar conclusions." (AMA Guides, p. 17) Yet, review of this data reveals that this is often not achieved.