The Evolution of the Medicare Program Integrity Manual

It’s sometimes better, sometimes worse for the provider.

The Medicare Program Integrity Manual (PIM) over the years has undergone a number of changes. Here, we will examine Chapter 8, Statistical Sampling for Overpayment Estimation. This is a complex and controversial part of the auditing world, because extrapolations sometimes are so large that they can bankrupt a provider.

Attorneys defending providers have struggled with defeating these extrapolations. Statistical experts are brought in to analyze the work of the auditor, and they are almost uniformly horrified by the lack of scientific discipline in the work. One statistician said: “if someone sent me a letter demanding more than a million dollars, based on this type of shoddy statistical work, I would laugh in their face. They could not be serious.”

But they are serious, and the pain inflicted on providers by shoddy statistical work continues.

Office of Medicare Hearings and Appeals (OMHA)

The Office of Medicare Hearings and Appeals (OMHA) had a budget of around $196 million in 2022, with a staff of133 Administrative Law Judges (ALJs), 339 attorneys, 256 legal assistants, and 68 legal administrators. OMHA has been working hard, as from 2015 until present, the number of pending appeals has been reduced by 93 percent, from 900,000 to around 60,000.

For providers, however, the number of favorable appeal outcomes has been falling, from a high of around 54 percent in 2012 to around 18 percent in the 2017-2020 period, although rising a little to 28 percent by 2022. (See Figure).

PIM Changes in Statistical Extrapolations

During this time period, Chapter 8 of the PIM has been revised at least 12 times. Not all of the earlier versions of this crucial document are easy to find, but using the Internet’s Wayback Machine, it is possible to unearth a number of them.

With such significant changes in the number of favorable decisions for providers, we can hypothesize that these changes have been the result of the cat-and-mouse game played between provider attorneys and OMHA. Attorneys over time have developed arguments that get extrapolations thrown out, then the PIM is revised to “plug the hole,” so to speak, leading to more wins for OMHA. Keep in mind that this is only a hypothesis. But it might be useful to examine a few of the changes that have been made to Chapter 8.

The Dependency Argument

Some provider attorneys and their statistical experts over time developed an argument that much of the sample-taking being done was faulty, because the items sampled, usually claims, were linked to one another. This could happen when there was some inter-dependency between the claims. Dependent claims are not independent of each other, and are not a valid sample.

This is covered in Chapter 8 §3.4.3.2.2, The Sampling Unit. In PIM revision 377 of 2011, there is no discussion of dependency. However, in a more recent revision, 11529 (2022), the same section admits that “there is always a certain amount of dependence” – but it then states that “correlations of characteristics in the target population do not imply dependence in sampling.” This helps “plug the hole” and nullify the argument. The PIM continues by stating that “independence means the selection of one sampling unit does not influence, or gives no information about, the outcome of another selection,” but since this appears to contradict the previous statement, it effectively blocks a provider from making this argument.

The Distribution Argument

Some statisticians working for providers in their appeals started to check the underlying distribution of the data being analyzed, and found that it was far from normal. A “normal” distribution is the standard bell curve that we all know about from Statistics 101. When there is a standard distribution, then the central limit theorem may be used reliably to understand the sample, and ultimately, the precision of the overpayment estimation. But when the distribution is not standard, then the standard formulas don’t work. Of course, there are ways to get around this, using more sophisticated mathematical techniques, but this was never done by the auditors. They continued to rely on the standard formulas, generating faulty results.

In the 2011 PIM, § 8.4.1.3, Steps for Conducting Statistical Extrapolation, contains six steps. Today’s PIM, however, has eight steps, including a requirement that the statistician working for the auditor assess “the distribution of the paid amounts in the sample frame to determine the sample design.” It notes that “it is very likely that the distribution of the overpayments will not be normal.” This helps to plug the hole by at least giving fair warning to the auditors to include a distribution analysis in their work, something never done in the past.

But this revision is not entirely straightforward. Note that the PIM  now requires a distribution analysis of the “paid” amounts, but it is the “overpayments” that is the variable being estimated. By using a distribution of the paid amounts, the distribution analysis will be off-base, but it is unlikely that an ALJ will know the difference, and instead will take a “distribution analysis” (even if the wrong variable is analyzed) as meeting this new requirement.

The Zero-Paid Claims Argument

The PIM always has required that when the analysis of claims in a sample is performed, the auditor should look for both overpayments as well as underpayments, “in whole or in part.” A partial underpayment is one in which the provider is paid less than they should have been. A whole underpayment is one in which the provider was paid nothing when they should have been compensated. “Zero-paid claims” are claims submitted that received zero payment.

In order to follow the PIM, the auditor should also sample these claims on the chance they will run across some “whole” underpayments. But auditors always screen out these claims from the universe file, so they never appear in the “frame” from which the sample is taken. This biases the extrapolation upwards, against the provider.

The new PIM has changed the rules a little bit. For Part B claims, § 8.4.3.2.1, Composition of the Universe, states that “the universe shall consist of all fully and partially paid claims.” It now adds, however, that “claims with no payment may be included in the universe from which the same frame is constructed.” This appears to address this issue, but in reality, it does little. Actually, the language is contradictory, but the auditor will cling to the first language of “fully and partially paid claims,” ignoring the second language as being optional. In any case, in my 22 years of examining extrapolations, never even once have I seen an auditor “discover” an unpaid claim that should have been paid, or even a single claim that the auditor was even partially underpaid.

The Infamous “Merit” Argument

For years, § 8.4.4.3, Determining Sample Size, has contained the infamous language noting that “a challenge to the validity of the sample that it … is too small … is without merit, as it fail[s] to take into account all of the other factors that are involved in the sample design.” This “Get Out of Jail Free” card for the auditors has allowed them successfully to argue that grossly inadequate samples that gave scientifically faulty precision in the extrapolation were acceptable. Criticism was “without merit.” The number of questionable extrapolations that were waved across the finish line by the ALJs, based on this “without merit” argument, is unknown, but substantial.

The new PIM has revised the language and seemingly allows the provider to make an argument with “reference to the actual sample methodology used.” How this is done is not specified, so we will need to wait to see if it has any real effect. Don’t hold your breath.

The Ebb and Flow of Overpayment Extrapolations

There have been a number of other changes to Chapter 8, including § 8.4.4.4.1, “Documentation of the Universe and Frame,” which has become more insistent on keeping electronic copies of the sample frame so that it can be recreated. This appears to help providers that have long suffered when auditors hid their work, refusing to produce electronic copies of all the data needed to reproduce the statistics. Often, for example, the original auditor would show its calculations for the extrapolation, but if at a higher level of appeal a few claims were turned around, and the extrapolation changed, this work would be kept hidden.

We are very far from having an ironclad documentation standard that would allow a provider and their statistician to actually reproduce the statistical work. Much depends on what is meant by the term. The risk remains that the ALJ will accept as being sufficient only a superficial reproduction of the calculation and sample, but ignore the screening and selective cherry-picking of claims that takes place when the “universe” file is extracted.

So, the cat-and-mouse game between the providers and their statisticians and attorneys, as they battle the auditors in the gladiatorial venue of OMHA, will continue. But we can be sure that the PIM has changed in response to various arguments made by provider attorneys over the years.