Resources

Resources

Hear from NextGen's expert advisors with insights and advice on the evolving COVID-19 pandemic and beyond.

Read Blog

The financial and clinical challenges you face now are evolving rapidly. Here are resources, solutions, and ideas we think will help.

Read Now

COVID-19

COVID-19: Does the Quality of the Test Matter?

By Dr. Martin Lustick

Subscribe to receive email updates as new information becomes available.   Subscribe now

Blog Home    COVID-19: Does the Quality of the Test Matter?
Showing of results

The pandemic has brought testing to the forefront of healthcare issues.  As we hear and see all of the media buzz about the accuracy, reliability, validity, and timeliness of tests for coronavirus, a review of the epidemiological concepts that underlie these discussions can shed some light on the best clinical approach to testing.  

Sensitivity and Specificity

Most of us are familiar and comfortable with an understanding of sensitivity and specificity expressed as the percent of true positives and true negatives, respectively. Unfortunately, having those numbers in a vacuum is of little value.  

First, it is important to understand the illness. Life threatening diagnoses that benefit from early intervention require tests that are maximally sensitive. While colon cancer is an obvious example of this, it is perhaps a little less obvious for COVID-19, as it is not always life threatening and there’s little if any early intervention that can change the course of the illness.  On the other hand, because it is contagious and potentially lethal, early detection is critical for public health interventions that can effectively reduce morbidity and mortality for the population as a whole.  

Specificity is critical when you’re trying to avoid a high risk intervention for a low risk condition.  No one would want open heart surgery if there were nothing wrong with their heart.  Again COVID-19 presents as an interesting special case. While a false positive isn’t likely to lead to a high risk intervention,  it does leave the patient subject to unnecessary quarantine and potentially social stigma as well.

The Importance of Prevalence

Unfortunately, sensitivity and specificity do not have much practical value until we understand the role of prevalence. The Positive Predictive Value (PPV) and Negative Predictive Value (NPV) are the first step in taking prevalence into account. These two values blend sensitivity and specificity with prevalence to provide a much more useful number to the clinician. 

Positive predictive value is the probability that subjects with a positive screening test truly have the disease. Negative predictive value is the probability that subjects with a negative screening test truly don't have the disease. Mathematically, PPV is the number of true positives divided by the total number of positives, and NPV is the number of true negatives divided by the total number of negatives. So even with a test that is both 99% sensitive and 99% specific, if the prevalence is only 1%, the PPV would be 50%. While this feels counter-intuitive, it’s obvious that if only one of 100 people are infected and you test all of them, it’s likely there will be one true positive and one false positive test. The good news is that by repeating the test on these two patients, there’s a 98% likelihood that the patients will know if they are truly infected. 

In this same example, the NPV would be 99.99%, so a negative test should be very reassuring.  Interestingly, even with a test that is only 80% sensitive and specific and in the context of 30% prevalence, the NPV would still be over 90%. 

Pretest Probability is Critical

Unfortunately, there are a variety of factors that can undermine the value of the reported overall prevalence, altering the pretest probability of disease. In the above example, imagine a patient who has recently attended an indoor event with 2,000 people. Further, assume that each of the 20 people (1% of 2000) who arrived at the event infected, spread the virus to nine people sitting near them. Two hundred people left the event infected, so this patient now has a 10% pretest probability of disease. That raises the PPV from 50% to greater than 90%.  

On the other hand, a person who is asymptomatic after 14 days in quarantine, would have a pretest probability much lower than the community prevalence, and a PPV much lower than 50%. In the end, it’s critical for clinicians to evaluate each patient’s risk for serious consequences of disease, together with their pretest probability and the sensitivity and specificity of the test they are using. Only by assessing all of these factors can clinicians be well positioned to counsel patients on testing for COVID-19.  


Dr. Martin Lustick

Senior Vice President, NextGen® Advisors

Dr. Martin Lustick is a principal and senior vice president with NextGen Healthcare focused on supporting provider organizations in their successful transition from volume to value-based care…

Dr. Martin Lustick is a principal and senior vice president with NextGen Healthcare focused on supporting provider organizations in their successful transition from volume to value-based care.

Dr. Lustick earned a BA in History from Cornell and an MD from Columbia. After completing his pediatric residency at Children’s Hospital National Medical Center in Washington, DC, he was in clinical practice for 17 years with Kaiser Permanente of the Mid-Atlantic States. While there, Dr. Lustick held various management and leadership roles, including chief operating officer for the 800-physician medical group. He oversaw development of their hospitalist program, population health capability, and open access delivery model.

Dr. Lustick then served as chief medical officer for ThompsonHealth—a small health system in Canandaigua, NY—where he provided clinical oversight for hospital, SNF, nursing home, IT, and out-patient physician practices.

In 2005, Dr. Lustick assumed the role of SVP & CMO for Excellus BCBS which covers 1.6 million lives comprised of Medicare, Commercial, and Medicaid. In his 13+ year tenure there he led a variety of strategic initiatives, including a patient-centered medical home program which served as the foundation for the plan’s value-based payment strategy. He also led the implementation of an automated authorization program for care management services, development of a clinical quality improvement strategy, and creation of innovative programs in management of low back pain, screening and prevention, opioid addiction, and chronic disease management.

Dr. Lustick has also been very active in the community, serving on boards and committees confronting issues such as: healthcare capacity planning, Health Information Exchange, mental health, substance use disorders, social determinants of health, and childhood obesity.

Learn More

Get in touch with Dr. Martin Lustick

The financial and clinical challenges you face now are evolving rapidly. Here are resources, solutions, and ideas we think will help.

Read Now

Follow us