In the conclusion to the article “Proton Radiation Therapy for Lung Cancer: Is There Enough Evidence?” Dr. David Bush puts his finger on the critical issue underlying most debates about the value of new medical technologies. He notes that “The evidence required to bring new technology into clinical practice is poorly defined.”[1] In the specific context of this article, the answer to the question of whether or not there is enough evidence depends entirely on how one chooses to define “enough.” Some experts believe that biological modeling based on dose distributions is sufficient to conclude that proton therapy improves health outcomes. Others have argued that the evidence is only adequate once randomized clinical trials have been completed to directly compare alternative interventions. Whatever one’s position on this issue, the lack of clarity on the question of adequacy of evidence is a major contributor to the gaps in knowledge about the comparative effectiveness of many widely used clinical interventions. Accelerating the rate at which this knowledge is generated will require a serious and sustained effort to define these evidence thresholds. This would allow more energy to be channeled into generating the needed evidence and less energy to be devoted to debating whether or not the evidence we have today is good enough.
Dr. Bush notes that proton beam therapy has been used to treat certain lung cancers for more than a decade at some institutions, yet most published literature reports on the experience of a relatively small number of patients (compared to the total number treated) at single institutions. The two phase II studies of proton radiotherapy plus chemotherapy in Stage III non-small cell lung cancer suggest that separate trials at different institutions may be less informative than they might have been, had the protocols been more closely aligned. These studies differ in the primary outcome measure used as well as other important study design features, which will likely limit the ability to combine the evidence in useful ways once the studies are completed. The summary statement at the end of his article underscores the significant uncertainty that remains regarding the effectiveness of this intervention: “…there appears to be enough evidence to warrant the continued exploration of proton therapy, which most probably has not reached its full potential in the treatment of cancers of the lung.’[1] The important question here is whether or not we should really know more than that by now.
Rather unfortunately, this difficultly in generating the necessary evidence to inform clinical and health policy decisions related to the use of proton beam therapy is not unique to the case of lung cancer. Controversy also exists over whether there is a sufficient level of evidence regarding the comparative clinical effectiveness of proton beam therapy, and virtually all other radiation therapy modalities used for the treatment of early stage prostate cancer.[2] We believe that the body of evidence will mature more rapidly only when there is more attention to greater alignment across studies and better definition of the evidence thresholds for broad clinical use.
One means of accomplishing this would be to develop methodological guidance documents for researchers and product developers indicating what level and type of evidence is necessary for clinical and health policy decision making. These guidance documents should be structured so that they compliment guidance documents that have been published by the FDA indicating the type and level of evidence necessary for regulatory approval. The Center for Medical Technology Policy, a Baltimore-based non-profit organization, is developing a process for producing these documents through the structured engagements of a diverse group of stakeholders, including payers, patients, consumers, regulatory officials, and academic researchers.[3] Engaging stakeholders and developing these guidance documents using a transparent process would help to ensure that evidence generation on new medical technologies is more consistent and timely.
Even if “Historically, new technology that improved dose delivery to the surrounding healthy tissues has passed as sufficient evidence for many,”[1] there has been increased recognition over the past five years that post-regulatory decision-makers need better evidence to inform their decisions. This was made apparent with the recent increase in funding for comparative effectiveness research through the American Reinvestment and Recovery Act and the Patient Protection and Affordable Care Act. To ensure that the necessary evidence is generated in a timely and reliable fashion, it is necessary for clinical researchers to work closely with clinicians, patients, payers, and other policy makers to develop consensus on the type and level of evidence needed. The radiation oncology community must take greater responsibility for improving the quality of evidence in this field, as the best possible care of patients depends on the rapid generation of evidence about which treatments work best and for whom.
Financial Disclosure: The authors have no significant financial interest or other relationship with the manufacturers of any products or providers of any service mentioned in this article.
