Advertisment ACS-IndiaSymposium
 
Journal of Pharmaceutical Negative Results
  Print this page Email this page Small font sizeDefault font sizeIncrease font size 
Search Article 
  
Advanced search 
 Home | About us | Editorial board | Search | Ahead of print | Current issue | Archives | Submit article | Instructions | Subscribe | Contacts  
 


 
  Table of Contents  
LETTER TO EDITOR
Year : 2011  |  Volume : 2  |  Issue : 2  |  Page : 121-122  

Is statistical significance a relevant tool for assessing clinical significance?


Discipline of Clinical Pharmacy, School of Pharmaceutical Sciences, Universiti Sains Malaysia, Penang, Malaysia

Date of Web Publication25-Nov-2011

Correspondence Address:
Syed Wasif Gillani
Discipline of Clinical Pharmacy, School of Pharmaceutical Sciences, Universiti Sains Malaysia 11800, Penang
Malaysia
Login to access the Email id

Source of Support: None, Conflict of Interest: None


DOI: 10.4103/0976-9234.90227

Rights and Permissions

How to cite this article:
Gillani SW. Is statistical significance a relevant tool for assessing clinical significance?. J Pharm Negative Results 2011;2:121-2

How to cite this URL:
Gillani SW. Is statistical significance a relevant tool for assessing clinical significance?. J Pharm Negative Results [serial online] 2011 [cited 2019 Sep 22];2:121-2. Available from: http://www.pnrjournal.com/text.asp?2011/2/2/121/90227

Sir,

In medicine and psychology, clinical significance refers to either of two related but slightly dissimilar concepts, whereby certain findings or differences, even if measurable or statistically confirmed, either may or may not have additional significance, either by [1] being of a magnitude that conveys practical relevance (a usage that conflates practical and clinical significance interchangeably) or [2] more technically and restrictively addresses whether an intervention or treatment may or may not fully correct the finding. Commentators who utilize the second, more restrictive usage designate the broader usage as linguistically imprecise and thus erroneous. Now a days, there is an element of statistics in the clinical assessment of therapeutic or outcome analysis.

During the period 1978-79, a total of 760 articles were published in NEJM. Statistical proportion was divided with 40% of descriptive statistics (%, mean and SD) and 56% element of t-test for continuous variables. The contingency tables (chi-square, Fisher's exact) increased to about 66%. [3] Similar study by another author describes the statistical patterns during the period 2003-2004. 311 articles were published in NEJM. There was a major shift of analysis with high increase in survival analysis (61%), multiple regression (51%), power analysis (39%) and multiple comparisons (23%). There was moderate increases in contingency tables (53%), epidemiologic stats (35%), and nonparametric tests (35%), ANOVA (16%). Steady level with descriptive or no stats, transformation, and simple linear regression. In contrast, there was a moderate decrease in t-test (26%), and significant decrease in Pearson correlation (3%). The author reported increase from 1.9 to 4.2 statistical methods per article. [4]

A minor change in response or treatment outcome will not produce statistical significance results but is considered as clinical significance in terms of pharmacovigilance issues. Similar conclusion was made by Van et al. in 2007, "Many conclusions of studies of exercise therapy for chronic low back pain have been based on statistical significance of results rather than on clinical importance and, consequently, may have been too positive. Authors of trials should report not only statistical significance of results but also clinical importance". [5]

The point of concern here is statistical analysis is needed to justify the objectives of the study and not to make the findings complicated with bulk amount of statistics. The current available research material seems to be statistical research with misleading significances. Statistical significance tends to be used in the context of null hypothesis significance testing (NHST). NHST answers the question: If a hypothesis that an effect is zero in the population is true (the null hypothesis), what is the probability of obtaining data that indicate the effect is not zero? [1] NHST is often misunderstood in several ways: that the P-value is the probability that the null hypothesis is false; that it is related to probability of replication; and that if the null hypothesis is rejected, the proposed alternative hypothesis should be accepted [Table 1].
Table 1: Concept of significance and expected interpretation


Click here to view


Given the nature of NHST, and its common misuse, statistical significance does not yield information about magnitude of effect, practical significance, or clinical significance. [2] NHST only yields information about whether results are statistically likely, given some assumption about the population. [6] In terms of testing clinical treatments, statistical significance can only provide an answer to the question: If a treatment is actually ineffective, how likely it is that the statistical test of the treatment would erroneously indicate that the treatment is effective?

In terms of clinical treatment studies, clinical significance answers the question: Is a treatment effective enough to cause the patient to be normal? For example, a treatment might significantly change depressive symptoms (statistical significance), the change could be a large decrease in depressive symptoms (practical significance-effect size), and 40% of the patients no longer met the diagnostic criteria for depression (clinical significance). It is very possible to have a treatment that yields a significant difference and medium or large effect sizes, but does not move a patient from dysfunctional to functional. Clinical significance, first proposed by Jacobson, Follette, and Revenstorf [7] as a way to answer the question, is a therapy or treatment effective enough such that a client does not meet the criteria for a diagnosis.

Jacobson and Truax later defined clinical significance as "the extent to which therapy moves someone outside the range of the dysfunctional population or within the range of the functional population." [8] Later, they proposed two components of this index of change: the status of a patient or client after therapy has been completed and how much change has occurred during the course of therapy. [8] Differences that are common in the population are also unlikely to be clinically significant because they may simply reflect a level of normal human variation. Additionally, clinicians look for information in the assessment data and the client's history that corroborates the relevance of the statistical difference, to establish the connection between performance on the specific test and the individual's more general functioning. [9],[10]

At this point of discussion, majority of researchers think what we will do if statistical concerns are consider as biased in clinical studies. So, there are many ways to calculate statistical significance and practical significance, and there are various ways to calculate clinical significance. Five common methods are the Jacobson-Truax method, the Gulliksen-Lord-Novick method, the Edwards-Nunnally method, the Hageman-Arrindell method, and hierarchical linear modeling. [11]

 
   References Top

1.Cohen J. The earth is round (P< .05). Am Psychol 1997;49:997-1003.  Back to cited text no. 1
    
2.Haase RF, Ellis MV, Ladany N. Multiple criteria for evaluating the magnitude of experimental effects. J Couns Psychol 1989;36:511-6.   Back to cited text no. 2
    
3.Emerson JD, Colditz GA. Use of statistical analysis in the New England Journal of Medicine. N Engl J Med 1983;309:709-13.  Back to cited text no. 3
    
4.Horton NJ, Switzer SS. Statistical methods in the journal. N Engl J Med 2005;353:1977-9.  Back to cited text no. 4
    
5.Van Tulder M, Malmivaara A, Hayden J, Koes B. Statistical significance versus clinical importance: trials on exercise therapy for chronic low back pain as example. Spine (Phila Pa 1976) 2007;32:1785- 90.  Back to cited text no. 5
    
6.Clinical Significance: "Clinical" Significance and "Practical" Significance are NOT the Same Things. Online Submission, Paper presented at the Annual Meeting of the Southwest Educational Research Association (New Orleans, LA, Feb 7, 2008).  Back to cited text no. 6
    
7.Jacobson NS, Follette WC, Revenstorf D. Psychotherapy outcome research: Methods for reporting variability and evaluating clinical significance. Behav Ther 1984;15:336-52.  Back to cited text no. 7
    
8.Jacobson NS, Truax, P. Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol 1991;59:12-9.  Back to cited text no. 8
    
9.Sattler JM. Assessment of children: Cognitive foundations (5/e). San Diego: Sattler Publications. ISBN 978-0-9702671-6-0; 2008.  Back to cited text no. 9
    
10.Kaufman AS. Lichtenberger E.Assessing Adolescent and Adult Intelligence 3 rd ed. Lay summary on 22 Aug 2010. Hoboken (NJ): Wiley. ISBN 978-0-471-73553-3; 2006.  Back to cited text no. 10
    
11.Peterson L. "Clinical" Significance: "Clinical" Significance and "Practical" Significance are NOT the Same Things. Online Submission, Paper presented at the Annual Meeting of the Southwest Educational Research Association. New Orleans, LA, Feb 7, 2008.  Back to cited text no. 11
    



 
 
    Tables

  [Table 1]



 

Top
  
 
  Search
 
    Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
    Access Statistics
    Email Alert *
    Add to My List *
* Registration required (free)  

 
  In this article
    References
    Article Tables

 Article Access Statistics
    Viewed2416    
    Printed150    
    Emailed0    
    PDF Downloaded357    
    Comments [Add]    

Recommend this journal