StatTools : Sample Size for Receiver Operator Characteristics (ROCs)
Explained and Tables

Links : Home Index (Subjects) Contact StatTools

Related Links:
Receiver Operator Characteristics (ROC) Explained Page
Sample Size for Receiver Operator Characteristics (ROC) Program Page

Introduction Sample Size (1 ROC) Sample Size (2 ROCs) References
StatTools provides two sample size algorithms for Receiver Operator Characteristics (ROC). Calculations are provided in the Sample Size for Receiver Operator Characteristics (ROC) Program Page , and tables provided on this page. The ROC value is represented by θ

Sample Size and Standard Error of a Single ROC is based on the calculation of the Standard Error (SE) from θ, as described by Hanley and McNeil (see references). The sample size is calculated by iterative approximation until the required Standard Error is obtained. The confidence interval is θ±z(SE)

The sample size table for a single ROC in the next panel is based upon the following arguments

  • The Standard Error (SE) from a nominated θ is calculated so that its one tail 95% confidence interval reaches but does not overlap the null value of 0.5.
  • As the one tail z value for 95% confidence interval is 1.65, θ-1.65SE=0.5, so SE = (θ-0.5) / 1.65
  • Sample size is then calculated using θ and SE
This sample size is the number of cases in each of the two groups (Outcome Positive (O+) and Outcome Negative (O-), so the total sample size is twice that presented.

This sample size is the minimum required to obtain a significant θ. In most cases, where a precise θ is required, or comparison between multiple θ is intended, a narrower SE and therefore a larger sample size is used.

Sample size and power calculations for comparing two ROCs is also based on the formula described by Hanley and McNeil (see references). It assumes that θ is a population measurement and approximately normally distributed, so that the difference is Diff=θ12, and the Standard Error of the difference is the root sum square of the individual Standard Errors SEdiff = sqrt(θ1222)

The sample size calculated is for the one tail model. Those requiring a two tail model can use the same calculations but the Type I Error value (α) is halved.

The sample size is for the total number of cases (number of Outcome Positive + number of Outcome Negative) in each of the two ROCs.

Instead of using the z=diff/SEDiff or the 95% confidence interval of the difference to assess statistical significance, the power calculation can also be used to see if the data contains sufficient power to satisfy the user's requirement. Commonly a power in the data exceeding 0.8 for a nominated α of 0.05 is taken as a statistically significant difference.

The table in the next panels provides sample size requirements for 3 levels of power (0.8, 0.9, 0.95), 3 levels of α (0.1, 0.05, 0.01), and difference between two θ values from 0.5 upwards at 0.02 intervals.