OMB#2-appendix E

APPENDIX E
VARIANCE AND POWER TABLES

In these tables, we assume 80 percent power and various sample and subgroup sizes, and
different assumptions about the impact of weighting and clustering on the variance of estimates
from the child assessments. We assume an intracluster correlation of .05 and, for the change
over time estimates, an average correlation between measures at baseline and age 3 of 0.5. We
also adjust the nominal sample size for design effects due to clustering and unequal weighting
according to the oversampling design, using a stratified variance formula, to yield the effective
sample sizes in the table.
As depicted in Table E.3, at the child level, if we compared normalized assessment scores
(mean of 100, standard deviation of 15) of perinatal cohort children at age 3 for two
approximately equal-sized program-defined subgroups (that is, each having about half the
programs, 45 out of 90, and about half the total sample, or about 184 children), this design would
allow us to detect a minimum difference of 4.4 points with 80 percent power (or an effect size of
.29). Table E.4 shows comparable minimum differences for subgroups defined at the child level,
where all 90 programs would be included. One would use Table E.3 to get sense of what size
differences in program-level variables (for example, home- vs. center-based or average teacher
education level) would need to be observed to be significant predictors of child-level assessment
outcomes in a regression model. Table E.4 gives a sense of what size differences in child-level
variables (for example, attendance rate) would need to be observed to be significant predictors of
child-level assessment outcomes. Classroom-level predictors (for example, classroom quality or
teacher qualifications) would fall somewhere in between.

E.3

TABLE E.1
HALF-CONFIDENCE INTERVALS (95 PERCENT)—CHILD ASSESSMENTS
Half-Confidence Intervals

Time Period

Nominal
Sample Size

Effective Sample
Size (Accounting
for Sample
Design)

Perinatal

Age 1
Age 3

869
509

546
368

.042
.051

1.258
1.533

Age 1

Age 1
Age 3

851
498

547
368

.042
.051

1.257
1.533

Cohort

Proportiona
p = 0.50
Std. Dev. = 0.50

Normalized
Variable
Mean=100
Std. Dev. = 15

Note: Two-sided α = .05.
These values would be used for estimating confidence intervals around descriptive statistics.
a

We show the most conservative situation here—an estimated proportion of 0.5 has the largest variance among all
proportions. Proportions that are higher or lower than 0.5 will have a smaller variance and, therefore, a smaller
margin of error than shown here. The same holds for Table 8. For Tables 6A, 6B, 9A, and 9B, the smaller variance
for other proportions will allow for the detection of smaller differences between subgroups. For Tables 7 and 10, the
smaller variance for other proportions will allow for the detection of smaller changes over time.
TABLE E.2
QUALITY MEASURES HALF-CONFIDENCE INTERVALS (95 PERCENT)

Effective Sample
Size (Accounting
for Sample
Design)

Half-Confidence Intervals
Proportion
p = 0.50
Std. Dev. = 0.50

Quality Variable
Mean = 5
Std. Dev. = 1

Time Period

Nominal
Sample Size

Perinatal

Age 1
Age 3

435
254

310
200

.056
.069

.111
.139

Age 1

Age 1
Age 3

426
249

310
200

.056
.069

.111
.139

Cohort

Note: Two-sided α = .05.
These values would be used for calculating confidence intervals around descriptive statistics.

E.5

TABLE E.3
CHILD ASSESSMENT MINIMUM DETECTABLE DIFFERENCES AND EFFECT SIZES
COMPARING TWO PROGRAM-DEFINED SUBGROUPS AT A POINT-IN-TIME

Effective Sample Sizes

Subgroup 1

Subgroup 2

Proportion
p = .50
Std. Dev. =
0.50

1/2, 1/2
1/3, 2/3
1/2, 1/2
1/3, 2/3

273.0
182.0
184.0
122.7

273.0
264.0
184.0
245.3

.120
.127
.146
.155

3.595
3.813
4.379
4.644

.24
.25
.29
.31

1/2, 1/2
1/3, 2/3
1/2, 1/2
1/3, 2/3

273.5
182.3
184.0
122.7

273.5
364.7
184.0
245.3

.120
.127
.146
.155
.097

3.592
3.809
4.379
4.644

.24
.25
.29
.31

1/2, 1/2
1/3, 2/3
1/2, 1/2
1/3, 2/3

415.0
276.7
303.5
202.3

415.0
553.3
303.5
404.7

2.916
3.093
3.409
3.616

.19
.21
.23
.24

Cohort
Perinatal

Age 1
Age 3

Age 1

Age 1
Age 3

Combined

Age 1
Age 3

Minimum Detectable Differences Between
Subgroups

.103
.114
.121

Normalized
Variable
Mean = 100
Std. Dev. = 15

Effect Size
(ES)

Note: Two-sided α = .05. Power = .80.
An example would be comparing average child cognitive outcomes for children in center-based versus other
program options (most closely represented by the 1/3, 2/3 rows).

E.6

TABLE E.4
CHILD ASSESSMENT MINIMUM DETECTABLE DIFFERENCES AND EFFECT SIZES
COMPARING TWO CHILD-DEFINED SUBGROUPS AT A POINT-IN-TIME, BY COHORT

Effective Sample Sizes

Subgroup 1

Subgroup 2

Proportion
p = .50
Std. Dev. =
0.50

1/2, 1/2
1/3, 2/3
1/2, 1/2
1/3, 2/3

324.4
230.8
206.0
143.0

324.4
407.0
206.0
264.2

.110
.116
.138
.146

3.298
3.461
4.138
4.360

.22
.23
.28
.29

1/2, 1/2
1/3, 2/3
1/2, 1/2
1/3, 2/3

325.1
231.3
206.0
143.0

325.1
407.9
206.0
264.2

.110
.115
.138
.146

3.294
3.457
4.138
4.360

.21
.23
.28
.29

Cohort
Perinatal

Age 1
Age 3

Age 1

Age 1
Age 3

Minimum Detectable Differences Between
Subgroups
Normalized
Variable
Mean = 100
Std. Dev. = 15

Effect
Size (ES)

Note: Two-sided α = .05. Power = .80.
An example would be comparing average child cognitive outcomes for children receiving higher intensity services
to those receiving lower intensity services.

TABLE E.5
CHILD ASSESSMENT MINIMUM DETECTABLE DIFFERENCES AND EFFECT SIZES FOR
COMPARISONS OVER TIME (AGE 1 TO AGE 3)
Effective Sample Size

Minimum Detectable Differences Over Time
Proportion
p = .50
Std. Dev. = 0.50

Normalized
Variable
Mean = 100
Std. Dev. = 15

Effect
Size (ES)

Time 1
(Age 1)

Time 2
(Age 3)

Perinatal

546

368

.077

2.307

.15

Age 1

547

368

.077

2.307

.15

Cohort

Note: Two-sided α = .05. Power = .80.
Assume correlation over time = 0.5.

E.7

TABLE E.6
QUALITY MEASURES MINIMUM DETECTABLE DIFFERENCES AND EFFECT SIZES
COMPARING TWO PROGRAM-DEFINED SUBGROUPS AT A POINT–IN-TIME
Effective Sample Size

Cohort

Time
Period

Perinatal

Age 1
Age 3

Age 1

Age 1
Age 3

Combined

Age 1
Age 3

Minimum Detectable
Differences Between
Subgroups
for a Proportion
p = .50

Minimum
Detectable
Differences Between
Subgroups
for Quality Variable
with Mean = 5 and
Std. Dev. = 1

Subgroups

Subgroup 1

Subgroup 2

1/2, 1/2
1/3, 2/3
1/2, 1/2
1/3, 2/3

155.0
103.3
100.0
66.7

155.0
206.7
100.0
133.3

.160
.169
.199
.211

.318
.337
.396
.420

1/2, 1/2
1/3, 2/3
1/2, 1/2
1/3, 2/3

155.0
103.3
100.0
66.7

155.0
206.7
100.0
133.3

.160
.169
.199
.211

.318
.337
.396
.420

1/2, 1/2
1/3, 2/3
1/2, 1/2
1/3, 2/3

253.0
168.7
175.0
116.7

253.0
337.3
175.0
233.3

.125
.132
.150
.159

.249
.264
.299
.317

Note: Two-sided α = .05. Power = .80.
An example would be comparing average program quality for children in programs with higher average staff
education to those in programs with lower average staff education.

E.8

TABLE E.7
QUALITY MEASURES MINIMUM DETECTABLE DIFFERENCES AND EFFECT SIZES
COMPARING TWO CHILD-DEFINED SUBGROUPS AT A POINT–IN-TIME
Effective Sample Size

Cohort

Time
Period

Perinatal

Age 1
Age 3

Age 1

Age 1
Age 3

Minimum Detectable
Differences Between
Subgroups
for a Proportion
p = .50

Minimum
Detectable
Differences Between
Subgroups
for Quality Variable
with Mean = 5 and
Std. Dev. = 1

Subgroups

Subgroup 1

Subgroup 2

1/2, 1/2
1/3, 2/3
1/2, 1/2
1/3, 2/3

174.7
121.6
107.8
73.8

174.7
223.4
107.8
140.1

.150
.158
.192
.203

.300
.316
.381
.403

1/2, 1/2
1/3, 2/3
1/2, 1/2
1/3, 2/3

174.7
121.6
107.8
73.8

174.7
223.4
107.8
140.1

.150
.158
.192
.203

.300
.316
.381
.403

Note: Two-sided α = .05. Power = .80.
An example would be comparing average program quality for children receiving higher intensity services to those
receiving lower intensity services.

E.9

TABLE E.8
QUALITY MEASURES MINIMUM DETECTABLE DIFFERENCES AND EFFECT SIZES FOR
COMPARISONS OVER TIME (AGE 1 TO AGE 3)
Effective Sample Size
Time 1
(Age 1)

Time 2
(Age 3)

Minimum Detectable Pre-Post
Differences for a Proportion p=.50

Minimum Detectable Pre-Post
Differences for Quality Variable
with Mean = 5 and Std. Dev. = 1

Perinatal

310

200

.105

.209

Age 1

310

200

.105

.209

Cohort

Note: Two-sided α = .05. Power = .80.
Assume correlation over time = 0.5)

E.10