FR3077_20171102_omb_B

Survey of Household Economics and Decisionmaking
(FR 3077; OMB No. 7100-NEW)
Panel Selection Methodology
The Board anticipates conducting the SHED annually with as many as 17,000
respondents per survey. The SHED data collection is conducted through a vendor who maintains
an online probability-based Internet panel. An online probability-based Internet panel is defined
here as a panel of voluntary respondents that have been recruited through an address-based
sampling methodology (ABS) using the Delivery Sequence File of the United States Postal
Service technique or other similar technique that would allow for equal probability of selection
into the panel for all potential respondents. There are several reasons that a probability-based
Internet panel was selected as the method for this survey. First, these types of Internet surveys
that employ address based sampling (ABS) (or a similar sampling technique) for recruitment
have been found to be representative of the general population. Second, the ABS Internet panel
allows the same respondents to be re-interviewed in subsequent surveys with relative ease, as the
respondents remain active in the panel for several years. Third, Internet panel surveys have
numerous existing data points on respondents from previously administered surveys, including
detailed demographic and economic information, allowing for the inclusion of additional
information on respondents without increasing respondent burden. Finally, collecting data
through an ABS Internet panel survey is cost-effective and can be done relatively quickly. The
resulting samples would behave as Equal Probability of Selection Method (EPSEM) samples.
The questions in the survey have been designed to better illuminate the activities,
experiences, and attitudes of individual consumers regarding their financial lives and the
financial wellbeing of those in their household. They are intended to complement and augment
the existing base of knowledge from other data sources.
The SHED would be a general population survey such that it would behave as an
EPSEM. The methodology used to develop a respondent universe that behaves as an EPSEM
would start by weighting the respondent universe to detailed geodemographic benchmarks of
adults living in the United States from, for example, the latest March supplement of the Current
Population Survey (CPS). The weights would then be used as the measure of size (MOS) for
each respondent within the respondent universe, allowing for a probability proportional to size
(PPS) procedure to select the 2017 SHED respondent sample, in turn allowing each respondent
to carry a design weight of unity. To accommodate for the oversample of low- or moderateincome respondents, the corresponding design weights would be manually adjusted to correct for
this departure from a representative sample of the general population.
The survey data would be weighted to produce reliable estimates of population
parameters. It is expected that the 2017 SHED would be weighted to compensate for limitations
such as differential nonresponse and undercoverage within the respondent universe.
To further compensate for limitations within the panel when sub-populations vary
considerably, each subpopulation (stratum) would be sampled independently. The strata would

be mutually exclusive (i.e., members must be assigned to only one stratum) and collectively
exhaustive (i.e., no members can be excluded).
Random or systematic sampling would then be applied within each stratum.
Stratification1 would likely improve the representativeness of the sample by reducing sampling
error. It would also likely produce a weighted mean that has less variability than the arithmetic
mean of a simple random sample of the population.
In order to identify the relevant strata for the SHED, the universe of respondents could be
enhanced with various ancillary data (maintained by the vendor) to facilitate a stratification plan.
This ancillary data could allow for a disproportionate stratified sampling methodology across
such strata as:
Stratum 1: Hispanic households with at least one 18 to 24 year-old
Stratum 2: Remaining Hispanic households
Stratum 3: Remaining households with at least one 18-24 year-old
Stratum 4: All remaining households
Initial, follow-up, and survey initiation contact with the sample respondents within the
respondent universe would be conducted by the vendor. The exact form of each of these
contacts would vary somewhat, depending upon vendor preference.
Sample Respondent Contact
Steps
Description
1
Adults from address-based sampled households are invited to join the vendor panel
through a series of mailings or other initial contact methods. Households who
receive the initial contact have the opportunity to respond and join the panel.
2
The subset of addresses that match to a corresponding telephone number and have
not responded to the initial contact (mailing or other form) receive a follow-up phone
call. Households who receive the follow-up contact call have the opportunity to
respond and join the panel.
3
Respondents who join the panel receive surveys through the processes and
technology established by the vendor.
The respondents receive e-mails when there is a survey available for them to take. The
surveys would be posted on a secure website developed and maintained using the vendor’s
proprietary web survey delivery system. The software would easily accommodate different
question formats, including open-ended response fields. It would also allow participants to skip
questions. Development and testing of the web survey would follow well-established,
documented best methods. If a respondent could not be reached through the web, an in person or
phone survey could be conducted.

1

Stratification is the process of grouping members of the population into relatively homogeneous sub-groups before
sampling.

2

These surveys could be qualitative or quantitative in nature. It is expected that the
quantitative and some of the qualitative aspects of this survey would be conducted online.
Qualitative data collected could include questions that are categorical, yes-no, ordinal, and openended. Quantitative data collected could include dollar amounts, percentages, numbers of items,
and other such information pertaining to the financial health of the consumer.
To ensure that the questions are clearly written and would produce accurate and valid
results, the Board would conduct cognitive testing on a subset of the new or revised questions.
Cognitive testing is a well-established qualitative research method intended to identify problems
respondents have with comprehension of survey questions (Willis 2005)2. Efforts would be
made to recruit respondents who are demographically representative of the population being
surveyed.
The Board expects that the respondents would include non-institutionalized individuals3
who are 18 years of age and older with the respondent components including an oversample of
low- or moderate-income individuals (such as households who make $40,000 or less per year), a
sample of re-interviewed respondents from the previous survey, and a fresh, nationally
representative sample of respondents. The oversample of low- or moderate- income respondents
would allow a deeper analysis into segments of the population most likely to experience financial
hardship. The sample of re-interviewed respondents would allow for evaluating changes in
respondent’s economic conditions as well as time series analysis. The vendor could use
incentives such as modest cash deposits, raffles and lotteries with cash, and other prizes to
enhance the completion rate. Because the 2017 SHED would be longer than 15 minutes, the
Board could use additional cash incentives (expected to be between $5 and $10), to be paid
through the same system that the vendor provides its incentives, to enhance completion rates.
Prepaid cash incentives have been found to increase response rates.4
The Board expects to retain all final reports, final survey instruments, and non-restricted
data (without PII) on the public website. Restricted data associated with the final report will be
retained for at least four years; drafts of the final report will be retained for at least two years; the
data collected from the survey instruments will be retained by the third party vendor for at least
six months; recruiting and participant lists will be maintained by the third party vendor who
fielded the instrument; and Contracting Officer’s Technical Representative contract records will
be retained for at least six years after final payment.

2

Willis, G.B. (2005). Cognitive Interviewing: A Tool for Improving Questionnaire Design. Thousand Oaks, CA:
Sage Publications.
3
Non-institutionalized individuals refers to individuals who are not inmates of institutions. This would include
those who are incarcerated, live in a retirement home, a hospital or other medical institution, and active duty
military.
4
Church, A. H. (1993). Estimating the effect of incentives on mail survey response rates: A meta-analysis. Public
Opinion Quarterly, 57, 62-79.
Edwards, P., Roberts, I., Clarke, M., DiGuiseppi, C., Pratap, S., Wentz, R., & Kwan, I. (2002). Increasing response
rates to postal questionnaires: Systematic review. British Medical Journal, 324, 1183-1191.

3