CMS-10441-Medicare_Plan_Finder_OMB_Supporting_Statement

TABLE OF CONTENTS
Introduction ..................................................................................................................................... 1
A.

Justification ...................................................................................................................... 1
A1. Circumstances Making the Collection of Information Necessary ................................ 1
A2. Purpose and Use of the Information Collection .......................................................... 12
A3. Use of Improved Information Technology and Burden Reduction ............................. 12
A4. Efforts to Identify Duplication and Use of Similar Information ................................. 13
A5. Impact on Small Businesses or Other Small Entities .................................................. 13
A6. Consequences of Collecting the Information Less Frequently.................................... 13
A7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5 .......................... 13
A8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside
the Agency .......................................................................................................................... 13
A9. Explanation of Any Payment or Gift to Respondents ................................................. 13
A10. Assurance of Confidentiality Provided to Respondents ............................................ 14
A11. Justification for Sensitive Questions ......................................................................... 14
A12. Estimates of Annualized Burden Hours and Costs.................................................... 14
A13. Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers 16
A14. Annualized Cost to the Federal Government ............................................................ 16
A15. Explanation for Program Changes or Adjustments ................................................... 17
A16. Plans for Tabulation and Publication and Project Time Schedule ............................ 17
A17. Reason(s) Display of OMB Expiration Date is Inappropriate ................................... 17
A18. Exceptions to Certification for Paperwork Reduction Act Submissions................... 18

B.

Collection of Information Employing Statistical Methods ............................................ 18
B1. Respondent Universe and Sampling Methods ............................................................ 18
B2. Procedures for the Collection of Information ............................................................. 19
B3. Methods to Maximize Response Rates and Deal with Nonresponse ......................... 19
B4. Test of Procedures or Methods to be Undertaken ...................................................... 19
B5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or
Analyzing Data ................................................................................................................... 20

References ..................................................................................................................................... 23
List of Attachments: ...................................................................................................................... 25

SUPPORTING STATEMENT
MEDICARE PLAN FINDER EXPERIMENT

Introduction
The Centers for Medicare & Medicaid Services (CMS) request a one year clearance from the
Office of Management and Budget (OMB) under the Paperwork Reduction Act of 1995 for the
Medicare Plan Finder Experiment) Survey. This request for approval takes the OMB control
number XXXX-XXXX.

A. Justification
A1. Circumstances Making the Collection of Information Necessary
The mission of the Centers for Medicare & Medicaid Services (CMS) is to ensure the provision
of health care to its beneficiaries. Recent legislative mandates, including the Medicare
Prescription Drug, Improvement, and Modernization Act of 2003, require CMS to provide
information to beneficiaries about the quality of the Medicare health and prescription drug plans.
To provide that information, all Medicare health and prescription drug plans with an enrollment
of 600 or more are required to collect and report data following protocols that CMS has
established. CMS has also contracted with various organizations to develop valid and reliable
quality measures and to consider how best to report those measures to beneficiaries.
A primary vehicle for reporting quality information to beneficiaries is the Medicare Plan Finder,
a section of the Medicare website that is intended to help beneficiaries make informed choices
among health and prescription drug plans. The Medicare Plan Finder tool contains a great deal
of potentially useful information, including extensive data on the fixed and variable costs
associated with being enrolled in plans, the benefits and coverage that plans offer, and the quality
of service that plans provide, as revealed by member experience data, disenrollment statistics,
and a variety of measures of clinical processes and outcomes.
One of the key challenges that CMS has faced is how to engage beneficiaries with the quality
information provided in the Medicare Plan Finder. Among the possible reasons that beneficiaries
may fail to engage with this information are first, that several steps are required for a user of the
Medicare Plan Finder to gain access to comparative plan information, and second that once the
user does reach a data display, the amount of information presented is voluminous, and can seem
overwhelming.
This study will use an experimental design to assess the effectiveness of two potential
enhancements to the Medicare Plan Finder tool that may help address these barriers to
engagement and use of quality information.

Overview

1

The purpose of this experiment is to test the effects of two prospective enhancements to the
Medicare Plan Finder (MPF) website. We refer to these prospective enhancements as the ―Quick
Links‖ home page and an ―enhanced data display.‖
The Quick Links home page, which was developed as an alternative to the current MPF home
page, is intended to provide a quick overview of all of the most common uses of MPF data, a
succinct explanation of each of those uses, and a direct conduit to comparative data on plans.
The enhanced data display is designed as a more consumer-friendly alternative to the current
data display. In particular, the enhanced data display is meant to make plan data more easily
evaluable and operable, and to draw greater attention to plan quality data Each of these
enhancements is described in greater detail below.
Each of these prospective enhancements will be tested in the context of an experiment conducted
with members of a national online panel. Participants in this experiment will be randomly
assigned to view either pages that are currently on the MPF website or enhanced versions of
these pages. Independently of this assignment to different versions, they will also be randomly
assigned to one of two tasks (browsing among plans available in an area or comparing ―their‖
plan to other plans) and to viewing one of two types of plans (standalone prescription drug plans
or Medicare plans with drug coverage). Participants will use the MPF tool to browse or compare
plans as instructed. They will then complete a questionnaire that measures their understanding of
quality measures and their experience using the MPF tool. We will assess the enhancements by
comparing process and outcome measures of participants’ use of the web site with versus
without the enhancements.
Quick Links Home Page

Before discussing in detail the design of the Quick Links home page, it is necessary to describe
the current MPF home page (to which it is meant as an alternative) and the steps necessary to get
from the current home page to a display containing comparative plan data. The current MPF
home page is dominated by two boxes—stacked one atop of the other—that occupy about threequarters of the screen (see Attachment A). The top box presents the option of a ―general‖ plan
search and the bottom box presents the option of a ―personalized‖ plan search. The main
distinction between these two search modes is that the personalized search requires the user to
enter his or her name, birthdate, Medicare number, ZIP code, and date of enrollment in
Medicare. The advantage of a personalized search is that it automatically retrieves information
(to the extent it is available in the Medicare database) about a beneficiary’s current health plan,
health status, current prescription drugs, and other information that enhance the accuracy of
estimates of the costs and benefits of plans to be considered. In conducting a general search,
users are required to choose their current plan from a list of plans (Step 1 in a 4-step process) and
to indicate (or not) one-by-one the prescription drugs that they currently take (Step 2) and the
pharmacy or pharmacies at which they fill their prescriptions (Step 3; each of the latter two steps
may be skipped, in which case cost estimates are based on averages). The option to enter
prescription drugs and preferred pharmacy information is still provided to beneficiaries who
choose a personalized search, but only as a way to update their drug list or selected pharmacy or
pharmacies. Step 4 of the 4-step process to getting to plan data allows users the option to ―refine
your plan results (see Attachment B).‖ This step involves the use of filters or check boxes to
2

narrow or expand the list of plans that are available in the beneficiary’s area. A likely
disadvantage of this approach is that, at this point in their search, users are not aware of the
meaning of attributes on which they are filtering or of the benefits of applying filters. Moreover,
all of the filters on the ―refine your results page‖ are closed by default, so users are not aware of
the possible values of the plan attributes. When a user is finished filtering, he or she clicks a
button at the bottom of the filtering feature to ―continue to plan results.‖ It is at this point that
the user is shown a data display. On the MPF home page, in addition to the options to conduct a
general or personalized search is a box of additional tools that allow users to ―Find and Compare
Medigap Policies,‖ ―Search by Plan Name or ID,‖ ―Enroll Now,‖ ―Find formularies in your
area‖ or access a ―Medicare Complaint Form (see Attachment A).‖ None of these functions is
explained on the page and whether and how these functions are distinct from the general and
personalized search is not clear.
The style of the Quick Links page mimics the style of the current MPF home page (see
Attachment C for an image of the Quick Links page, see Attachment A for an image of the
current MPF home page). Listed prominently in the center of the Quick Links page are five
pathways related to the MPF. No toolbox is presented on the side of the Quick Links page, as all
of the functions provided by the toolbox are incorporated in the list of pathways. The pathways
include: (1) see what plans are available in my area, (2) compare my current plan to other plans
in my area, (3) find a plan that covers my drugs, (4) enroll in a plan, and (5) find and compare
Medigap (supplement) policies. Clicking on any of these pathway names highlights the pathway
and brings up a callout box that explains the purpose of the pathway and shows the initial steps
required to follow it. The callout box appears to the right of the list of pathways and does not
obscure the list so that choosing another pathway requires nothing more than clicking on another
pathway name (see Attachment C). Experimental participants who are shown the Quick Links
page will be able to click on any pathway and view its callout box; however, the focus of the
experiment is on the initial two pathways, ―See What Plans Are Available in My Area‖ and
―Compare My Current Plan to Other Plans in My Area.‖
The callout box that appears when users click the pathway, ―See What Plans Are Available in
My Area,‖ provides a simple explanation of the purpose served by the pathway: ―Browse a list of
health and prescription drug plans that are available to people in your area. Use filters to narrow
the list by cost and coverage (see Attachment C).‖ Beneath this statement is an action field in
which users are required to enter their ZIP code to indicate the area in which they want to search,
followed by a button that users must click (Find Plans) to begin the process of finding available
plans. The subsequent three steps mimic ones that appear on the MPF site as it is currently
configured (see above). These steps require users to indicate any prescription drugs they
currently take (as a way to customize cost estimates), select their preferred pharmacy from a list
of pharmacies in their area (also a way to customize cost estimates), and then narrow their list of
plan results by specifying (or not) the types of plans they would like to see. Importantly, this
final step (narrowing the list of plan results) will only be seen by participants in conditions that
do not include the enhanced data display. For participants in conditions that include the
enhanced data display, the option to narrow search results will be incorporated into the data
display page (this is explained fully in the subsequent section).

3

The callout box that appears when users click the pathway, ―Compare My Current plan to Other
Plans in My Area,‖ begins with a statement of the primary purpose of the pathway: ―See how the
plan in which you are currently enrolled compares with other plans in your area on cost, benefits,
and quality (see Attachment D).‖ Beneath this statement is an action field in which users are
required to enter their ZIP code to indicate the area in which they would like to search. Users are
also presented with the option (via radio buttons) to either enter their Medicare information
(member ID number, date of birth, and other details) on a secured page (a personalized search,
see above) or to select their current plan manually from a list (a general search, see above).
Because of the advantages to the user of a personalized search, i.e., in requiring fewer steps to
get to the plan data and generating more accurate estimates, we have made this the default
option. In either case (personalized or general search), users are eventually given the option of
narrowing their list of plan results by specifying (or not) the types of plans they would like to see
(see above). Once again, this final step (narrowing the list of plan results) will only be seen by
participants in conditions that do not include the enhanced data display. For participants in
conditions that include the enhanced data display, the option to narrow search results will be
incorporated into the data display page (see subsequent section).
Enhanced Data Display

The enhanced data display is shown in Attachment E. As explained in the Overview, the main
goals of the enhanced data display are to make plan data more easily evaluable and operable (i.e.,
easier to manipulate based on one’s preferences), and to draw greater attention to plan quality
data.
Two key features of the enhanced data display distinguish it from the data display on the current
MPF site, which is shown in Attachment F. First, the enhanced display is designed so that only
essential information appears in the cells of the display (though all of the plan attributes
represented in the current MPF display are also represented in the enhanced display). Some of
the information in the cells of the original display has been moved into the column headers of the
enhanced display. Other information has been omitted entirely. Another way in which we have
simplified the amount of information in a single cell is by providing information on only a one
variable per column. Some of the data columns in the current MPF data display provide
information on multiple plan attributes, which contributes to the overall crowded look of the
display. A second distinguishing feature of the enhanced data display is that it includes a faceted
filtering feature that allows users to see information about the distribution of values for each of
the plan attributes and allows them to shift the content of the data display according to their
preferences. Filters are presented for plan type, monthly premium, overall plan rating, and other
plan features. What filters are shown will depend on the types of plans that a user chooses to
view. Three filters will always be present and ―open‖ by default: plan type, monthly premium,
and overall plan rating. This is meant to call attention to the plan ratings. Use of the filters is
explained briefly at the top of the data display along with other instructions that largely replicate
ones presented on the current MPF plan results page. Other features of the enhanced display that
distinguish it from the current MPF data display include (a) a more prominent feature to select
plans for comparison, (b) a plan sorting feature that is more optimally located on the page (i.e., at
the top of the table rather than embedded in the middle of it), (c) a clearer indication of plan type
(above the plan name), (d) labels for the number of stars and the indicator of low plan
performance in the overall plan quality column, and (e) a link through which a user can view
4

additional details about any one plan. Finally, the enhanced data display includes an option to
collapse or expand the table width-wise so that fewer or more plan attributes are shown. By
default, the collapsed view will be shown. In this view, some of the details of plan coverage and
cost are hidden so that consumers can more quickly scan the table and get a sense for the
available plans. The expanded view (See Attachment G) was included as an option mainly to
accommodate the needs of CMS call-in center representatives and SHIPs counselors who prefer
to have in-depth information about plans available in a single table. Their experience with plan
details and the CMS website prevents them from being overwhelmed by so many details.
Experimental Design

Participants in our experiment will be randomly assigned to one of sixteen conditions in a 2
(Quick Links page: yes, no) x 2 (enhanced data display: yes, no) x 2 (plan type: MAPD, PDP) x
2 (assigned task: browse plans in your area, compare your plan to others) fully factorial design.
With 600 total participants, this means that 37-38 participants will be assigned to each condition.
The sixteen conditions of our experiment are summarized in Exhibit 1 below.
With this design, we will be able to test the effect of the Quick Links page on each of our
outcomes by comparing the responses of participants in Conditions 1-8 (n = 300) with those of
participants in Conditions 9-16 (n = 300). We will also be able to test the effects of the enhanced
data display on each of our outcomes by comparing the responses of participants in Conditions
1-4 and 9-12 (n = 300) with those of participants in Conditions 5-8 and 13-16 (n = 300). This
design will also allow us to test the synergistic effect of the quick links page and the enhanced
data display by testing for an interaction between these two variables. Because the enhanced
data display incorporates the filtering and sorting step that is currently a part of the process of
getting to plan results, it not only has the (intended) effect of making the sorting and filtering
process more transparent but also allows participants to get to the plan data more quickly. Given
that the main purpose of the quick links display is likewise to provide a more direct route to the
plan data, having these two features in place may have effects on consumer experience and
decision-making that are multiplicative rather than additive (i.e., an interactive effect).
Another important question that we will be able to address with this design is whether the
enhanced data display works as well (i.e., produces a similar net benefit beyond the nonenhanced display) for consumers comparing Medicare Advantage plans with drug coverage
(MAPD) as it does for consumers comparing standalone prescription drug plans (PDPs). As
MAPD plans are significantly more complex in the benefits that they provide (and thus require a
more elaborate data display), it is important to determine whether an enhancement benefits
consumers in understanding a data display that portrays MAPD plans. We will address this
question by testing for a 2-way interaction between the type of data display participants see and
the type of plans that they are assigned to compare. We will also be able to test whether any
effects of the quick links page on user experience and decision-making generalizes beyond a
single task by testing for a 2-way interaction between the quick links page and the task to which
participants are assigned (i.e., browse plans in your area or compare your current plan to other
plans in your area). Finally, we will be able to test whether the impact of either of our
enhancements is moderated by patient activation, level of numeracy, past experience with
choosing a Medicare plan, using the Internet to search for health information, or seeing reports
on health care quality.
5

Exhibit 1: Overview of Experimental Design
Condition
Quick
Enhanced
Plan Type
Links Page
Data
Display
1
Yes
Yes
MAPD
2
Yes
Yes
MAPD
3
Yes
Yes
PDP
4
Yes
Yes
PDP
5
Yes
No
MAPD
6
Yes
No
MAPD
7
Yes
No
PDP
8
Yes
No
PDP
9
No
Yes
MAPD
10
No
Yes
MAPD
11
No
Yes
PDP
12
No
Yes
PDP
13
No
No
MAPD
14
No
No
MAPD
15
No
No
PDP
16
No
No
PDP

Assigned Task

Browse plans in your area
Compare your plan to others
Browse plans in your area
Compare your plan to others
Browse plans in your area
Compare your plan to others
Browse plans in your area
Compare your plan to others
Browse plans in your area
Compare your plan to others
Browse plans in your area
Compare your plan to others
Browse plans in your area
Compare your plan to others
Browse plans in your area
Compare your plan to others

Procedures

Participants will begin by completing a brief pre-exposure survey (see Attachment H) in which
they will report on prior experience choosing a Medicare health plan, prior exposure to
comparative quality information on health plans, doctors, or hospitals, use of the internet to seek
health information, health status and health care utilization, and they will complete a scale
measuring patient activation (see Measures section below for more detail). They will then be
given a set of instructions for the plan choice task that follows (see below). After clicking a
button to indicate that they have read and understand these instructions, participants will be
directed from the Knowledge Networks website to a fictitious website, the Medicare Plan
Locator, maintained on RAND’s web server. On this website, they will be asked to compare a
set of plans and make a hypothetical choice for themselves. After choosing a plan, participants
will be directed back to the Knowledge Networks website where they will complete a postexposure survey (see Attachments I-P) in which they will report on their experience with and
reactions to the Medicare Plan Finder and will answer a few questions designed to assess
numeracy (see Measures section below).
Instructions to Participants
All participants will be told that the purpose of the study is to evaluate a new web tool, called the
Medicare Plan Locator, which is being developed for Medicare beneficiaries who are looking to
select a health or prescription drug plan. Participants will be told further that their task is to roleplay a Medicare beneficiary making this type of choice. Half of participants will be told to
imagine that they are a beneficiary who is searching for a Medicare prescription drug plan (i.e., a
6

PDP; Conditions 3, 4, 7, 8, 11, 12, 15, and 16); the other half will be told to imagine that they are
a beneficiary who is searching for a Medicare health plan with drug coverage (i.e., an MAPD
plan; Conditions 1, 2, 5, 6, 9, 10, 13, and 14). Furthermore, half of each of these subsets will be
told to imagine that they are not currently enrolled in a PDP/MAPD but looking to enroll in one
for the first time. These participants will be told that their task is to ―browse plans in their area‖
and choose a plan that seems to get them the best value for their money (not necessarily to
minimize costs). Alternately, participants will be told to imagine that they are currently enrolled
in a plan and that they are considering switching to a new plan that gives them better value for
their money. These participants will be told that their task is to ―compare their current plan to
other options available in their area.‖
Participants told to browse PDP or MAPD plans available in their area (Conditions 1, 3, 5, 7, 9,
11, 13, and 15) will be given a zip code, told that they take two specific drugs and that they do
not have a pharmacy preference. This information, which we need to standardize across
participants, will be necessary in completing the multi-step process of getting to the data.
Participants who see the Quick Links page (Conditions 1, 3, 5, and 7) will presumably click the
link to ―browse plans available in your area.‖ Doing so will provide them with the option of
conducting a general or personalized search (they will be told to choose a general search). After
entering the zip code provided, these participants will then be given the option of entering drug
and pharmacy information (the latter of which they will decline) and then they will be taken
either to the refine your results page (Conditions 5 and 7) or to the enhanced data display
(Conditions 1 and 3, from which they can refine their results). Participants who do not see the
Quick Links page (Conditions 2, 4, 6, and 8) will navigate a replica of the current MPF home
page and go through the same steps as participants in the Quick Links condition (plus the
additional step of indicating that they are not currently enrolled in a plan at Step 1).
Participants told to compare their current PDP or MAPD plan with other options available in
their area (Conditions 2, 4, 6, 8, 10, 12, 14, and 16) will be given a zip code, told that they
belong to a specific plan, take two specific drugs, and that they have no pharmacy preference.
Participants who see the Quick Links page (Conditions 2, 4, 6, and 8) will presumably choose
the option to ―compare my current plan to other plans in my area). They will then be required to
find their assigned plan in a list of fictitious plan names, after which they will enter their drugs,
decline to enter pharmacy information, and be taken either to the refine your results page
(Conditions 6 and 8) or to the enhanced data display (Conditions 2 and 4, from which they can
refine their results).
Plan Data
Participants in the MAPD conditions will be provided with data on 20 plans; participants in the
PDP conditions will be provided with data on 30 plans. Respectively, these numbers represent
the average number of MAPD plans available to Medicare beneficiaries in 2011 (Gold,
Jacobson, Damico, & Neuman, 2011), and the median number of PDP plans that were available
in a given state in 2011 (KFF, 2011). Aside from increasing the realism of the task, presenting
20+ plans makes using the sorting and filtering options appealing (or at least relevant). All
participants who are tasked with considering MAPD plans will see the same 20 plans (though the
order of the plans will be randomized), and the data associated with each of these plans will be
7

constant across participants. Likewise, all participants who are tasked with considering PDPs
will see the same 30 plans (in random order), and the data associated with each plan will be
constant. MAPD plan attributes will include information about plan type, total estimated annual
costs, monthly premium, annual deductible, the extent of doctor choice offered by the plan,
whether all of the beneficiary’s drugs are included on the plan’s formulary, pharmacy status
(whether the pharmacy selected by the user falls in the plan’s network), and an overall plan
quality rating. In the expanded view, this list of MAPD attributes will also include information
about drug co-payment and co-insurance amounts, limits on out-of-pocket spending, information
about drug restrictions, and information about gap coverage (if any) provided by the plan. PDP
plan attributes will include all of these same variables except doctor choice. Moreover, the
columns for monthly premium and annual deductible will only include information pertaining to
prescription drugs, whereas for MAPD plans these columns will include information about both
the health plan and prescription drug portions of the plan.
The data presented in the displays will be based on actual data on plans from a single mid-sized
market area. The only exception to using real data is that we will adjust plan ratings as necessary
to ensure that there are among the alternatives some plans that are more expensive and have
lower quality ratings and other plans that are less expensive and have higher quality ratings.
Having these plans among the alternatives will allow us to evaluate the degree to which our two
enhancements increase the likelihood that participants will pay attention to and use plan quality
data.
Measures

Unless otherwise noted, the measures described in this section will be included in the survey that
participants complete after visiting the Plan Finder website and choosing a plan (also referred to
as the post-exposure survey). Attachment Q explains how the measures described in this section
map onto the items in the pre- and post-exposure surveys.
Process Variables
Perceived ease of understanding information on the website. We will ask participants to rate how
easy or difficult it was for them to understand the information that was provided about the health
(prescription drug) plans (1 = very difficult to 5 = very easy). This measure is based on a similar
one reported by Greene and colleagues (2008) that was sensitive to different ways of presenting
comparative quality data on health plans.
Amount of information acquired. Consistent with Hanoch et al. (2011), we will use the web
tracking data to compute a measure of amount of information required that is based on the
number of times a participant drills down for detailed data and the number of times the
participant clicks on the names of plan attributes for information about the attribute. In Hanoch
et al. (2011), this measure was sensitive to the experimental manipulation of choice-set size in a
study of drug plan choice. In particular, increasing choice-set size was associated with
consideration of less of the available information on a website that presented comparative quality
data on Medicare prescription drug plans.

8

Emotional reaction to the decision-making task. Based on a measure by Mikels et al. (2010), we
will ask participants to indicate the extent to which choosing a plan made them feel each of
seven positive emotions (e.g., relaxed, calm, certain, and capable) and each of seven negative
emotions (e.g., confused, overwhelmed, doubtful, and frustrated). We anticipate combining the
ratings of the positive emotions into a one scale and the ratings of the negative emotions into a
second scale. In the study by Mikels and colleagues (2010), this measure was sensitive to the
experimental manipulation of the framing of a health care choice task.
Salience of decision attributes. To measure the salience of each plan attribute that was included
in the data display, we will show participants a list of these attributes and ask participants if they
recall seeing the information (yes, no). This measure is based on one developed for our AHRQfunded SelectMD online experiment.
Perceived usefulness of decision attributes. For those plan attributes that participants recall
seeing, we will ask them to rate how easy it was distinguish among plans on the basis of the
attribute (1 = very difficult to 4 = very easy). We will create an index by averaging across the
attributes they remember. This measure is based on one developed for our AHRQ-funded
SelectMD online experiment.
Comprehension of data display. As part of the post-exposure survey, we will show participants a
picture of the data display that they saw on the website to which they were exposed (i.e., either
the enhanced data display or the current MPF data display) and ask them to use the data display
to answer three relatively simple comprehension questions (e.g., ―Which plan has the lowest
monthly premium for prescription drug benefits?‖) and three more difficult comprehension
questions (e.g., ―If you wanted a plan that is above average quality, allows you to see doctors
outside of your plan, and would cost you no more than $2,500 per year, which plan would you
choose?‖). Answering a simple comprehension questions requires that the participant look at
one plan attribute only. Answering a complex comprehension question requires that the
participant consult information about two or more plan attributes. This measure is based on one
reported in Hibbard et al. (2007); in that study, comprehension was strongly related to the quality
of decision-making by consumers who were asked to read a simulated hospital quality report.
Outcome Variables
Overall evaluation of website. Participants will report their overall evaluation of the website by
completing the following two items that were developed for our AHRQ-funded SelectMD online
experiment: ―If you could have free access to a website like this one when you need to choose a
health plan (prescription drug plan) in real life, how likely would you be to use this website (1 =
Definitely would not use to 5 = Definitely would use),‖ and ―Would you recommend that your
friends and family use a website like this one when they make their own choices about a health
(prescription drug) plan (1 = Definitely not recommend to 5 = Definitely recommend)?‖
Evaluation of home page (Quick Links or current MPF home page). As part of the post-exposure
survey, we will show participants a picture of the home page of the website that they visited (i.e.,
either the Quick Links page or the current MPF home page) and ask them two questions about
the page: ―Thinking about when you first saw this web page, how good an idea did the page give
9

you about what information was available on the Plan Finder website,‖ and ―How goo an idea
did this page give you about what you might be able to do on the website?‖ The end points of
the 4-point response scale for each of these items are labeled ―no idea at all‖ and ―a very clear
idea.‖
Elapsed time to get to a data display. As a measure of how easily participants are able to
navigate the website, we will use the web tracking data to create a measure of the time (in
seconds) elapsed between when participants first access the website and when they first reach a
data display.
Perceived usability and navigability of the website. Participants will rate the ease of navigability
of the website by responding to the question, ―How easy or difficult was it for you to find
information that you were looking for (1 = very difficult to 5 = very easy).‖ Participants will also
rate the overall usability of the website by responding to the question, ―How easy or difficult was
it for you to use the website in general (1 = very difficult to 5 = very easy).‖
Proactive engagement with decision task. We will use the web tracking software to compute
measures of whether and how often participants used the sorting, filtering, and plan comparison
features on the website. We will also compute a measure of the total amount of time participants
spent on the website. We will analyze these measures separately and combined (provided
adequate correlation among them) as indices of participants’ engagement with the decision task.
Perceived ease of decision-making. Participants will use a 5-point scale (1 = very difficult to 5 =
very easy) to rate how easy or difficult it was to choose a plan. This measure is based on one
reported in Tanius et al. (2009); in that study, perceived ease of decision-making in choosing
among drug plans was sensitive to the experimental manipulation of the size of the choice set. In
particular, increasing choice-set size was associated with greater perceived difficulty in choosing
a Medicare prescription drug plan.
Post-decision confidence. Participants will use a 5-point scale (1 = not at all confident to 5 =
extremely confident) to rate how confident they are about their choice of plan. This measure is
based on one reported in Uhrig et al. (2006); in that study, this measure was sensitive to the
experimental manipulation of the content and formatting of a report on health plan quality.
Quality of decision. Among the plans from which participants will be asked to choose, there will
be one superior plan (in higher quality and more generous benefits) within each stratum of
estimated annual costs. Consistent with Hibbard et al. (2005), we will consider participants to
have made a ―quality choice‖ if they choose the best (highest quality, most generous benefits)
plan within any cost stratum.
Perceived value of information on the website. We will ask participants to rate their satisfaction
(1 = very dissatisfied to 5 = very satisfied) with the information provided about each plan. Uhrig
et al. (2006) found that this measure was sensitive to an experimental manipulation of content
and formatting of a report on health plan quality. We will also ask participants to indicate how
they feel about the amount of information provided about each plan (with response options of not
enough information, about the right amount of information for me to handle, and more
10

information than I could handle). This measure is based on one reported in Sainfort and Booske
(2000); in that field study, individuals who perceived that the amount of information provided in
a health plan quality report was ―about the right amount of information‖ (vs. too much or too
little) were significantly more likely to be satisfied with their choice of a health plan and to feel
that the information in the report was useful and adequate for their decision needs.
Average importance of attributes. For each plan attribute that participants recall seeing, we will
ask them to rate how much they relied upon that attribute in making their choice of a plan (1 =
not at all to 4 = a great deal). We will use this information to create an overall index (averaging
across all items) of the perceived importance of the attributes for decision-making. Participants
who do not recall seeing information about a specific attribute will be assigned a score of 1 (not
at all) on that attribute.
Reliance on quality information. From the ratings of reliance on plan attributes described above,
we will create a measure of the importance of overall quality in choosing a plan. Participants
who do not recall seeing information about overall quality will be assigned a score of 1 (not at
all).
Hypothetical Moderating Variables
Experience using the Internet for health or medical information. As part of the pre-exposure
survey, participants will complete a four-item measure of use of the Internet for information
about health or medical care (e.g., ―In the past 12 months, how often did you use the Internet to
look for health or medical information for yourself or someone else?‖). These items are from the
National Cancer Institute’s Health Information National Trends Survey (the measure can be
accessed at http://hints.cancer.gov/topics.aspx?section=Internet+Use).
Experience with health care quality reports. As part of the pre-exposure survey, participants will
also indicate whether in the past 12 months they have seen information comparing the quality of
different doctors, hospitals, or health plans. This measure was developed for our AHRQ-funded
SelectMD online experiment.
Experience choosing a Medicare health or prescription drug plan. As part of the pre-exposure
survey, participants will indicate whether they have ever had to choose a Medicare health or
prescription drug plan for themselves or another person.
Patient activation. As part of the pre-exposure survey, participants will complete the ―Believes
active role important (2 items)‖ and ―Confidence and knowledge to take action (4 items)‖
subscales of the Short Form Patient Activation Measures (PAM). Two other facets of patient
activation that are measured by this scale but that we do not plan to include in our study are
taking action to address a health problem (3 items) and maintaining action when one’s health
condition or psychological state worsens (4 items). We do not plan to include these subscales
because the items in them presume the existence of a health problem and thus will not be
relevant for some of the participants in our study. Alpha reliability for this scale is reported to be
above 0.80 (Hibbard, Mahoney, Stockard, & Tusler, 2005). As part of the pre-exposure survey,
participants will also complete a 2-item measure that was included in the 2007 Medicare CAHPS
11

survey to assess patient activation status. These two items are: ―How confident are you that you
can identify when it is necessary for you to get medical care (not at all confident, somewhat
confident, confident, or very confident)‖ and ―How frequently do you bring to your doctor visits
a list of questions or concerns you want to cover (never, sometimes, usually, or always)?‖
Information about the psychometric properties of this scale is included in Williams and Heller
(2007). If the two-item measure is highly correlated with the 6 items from the PAM, we will
combine these 8 items to form a single scale.
Numeracy. Participants will complete a 7-item risk numeracy scale by Lipkus and colleagues
(e.g., ―If Person A’s risk of getting a disease is 1% in ten years, and Person B’s risk is double
that of A’s, what is Person B’s risk?‖). Alpha reliability for this scale has been reported to be
between 0.70 and 0.75 (Lipkus, Samsa, and Rimer, 2001).
Control Variables
Demographics. Demographic measures collected by Knowledge Networks include age, sex,
race/ethnicity, and level of education. Measures of race and ethnicity will conform to the data
standards required by Section 4302 of the Affordable Care Act.
Self-rated health status. As part of the pre-exposure survey, participants will rate their overall
health status using a single-item measure from the SF-36 Health Survey (Ware & Sherbourne,
1992).
Chronic health condition. Knowledge Networks collects extensive data from members of its
Knowledge Panel on chronic health conditions (e.g., high blood pressure, asthma, diabetes,
chronic pain, kidney disease, heart disease, osteoarthritis, and rheumatoid arthritis). We will use
this information to create a single indicator of whether or not a participant has a chronic health
condition.
Level of use of health care system. Also as part of the pre-exposure survey, participants will
report the number of times in the past 12 months that they went to a doctor’s office or clinic to
get health care for themselves.
A2. Purpose and Use of the Information Collection
The results of this study will be used to develop recommendations to CMS for enhancements to
the Medicare Plan Finder tool on the Medicare.gov website that will help consumers better
understand and more effectively use the information on the site to select health plans. The aim is
to make the tool and the information it contains more accessible, useful, and transparent to the
public. The information collected will therefore serve functions related both to program
evaluation and quality improvement.
A3. Use of Improved Information Technology and Burden Reduction
Participants will complete the experiment through a secure online connection from their homes.
Survey data are collected by a web-based survey system (internally referred to as ―Dimensions‖).
12

This application runs on top of a secured Windows environment that has been hardened through
various network and hosted-based security techniques. Participants take online surveys by using
a web-browser to access a unique, secured web URL that is both emailed to them and made
available through a secured web-portal. The URL provides access to click through to a highlyavailable load-balanced farm of web servers that hosts the online survey. This survey URL can
be exposed via either standard http or over SSL and TLS encrypted https, depending on the client
requirements. Throughout the interview process, questionnaire data are copied to a secured,
centralized database for data processing.
Collection of information from respondents through online connection from their homes reduces
the ancillary burden associated with participating at a study site, since no time is required for
travel. Respondents are also able to schedule their participation at their own convenience.
A4. Efforts to Identify Duplication and Use of Similar Information
Work carried out under this clearance has been designed to reflect specific needs of the
population for which this work is being conducted and will not duplicate any other work being
done by CMS or other Federal agencies.
A5. Impact on Small Businesses or Other Small Entities
Respondents are seniors of Medicare age and caregivers who assist seniors of Medicare age with
their health care decisions. The study will not have a significant impact on small businesses or
other small entities.
A6. Consequences of Collecting the Information Less Frequently
This is a one-time information collection. The consequences of not collecting this information at
all would be that CMS would be in the position of needing to make decisions about changes to
the Medicare.gov website and Medicare Plan Finder tool in the absence of the valuable evidence
that this experiment will provide on how enhancements that are contemplated for the website are
likely to affect user experience.
A7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
This request is consistent with the general information collection guidelines of 5 CFR
1320.5(d)(2). No special circumstances apply.
A8. Comments in Response to the Federal Register Notice and Efforts to Consult
Outside the Agency
The Federal Register notice was published on date X page Y.
A9. Explanation of Any Payment or Gift to Respondents
Consistent with Knowledge Networks’ practice in projects that require more than 20 minutes of
participants’ time, panel members who participate in this experiment will receive 5,000 points as
13

an incentive payment. This is the equivalent of $5. Points are redeemed in the form of a check to
the panel member at a later date.
A10. Assurance of Confidentiality Provided to Respondents
Individuals contacted as part of this data collection will be assured of the confidentiality of their
replies under 42 U.S.C. 1306, 20 CFR 401 and 422, 5 U.S.C. 552 (Freedom of Information Act),
5 U.S.C. 552a (Privacy Act of 1974), and OMB Circular A-130.
A11. Justification for Sensitive Questions
The survey does not include any questions of a sensitive nature.
A12. Estimates of Annualized Burden Hours and Costs
Exhibit 2 shows the estimated annualized burden hours for the respondents' time to participate in
this experiment. The entire experiment (including the design phase) will not exceed two years.
All participants will complete the pre-exposure questionnaire, which is estimated to require 3.8
minutes (assuming an average response pace of 5 items per minute). As explained above, the
experimental website varies by experimental condition; however, based on preliminary testing,
each participant will require about 10 minutes to review the information on the site. Exhibit 2
provides an average time required to complete the post-exposure questionnaires (estimated at
11.4 minutes assuming an average response pace of 5 items per minute). The total burden hours
are estimated to be 252.2 hours.

14

Exhibit 2. Estimated annualized burden hours
Number of
Total
Number of
responses Hours per
Surveys/Response Tasks
Burden
Respondents
per
response
hours
respondent
Pre-exposure survey
600
1
0.06333
38.0
Experimental Website
600
1
0.167
100.2
Condition #1 Post-exposure
38
1
0.19
7.22
Condition #2 Post-exposure
37
1
0.19
7.03
Condition #3 Post-exposure
38
1
0.19
7.22
Condition #4 Post-exposure
37
1
0.19
7.03
Condition #5 Post-exposure
38
1
0.19
7.22
Condition #6 Post-exposure
38
1
0.19
7.22
Condition #7 Post-exposure
37
1
0.19
7.03
Condition #8 Post-exposure
37
1
0.19
7.03
Condition #9 Post-exposure
38
1
0.19
7.22
Condition #10 Post-exposure
37
1
0.19
7.03
Condition #11 Post-exposure
38
1
0.19
7.22
Condition #12 Post-exposure
37
1
0.19
7.03
Condition #13 Post-exposure
37
1
0.19
7.03
Condition #14 Post-exposure
38
1
0.19
7.22
Condition #15 Post-exposure
37
1
0.19
7.03
Condition #16 Post-exposure
38
1
0.19
7.22
Total
600
1
0.420*
252.2
* We estimate the total response hours per individual to be 0.420 hours (25.2 minutes) regardless
of which study condition the individual is assigned to.

15

Exhibit 3 shows the respondents' cost burden for their time to participate in this experiment. The
total cost burden is estimated to be $5,382.

Exhibit 3. Estimated annualized cost burden
Surveys/Response Tasks

Number of
respondents

Total
burden
hours

Average
hourly wage
rate*

Total cost
burden

Pre-exposure survey
600
38.0
$21.35
$811
Experimental Website
600
100.2
$21.35
$2,139
Condition #1 Post-exposure
38
7.22
$21.35
$154
Condition #2 Post-exposure
37
7.03
$21.35
$150
Condition #3 Post-exposure
38
7.22
$21.35
$154
Condition #4 Post-exposure
37
7.03
$21.35
$150
Condition #5 Post-exposure
38
7.22
$21.35
$154
Condition #6 Post-exposure
38
7.22
$21.35
$154
Condition #7 Post-exposure
37
7.03
$21.35
$150
Condition #8 Post-exposure
37
7.03
$21.35
$150
Condition #9 Post-exposure
38
7.22
$21.35
$154
Condition #10 Post-exposure
37
7.03
$21.35
$150
Condition #11 Post-exposure
38
7.22
$21.35
$154
Condition #12 Post-exposure
37
7.03
$21.35
$150
Condition #13 Post-exposure
37
7.03
$21.35
$150
Condition #14 Post-exposure
38
7.22
$21.35
$154
Condition #15 Post-exposure
37
7.03
$21.35
$150
Condition #16 Post-exposure
38
7.22
$21.35
$154
$5,382
Total
600
252.2
na
*Based upon the mean of the average wages, ―May 2010 National Occupational Employment and Wage Estimates,
United States‖ U.S. Department of Labor, Bureau of Labor Statistics.

A13. Estimates of Other Total Annual Cost Burden to Respondents and Record
Keepers
There are no direct costs to respondents other than their time to participate in the study.
Capital and maintenance costs include the purchase of equipment, computers or computer
software or services, or storage facilities for records as a result of complying with this data
collection. There are no capital or maintenance costs for this study.
A14. Annualized Cost to the Federal Government
Exhibit 4 shows the total and annualized cost for developing and conducting the experimental
study, including the cost of designing the experiment, developing the Quick Links and enhanced
data display web pages, developing a simulated Plan Finder site, conducting usability testing of
the new Web-pages, pilot testing the experiment, collecting the data, analyzing the data, and
preparing reports to CMS and papers for journal submission. The total and annual costs are
16

identical since data collection will not exceed one year. The total cost is estimated to be
$352,000.
Exhibit 4. Total and Annualized Costs

Cost Components
Experimental design
Development of enhanced pages for testing
Development of simulated Plan Finder site
Pilot testing
Usability testing of Web pages
Data collection via Knowledge Networks
Data analysis
Preparation of reports and journal articles
Total

Total
Cost
($1,000s)
$104
$45
$45
$33
$15
$50
$75
$15
$352

Annual
Cost
($1,000s)
$104
$45
$45
$33
$15
$50
$75
$15
$352

A15. Explanation for Program Changes or Adjustments
There are no program changes or adjustments.
A16. Plans for Tabulation and Publication and Project Time Schedule
We will produce two types of reports on findings from this experiment. First, we will prepare a
report to CMS that recommends changes to the Medicare.gov website, based on results of the
experiment along with results of a literature review that has informed the design of this
experiment. Second, we will prepare one or more reports in a form suitable for publication in
peer-reviewed journals,
Exhibit 5 details the timeline for sample selection, data collection, data analysis, and delivery of
analytic reports.
Exhibit 5. Timeline
Task
Sample selection
Data collection
Data analysis
Prepare and submit data
analysis report

Planned Start Date
OMB approval
11 days after OMB
approval
45 days after OMB
approval
60 days after OMB
approval

Planned End Date
10 days after OMB
approval
41 days after OMB
approval
157 days after OMB
approval
187 days after OMB
approval

A17. Reason(s) Display of OMB Expiration Date is Inappropriate
17

The OMB expiration date will be displayed on the online survey.
A18. Exceptions to Certification for Paperwork Reduction Act Submissions
There are no exceptions to the certification statement identified in item 19 of OMB Form 83-I
associated with this data collection effort.

B. Collection of Information Employing Statistical Methods
B1. Respondent Universe and Sampling Methods
As noted in Section A1, the Medicare Plan Finder is intended to help Medicare beneficiaries
make informed choices among health plans. Medicare beneficiaries themselves are therefore a
primary target audience for the Medicare.gov website and the Medicare Plan Finder in particular.
However, many beneficiaries also receive help through family members who visit the web site
and obtain information on their behalf. Many others receive help through trained intermediaries
such as staff at call centers and State Health Insurance Assistance Program (SHIP) counselors.
These professional intermediaries make extensive use of the Medicare Plan Finder in working
with beneficiaries. However, intermediaries’ familiarity with the tool and with the information it
contains makes them sophisticated users whose experiences with and challenges in using the tool
are likely to be quite different from those of lay users. The enhancements to be tested in this
experiment are designed with lay users the Medicare Plan Finder in mind. The target population
for the experiment is therefore Medicare beneficiaries and adult ―caregivers‖ of Medicare
beneficiaries who assist beneficiaries with their decisions about health insurance.
Respondents for this experimental study (n = 600) will be drawn from the Knowledge Networks
Internet panel. This web-enabled panel, recruited from an address-list sampling frame that covers
97% of U.S. households, represents the broad diversity and key demographic dimensions of the
U.S. population. The panel closely tracks the U.S. population on age, race/ethnicity, geography,
employment, and other demographics, with the small differences further reduced by nonresponse adjustment. The Knowledge Networks panel (~ 50,000 adults, 16% age 65 and over) is
large and diverse enough to provide a geographically, racially, ethnically, and educationally
diverse sample of seniors of Medicare age and people who assist them for this experiment. Use
of an existing panel substantially lowers recruitment costs relative to a sample recruited through
a focus group recruiting firm, allowing larger sample sizes for a given budget. Kanouse and
Martino have recently conducted a successful web-based physician choice experiment using a
sample recruited from the Knowledge Networks Panel.
Half of the participants recruited for this experiment will be seniors of Medicare age (65 and
older) and half will be ―caregivers‖ of seniors of Medicare age (defined as adults who have
helped someone of Medicare age with decisions about health insurance or who have retrieved
information via paper, phone, or the web about health insurance for someone of Medicare age).
We have defined all seniors of Medicare age as eligible to participate regardless of whether they
are enrolled as Medicare beneficiaries, and we will restrict screening of younger adults for
eligibility as caregivers to those aged 35 to 59. We estimate an eligibility rate of about 20 percent
for those in this 25-year age bracket.
18

To assure a balance of seniors of Medicare age and caregivers in each condition, random
assignment to experimental condition will be made within strata.
The Knowledge Networks Panel is constructed to include those who do not otherwise have
internet access (by providing them with a free netbook computer and Internet service in return
for their participation on the panel).
We do not intend to generate nationally or locally representative results or precise estimates of
population parameters from this study. The sample used is best understood as a convenience
sample, rather than a probability sample. The Knowledge Networks panel is large and variegated
enough to produce samples with a reasonable degree of diversity in key demographic
characteristics. Furthermore, no legitimate weights can be constructed from non-probability
samples such as the one used here. Hence, we will not in any publications emerging from this
work construe this sample or the results generated from this sample as nationally or locally
representative. The strength of the experimental study lies in its internal validity, on which
meaningful estimates of differences across the experimental exposures (type of task, type of
plan, and presence or absence of enhancements) can be produced and generalized.
B2. Procedures for the Collection of Information
Study participants will be randomly assigned to one of 16 conditions (see Part A Section 1 for a
description of the conditions) that vary according to the task the respondent is assigned, the type
of plan he or she will be reviewing (MA-PD or PDP), and whether the Plan Finder includes each
of two prospective enhancements. Participants will complete the experiment through a secure
online connection from their homes. Data will be derived from pre and post-test questionnaires
and from server logs that record the web pages visited and viewing times.
B3. Methods to Maximize Response Rates and Deal with Nonresponse
The response rate is estimated at about 75% based on results obtained from the past projects
conducted by KN. Procedures for maximizing response rates include:





Field period of 3 to 4 weeks
Use of the Federal agency name in the email invitation
Email reminders
Telephone reminder calls to non-responders

The initial analysis of response rate of 75% or better indicates that this response rate, in
combination with the size of the population selected for each experiment (described the section
titled Respondent Universe and Sampling Methods), will provide sufficient power to test for the
experimental differences.
B4. Test of Procedures or Methods to be Undertaken

19

This experiment has a multifactorial design that is fully crossed, such that experimental cells take
on every possible combination of the four manipulated factors. This allows estimation of both
the main effects of each factor on outcome variables and interaction effects.
We will use regression analysis in an analysis of variance (ANOVA) style to assess the effects of
main effects and interactions. Some outcome variables we will examine will be dichotomous
(e.g., whether the respondent remembers seeing a particular type of information on the web site)
while others will be ordinal scales that can be treated as continuous (e.g., reported ease or
difficulty of understanding the information). We will use logistic regression for dichotomous
outcomes and ordinary least squares (OLS) regression for outcomes treated as continuous, and
will consider ordered logistic regression and multinomial logistic regression if diagnostics and
distributions suggest they are the best models.
Our primary interest is in the main effects associated with each of the two enhancements.
However, we hypothesize that the two enhancements will have synergistic effects on outcomes
related to ease of use, understanding of content, and quality of the decision. This hypothesis will
be tested with a planned contrast for the two-way interaction between the two factors
representing the enhancements on the outcomes that should be affected. For other interactions
(five other two-way interactions, four three-way interactions, and one four-way interaction), we
will conduct omnibus tests of the joint incremental predictive effect of these interactions
compared with lower order effects using partial-F tests and Wald test for OLS and logistic
regression forms (also examining the corresponding changes in R2 and c-statistic, respectively).
Should an omnibus test be statistically significant, we will follow-up with contrasts to identify
the specific interaction(s) contributing to this effect. Significant two-way interactions involving
enhancements will be reported as main effects of the enhancement within each level of the
interacting variable. For example, we might report the main effect of the enhanced display for
those subjects instructed to browse among available plans and for those subjects instructed to
compare their plan with other available plans.
For main effects on continuous outcomes, given a sample size of 600, we will have 80% power
to detect a small to medium effect of 0.23 SD; for two-way interactions, the minimum detectable
effect size is 0.47 SD when all four cells are of equal size, as will be the case for the planned
interaction of greatest interest. For a dichotomous outcome with prevalence of 10-90%, the
minimum detectable main effect corresponds to a difference of approximately 6.9-11.5
percentage points and the interaction to a difference-of-differences of approximately 14.1-23.5
percentage points, so that continuous outcomes will be most useful in assessing interactions.
We will conduct the analyses described above on the entire sample, pooling seniors of Medicare
age and caregivers. However, we will conduct additional exploratory analyses examining both
main effects and interactions involving the type of respondent (senior vs. caregiver) as well as
other individual difference variables measured in the pre- or post-exposure surveys (e.g.,
experience choosing a Medicare health plan, prior exposure to quality information, patient
activation, numeracy).
B5. Individuals Consulted on Statistical Aspects and Individuals Collecting
and/or Analyzing Data
20

The survey, sampling approach, and data collection procedures were designed by the RAND
Corporation under the leadership of:
Marc Elliott, Ph.D.
Senior Statistician
RAND Corporation
1776 Main Street
PO Box 2138
Santa Monica, CA 90407-2138
310-393-0411
Dr. Elliott is a statistician and has provided oversight on statistical aspects.
Data will be collected by Knowledge Networks under the direction of:
Jordan Peugh
VP, Health Care and Policy Research
Government and Academic Research
Knowledge Networks
440 Park Avenue South, 6th Floor
New York, New York 10016
646-742-5334
jpeugh@knowledgenetworks.com

21

The following individuals will analyze data:
Steven C. Martino, Ph.D.
Senior Behavioral Scientist
RAND Corporation
4570 Fifth Avenue, Suite 600
Pittsburgh, PA 15213-2665
David E. Kanouse, Ph.D.
Senior Behavioral Scientist
RAND Corporation
1776 Main Street. P.O. Box 2138
Santa Monica, CA 90407-2138

22

References
Gold M, Jacobson G, Damico A, Neuman T. Medicare Advantage 2012 spotlight: Plan
availability and premiums. Menlo Park, CA: Henry J. Kaiser Family foundation; November
2011.
Greene, J., Peters, E., Mertz, C. K., & Hibbard, J. H. (2008). Comprehension and choice of a
consumer-directed health plan: An experimental study. American Journal of Managed Care,
14(6), 369-a379.
Hanoch, Y., Wood, S., Barnes, A., Liu, P.J., & Rice, T. (2011). Choosing the right medicare
prescription drug plan: The effect of age, strategy selection, and choice set size. Health
Psychology, 30(6), 719-727.
Hibbard, J. H., Mahoney, E. R., Stockard, J., & Tusler, M. (2005). Development and testing of a
short form of the patient activation measure. Health Services Research, 40, 1918-1930.
Hibbard, J. H., Stockard, J., & Tusler, M. (2005). It isn't just about choice: The potential of a
public performance report to affect the public image of hospitals. Medical Care Research &
Review, 62, 358-371.
Hibbard, J. H., Peters, E., Dixon, A., & Tusler, M. (2007). Consumer competencies and the use
of comparative quality information. It isn’t just about literacy. Medical Care Research and
Review, 64, 379-394.
Kaiser Family Foundation. The Medicare prescription drug benefit (fact sheet). Menlo Park, CA:
Henry J. Kaiser Family foundation; November 2011.
Lipkus, I. M., Samsa, G., & Rimer, B. K. (2001). General performance on a numeracy scale
among highly educated samples. Medical Decision Making, 21, 37-44.
Mikels, J. A., Lockenhoff, C. E., Maglio, S. J., Goldstein, M. K., Garber, A., & Carstensen, L. L.
(2010). Following your heart or your head: Focusing on emotions versus information
differentially influences the decisions of younger and older adults. Journal of Experimental
Psychology: Applied, 16, 87-95.
Sainfort, F., & Booske, B. C. (1996). Role of information in consumer selection of health plans.
Health Care Financing Review, 18(1), 31-54.
Tanius, B. E., Wood, S., Hanoch, Y., & Rice, T. (2009). Aging and choice: Applications to
Medicare Part D. Judgment and Decision Making, 4(1), 92-101.
Uhrig, J. D., Bann, C. M., McCormack, L. A., & Rudolph, N. (2006). Beneficiary knowledge of
original medicare and medicare managed care. Medical Care, 44(11), 1020-1029.

23

Ware, J. E., Jr., Sherbourne, C. D. (1992). The MOS 36-Item Short Form Health Survey (SF 36).
1. Conceptual framework and item selection. Medical Care, 30, 473-483.
Williams, S. S., & Heller, A. (2007). Patient activation among Medicare beneficiaries:
Segmentation to promote informed health care decision-making. International Journal of
Pharmaceutical and Healthcare Marketing, 1, 199 –213.

24

List of Attachments:
A.
B.
C.
D.
E.
F.
G.
H.
I.
J.
K.
L.
M.
N.
O.
P.
Q.

25

Screen shot of current MPF home page
Screen shot of Step 4 (Refine Your Plan Results) from the current MPF plan search process
Representation of the Quick Links home page
Quick Links page with ―compare my current plan to other plans in my area‖ path activated
Representation of the enhanced data display, collapsed view
Screen shot of the current data display on the MPF site
Representation of the enhanced data display, expanded view
Pre-exposure survey
Post-exposure survey, experimental conditions #1 and #2
Post-exposure survey, experimental conditions #3 and #4
Post-exposure survey, experimental conditions #5 and #6
Post-exposure survey, experimental conditions #7 and #8
Post-exposure survey, experimental conditions #9 and #10
Post-exposure survey, experimental conditions #11 and #12
Post-exposure survey, experimental conditions #13 and #14
Post-exposure survey, experimental conditions #15 and #16
Measures cross-walk