OMB #1545-1349
IRS W&I Injured Spouse Customer Satisfaction Survey Usability Testing-OMB #: 1545-1349
Screener Guide
Ten 60 minute usability sessions total. Recruit 13 for 10 to show for total number of sessions.
Use survey participants that have indicated that they are interesting in participating in additional IRS-related research by providing their phone numbers at the end of the Injured Spouse survey. We would like a mix of:
Census geographic locations
Both Satisfied and Dissatisfied participants (this information will be pulled from survey responses to the overall satisfaction item Q4).
Hello, my name is __________________and I work for Fors Marsh Group. We are a market research firm that is working with the IRS to schedule people to participate in one-on-one online interview sessions. We received your contact information from a survey you completed in [INSERT SURVEY DATE] regarding the Injured Spouse process that you went through. We are contacting you today because we are looking for participants in an upcoming study. The study will be conducted remotely via phone and Internet.
Your identity will not be shared with the IRS or other agencies. We would like to ask you a couple of questions to see if you qualify for this study. Your participation is voluntary, but your help on this project would be very much appreciated.
Note to recruiter: If respondent asks were we got their contact info from tell them: “At the end of the Injured Spouse survey, you were given the option to provide your contact information if you were interested in participating in future research.”
First, we would like to know if you are interested in participating in this study.
1 Yes
2 No Thank and terminate
Thank you, we would like to now ask you a few questions for qualification purposes.
Do you recall your experience with the Injured Spouse process?
Yes
No Thank and Terminate
Do you feel that you can recall this experience well enough to answer some detailed questions on the whole process?
1 Yes
2 No Thank and Terminate
Do you have a computer at home?
1 Yes
2 No Thank and Terminate
Do you have Internet access at home?
1 Yes
2 No Thank and Terminate
How often do you use the Internet other than using e-mail?
1 Very often, daily
2 Often, several times a week
3 Not very often, once a week
4 Hardly ever, less than once a week Thank and Terminate
Have you ever used an online video chat program such as Skype or Google Video Chat?
1 Yes
2 No
Do you feel confident that you could download and use a video chat program if provided with detailed instructions?
1 Yes
2 No Thank and Terminate
If respondent qualifies for the study:
This study will take about 60 minutes to complete and we will give you $40.00 for your time. We would like to go ahead and schedule an interview time for you. [Provide Options].
Would you please provide us with your email address so that we can send you a reminder and instruction?
[EMAIL]
What online video chat program do you prefer?
[TEXT BOX]
The Paperwork Reduction Act required that IRS provide an OMB Control Number on all approved public information requests. That number is OMB 1545-1349. Also, if you like, I can give you a name and address where you can send comments and questions regarding this process or suggestions for making it simpler.
[Provide the following information only if respondent asks for address:]
Internal Revenue Service
Tax Products Coordinating Committee, Room #IR-6406
1111 Constitution Avenue, NW
Washington, DC 20224
Thank you and have a nice day. We look forward to speaking with you on (DATE) at (TIME)
IRS W&I Injured Spouse Customer Satisfaction Survey UsabilityTesting
Moderator’s Guide
Protocol – Usability Testing for Injured Spouse Survey
Introduction
“Hello. Thank you for joining us today. My name is [Insert name of moderator] and I will be working with you today. We’ve invited you here so we can discuss your experience with the Injured Spouse Allocation process at the Internal Revenue Service. What we are doing today is part of a larger effort being conducted by the Internal Revenue Service. I do not work for the Internal Revenue Service. I am an independent researcher who has been asked to help them learn more about taxpayers’ experiences with a specific branch of the IRS. Please be assured that nobody from the Internal Revenue Service will attempt to contact you after your participation today to discuss this topic and I will not be providing any of your personal information to them.
Today, we’ll ask you to answer questions that will be used in a national survey. The questions are about your correspondence and interactions with the IRS regarding the Injured Spouse Allocation process.
Since we are interested in your thoughts and reactions, I would like you to think aloud as you are figuring out the responses to the questions. So I would like you to tell me what you are thinking while you are working on your answers. Tell me how you are coming up with your answer and if words or terms in the question are confusing or unclear. This is important because we want to ensure that the questions are clear to people who are responding to them. Do you have any questions?”
“Let’s begin with a practice task. I am going to ask you a question, and I would like you to think aloud while you come up with your answer. Tell me what you are thinking as you determine your final answer. Ready?”
“How many doors are in your home?” Ensure that respondents are discussing how they are determining their answer. For example, they may say that there is a front and back door, and there is a door at the bathroom and at the bedroom…any others? No, that is all…So there are 4 doors total. Then ask if there are any more, such as on a closet or laundry room. Then ask if they would include cabinet doors and garage doors. The point is to get them to think through this question and consider all possible options.
When participants indicate something is unclear, probe about the perceived meaning of the item. Some potential probes are:
What does this mean to you?
Is this word/phrase/question clear?
How would you say this in your own words?
Why do you think that?
“Please take a moment to recall your most recent experience with the Injured Spouse program.
What are a couple things that you can remember going right with the process?
What are a couple things that you can remember going wrong with the process?”
“We will now start working our way through a new version of the survey that you completed a few months back. Please keep in mind that we would like to hear about your experience with the Injured Spouse process throughout the interview. For the items on the survey we would like you to let us know if they were applicable to your experience or if you feel that they don’t relate to the process.”
Regardless of whether you agree or disagree with the final outcome, how would you rate your overall satisfaction with the service received from the IRS about your Injured Spouse Allocation?
1 = Very Dissatisfied
2 = Dissatisfied
3 = Neither Satisfied nor Dissatisfied
4 = Satisfied
5 = Very Satisfied
In your own words, what do you think this question is asking?
What experiences or criteria did you use to develop your answer?
What does overall satisfaction mean?
For the following questions, please focus on the outcome of the issue you recently resolved with the IRS.
2a) To what extent do you agree with the final outcome of your recent issue with the IRS?
1 = Strongly Disagree
2 = Disagree
3 = Neutral
4 = Agree
5 = Strongly Agree
What does “final outcome” mean to you?
Why do you agree/disagree with the outcome?
2b) To what extent do you agree that your outcome was reasonable given what you provided to the IRS?
1 = Strongly Disagree
2 = Disagree
3 = Neutral
4 = Agree
5 = Strongly Agree
In your own words, what do you think this question is asking?
Was it difficult for you to select a response option?
At the end of this section ask:
Why do you agree/disagree with the outcome of your case?
Do you understand why this outcome was reached?
For the next set of questions, regardless of your satisfaction with the outcome of your issue, please focus on the process and procedures that were applied to address and resolve your issue.
In your own words, please explain what “regardless of your satisfaction with the outcome of your issue” means.
What do “process” and “procedures” mean?
3a) How satisfied were you with the ease of gathering information requested by the IRS?
1 = Very Dissatisfied
2 = Dissatisfied
3 = Neither Satisfied nor Dissatisfied
4 = Satisfied
5 = Very Satisfied
What all did you think about in answering this question?
How did you provide this information?
3b) How satisfied were you with the time you were given to respond to the IRS?
1 = Very Dissatisfied
2 = Dissatisfied
3 = Neither Satisfied nor Dissatisfied
4 = Satisfied
5 = Very Satisfied
Do you feel that you were given enough time to respond?
3c) How satisfied were you with the length of time to resolve your issue?
1 = Very Dissatisfied
2 = Dissatisfied
3 = Neither Satisfied nor Dissatisfied
4 = Satisfied
5 = Very Satisfied
How do you interpret the word “resolve”?
How long did you expect it to take to resolve your issue?
Did any correspondence with the IRS indicate that your issue would be resolved in a shorter timeframe?
At the end of this section ask:
How do you see the process working? What steps were involved for you?
Did you contact the IRS to receive instructions on the preparation of the Form 8379, Injured Spouse Allocation?
Yes
No
Was it difficult for you to answer this question?
Were you able to remember the form 8379?
[If “yes”] how did you contact the IRS to receive instructions?
For the next set of questions, regardless of your satisfaction with the outcome of your issue, please focus on your interactions with the IRS regarding the instructions for preparing Form 8379, Injured Spouse Allocation.
Please rate your satisfaction with…
5a) Getting through to someone who could answer your questions about Form 8379?
1 = Very Dissatisfied
2 = Dissatisfied
3 = Neither Satisfied nor Dissatisfied
4 = Satisfied
= Very Satisfied
Do you know which form the question was referring to?
Was it difficult for you to answer this question?
5b) Professionalism of the IRS representatives?
1 = Very Dissatisfied
2 = Dissatisfied
3 = Neither Satisfied nor Dissatisfied
4 = Satisfied
5 = Very Satisfied
What does “professionalism” mean?
What criteria did you use to develop your answer?
How many representatives did you interact with?
Did the different representatives you interacted with show different levels of professionalism, or were they all about the same?
When you answered, were you thinking of all of the representatives you interacted with?
5c) IRS representatives’ knowledge of the Injured Spouse process?
1 = Very Dissatisfied
2 = Dissatisfied
3 = Neither Satisfied nor Dissatisfied
4 = Satisfied
5 = Very Satisfied
Can you recall what questions you asked the representatives?
[If you were dissatisfied], what information did you request that the representatives could not provide?
Did the different representatives you interacted with show different levels of knowledge, or were they all about the same?
5d) Ease of understanding the information supplied by the IRS?
1 = Very Dissatisfied
2 = Dissatisfied
3 = Neither Satisfied nor Dissatisfied
4 = Satisfied
5 = Very Satisfied
What type of information did the IRS supply?
[If dissatisfied] why were you dissatisfied with the ease of understanding the information supplied?
5e) Completeness of the instructions you received?
1 = Very Dissatisfied
2 = Dissatisfied
3 = Neither Satisfied nor Dissatisfied
4 = Satisfied
5 = Very Satisfied
What type of instructions did you receive?
[If dissatisfied] why were you dissatisfied with the instructions? What additional information would you have wanted?
What was the final outcome of your Injured Spouse case? (select only one)
Full refund
Partial refund
No payment (refund was offset)
Case was denied
Do you understand the difference between “no payment” (refund was offset) and “case was denied”?
Are there any response options on this list that you find confusing?
When you received all or part of your refund, or a letter describing the outcome of your case, did you contact the IRS for an explanation?
Yes
No
Was it difficult for you to remember whether you contacted the IRS for an explanation?
[If yes] how did you contact the IRS?
For the next set of questions, regardless of your satisfaction with the outcome of your issue, please focus on your interactions with the IRS regarding the outcome of your case.
What does “interactions” mean to you?
How satisfied were you with…
8a) Getting through to someone who could explain the outcome of your case?
1 = Very Dissatisfied
2 = Dissatisfied
3 = Neither Satisfied nor Dissatisfied
4 = Satisfied
5 = Very Satisfied
In your own words, what do you think this question is asking?
8b) Professionalism of the IRS representatives?
1 = Very Dissatisfied
2 = Dissatisfied
3 = Neither Satisfied nor Dissatisfied
4 = Satisfied
5 = Very Satisfied
What does “professionalism” mean?
Do you feel that you already answered this item in another section?
What criteria did you use to develop your answer?
Did the different representatives you interacted with show different levels of professionalism, or were they all about the same?
8c) IRS representatives’ knowledge of the Injured Spouse process?
1 = Very Dissatisfied
2 = Dissatisfied
3 = Neither Satisfied nor Dissatisfied
4 = Satisfied
5 = Very Satisfied
8d) The explanation provided of your case outcome?
1 = Very Dissatisfied
2 = Dissatisfied
3 = Neither Satisfied nor Dissatisfied
4 = Satisfied
5 = Very Satisfied
What was the explanation provided of your case outcome?
At the end of this section ask:
Can you please describe some of the interactions that you had with the IRS during this process?
What was positive about them? What was negative?
Repeat if participant needs additional prompting: “Again, please keep in mind that we would like to hear about your experience with the Injured Spouse process throughout the interview. As with the previous items, we would like to you to let us know if they were applicable to your experience or if you feel that they don’t relate to the process.”
How many times did you contact the IRS regarding your Injured Spouse Allocation?
1
2
3
4 or more
I did not contact the IRS regarding my Injured Spouse Allocation
Was it difficult for you to remember how many times you contacted the IRS regarding your allocation?
[If 4 or more] exactly how many times did you contact the IRS?
What reasons did you contact them? What did you take into account when you answered this item?
How satisfied were you with the update you received when you contacted the IRS regarding the status of your resolution?
1 = Very Dissatisfied
2 = Dissatisfied
3 = Neither Satisfied nor Dissatisfied
4 = Satisfied
5 = Very Satisfied
If you were dissatisfied for any reason, please explain why in the space below:
What was the update you received when you contacted the IRS regarding the status of your resolution?
If you were dissatisfied, how did you decide what explanation to give?
The questions that follow ask your opinion regarding the Form 8379, Injured Spouse Allocation, preparation process. For each question, please indicate your answer by checking the box that best represents your opinion.
How satisfied were you with the…
11a) Ease of understanding Form 8379, Injured Spouse Allocation?
1 = Very Dissatisfied
2 = Dissatisfied
3 = Neither Satisfied nor Dissatisfied
4 = Satisfied
5 = Very Satisfied
In your own words, what is this question asking? [Probe on whether they reference the preparation process or not.]
11b) Amount of time you spent on Form 8379, Injured Spouse Allocation?
1 = Very Dissatisfied
2 = Dissatisfied
3 = Neither Satisfied nor Dissatisfied
4 = Satisfied
5 = Very Satisfied
How much time did you expect it would take to complete the form?
How long did it actually take you to complete the form? [Probe on whether they reference the preparation process or not]
How did you establish how long it took you to complete the form?
At the end of this section ask:
What did you find confusing about the form? Were there any difficult sections for you?
How did you find out about the possibility of obtaining Injured Spouse relief?
Letter or Notice
IRS Customer Service Representative
IRS Taxpayer Advocate Office
IRS website
Family/Friend/Colleague
Lawyer or legal counsel
Tax professional (e.g., CPA, accountant)
Tax software
Other (please specify) ___________
Please describe your experience with finding out about the Injured Spouse program.
Are there response options missing from this list (besides “Other”) that you or someone you know might choose?
Are there any items on this list that you find confusing?
When did you file Form 8379, Injured Spouse Allocation?
I submitted it with my tax return
I submitted it after my tax return
Was it difficult for you to remember when you filed Form 8379?
How much time passed between your filing of Form 8379 and your submitting your tax return?
What type of letter or notice prompted you to begin the Injured Spouse Allocation process?
A letter saying that the IRS used all or part of your refund to pay a tax debt
A letter saying that the Department of Treasury, or Financial Management Service, used all or part of your refund to pay past due child support or spousal support, federal non-tax debt (student loans), or state income tax obligation
Other (Please specify) _________
Did not receive a notice
Are there response options missing from this list (besides “Other”) that you or someone you know might choose?
Are there any items on this list that you find confusing?
What type of debt issue required you to file Form 8379, Injured Spouse Allocation?
Federal tax debt issue
Other type of debt issue (other than Federal tax issue, e.g., child support, student loans)
Both (such as tax debt issue and other type of non-tax debt issue)
Was it easy to determine which category your debt issue fell into?
How satisfied were you with the ease of finding out about the Injured Spouse program?
1 = Very Dissatisfied
2 = Dissatisfied
3 = Neither Satisfied nor Dissatisfied
4 = Satisfied
5 = Very Satisfied
What steps did you take to find out information about the program?
What criteria did you use to develop your answer?
Where did you obtain the forms and instructions you filed for your Injured Spouse Allocation?
IRS via telephone
IRS office
IRS website
Lawyer or legal counsel
Tax professional (e.g., CPA, accountant)
Tax software
Other (Please specify) ____________
Are there response options missing from this list (besides “Other”) that you or someone you know might choose?
Are there any items on this list that you find confusing?
In number of months, how long did you expect the Injured Spouse Allocation process to take?
How did you develop your answer?
Why did you expect the process to take this long?
In number of months, approximately how long did it take to resolve your Form 8379, Injured Spouse Allocation, issue?
How did you develop your answer to this question?
Was it difficult for you to answer this question?
Who prepared your Form 8379, Injured Spouse Allocation? (Check only one response)
Yourself
Paid tax professional
Volunteer Income Tax Assistance (VITA)
Friend/Relative
IRS office
Are there response options missing from this list (besides “Other”) that you or someone you know might choose?
Are there any items on this list that you find confusing?
Regardless of the outcome of your case, how much do you agree with the following statements?
21a) I received an adequate description of the Injured Spouse process
1 = Strongly Disagree
2 = Disagree
3 = Neutral
4 = Agree
5 = Strongly Agree
What does “adequate” mean?
Who or what provided you with a description of the Injured Spouse process?
21b) My experience reflected the described Injured Spouse process
1 = Strongly Disagree
2 = Disagree
3 = Neutral
4 = Agree
5 = Strongly Agree
Who or what are you referring to when it asks for the “described Injured Spouse” process?
Why did you agree or disagree?
21c) I had the opportunity to provide information important to my case
1 = Strongly Disagree
2 = Disagree
3 = Neutral
4 = Agree
5 = Strongly Agree
How could you provide information about your case?
21d) I was treated with respect during the Injured Spouse process
1 = Strongly Disagree
2 = Disagree
3 = Neutral
4 = Agree
5 = Strongly Agree
What experiences or interactions did you take into account while answering this question? What do you consider being “treated with respect”?
Overall, how well did the IRS meet your expectations while handling your Injured Spouse case?
1 = Much Worse Than Expected
2 = Worse Than Expected
3 = As Expected
4 = Better Than Expected
5 = Much Better Than Expected
Can you describe where your “expectations” were drawn from?
Can you explain what factors you considered while answering this question?
If you answered “Worse than expected” or “Much worse than expected” to the above question, can you describe what caused you to feel that way?
At the end of the survey ask the participant items to better understand the experience that they had with the Injured Spouse Program:
Does this survey seem to capture your experience with the Injured Spouse process?
Did this order of the survey items make sense to you?
Please describe some of the difficulties that you experienced.
Are there any aspects of the Injured Spouse allocation process that you feel are missing from this survey that you would suggest including?
Was there anything that you feel the IRS did well on that was not included in this survey?
Were there any shortcomings that you experienced throughout the process? What were they? Do you feel that they were captured on the survey?
Do you have any suggestions on how the IRS could better your experience with the Injured Spouse process?