Summary of Recommendations from Round 1 Cognitive Testing of the MEPS Cancer SAQ
The National Cancer Institute, the Centers for Disease
Control and Prevention, and the American Cancer Society are developing a
self-administered questionnaire, Your Experiences with Cancer,
for current cancer patients and cancer survivors identified in the Medical
Expenditure Panel Survey sponsored by the Agency for Health Care Research and
Quality. The questionnaire will be fielded in Spring 2012 (MEPS Panel 16 Round 3
and Panel 15 Round 5).
The Medical Expenditure Panel Survey produces national
estimates of healthcare use and expenditures. The Cancer SAQ will supplement
MEPS data, providing information about the burden of cancer in the US, long
lasting effects of the disease, financial impacts, and employment outcomes for
cancer survivors and their families. Findings from this research will provide
key information needed for the development of interventions to improve the
quality of the cancer survivorship experience, as well as pivotal information
for policy makers in understanding and improving the burden of cancer.
As part of the development process, Westat plans to
conduct 2 Rounds of cognitive testing to assess how well respondents are able to
understand and answer the questionnaire items. Round 1 cognitive interviews with
24 English speaking respondents were conducted between June 7 and June 30, 2011.
This memo summarizes findings and recommendations from that testing effort.
Working from our own as well as sources provided by
the SAQ inter-agency workgroup, Westat recruited respondents in Washington,
D.C., Baltimore, and the surrounding areas using several methods. We posted
flyers at local cancer and community clinics. We networked with support groups
such as Relay for Life, Patient Family Support Group at Suburban Hospital and
Hopewell Cancer Support in Baltimore. We created a Facebook
advertisement targeted to users who live within 10 miles of Washington, D.C.,
Baltimore, MD, Rockville, MD, or Frederick, MD and who "like" the following
cancer related categories:
- Breast cancer survivors;
- Cancer research institute;
- Relay For Life;
- Susan G. Komen foundation;
- Cancer awareness (brain, breast, colon, leukemia, lung, ovarian); and
- Various cancer research groups.
These criteria yielded a pool of 35,340 Facebook
users. When users clicked on the ad, they were directed o a Zoomerang survey
that described the research in more detail and asked for contact information.
Westat monitored the Zoomerang results on a daily basis and followed up with all
"hits" to screen and, if eligible, schedule an interview. Early in the
recruiting effort, several potential eligible respondents indicated that they
were willing to participate in the pretest but were unable to travel to Westat’s
Rockville location to be interviewed. As a result, we revised the Round 1
protocol to include alternative interview locations,
To be eligible,
potential respondents had to have a current or past cancer diagnoses. In
addition, participants were screened and selected based on type of cancer, time
since last diagnosis, employment status, and number of caregivers. Table 1 shows the distribution of demographic characteristics across Round 1
About half the respondents had a current diagnosis and the rest were spread
fairly evenly among categories representing diagnoses in the recent and distant
past They were also split evenly between employed and unemployed at the time of
diagnosis. The Round 1 respondents represented a good mix of education levels.
However, they skewed heavily female, older and White. Finally, although not
evident from the demographic characteristics, we noticed that many of these
respondents were savvy about the topic of cancer in general as well as their own
particular medical situations. For example, one respondent writes a cancer blog
and two others write articles, short stories, and novels about their experiences
with cancer. This is likely a result of the recruiting sources used for this
effort and may not accurately reflect the experiences of typical MEPS
Table 1. Distribution of Round 1 demographic characteristics
| HS or less
| Some College
| College Degree
| Advanced Degree
| < 40 years old
| 40 – 49 years old
| 50 – 59 years old
| 60 – 69 years old
| 70 + years old
|Time Since Last Treatment
| Current Diagnosis (5 w/multiple diagnoses)
| 2 years or less (1 w/multiple diagnoses)
| 2 – 6 years
| 6 + years (1 w/multiple diagnoses)
|Employment Status when Diagnosed**
| In School/too young to work
*One respondent did not provide her age.
**One respondent with multiple diagnoses was employed
at the time of his first diagnosis and retired at the time of his second
With input from the workgroup, Westat developed an
interview protocol that explored issues of concern using scripted probes and
guidelines for unscripted probes. General issues for testing included item
wording and response option appropriateness; feasibility and accuracy of
respondent recall for various items; appropriate use of words and phrases in the
question stems; and ease or difficulty answering some items. Examples of
specific issues for testing appear below. More detail appears in the final
interview protocol, Attachment D.
- Appropriateness of the Cancer SAQ for those with a wide variety of
cancer experiences, including current diagnosis, multiple diagnoses, and
last treatment in the recent and in the distant past.
- Ability of respondents to answer for all three survey concepts – cancer,
its treatment, and the lasting effects of that treatment – across the entire
- Respondents’ interpretation of specific words and phrases such as
"lasting effects," "flexible work schedule," "extended paid time off," and
- Ability to answer questions about caregivers’ employment experiences.
- Recall of certain life events, such as the duration of employment
As part of the protocol, Westat also developed an
interview introduction script (Attachment E), which included informed consent
language, and the informed consent form used to document written consent
The cognitive interviews were conducted by a team of
five skilled qualitative researchers with survey design experience and two MEPS
project staff who received intensive cognitive interview training for this task.
All interviewers reviewed the SAQ and interview protocol prior to a more formal
3-hour training session held a few days before the interview. The training
agenda included the study background and purpose; overview of the testing
effort, discussion of questions; and logistics of interview administration. The
team leader and lead analyst conducted the first four interviews. Interviewers
were required to observe at least one of these "model" interviews before
conducting their own interviews. The team leader listened to approximately 20
percent of the interviews and provided feedback as needed.
As well, the team leader and lead analyst communicated to the interviewing team
throughout the testing period to provide updates and instructions. Each
interviewer conducted approximately 2-5 interviews.
Conducting Cognitive Interviews
Respondents participated in a one-hour in-person
interview either at Westat’s Rockville offices or at a location of their choice
(usually in their homes, sometimes at a neutral location such as an office at a
local university). They received $40 for their time as well as a $10 travel
allowance. After introducing the study, its purpose and the cognitive testing
procedures, interviewers asked respondents to sign the informed consent form.
The interviewer then asked the respondent to read and react to the cover page,
after which he/she was instructed to begin filling out the questionnaire. Using
the protocol, interviewers administered a combination of mostly concurrent and
some retrospective probes to obtain respondent feedback on how easy or difficult
it was to understand and answer the survey questions. The technique of asking
respondents to describe their thought processes immediately after answering
selected survey items allowed the cognitive interviewer to focus on
comprehension and response processes. In addition to administering the scripted
concurrent probes, interviewers also noted any unanticipated reactions or
problems with the items and followed up on those issues as needed.
Analysis of the Interview Data
Westat analyzed the interview data in a series of
steps. The first step was for each interviewer to review and synthesize the data
(from the audio-recordings and from any notes she may have taken) from her own
interviews. During this step, the interviewer identified themes within each
respondent’s interview and across all the interviews she conducted. The themes
were organized by overall survey issues, individual survey items and sections,
and respondents’ overall reactions to the survey. In the second step, all
interviewers met and discussed the themes they had identified. As a group and
with the guidance of the lead analyst, the team identified common themes (as
well as unique situations worthy of notice) within and across all interviews.
The team also began to identify recommendations for revising the survey based on
the cognitive interview results. In the final step, the lead analyst conducted
another review of all interview data and organized the themes into the report of
findings and recommendations. This final review ensured that the recommendations
are thoroughly grounded in the cognitive interview data.
Findings and Recommendations
The Cancer SAQ attempts to capture a broad range of
experiences from a wide variety of cancer patients and cancer survivors.
Overall, it met those objectives fairly well. Most respondents understood and
interpreted most items as intended. The issues that arose in testing can for the
most part be addressed with wording revisions, formatting adjustments, or skip
pattern updates that can be seamlessly incorporated into the existing survey
The remainder of this memo presents findings and
recommendations from the Round 1 cognitive interviews. The first section
identifies issues experienced by certain types of respondents and describes some
broad observations. Note that, with the exception of the duration questions, we
do not include recommendations in the broad observations section as they are
addressed in the detailed recommendations. The next section is organized by
survey section and item. It provides detailed findings and recommendations to
address problems that surfaced during testing.
Broad Observations: Issues Experienced by Selected
The Cancer SAQ is targeted to respondents who are
currently undergoing treatment and those whose treatment occurred at any time in
the past. It is intended for those who have had one cancer occurrence as well as
those with multiple cancer diagnoses. Those with one cancer occurrence,
either current or in the recent past, had the least trouble navigating through
the survey. The remainder of this section describes issues experienced by those
with multiple diagnoses, those whose last occurrence was in the distant past,
and those with childhood diagnoses.
Multiple Cancer Diagnoses
Of the 24 respondents participating in the cognitive
interviews, three reported a recurrence of their cancer and eight had multiple
cancer diagnoses (which for one respondent included a recurrence). For these
respondents, answering the questionnaire was not a straightforward process, nor
did the survey items always do a good job of accurately capturing their diverse
- Generally, respondents with multiple experiences of cancer did not know
whether to answer items based on their first cancer, or subsequent
cancer(s). This was especially true for respondents whose first diagnosis
occurred in the distant past or in childhood.
- Some respondents focused on their most severe diagnosis, or cancers
requiring extensive treatment and recovery. These respondents tended not to
answer for cancer diagnoses that they considered to be less severe, or that
required only short, outpatient procedures and minimal recovery time. For
example, one respondent who had been diagnosed with melanoma and then later
with prostate, often selected the answer that fit the scenario for the
prostate treatment since he felt the melanoma "wasn’t that serious."
- Respondents with more than one severe cancer had difficulty deciding
which one to talk about. When formulating answers, some respondents
aggregated their cancers while others referred to different cancers for
different items. Still others indicated confusion at various points in the
questionnaire, asking the interviewer "which one?" throughout.
Cancer Diagnoses in the Distant Past
Four out of the five respondents recruited in the "6
years +" category had their last treatment ten or more years ago. Recall was
often difficult for them in the finance and medical care sections, and
especially difficult for less salient items, such as the content of their
discussions with their doctor. Additionally, these respondents had difficulty
remembering details such as the amount of money they borrowed or how much time
they worked at home. Some examples are described below:
- One respondent, whose treatment was 25 years ago, struggled to remember
information for answering the medical debt and medical care questions.
At the Q74 series that asks about discussions with the doctor, the
respondent said she "didn’t remember discussing" several of the topics with
her doctor because it was such a long time ago, but marked "did not discuss
at all" rather than "I don’t remember" as her answers. This is troublesome
because a similar pattern in the survey population would end up inflating
the proportion of "did not discuss" answers and thereby bias the survey
- Another respondent who had multiple bouts of cancer 35 years ago said
that the questionnaire was "problematic" for him because he felt that many
of the questions assume a more recent cancer experience. The respondent had
difficulty recalling how his cancer impacted his caregiver’s employment, or
whether his doctors discussed the topics at Q74 in detail with him.
- For one respondent, difficulty recalling her last treatment 17 years ago
may have been confounded by the fact that she was only three years old when
she was diagnosed.
Cancer Diagnosis as a Child
Four respondents were diagnosed with cancer as
children. These respondents explained that because they were so young during
treatment, they didn’t remember enough to answer some of the questions, they
were unaware of what actually happened, or their parents made all of the
decisions related to their cancer.
- A respondent who was diagnosed with cancer when she was three said,
"I’m finding it probably more difficult than other people might find it just
because it was a long time ago…a lot of this stuff is based on what my
parents have told me…I don’t think I have the whole, clear picture…This
would be way easier for me to fill out with my mom by my side." She had
particular difficulty in the insurance section because she said her mother
would have dealt with those issues.
- Another respondent echoed this feeling and had difficulty answering some
of the questions since his first experience with cancer was when he was in 5th
grade. He stated he does not remember many things from that episode of
cancer, and answered the questions thinking mostly about his most recent
experience with cancer (5 years ago).
- A third had the most difficulty in Section 5. He reported "no" for all
of the questions, but after talking with the interviewer, it was clear he
did not actually know what the answers were and that he would have had to
consult with his parents in order to get them.
- Respondents with childhood cancer also had difficulty with the finance
section because they "don’t know specifics". Respondents commented that
their parents were the ones who dealt with the financial matters.
Some respondents with childhood diagnoses noted that
the questionnaire seemed to have been designed for those who were diagnosed with
cancer as an adult, particularly items in the Employment and Caregiver sections.
- At Q8 (…were you working for pay at a job or business), one respondent
initially answered "no" because she was not working when she was
diagnosed with cancer at age 3. In explaining her answer she said,
"I was thinking there are a lot of people who are older and they have cancer
and they’ve been working and it kind of disrupts things…I was three …I
wasn’t affected by it; but I have been employed since." In talking through
her rationale, she realized that the question was asking about employment
since her diagnosis and changed her answer to "yes."
- Another respondent felt that most of the employment and caregiver
questions did not apply to his situation since he experienced cancer when he
was a child. He pointed out that he was not working and he was not aware of
the situation his parents had in regards to their work schedules while he
was undergoing treatment.
- A respondent who has held a number of jobs as an adult, long after his
cancer diagnoses at ages 15 and 16, commented that Q8, "…doesn’t take into
account someone who was 15 at the time of diagnosis. At 15, I was going to
school". He also said that the question was "vague", adding, "I didn’t start
working until I was 20". The respondent also had difficulty in the caregiver
section and felt that the questions were designed for people who were
diagnosed more recently.
Broad Observations: Survey-wide Issues of Note
Respondents’ interpretations of "lasting effects"
varied, but for the most part they seemed to consistently apply their own
interpretations across questionnaire items. Below are examples of what
respondents did and did not consider "lasting effects" of cancer.
- When asked to interpret the meaning of "lasting effects", one respondent
said that she considered them to be effects that occurred up to one year
from her last treatment. Conversely, two respondents, one with terminal
cancer, and one with a recurrence said "…It never goes away…It’s forever".
- Some respondents did not differentiate between "lasting effects" of
cancer and its treatment, and "side effects" of treatment, such as "chemo
- A respondent described chronic arm lymphedema related to her cancer, the
expense of buying a special sleeve for her arm, and the expense of medical
appointments as "lasting effects".
- An osteogenic cancer survivor whose leg was amputated in childhood as
part of his cancer treatment described at various points in the survey how
lack of mobility and prosthesis were his main problems in life. However, he
did not consider them to be "lasting effects" of the cancer treatment.
- One respondent described chronic stomach problems resulting from a
poorly performed cancer surgery. However, she did not consider her stomach
problems to be "lasting effects" of cancer but rather the result of medical
- Another respondent whose chemo tube was not removed until five years
after her chemotherapy was over noted that she developed a blood clot, which
still today affects her arm. However, when asked if she considered this a
lasting effect of the cancer treatment, she said no, because she felt it was
related to her doctor’s malpractice rather than to the cancer or treatment
- At least three respondents who were currently in treatment for their
first cancer diagnosis explicitly said they did not include "lasting
effects" in their answers because they have not yet experienced any.
Of key analytic interest are the "lasting effects" of
cancer treatment on the various aspects of respondents’ lives. It is therefore
important that respondents consistently include "lasting effects" in their
answers throughout the survey. For the most part, it appeared that when they did
think about lasting effects, they were doing so throughout the entire survey.
those who were not thinking about lasting effects seemed not to do so throughout
the entire survey.
Some respondents had difficulty differentiating
between the effects of aging or other health conditions and those of their
cancer treatments. This problem manifested itself most obviously at the
questions about the impact of lasting effects on physical and mental tasks.
Because they did not always know which one was the cause of any given problem,
individual respondents sometimes had variable interpretations of "lasting
effects" across their own survey answers. If this phenomenon is replicated in
the survey population, data from these items will lack reliability.
- A survivor of colorectal cancer who underwent radiation therapy reported
chronic back pain from degenerative disc disease. However, he was unsure if
his condition was mainly due to bone damage from radiation, or simply due to
- A respondent described some of her cognitive deficits related to "chemo
brain". However, it was unclear if these deficits were related to
chemotherapy, or to a recent stroke.
- Some elderly respondents found it difficult to pinpoint the "lasting
effects" of cancer. For example, respondents also suffered from arthritis or
other age-related chronic conditions, and could not separate these
conditions from those related to cancer and its treatment.
Seventeen respondents expressed difficulty with the
duration questions at some point in the questionnaire. These questions occurred
in Section 2, Section 4, and at Q79. Thirteen of them were confused about how to
record their answers in the split "amount" and "unit" format.
- One respondent, who had switched to a flexible work schedule, commented
at Q13, "It’s a little bit confusing right now… Do you want to put months in
it, years in it, weeks in it?"
- Another respondent described her difficulty in the caregiver employment
section, "I was confused about what kind of number they’re looking for…in my
mind I was thinking my mom worked part-time for about a year…I didn’t know
what number to put there, but it was for the whole year…I guess I could have
put seven per week or five per week."
The word "per" was also confusing for some respondents
because it implies a rate. One asked, "Would that be 5 times per week?" while
another who was trying to find a way to record her answer of "2 years" wondered,
"Is this days per week, days per month, or days per year?"
Many respondents said that they were frustrated with
these questions and some said they would have skipped them if the interviewer
had not been there. They dealt with answering the duration questions in a
variety of ways, as indicated below:
- Four respondents ignored the boxes next to "amount" but wrote in a
number in the box next to the unit that applied to them. Sometimes the
respondent pluralized the unit, for example, changing "month" to "months."
- Several respondents wrote in their own answers outside of the designated
- One respondent checked the box next to year and then wrote a (+) next to
it to indicate it was a few years.
- One respondent checked off multiple unit boxes indicating both months
Eight respondents struggled with making the
calculations to come up with their answers. This was especially true for those
with different episodes of cancer or job situations.
- One respondent commented that, given how long ago her cancer occurred,
it was too difficult to provide the specific kind of answer the question
seemed to be looking for.
- One respondent had to do extensive calculations for this item because
she was out of work several times over the course of her diagnosis and
treatment, each time varying in length.
- One respondent with multiple cancers was not sure how to calculate the
duration because her employment change lasted the entire time between her
first and second cancer diagnoses, which were 14 years apart. She thought it
would be misleading to mark "14 years" because the reason she did not go
back to her original employment status had nothing to do with her cancer or
its lasting effects.
- One respondent had difficulty calculating several instances of extended
paid time off into one amount in the duration part of the question.
- Two respondents were still experiencing their employment change. One
struggled with Q13 because she is still on the flexible work schedule she
changed to at her first diagnosis, then stayed on after treatment and for
her second diagnosis. The other respondent pointed out that the questions
assume the employment change happened in the past. She answered by counting
the total time up to the date of the survey. In both of these situations,
the duration questions do not capture the fact that the employment change is
- On the flip side of wanting to convey an ongoing change, at least one
respondent wanted to indicate that she had gone back to her original
employment status. She answered both the part-time to full-time and
full-time to part-time questions, using the latter to indicate that the
change was temporary and she is now back on her regular schedule. This
respondent’s cancer treatment had happened in the distant past.
RecommendationsWe suggest dropping the split "amount" and
"unit" format and using the following response categories for all
duration questions in the employment section:
Less than 3 weeks
3 weeks to < 2 months
2 months to < 6 months
6 months to < 1 year
1 to 3 years
More than 3 years
In the caregiver section, the response categories would be:
2 months to < 6 months
6 months to < 1 year
1 to 3 years
More than 3 years
We need further guidance from the workgroup
before providing a recommendation for Q79 (see item-specific discussion
There are numerous skip patterns and instructions in
the questionnaire, which was distressing for a significant number of
respondents. They had a particularly difficult time in Section 1, where 10
people made at least one skip pattern error, although that was not the only
place that caused confusion. All but five respondents incorrectly skipped or
were confused by a skip pattern at least once in the questionnaire.
Additionally, the switch between arrows on the left and brackets on the right
was confusing for a small number of them.
- One respondent was confused by the skip instruction at Q3. He initially
thought it was an error because he was familiar with skips that apply to
either the "yes" or "no" but not to both response options.
- One respondent said she was "confused about what this is telling me to
do" in reference to the skip instruction with brackets at Q3.
The questionnaire was formatted using design
principles developed and successfully tested by Dillman. As well, Westat has so
far found very few errors of omission or commission in two large-scale field
tests for the National Household Education Survey that used the same format for
self-administered questionnaires. We are somewhat puzzled as to why the
formatting should have posed such a problem for this population. We speculate
that in part it may be because respondents are expecting more of the
questionnaire to apply to them, or that there are simply too many skips in an
effort to accommodate the wide variety of cancer experiences the survey wishes
to capture. Specific issues with the skips are discussed further in the relevant
Section- and Item-Specific Findings and Recommendations.
Respondents, in general, appeared to have positive
impressions of the questionnaire. Below are examples of their overall reactions
after completing it, along with a few of their suggestions.
- One respondent stated that people often don’t think about these
experiences because all the focus is on finding a cure. She approves of the
questionnaire because she thinks it will help address other cancer-related
issues and perhaps ultimately help others.
- Another respondent also felt the survey would help others. By filling it
out, he said, "…It is something I can pass down to other people who are
going through the same experience…it might help someone else."
- Two respondents thought that the survey was "very good"; one liked that
it asked about the financial burdens a cancer patient has to suffer; the
other liked the focus on patient experiences in hopes that it will help
others through their cancer journey. Another respondent thought the
questionnaire covered "great topics".
- One respondent commented that she thought the font size was good and
that the bold font and blue background made it easy to read.
- One respondent thought the survey was good, but noted that it was "very
extensive and had a lot of words." The respondent felt that others with low
literacy levels may struggle or have difficulty and suggested making it
briefer, especially since it is a self-administered questionnaire.
- One respondent, who was diagnosed with cancer as a child, suggested that
there should be more questions about the mental and social aspect of dealing
with the disease, for instance, how it affects the interaction with friends
and other people.
- Two respondents felt that the questionnaire should focus more on the
emotional impact of cancer on your life. One commented that "the emotional
aspect plays a great deal in the process of getting back into your life."
In general respondents seemed to feel that the cover
page was not informative about the survey contents.
- Some respondents surmised from the cover that the survey was about their
"experiences with cancer", "…What I was going through", or "…My cancer
- One respondent had the impression from the cover that the questions
would ask about more intimate details of her life and her cancer experience.
- One respondent said that the cover page should provide more information,
and should specify how filling out this questionnaire would help other
people with cancer.
- Some respondents assumed from the cover that the questionnaire would
focus more on medical experiences, such as experiences with doctors,
hospitals, and cancer treatment (what kind, how long it took,
recovery/set-backs) while others thought the questionnaire was more about
the "effects of cancer that are long-lasting" or how cancer has impacted
their lives and the lives of their families.
Add a second sentence to the introductory paragraph on the cover
page that reads, "The survey will ask about the effect of cancer on your employment,
finances, health insurance coverage, and life in general."
Section 1 – Cancer History
Eleven of twenty-three respondents made a skip pattern
mistake at some point in Section 1. Respondents may have had difficulty because
of the large number of skips and the variety of different kinds of skip
instructions (left-hand arrows, large "stop box", right-hand arrows, right-hand
brackets) so early in the survey.
Also, the idea of passing over more than half a section very early in the survey
seemed jarring for those who were skipped out of the section at the third
Questions about treatment can be confusing because the
word can refer either to an entire course of treatment, such as all four cycles
of a chemotherapy treatment, or one particular aspect of treatment, such one
dose of the chemotherapy. As well, it may be unclear to respondents whether to
include in treatment other aspects of cancer care, such as biopsies, follow-up
exams, and preventative measures. All of these issues appeared at various items
in Section 1, which contributed further to the skip pattern problems, as
Q2 – Are you currently being treated for cancer?
"Currently being treated" at Q2 refers to radiation,
chemotherapy, surgery or other courses of treatment administered at the time of
or shortly after diagnosis. Some respondents, who are currently being treated
for cancer, were interpreting Q2 as intended.
- Two were thinking about their hormone treatment for prostate cancer. One
of these respondents was also thinking about radiation.
- A third respondent was thinking of her chemotherapy.
Many others, however, did not appear to have
interpreted Q2 as intended.
- One respondent whose course of chemotherapy ended in April 2010 answered
"yes" at Q2 because she was including her follow-up CT scans, blood tests,
and pills "to keep [her] from getting cancer."
- Another respondent who has completed his chemotherapy and radiation
included his follow-ups with the oncologist and urologist as his "current
- One respondent said she considered taking "standard preventative
measures" to be treatment.
- One respondent asked, "What does being treated for cancer mean?" She
decided to answer "yes" at Q2, thinking about the medication she is
currently taking. One other respondent was thinking of pills she takes as
her "current treatment."
- One respondent said she was still in her five year follow-up and so
considers herself as "currently being treated."
In some instances, respondents’ may be thinking of
these more preventative measures because of the language their doctors are
- One respondent who has completed her chemotherapy marked "yes" at Q2 and
explained that, because her cancer is slow growing, her doctor has
instructed her to see him in 6 months so he can decide if she needs
- Similarly, another respondent marked "yes" at Q2 because her doctor told
her "I am still treating you" in regard to her 6-month follow-up visits and
- One respondent answered "no" at Q2 because she had just finished the
second of four cycles of chemotherapy.
With the exception of the last respondent, the broad
interpretation of "currently being treated" at Q2 led respondents to skip out of
Section 1 before receiving Q5, which asks when they received their last cancer
treatment (meaning, last entire course of treatment). Although we do not
have direct evidence of this from the cognitive interviews, misinterpretation of
"current treatment" could also lead respondents to under-report lasting effects
of cancer, if they believe they have not yet experienced lasting effects because
they are still currently in treatment. For the respondent who is in the middle
of a four-cycle course of chemotherapy, the categories at Q5 seemed too broad
(she answered "less than 12 months") and Q6 and Q7 did not make sense.
Consider revising Q2 to better convey the
intended definition of "currently being treated." An example from the
National Cancer Institute’s Adolescent and Young Adult Health Outcomes
and Patient Experience (AYA HOPE) Follow-up Survey provides more detail
about what to include and what to exclude. It is also much lengthier than Q2.
"What was the date of your last cancer
treatment – that is, surgery, radiation, chemotherapy, bone marrow, or
stem cell transplant? Please DO NOT consider a bone marrow biopsy to be
a bone marrow transplant. If you are currently undergoing
treatment, please indicate the current month and year."
Q3 – Is this the first time you have been treated
for any type of cancer?
Of the 14 respondents who answered Q3, 10 continued to
answer questions in Section 1 instead of skipping to Section 2. Most simply did
not see the skip instruction. Others saw it but, in glancing ahead at the rest
of the section, felt the questions applied to them and wanted to answer anyway.
- After completing Q3, one respondent read "if ever treated" at the
beginning of Q4, and stated she thought she should continue on because she
has been treated before. Additionally, she wanted to answer the remainder of
the questions to show her cancer had come back (Q4 and Q7).
- One respondent wanted to answer Q7 in order to say that her cancer had
- One respondent who had multiple cancer diagnoses thought Q3 was asking
her just about her breast cancer (more recent), but didn’t understand why
she couldn’t answer Q4-Q7.
To reinforce at Q3 that respondents need not
answer any further questions in Section 1, revise the skip instruction
to read, "You are currently being treated for cancer, please skip to
Q4 – (If ever treated for cancer.) To the best of
your knowledge, are you now free of cancer?
Although Dillman’s design approach uses parenthetical
instructions in combination with arrows and skip instructions, the strategy does
not appear to work at Q4. In part we believe this is because there is no arrow
associated with the parenthetical. As noted above, at least one respondent who
should have skipped to Section 2 read the parenthetical instruction and,
because it appeared to apply to her (she was "ever treated for cancer"),
A separate issue with the parenthetical phrase is that
it does not actually apply to all respondents who answer "no" at Q2. There are
those who have been diagnosed with cancer but either have not yet begun their
treatment or never got treatment. In fact, one of our respondents chose not to
undergo treatment because her insurance will not cover the cost.
Remove the parenthetical phrase from this
question. The new question should read, "To the best of your knowledge
are you now free of cancer?"
To accommodate those respondents who have not
been treated for their reported cancer diagnosis, add an additional
response category at Q5 that reads, "I have not been treated for cancer."
Q5 – About how long ago did you receive your last cancer treatment?
Only eight respondents arrived at Q5 correctly. All of
these respondents were thinking about their "primary course of treatment" or
"active treatment" and mentioned when their last chemotherapy or radiation
treatment occurred. These respondents noted that they were not including
biopsies, follow-up visits, or medication as "treatment" but were including
chemotherapy, radiation, and MRIs and CAT scans after radiation (to see if the
treatment was successful).
As noted above, add a category "I have not
been treated for cancer." Otherwise, leave Q5 as is.
Q7– What was the most recent year a doctor or
health professional told you that your cancer had come back?
Respondents did not appear to have any trouble
answering this question with "year."
When probed, only two respondents, both of whom were diagnosed with cancer at a
young age, said they preferred answering with something other than the year
their cancer came back. One respondent said it was easier for him to remember
his age; the other remembered what grade he was in (12th) when he was
Leave as is and continue to test in Round 2.
Section 2 – Changes to Your Work Schedule
The majority of respondents tied most of their answers
in Section 2 to their cancer, treatment and lasting effects of that treatment.
As well, most seemed to understand the questions as intended. However,
respondents did encounter difficulties within the section related to skip
patterns and the duration questions.
Q8 – At any time since you were first diagnosed
with cancer, were you working for pay at a job or business?
Sixteen of the twenty-four respondents said "yes" at
Q8. These respondents all seemed to be interpreting the question as intended.
Five of the eight respondents who answered "no" at Q8 seemed to have interpreted
the question as intended.
- One respondent was looking for a job at the time of diagnosis, stopped
after her diagnosis, and never looked for employment again.
- Several respondents were not working at the time of diagnosis (one
because he was diagnosed as a teenager) and have not worked since their diagnosis.
- One respondent was retired at the time of diagnosis and has not worked
The other three respondents had worked since their
initial cancer diagnosis but in answering the question were thinking only about
their employment status at the time they had cancer or simply did not think the
question applied to their unique situations.
- One respondent, who was diagnosed as a child, did not think the question
applied to him because he was not working when he was diagnosed. He
commented that the question "doesn’t take into account someone who was 15 at
the time of diagnosis…at 15, I was going to school."
- One respondent was diagnosed 25 years ago but did not start working
until 5 years after her treatment ended. She answered "no" at Q8, explaining
that any work changes she might have experienced were unrelated to her
- One respondent who was diagnosed 8 years ago was not working at the time
of her diagnosis and has only worked one year at a job since that time. She
felt the question did not really apply to her.
- One respondent who had cancer as a child and was not working when she
was diagnosed originally said "no" at Q8. However, after the interviewer
asked her to explain her answer, she re-read the question and decided that
she should in fact mark "yes" because she has worked since the diagnosis.
She went on to comment that the question seemed to apply more to adults
whose cancer might disrupt their job, than to those who were diagnosed as
To reinforce the reference period respondents
should be thinking about, revise Q8 to read, "At any time from when you
were first diagnosed with cancer until now, were you ever working for
pay at a job or business?"
Q9 – At any time since your first cancer
diagnosis, did you ask for extended sick leave, leave without pay, or a change
in your hours, duties, or employment status?
Nine of the sixteen respondents who received Q9
answered "yes" to having had an employment change. Of these nine, only one had
difficulty remembering if there was an employment change.
Of the seven who answered "no," most had either
continued to work their regular schedules through their treatment or had enough
sick leave accumulated that they would take days off here and there without
changing their overall schedules. However, a few selected "no" for Q9 even if
they did change their work schedules around to accommodate their treatment
because they felt the arrangements were more informal than their interpretation
- One respondent who was working part-time moved his schedule around but
did not consider it a change because he continued to work part-time.
- One respondent changed to a more flexible job after he was diagnosed so
that he would have an easier time working and undergoing treatment
simultaneously. However, he did not include this change in the employment
- One respondent, who was a teacher, had to arrange for a long term
substitute while she was undergoing treatment. The respondent was working
from home with the substitute, once a week but was not being paid.
A few of those who said "no" at Q9 had difficulty with
the word "ask."
- One respondent, who worked as a housekeeper, said that she did not have
to ask for time off because she was given time off by her
- One respondent was self-employed and so did not have to "ask" anyone
when she wanted to change her hours.
- Another respondent worked at a nursery in a bowling alley and her boss
allowed her to leave early or not come in if she was feeling sick. Her boss
would "clock her out" so she would get full pay. She never had to "ask" for
any employment change.
To accommodate respondents who did not have to
"ask" for leave, as well as to better capture the experience of those
who asked for but did not receive leave, consider the following revisions.
Revise Q9 to read:
At any time since your first cancer diagnosis,
did you take extended sick leave, leave without pay, or a change in your
hours, duties, or employment status?
Yes-Skip to Q11
Add a new question after Q9 to determine whether
or not respondents who did not take leave actually asked for the leave:
At any time since your first cancer diagnosis,
did you ask for extended sick leave, leave without pay, or a
change in your hours, duties, or employment status?
No-Skip to Q32 (see recommendation at the discussion of Q33)
Revise Q10 to read, "Did you ask for or take
these work changes…" This will require an instruction for those who said
"no" at Q9 to skip to Q32.
Note: The following results and recommendations are
based on the responses from the nine respondents who completed the remainder of
the employment section and, where applicable, seven respondents who completed
similar questions in the caregiver section.
In the caregiver section, a few respondents answered
"yes" to the question about whether their caregivers had made employment changes
but then marked "no" at all questions about the specific employment changes
because their caregivers’ situation did not fit anywhere. While this did not
happen in the employment section, we believe it is still worth addressing in
both sections. We recommend adding a closed-ended "catch-all" question after Q31
(that asks about "less demanding job"). This new question should include a
reference to "cancer, its treatment or the lasting effects of that treatment" as
a reminder of the reason for the change. Furthermore, for consistency, the
remaining questions in Section 2 should include that same reference. We believe
this will help with some of the issues described below (e.g., forgetting to
include those factors when answering about taking disability or making
retirement decisions). Related to these changes, we recommend that the skips at
Q9 and Q10 point to Q32 instead of Section 3. Although this did not come up in
testing, it seems unlikely that respondents would think be thinking about
advancements or promotions when answering Q9.
Insert a new question after Q31:
Did you make any other type of work
arrangements because of your cancer, its treatment, or the lasting
effects of that treatment?
Revise Q32 and Q33 to include "because of your
cancer, its treatment or the lasting effects of that treatment" at the
beginning of each.
See discussion at Q36 for suggested revisions
to the retirement questions.
Q11 – Did you ever change from a set work
schedule, where you start and end at the same time every day, to a flexible work
schedule, where your start and end times vary from day-to-day?
Respondents appeared to be interpreting "flexible work
schedule" as intended.
- Set hours per day and per week but not at the same time everyday;
- Being able to leave when you are tired;
- Working when you feel well enough to;
- Taking different shifts at the grocery store to accommodate treatment
One respondent who answered "yes" to both the flexible
schedule and full-time to part-time questions was thinking of the same change,
when he requested what he described as a flexible work schedule of 30 hours,
reduced from his 40-hour work week.
Leave item wording as is, but move item in the
order of employment change questions. See recommendation at Q23/Q52 below.
Q23 – Did you ever take extended paid time off
from work (vacation and/or sick time)? By extended time off, we mean more than
an occasional day off here and there.
Q52 – Did any of your caregivers take extended
paid time off from work (vacation and/or sick time)?
Most respondents seemed to interpret "extended paid
time off from work" as intended.
- One respondent included vacation time he took after receiving the
- One respondent took five months of sick leave from his "sick leave bank"
that he had been contributing to for the past 30 years.
- One respondent took one year of leave from his job. He was including
sick-leave as well as short- and long-term disability leave.
- One respondent described how her cousin took her to the beach for a
month. She used two weeks of vacation time and her boss gave her two extra
- One respondent did not consider the time her friends took off as
"extended" because it was usually just for a day. Similarly, another said
that a vacation day her husband took here and there to take her to a
doctor’s appointment was not extended time off.
A few respondents were confused about whether to
report disability leave at this question, described in further detail at the
discussion of Q33 below.
One source of confusion over where to include
different types of employment changes may stem from the current order in
which the specific changes are asked about. It may help respondents to
think first about discrete and finite types of changes such as extended
leave first, before tackling more amorphous changes such as flexible
work schedules and working from home. As well, the proposed revision has
the advantage of consistency with the order in which changes are
described in Q9. We recommend the revised order for both the employment
and caregiver sections.
Extended Paid time off
Unpaid time off
Switch from Part-time to full-time
Switch from full-time to part-time
Flexible work schedule
Work from home
Less demanding job
Q29 – Did you ever change to a less demanding job?
Q58 – Did any of your caregivers change to a less
Most respondents seemed to interpret the question as
- One respondent who checked "no" reported that she is an accountant and
continues to perform the same type of work she did before her diagnosis. She
works fewer days a week and fewer hours a day now, but does not consider
this to be less demanding. To her, a less demanding job would be changing to
a different position, reporting to someone else, or performing a different
- One respondent who answered "no" noted that she thought a less demanding
job "requires adjustments, like less hours, less straining tasks, or less
Ultimately, only two respondents answered "yes" to Q29
and none answered "yes" to Q58.These two struggled somewhat with whether their
employment change could be considered "less demanding."
- One respondent, who answered "yes,"
said that she changed from working as a cashier to working in the back of
the grocery store, which she considered to be an easier job. While she did
not make this change because she needed something less demanding, she did
make the change because of her cancer diagnosis.
- Another respondent who answered "yes" noted that although her job title
remained the same, she felt the workload she was given after her diagnosis
was lighter. Additionally, she no longer can get raises or promotions.
Leave as is.
Q33 - Did you ever leave work on disability?
Several respondents were unsure about what to include
under "leave work on disability". Several respondents thought that disability
and extended time off were similar, if not interchangeable.
- One respondent could not distinguish between disability and extended
time off and so decided to answer "yes" at this question and at Q23 for the
same employment change.
- One respondent noted that she took extended time off but only received
60% of her salary. Further probing revealed she was likely receiving
disability. She answered "yes" at Q23 but "no" at Q33.
- One respondent answered "yes" to Q33 but for a disability leave that was
not related to her cancer.
She described the question as "very confusing" and suggested it clarify
whether cancer-related disability leave should be included.
- One respondent at first thought she should report her three-month
disability leave at Q23, but ultimately decided to report it at Q33 instead.
We would like to further discuss with the
workgroup the intended meaning of "leave work on disability" before
providing additional recommendations for Q33.
Q36 – Has your experience with cancer, its
treatment or the lasting effects of that treatment had a mostly positive, mostly
negative or an equally positive and negative effect on your retirement decision?
Respondents reported a variety of issues with this
item. Two left it blank and others suggested adding a "no effect" category
because they felt their retirement decisions had nothing to do with their cancer
or they were not yet thinking about retirement. Additionally, respondents seemed
to be answering more generally about their work experience instead of tying
their retirement decision to their cancer diagnosis.
To help focus respondents on their retirement
decision as it relates to cancer, consider the following revisions.
Skip those who say no to both Q34 and Q35 past
Q36. That is, if they have not made a retirement decision, do not ask
about the effect of cancer on their retirement decision.
Add "because of your cancer, its treatment, or
the lasting effects of that treatment" to Q34 and Q35.
34. Because of your cancer, its treatment, or
the lasting effects of that treatment, did you retire earlier than you
Yes-Go to Q36
35. Because of your cancer, its treatment, or
the lasting effects of that treatment, did you delay your retirement
beyond when you had planned?
No Go to Section 3
36. Thinking about your retirement decision,
what effect has your cancer, its treatment or the lasting effects of
that treatment had on it?
Mostly positive effect
Mostly negative effect
Equally positive and negative effect
Neither positive or negative effect
Section 3: Other Aspects of Work
All 16 employed respondents answered the section about
other aspects of work.
Q37 – Did you ever feel that your cancer, its
treatment or the lasting effects of that treatment interfered with your ability
to perform any physical tasks required by your job?
Two respondents marked that they did not have to
perform physical tasks for their job and eight answered "yes" to this question.
Their interpretation of "physical tasks" ranged from light to strenuous,
depending on the type of job they had.
- Walking around (teacher; car salesman);
- Lifting or pulling when putting away supplies or sitting for long
periods (office manager);
- Standing for long periods (appliance salesman)
- Doing laundry, cleaning, making beds (housekeeper);
- Vacuuming, sterilizing toys, holding children (nursery worker);
- Lifting heavy bags, transferring passengers in wheelchairs (airline
One respondent who answered "yes" was thinking of the
time before her cancer diagnosis. She related that before she knew she had cancer,
"I became so physically debilitated that I was unable to do a lot of the physical things
I was normally able to" in her job as a teacher. Her energy returned after she finished
her treatment such that she was able, once again, to perform the physical tasks required by
her work. Another respondent who also considered the time before her diagnosis ultimately
decided not to include it in her answer, which was "no."
If the workgroup is satisfied with respondents’ broad interpretation
of physical tasks, leave the question as is.
Q38 – Did you ever feel that your cancer, its
treatment, or the lasting effects of that treatment interfered with your ability
to perform any mental tasks required by your job?
Eleven respondents answered "no" and five said "yes"
to this question. Both those who did and did not report interference seemed to
be interpreting "mental tasks required by your job" as intended.
- Respondents included in "mental tasks required by your job" things such
as memory, composing emails, making calls to customers, maintaining
"reading or doing any kind of math or computing", multitasking, getting work
done on time, making judgments, and concentrating.
- Those who answered "yes" noted that their cancer and its treatment
caused them to take longer to process complex tasks, become "very
forgetful", be more easily distracted, and have difficulty remembering
- Many respondents, regardless of how they answered the question,
mentioned "chemo brain" or "chemo fog", or described how chemotherapy can
make you "slower" for awhile.
Leave question as is.
Q39 – Did you ever feel that, because of your
cancer, its treatment, or the lasting effects of that treatment, you were less
productive at work?
Overall, respondents appeared to understand and
interpret "less productive at work" as intended. Only one respondent, with
limited English fluency, did not tie her decrease in productivity to her cancer,
but rather explained that she became less productive at work because of the
Some respondents acknowledged feeling less productive:
- A male respondent who is employed at a car dealership said he was
productive mentally but "…couldn’t do everything the way I wanted it
done…having the cars looking the way I wanted…"
- One respondent reported being less confident in her work, causing her to
check her work more often which she said resulted in decreased productivity.
Others reported no change, or increased productivity:
- One respondent who answered "no" said that while she could feel the
effects of her treatment, she had "not yet" noticed herself being less
productive at work because of it.
- A hairdresser answered "no" because she can still mix colors and style
- Two respondents who work with children, one as a teacher and the other
as a day care provider, said they did not consider themselves to be "less
productive" because they were still able to work with children. The teacher
said that physically, she was less productive, but mentally and emotionally
she pushed herself "harder than ever,"
and she could still "be there emotionally" for her students. The day care
provider said she could not vacuum, sterilize toys, or clean up, but was
still able to have contact with the children.
- One respondent, a writer, described how his experience with cancer had
made him more creative and helped to improve his writing.
Leave question as is.
Q41 – Did you ever stay at a job in part because
you were concerned about losing your health insurance?
Four of the five respondents who answered "yes" to
this question did not relate their answers to their cancer.
- One respondent described an experience many years ago of keeping a job
due to health issues, but not any that were related to her cancer. She went
on to point out that the question does not specify that staying in one’s job
because of concerns about health insurance be related to cancer.
- Another explained that she stayed at a job because she was afraid of
losing her health insurance due to other health conditions unrelated to
- The third tied her answer to her responsibility toward her family,
stating, "…If I don’t work, my children and my husband will not have health
- One respondent said his concerns are related to "the exorbitant cost of
health care in general."
Add a follow-up question for those who answer
"yes" at Q41 that reads:
Were you concerned about losing your health
insurance because of your cancer?
Q42 – Has your experience with cancer, its
treatment or the lasting effects of that treatment had a mostly positive, mostly
negative, or an equally positive and negative effect on your work life or career?
Some respondents seemed to be answering Q42 as
- The respondent who is a hairdresser felt her cancer had a positive
effect on her work life because she now talks with clients about skin cancer
and encourages them to have their moles looked at.
- One respondent explained that she selected "mostly negative" because she
is more insecure about work now. She did not feel that her employers were as
helpful as they could have been nor that they adjusted their expectations of
Others appeared to be answering for how cancer, its
treatment or the lasting effects of that treatment impacted their life in
general, not just their work.
One respondent asked why there was not a response
option for cancer having neither a positive nor a negative effect. Another
respondent chose not to check any of the response options but wrote "neither"
below the response options.
To help focus respondents on their work revise
the question to read:
Thinking about your work life or career, what
effect has your cancer, its treatment or the lasting effects of that
treatment had on it?
Mostly positive effect
Mostly negative effect
Equally positive and negative effect
Neither positive or negative effect
Q43 – Did your spouse or significant other ever
stay at a job in part because he/she was concerned about losing health insurance
for the family? Yes, No, Don’t have spouse/significant other, Spouse was not
employed at any time after I was diagnosed
This questions makes multiple assumptions about
insurance, marital, and employment status that sometimes proved challenging for
respondents when thinking about their answers.
- One respondent, who had had childhood cancer hesitated and then checked,
"Don’t have a spouse/significant other", explaining, "…of course I didn’t
have a spouse. I was a kid!"
- Another respondent with childhood cancer remarked, "…I was only 15!"
- One respondent left the question blank because, although her husband is
employed, he does not have health insurance, so she felt none of the
response options applied to her.
Revise question to read:
Did your spouse/significant other or
parent/guardian ever stay at a job in part because he/she was concerned
about losing health insurance for the family?
Does not apply
Section 4 – Caregivers
Seven respondents reported that their caregivers made
employment changes. The workgroup was concerned that respondents have in mind a
broad definition of caregivers when answering the questions in this section. For
the most part, we found that to be the case. Respondents included friends,
neighbors, significant others/spouses, co-workers, and relatives (daughter,
cousin, sister, grandson, parents). One respondent counted her doctor as a
caregiver, in addition to friends and family. Another respondent was only
thinking of medical professionals until he read the section introduction, at
which point he revised his answer to include friends and family.
When asked to describe the type of caregiving
provided, respondents mentioned bringing and cooking food, praying for them to
get better, taking care of house plants, performing chores, shopping, taking
children to events, keeping the respondent focused, driving them to
appointments, taking notes and asking questions at appointments, visiting
respondents at the hospital, taking care of them after surgery, picking up their
medicine, being supportive, listening, doing heavy lifting, giving haircuts, and
changing bedpans and other medical tasks (emptying drain tubes, cleaning
Q45 – Because of your cancer, its treatment or the
lasting effects of that treatment, did any of your caregivers ever take extended
sick leave, leave without pay, or change their hours duties, or employment
Three respondents reported that their caregiver
experienced an employment change, but did not select any of the changes listed
in the remainder of the section.
- One respondent was thinking of her daughter’s availability due to her
flexible work schedule, which did not require any "drastic changes."
Consequently, the specific changes described in the subsequent items in
Section 4 did not apply to her situation.
- Another respondent said her spouse worked fewer hours, but did not
"officially" take time off from his job.
- The husband of another respondent worked part-time for 2 months while
she was in the hospital, but it was not an official change so she did not
feel Q46 adequately captured his particular situation.
One respondent who was diagnosed as a child selected
"no" because he did not remember whether his parents had to make employment
changes at the time, and another with a child diagnosis answered "yes" but was
unable to recall the details of the employment changes.
As in Section 2, add a question before Q58 that reads:
"Did any of your caregivers make any other
type of work arrangements because of your cancer, its treatment, or the
lasting effects of that treatment?"
Revise Q58 throughQ61 to include "because of
your cancer, its treatment or the lasting effects of that treatment" at
the beginning of each.
To differentiate between a true "no" answer
and those who may not be aware of their caregiver’s employment situation, or may
not remember because it was in the distant past or because they were children, we
suggest including a "don’t know" option at Q45 (with a corresponding
skip out of the section).
Section 5 – Experiences with Health Insurance
Q62 – At any time from when you were first
diagnosed with cancer to now, were you covered by health insurance that paid for
all or part of your medical care, tests or cancer treatment?
Most respondents had little trouble understanding or
answering the health insurance question. Those who answered "yes" described a
variety of health insurance situations (such as through work, a
spouse/significant other/parent, or a government-funded program such as
Medicaid) and, with one exception, those who answered "no" did so with little
trouble. One survivor of childhood cancer was unsure at first how to answer
because his medical care was covered by an NIH research grant but he has never
had health insurance. He ultimately answered with "no."
Leave question as is.
Q63 through Q65 – Was there ever a time when
health insurance refused to cover…?
There were two issues with this series of questions.
First, the "no" answer could indicate either that respondents sought the benefit
and were denied or that they never sought it in the first place. For example,
one respondent who answered "no" at Q64 and at Q65 explained that she had never
asked for a second opinion or tried to use another facility. Two respondents
(one of whom is on Medicaid) explained that they have only ever seen doctors or
used facilities that they know are approved by their health insurance plans, so
the concept of having a "choice" about those things does not apply to them. The
second issue is that some respondents who answered "yes" actually ended up
receiving the benefit denied them by the health insurance company because they
decided to pay for it out of pocket or they found another way to get coverage.
We would like to discuss these issues further
with the workgroup before making a recommendation.
Q66 – Were you ever denied health insurance coverage
because of your cancer.
Q67 – Were you ever denied long-term care
insurance coverage because of your cancer?
Similar to the issue in Q63 through Q65, one
respondent was unsure how to answer Q66 because she had not applied for any
health insurance since the onset of her cancer. Some respondents also pointed
out that they had never been denied long-term care insurance because they had
never applied for it. In addition, some respondents did not know what long-term
care insurance is.
- One respondent asked, "…Is it senior insurance?"
- Another commented, "I’m not sure which long-term health care you mean"
but speculated it may be what someone has when they are older and need to be
in a facility.
- One respondent said, "I don’t think I’ve ever been denied it, but I know
I’ve been told that I will either have to pay a lot for insurance or it’s
going to be really, really hard to find…I’m not sure I really know what you
mean by long term care insurance."
Depending on the saliency of the issue of long
term care insurance for cancer survivors, consider dropping the question.
If it is retained, consider adding a
definition of long term care and a third response option, "I have not
applied for long term care insurance."
Section 6 – Finances
Seventeen respondents said "no" at Q68, 5 said "yes"
and one left it blank. She had cancer as a child and explained, "I wasn’t sure
about borrowing money or debt; I know we scraped by for a while. But I don’t
know specifics and I don’t know how to answer that."
Some respondents tied their financial difficulties
directly to their cancer, its treatment or the lasting effects of that
- One respondent said "yes" at Q71 (making other kinds of financial
sacrifices) because she could not afford to live on her own after paying for
- One respondent was mainly thinking about money to pay for the prosthesis
he received as a result of his cancer. He said that they are very expensive
and at times, he has had to "let other things go… my prosthesis is the only
cancer-related thing I’m worried about."
Others decided to exclude financial difficulties that
arose indirectly from their cancer experience. For these respondents, the term
"medical care" seemed to narrow their thinking in that regard.
- One respondent answered "no" at Q68 and at Q71 because all of her
medical expenses were covered by her health insurance. When asked to explain
her answer, the respondent revealed that she did have to borrow money for
gas, bills, food, and other daily expenses because she was on disability
leave for her cancer and only receiving 60 percent of her salary.
- One respondent who marked "no" for Q71 said he chose that answer
"because it is medical care related.
He explained that he has not made financial sacrifices to pay for medical
care, but has made sacrifices to pay for other things because his income
suffered after he retired early because of his cancer. Another respondent
had a similar situation where he had no medical debt because his insurance
covered everything, but he lost his job because of his cancer and could no
longer pay other bills,
like his mortgage.
For those whose cancer experience was in the distant
past or continued over a long period of time, recalling the amount of money
borrowed or debt incurred proved challenging. At least one chose to leave the
amount question blank for that reason and another answered "no" at Q68 because
she could not remember.
If the workgroup is interested in capturing
the indirect as well as direct financial impact of cancer, consider
refocusing this section to ask about cancer, its treatment and the
lasting effects of that treatment. For example, Q68 might read:
Have you or has anyone in your family had to
borrow money or go into debt because of your cancer, its treatment or
the lasting effects of that treatment.
Section 7 – Medical Care
Q74 – At any time since you were first diagnosed with
cancer, did any doctor ever discuss with you…
For respondents with recent or single cancer
diagnoses, this question series worked well. However, some respondents with
cancer diagnoses from the distant past struggle with recall. One said, "How
could I know whether I talked with my doctor in detail 25 years ago?" And a
respondent with multiple diagnoses said she has had so many different doctors
over such a long period that she did not know how to respond. She ultimately
decided to answer for the experiences that were most positive for her, choosing
"discussed in detail" or "briefly discussed" for all sub-items.
Several respondents mentioned talking with social
workers or counselors about their social and emotional issues, noting that
doctors are not usually the ones to discuss those topics. Some respondents
decided to include these other kinds of practitioners in their answers while
others answered only for doctors. For example, two respondents spoke with social
workers about their social and emotional issues, but answered "did not discuss"
We would like further guidance from the
workgroup about whether respondents’ answers should include other
healthcare practitioners before providing a recommendation.
Q75 – At any time since you were first diagnosed
with cancer, were you unable to obtain medical care, tests, or treatments that
you or your doctor believed were necessary?
Several respondents had trouble interpreting, "unable
to obtain", commenting that it was "stiff, awkward language."
For example, one respondent explained that she was able to get all the tests she
needed, but could not figure out how to answer the question, focusing her
frustrations on the word "unable."
Q75 is worded such that a "no" answer results
in the confusing idea that one was "not unable" to obtain medical care.
Consider revising the question to read:
At any time since you were first diagnosed,
did you get all of the medical care, tests, or treatments that you and
your doctor believed were necessary?
If this change is made, Q76 and Q77 would be
revised to read:
Which of these are reasons you did not get all
of the medical care, tests, or treatments that you or your doctor
believed you needed?
Which one of these is the main reason
that you did not get all of the medical care, tests, or treatments that
you or your doctor believed you needed?
Section 8 – Effects of Treatment on Life in General
Q78 – Did your cancer, its treatment or the
lasting effects of that treatment ever limit the kind or amount of activities
you do outside of work, such as shopping, child care, exercising, studying, work
around the house, and so on?
Respondents did not appear to have difficulty with Q78
and most reported some kind of limitation.
- A respondent who had osteogenic cancer as a child and is now an amputee
answered "yes," saying, "have I been limited… sure I have. It’s been a
- One respondent who had multiple cancer diagnoses noted that only one of
his cancers limited him, describing how it leaves him feeling drained and
fatigued to where he cannot do his usual exercise routine.
- One respondent reported not being able to do any heavy lifting and
needing to pace himself and take breaks when performing physical activities
such as doing the dishes, washing towels, or power washing. He also noted
that he cannot carry heavy things for long distances, go to football games,
or go for walks around the park.
- A respondent described not being able to garden and being tired from
One respondent who is currently undergoing treatment
said she did not feel limited at the moment, but answered "yes" at Q78 based on
her assumption that soon she would be. Two older respondents reported
limitations but were unsure whether they were the result of cancer or the aging
Q79 – How long were you limited, in weeks, months,
or years, in the kind or amount of usual daily activities?
Calculating the amount of time they were limited in
the activities was exceptionally challenging for many respondents.
- One respondent left this question blank, stating that the first year she
could not do a lot of things, but afterwards she could. She wrote on the
side of the response options, "couldn’t do 1st year" instead of
selecting an amount.
- One respondent asked, "When?...During chemo?...Here and there?...After
chemo?" in the end the respondent decided to put down 8 months, the length
of time she has been in treatment so far.
- Another respondent, with melanoma, remarked that she is limited in her
activities every day, since she must avoid exposure to sunlight. She marked
the "year" box without specifying how many.
Several respondents had difficulty with calculating
the limitations because they were not continuous. All three respondents in the
examples below left Q79 blank, as did others.
- One respondent who had cancer 14 years ago and had recently had a
recurrence could not answer this question because she found it impossible to
quantify her limitations over such a long period of time. She noted that she
experienced the effects from her cancer, but that they were not "tangible
- Another respondent who had cancer as an adolescent had difficulty
quantifying her answer because the effects were different at varying points
in her life. She reported, "I wasn’t sure because it’s varied over the
years…when I was younger it limited me most every day…since elementary
school the effects have waned out, so I wasn’t really sure what to put for
- One respondent wanted to answer with a percentage but couldn’t figure
out how to, commenting, "the time period…I don’t understand that at all."
Based on this first round of testing, we might
expect a significant amount of missing data on Q79. As well, it is
exceptionally burdensome for respondents given the vague concept
(limitation of daily activities) they are being asked to quantify.
Consider dropping the question. If the
workgroup wishes to retain it, revise with response categories similar
to those suggested for the employment and caregiver duration questions.
We would like to further discuss this item with the workgroup before
providing a specific recommendation.
Q80 – Did you ever feel that your cancer, its
treatment or the lasting effects of that treatment interfered with your ability
to perform any mental tasks as part of your usual daily activities?
Similar to Q38, most respondents did not have any
trouble with this question and seemed to be interpreting it as intended. Most
respondents did not report any interference with their mental abilities in
reference to their usual daily activities.
- Eleven respondents answered "yes" to this question. Examples of how they
defined the ability to perform mental tasks include decision making; trying
to learn new tasks and concepts; and relying on memory (e.g., remembering to
turn off the stove).
- One respondent who answered "no" said she felt "loopy and dizzy" right
after surgery, but it was so short that it did not interfere with her daily
activities during that period.
- Another respondent, who answered "no" said his treatment did not
interfere with any tasks, but just with his ability to think and
- One respondent reported that there has been no effect on his ability to
perform mental tasks because of his cancer. In fact,
just the opposite. He has been able to write several articles and a book
about people’s experiences with cancer.
There may be some social desirability associated with
this question and its counterpart in the employment section. Interviewers
noticed that some respondents who answered "no" would, over the course of the
interview, describe situations where their ability to perform mental tasks was
compromised. A few respondents emphatically stated that cancer and its treatment
did not interfere at all with their ability to perform their jobs. This biasing
effect may disappear, however, during data collection, when respondents will be
completing the survey on their own rather than in the presence of an
Leave question as is.
Q81 – Have you ever needed any of the following
kinds of help because of your cancer, its treatment or the lasting effects of
Several respondents pointed out that "needing help"
does not actually mean the help was received.
- One respondent explained that she needed help keeping track of her
medications and figuring out her insurance, but she had to do it herself
since there was no one available to help her. She answered "no" at both Q81b
- Another respondent related that he didn’t actually "need" his friends to
come to doctor’s’ appointments with him but that he had asked them to
Respondents who had childhood cancer also had an issue
with the concept of "needing help." It was difficult for them to think of
themselves as children needing "help" from their parents getting to the doctor
or taking medications. In those situations, parents are not "helping" their
children; they are simply performing their normal parental duties. For example,
one respondent who answered no to all of the items listed in Q81a through Q81d
explained that his parents took care of all of those things but that he did not
consider them as having helped him because they were fulfilling their
responsibilities as parents. For these kinds of respondents, "no" could either
mean they did not need the help because they did it themselves or they did not
need the help because their parents did it for them.
We would like to further discuss this item
with the workgroup before providing a recommendation.
Q84 – What do you think are the chances that your
cancer will come back or get worse within the next 10 years?
This question seemed odd to elderly respondents, some
of whom were old enough to be more concerned about whether they would still be
alive in 10 years, never mind their cancer coming back.
We also noticed that some respondents answered the
question based on what their doctors or others had told them, even if they
themselves disagreed. . This response pattern introduces a potential for bias,
failing to capture respondents’ true anxiety levels.
- One respondent felt the chances of her cancer coming back were quite
high, but checked "fairly low" because her doctor had given her that
- One respondent marked "moderate" because it is the middle ground between
what actually thinks (fairly low) and what her doctor thinks (very high).
- One respondent marked "very low" because this is what her husband, a
cancer genetic specialist, has told her. She herself believes her changes
are very high.
Two respondents simply did not know but could not find
a place to record such an answer. One of them decided to mark "moderate" because
its position at the mid-point of the list of choices was the closest she felt
she could come to "don’t know."
Revise Q84 read:
In your own opinion, what do you think are the
chances that your cancer will come back or get worse within the next 10 years?
Add a "don’t know" response category.
Q85 – Which of these, if any, have been positive things about your experiences
with your cancer, its treatment, or the lasting effects of that treatment?
Most respondents enjoyed answering this question and
talking about their positive experiences. One respondent left it blank, saying
emphatically that "there is nothing positive about being sick." Another felt
that none of the options really fit but did not want to leave the question
Respondents were asked if there were any other
positive things related to their cancer besides those listed. Examples of the
kinds of things they added include:
- I appreciate friends more
- I can help others going through the same experience
- I learned to do things I enjoy
- I have a better attitude ("laugh at more silly things")
- I gained different perspectives on life
- I met new people (in support groups)
- I live more spontaneously now
- I appreciate life more
For improved data capture on a self-administered questionnaire,
consider adding "yes" and "no" response boxes similar to those used
in Q81. Alternatively, add a response category, "There has been nothing positive
about my cancer experience."
 Facebook users post links to websites, videos,
photos, documents and any other content they "like" as a way of sharing their interests and activities with their friends.
Facebook users can also click on the "like" button located next to content on their friends’ pages as a way of acknowledging
a shared interest.
 Dillman, D. A. (2009). Mail and internet survey:
The tailored design method (2nd ed.). Hoboken, NJ: John Wiley and Sons.
 These are preliminary findings based on examination
of missing data. No articles or technical reports have yet been written about these findings.
 One respondent’s literacy level was so low that the
interviewer had to read the survey items to her. Even with that help she had excessive difficulty understanding many of the
items and often did not provide a codeable answer. For that reason we have not included her data in the reported counts except
for in selected areas.
Return To Top