Introduction

The National Cancer Institute, the Centers for Disease Control and Prevention, and the American Cancer Society are developing a self-administered questionnaire, Your Experiences with Cancer, for current cancer patients and cancer survivors identified in the Medical Expenditure Panel Survey sponsored by the Agency for Health Care Research and Quality. The questionnaire will be fielded in Spring 2012 (MEPS Panel 16 Round 3 and Panel 15 Round 5).

The Medical Expenditure Panel Survey produces national estimates of healthcare use and expenditures. The Cancer SAQ will supplement MEPS data, providing information about the burden of cancer in the US, long lasting effects of the disease, financial impacts, and employment outcomes for cancer survivors and their families. Findings from this research will provide key information needed for the development of interventions to improve the quality of the cancer survivorship experience, as well as pivotal information for policy makers in understanding and improving the burden of cancer.

As part of the development process, Westat plans to conduct 2 Rounds of cognitive testing to assess how well respondents are able to understand and answer the questionnaire items. Round 1 cognitive interviews with 24 English speaking respondents were conducted between June 7 and June 30, 2011. This memo summarizes findings and recommendations from that testing effort.

Methods

Recruiting

Working from our own as well as sources provided by the SAQ inter-agency workgroup, Westat recruited respondents in Washington, D.C., Baltimore, and the surrounding areas using several methods. We posted flyers at local cancer and community clinics. We networked with support groups such as Relay for Life, Patient Family Support Group at Suburban Hospital and Hopewell Cancer Support in Baltimore. We created a Facebook advertisement targeted to users who live within 10 miles of Washington, D.C., Baltimore, MD, Rockville, MD, or Frederick, MD and who "like"^[1] the following cancer related categories:

• Breast cancer survivors;
• Cancer research institute;
• Relay For Life;
• Susan G. Komen foundation;
• Cancer awareness (brain, breast, colon, leukemia, lung, ovarian); and
• Various cancer research groups.

These criteria yielded a pool of 35,340 Facebook users. When users clicked on the ad, they were directed o a Zoomerang survey that described the research in more detail and asked for contact information. Westat monitored the Zoomerang results on a daily basis and followed up with all "hits" to screen and, if eligible, schedule an interview. Early in the recruiting effort, several potential eligible respondents indicated that they were willing to participate in the pretest but were unable to travel to Westat’s Rockville location to be interviewed. As a result, we revised the Round 1 protocol to include alternative interview locations,

Respondent Characteristics

To be eligible, potential respondents had to have a current or past cancer diagnoses. In addition, participants were screened and selected based on type of cancer, time since last diagnosis, employment status, and number of caregivers. Table 1 shows the distribution of demographic characteristics across Round 1 respondents. About half the respondents had a current diagnosis and the rest were spread fairly evenly among categories representing diagnoses in the recent and distant past They were also split evenly between employed and unemployed at the time of diagnosis. The Round 1 respondents represented a good mix of education levels. However, they skewed heavily female, older and White. Finally, although not evident from the demographic characteristics, we noticed that many of these respondents were savvy about the topic of cancer in general as well as their own particular medical situations. For example, one respondent writes a cancer blog and two others write articles, short stories, and novels about their experiences with cancer. This is likely a result of the recruiting sources used for this effort and may not accurately reflect the experiences of typical MEPS respondents.

Table 1. Distribution of Round 1 demographic characteristics

*One respondent did not provide her age.

**One respondent with multiple diagnoses was employed at the time of his first diagnosis and retired at the time of his second diagnosis.

Protocol Development

With input from the workgroup, Westat developed an interview protocol that explored issues of concern using scripted probes and guidelines for unscripted probes. General issues for testing included item wording and response option appropriateness; feasibility and accuracy of respondent recall for various items; appropriate use of words and phrases in the question stems; and ease or difficulty answering some items. Examples of specific issues for testing appear below. More detail appears in the final interview protocol, Attachment D.

• Appropriateness of the Cancer SAQ for those with a wide variety of cancer experiences, including current diagnosis, multiple diagnoses, and last treatment in the recent and in the distant past.
• Ability of respondents to answer for all three survey concepts – cancer, its treatment, and the lasting effects of that treatment – across the entire SAQ.
• Respondents’ interpretation of specific words and phrases such as "lasting effects," "flexible work schedule," "extended paid time off," and "mental tasks."
• Ability to answer questions about caregivers’ employment experiences.
• Recall of certain life events, such as the duration of employment changes.

As part of the protocol, Westat also developed an interview introduction script (Attachment E), which included informed consent language, and the informed consent form used to document written consent (Attachment F).

Training

The cognitive interviews were conducted by a team of five skilled qualitative researchers with survey design experience and two MEPS project staff who received intensive cognitive interview training for this task. All interviewers reviewed the SAQ and interview protocol prior to a more formal 3-hour training session held a few days before the interview. The training agenda included the study background and purpose; overview of the testing effort, discussion of questions; and logistics of interview administration. The team leader and lead analyst conducted the first four interviews. Interviewers were required to observe at least one of these "model" interviews before conducting their own interviews. The team leader listened to approximately 20 percent of the interviews and provided feedback as needed. As well, the team leader and lead analyst communicated to the interviewing team throughout the testing period to provide updates and instructions. Each interviewer conducted approximately 2-5 interviews.

Conducting Cognitive Interviews

Respondents participated in a one-hour in-person interview either at Westat’s Rockville offices or at a location of their choice (usually in their homes, sometimes at a neutral location such as an office at a local university). They received $40 for their time as well as a $10 travel allowance. After introducing the study, its purpose and the cognitive testing procedures, interviewers asked respondents to sign the informed consent form. The interviewer then asked the respondent to read and react to the cover page, after which he/she was instructed to begin filling out the questionnaire. Using the protocol, interviewers administered a combination of mostly concurrent and some retrospective probes to obtain respondent feedback on how easy or difficult it was to understand and answer the survey questions. The technique of asking respondents to describe their thought processes immediately after answering selected survey items allowed the cognitive interviewer to focus on comprehension and response processes. In addition to administering the scripted concurrent probes, interviewers also noted any unanticipated reactions or problems with the items and followed up on those issues as needed.

Analysis of the Interview Data

Westat analyzed the interview data in a series of steps. The first step was for each interviewer to review and synthesize the data (from the audio-recordings and from any notes she may have taken) from her own interviews. During this step, the interviewer identified themes within each respondent’s interview and across all the interviews she conducted. The themes were organized by overall survey issues, individual survey items and sections, and respondents’ overall reactions to the survey. In the second step, all interviewers met and discussed the themes they had identified. As a group and with the guidance of the lead analyst, the team identified common themes (as well as unique situations worthy of notice) within and across all interviews. The team also began to identify recommendations for revising the survey based on the cognitive interview results. In the final step, the lead analyst conducted another review of all interview data and organized the themes into the report of findings and recommendations. This final review ensured that the recommendations are thoroughly grounded in the cognitive interview data.

Findings and Recommendations

The Cancer SAQ attempts to capture a broad range of experiences from a wide variety of cancer patients and cancer survivors. Overall, it met those objectives fairly well. Most respondents understood and interpreted most items as intended. The issues that arose in testing can for the most part be addressed with wording revisions, formatting adjustments, or skip pattern updates that can be seamlessly incorporated into the existing survey structure.

The remainder of this memo presents findings and recommendations from the Round 1 cognitive interviews. The first section identifies issues experienced by certain types of respondents and describes some broad observations. Note that, with the exception of the duration questions, we do not include recommendations in the broad observations section as they are addressed in the detailed recommendations. The next section is organized by survey section and item. It provides detailed findings and recommendations to address problems that surfaced during testing.

Broad Observations: Issues Experienced by Selected Respondent Types

The Cancer SAQ is targeted to respondents who are currently undergoing treatment and those whose treatment occurred at any time in the past. It is intended for those who have had one cancer occurrence as well as those with multiple cancer diagnoses. Those with one cancer occurrence, either current or in the recent past, had the least trouble navigating through the survey. The remainder of this section describes issues experienced by those with multiple diagnoses, those whose last occurrence was in the distant past, and those with childhood diagnoses.

Multiple Cancer Diagnoses

Of the 24 respondents participating in the cognitive interviews, three reported a recurrence of their cancer and eight had multiple cancer diagnoses (which for one respondent included a recurrence). For these respondents, answering the questionnaire was not a straightforward process, nor did the survey items always do a good job of accurately capturing their diverse experiences.

• Generally, respondents with multiple experiences of cancer did not know whether to answer items based on their first cancer, or subsequent cancer(s). This was especially true for respondents whose first diagnosis occurred in the distant past or in childhood.
• Some respondents focused on their most severe diagnosis, or cancers requiring extensive treatment and recovery. These respondents tended not to answer for cancer diagnoses that they considered to be less severe, or that required only short, outpatient procedures and minimal recovery time. For example, one respondent who had been diagnosed with melanoma and then later with prostate, often selected the answer that fit the scenario for the prostate treatment since he felt the melanoma "wasn’t that serious."
• Respondents with more than one severe cancer had difficulty deciding which one to talk about. When formulating answers, some respondents aggregated their cancers while others referred to different cancers for different items. Still others indicated confusion at various points in the questionnaire, asking the interviewer "which one?" throughout.

Cancer Diagnoses in the Distant Past

Four out of the five respondents recruited in the "6 years +" category had their last treatment ten or more years ago. Recall was often difficult for them in the finance and medical care sections, and especially difficult for less salient items, such as the content of their discussions with their doctor. Additionally, these respondents had difficulty remembering details such as the amount of money they borrowed or how much time they worked at home. Some examples are described below:

• One respondent, whose treatment was 25 years ago, struggled to remember information for answering the medical debt and medical care questions. At the Q74 series that asks about discussions with the doctor, the respondent said she "didn’t remember discussing" several of the topics with her doctor because it was such a long time ago, but marked "did not discuss at all" rather than "I don’t remember" as her answers. This is troublesome because a similar pattern in the survey population would end up inflating the proportion of "did not discuss" answers and thereby bias the survey estimates.
• Another respondent who had multiple bouts of cancer 35 years ago said that the questionnaire was "problematic" for him because he felt that many of the questions assume a more recent cancer experience. The respondent had difficulty recalling how his cancer impacted his caregiver’s employment, or whether his doctors discussed the topics at Q74 in detail with him.
• For one respondent, difficulty recalling her last treatment 17 years ago may have been confounded by the fact that she was only three years old when she was diagnosed.

Cancer Diagnosis as a Child

Four respondents were diagnosed with cancer as children. These respondents explained that because they were so young during treatment, they didn’t remember enough to answer some of the questions, they were unaware of what actually happened, or their parents made all of the decisions related to their cancer.

• A respondent who was diagnosed with cancer when she was three said, "I’m finding it probably more difficult than other people might find it just because it was a long time ago…a lot of this stuff is based on what my parents have told me…I don’t think I have the whole, clear picture…This would be way easier for me to fill out with my mom by my side." She had particular difficulty in the insurance section because she said her mother would have dealt with those issues.
• Another respondent echoed this feeling and had difficulty answering some of the questions since his first experience with cancer was when he was in 5^th grade. He stated he does not remember many things from that episode of cancer, and answered the questions thinking mostly about his most recent experience with cancer (5 years ago).
• A third had the most difficulty in Section 5. He reported "no" for all of the questions, but after talking with the interviewer, it was clear he did not actually know what the answers were and that he would have had to consult with his parents in order to get them.
• Respondents with childhood cancer also had difficulty with the finance section because they "don’t know specifics". Respondents commented that their parents were the ones who dealt with the financial matters.

Some respondents with childhood diagnoses noted that the questionnaire seemed to have been designed for those who were diagnosed with cancer as an adult, particularly items in the Employment and Caregiver sections.

• At Q8 (…were you working for pay at a job or business), one respondent initially answered "no" because she was not working when she was diagnosed with cancer at age 3. In explaining her answer she said, "I was thinking there are a lot of people who are older and they have cancer and they’ve been working and it kind of disrupts things…I was three …I wasn’t affected by it; but I have been employed since." In talking through her rationale, she realized that the question was asking about employment since her diagnosis and changed her answer to "yes."
• Another respondent felt that most of the employment and caregiver questions did not apply to his situation since he experienced cancer when he was a child. He pointed out that he was not working and he was not aware of the situation his parents had in regards to their work schedules while he was undergoing treatment.
• A respondent who has held a number of jobs as an adult, long after his cancer diagnoses at ages 15 and 16, commented that Q8, "…doesn’t take into account someone who was 15 at the time of diagnosis. At 15, I was going to school". He also said that the question was "vague", adding, "I didn’t start working until I was 20". The respondent also had difficulty in the caregiver section and felt that the questions were designed for people who were diagnosed more recently.

Broad Observations: Survey-wide Issues of Note

"Lasting Effects"

Respondents’ interpretations of "lasting effects" varied, but for the most part they seemed to consistently apply their own interpretations across questionnaire items. Below are examples of what respondents did and did not consider "lasting effects" of cancer.

• When asked to interpret the meaning of "lasting effects", one respondent said that she considered them to be effects that occurred up to one year from her last treatment. Conversely, two respondents, one with terminal cancer, and one with a recurrence said "…It never goes away…It’s forever".
• Some respondents did not differentiate between "lasting effects" of cancer and its treatment, and "side effects" of treatment, such as "chemo brain".
• A respondent described chronic arm lymphedema related to her cancer, the expense of buying a special sleeve for her arm, and the expense of medical appointments as "lasting effects".
• An osteogenic cancer survivor whose leg was amputated in childhood as part of his cancer treatment described at various points in the survey how lack of mobility and prosthesis were his main problems in life. However, he did not consider them to be "lasting effects" of the cancer treatment.
• One respondent described chronic stomach problems resulting from a poorly performed cancer surgery. However, she did not consider her stomach problems to be "lasting effects" of cancer but rather the result of medical error.
• Another respondent whose chemo tube was not removed until five years after her chemotherapy was over noted that she developed a blood clot, which still today affects her arm. However, when asked if she considered this a lasting effect of the cancer treatment, she said no, because she felt it was related to her doctor’s malpractice rather than to the cancer or treatment itself.
• At least three respondents who were currently in treatment for their first cancer diagnosis explicitly said they did not include "lasting effects" in their answers because they have not yet experienced any.

Of key analytic interest are the "lasting effects" of cancer treatment on the various aspects of respondents’ lives. It is therefore important that respondents consistently include "lasting effects" in their answers throughout the survey. For the most part, it appeared that when they did think about lasting effects, they were doing so throughout the entire survey. Similarly, those who were not thinking about lasting effects seemed not to do so throughout the entire survey.

Some respondents had difficulty differentiating between the effects of aging or other health conditions and those of their cancer treatments. This problem manifested itself most obviously at the questions about the impact of lasting effects on physical and mental tasks. Because they did not always know which one was the cause of any given problem, individual respondents sometimes had variable interpretations of "lasting effects" across their own survey answers. If this phenomenon is replicated in the survey population, data from these items will lack reliability.

• A survivor of colorectal cancer who underwent radiation therapy reported chronic back pain from degenerative disc disease. However, he was unsure if his condition was mainly due to bone damage from radiation, or simply due to aging.
• A respondent described some of her cognitive deficits related to "chemo brain". However, it was unclear if these deficits were related to chemotherapy, or to a recent stroke.
• Some elderly respondents found it difficult to pinpoint the "lasting effects" of cancer. For example, respondents also suffered from arthritis or other age-related chronic conditions, and could not separate these conditions from those related to cancer and its treatment.

Duration Questions

Seventeen respondents expressed difficulty with the duration questions at some point in the questionnaire. These questions occurred in Section 2, Section 4, and at Q79. Thirteen of them were confused about how to record their answers in the split "amount" and "unit" format.

• One respondent, who had switched to a flexible work schedule, commented at Q13, "It’s a little bit confusing right now… Do you want to put months in it, years in it, weeks in it?"
• Another respondent described her difficulty in the caregiver employment section, "I was confused about what kind of number they’re looking for…in my mind I was thinking my mom worked part-time for about a year…I didn’t know what number to put there, but it was for the whole year…I guess I could have put seven per week or five per week."

The word "per" was also confusing for some respondents because it implies a rate. One asked, "Would that be 5 times per week?" while another who was trying to find a way to record her answer of "2 years" wondered, "Is this days per week, days per month, or days per year?"

Many respondents said that they were frustrated with these questions and some said they would have skipped them if the interviewer had not been there. They dealt with answering the duration questions in a variety of ways, as indicated below:

• Four respondents ignored the boxes next to "amount" but wrote in a number in the box next to the unit that applied to them. Sometimes the respondent pluralized the unit, for example, changing "month" to "months."
• Several respondents wrote in their own answers outside of the designated boxes.
• One respondent checked the box next to year and then wrote a (+) next to it to indicate it was a few years.
• One respondent checked off multiple unit boxes indicating both months and years.

Eight respondents struggled with making the calculations to come up with their answers. This was especially true for those with different episodes of cancer or job situations.

• One respondent commented that, given how long ago her cancer occurred, it was too difficult to provide the specific kind of answer the question seemed to be looking for.
• One respondent had to do extensive calculations for this item because she was out of work several times over the course of her diagnosis and treatment, each time varying in length.
• One respondent with multiple cancers was not sure how to calculate the duration because her employment change lasted the entire time between her first and second cancer diagnoses, which were 14 years apart. She thought it would be misleading to mark "14 years" because the reason she did not go back to her original employment status had nothing to do with her cancer or its lasting effects.
• One respondent had difficulty calculating several instances of extended paid time off into one amount in the duration part of the question.
• Two respondents were still experiencing their employment change. One struggled with Q13 because she is still on the flexible work schedule she changed to at her first diagnosis, then stayed on after treatment and for her second diagnosis. The other respondent pointed out that the questions assume the employment change happened in the past. She answered by counting the total time up to the date of the survey. In both of these situations, the duration questions do not capture the fact that the employment change is ongoing.
• On the flip side of wanting to convey an ongoing change, at least one respondent wanted to indicate that she had gone back to her original employment status. She answered both the part-time to full-time and full-time to part-time questions, using the latter to indicate that the change was temporary and she is now back on her regular schedule. This respondent’s cancer treatment had happened in the distant past.

Recommendations

We suggest dropping the split "amount" and "unit" format and using the following response categories for all duration questions in the employment section:

Less than 3 weeks
3 weeks to < 2 months
2 months to < 6 months
6 months to < 1 year
1 to 3 years
More than 3 years

In the caregiver section, the response categories would be:

2 months to < 6 months
6 months to < 1 year
1 to 3 years
More than 3 years

We need further guidance from the workgroup before providing a recommendation for Q79 (see item-specific discussion below).

Skip Patterns

There are numerous skip patterns and instructions in the questionnaire, which was distressing for a significant number of respondents. They had a particularly difficult time in Section 1, where 10 people made at least one skip pattern error, although that was not the only place that caused confusion. All but five respondents incorrectly skipped or were confused by a skip pattern at least once in the questionnaire. Additionally, the switch between arrows on the left and brackets on the right was confusing for a small number of them.

• One respondent was confused by the skip instruction at Q3. He initially thought it was an error because he was familiar with skips that apply to either the "yes" or "no" but not to both response options.
• One respondent said she was "confused about what this is telling me to do" in reference to the skip instruction with brackets at Q3.

The questionnaire was formatted using design principles developed and successfully tested by Dillman^[2]. As well, Westat has so far found very few errors of omission or commission in two large-scale field tests for the National Household Education Survey that used the same format for self-administered questionnaires^[3]. We are somewhat puzzled as to why the formatting should have posed such a problem for this population. We speculate that in part it may be because respondents are expecting more of the questionnaire to apply to them, or that there are simply too many skips in an effort to accommodate the wide variety of cancer experiences the survey wishes to capture. Specific issues with the skips are discussed further in the relevant sections below.

Section- and Item-Specific Findings and Recommendations.^[4]

Respondents, in general, appeared to have positive impressions of the questionnaire. Below are examples of their overall reactions after completing it, along with a few of their suggestions.

• One respondent stated that people often don’t think about these experiences because all the focus is on finding a cure. She approves of the questionnaire because she thinks it will help address other cancer-related issues and perhaps ultimately help others.
• Another respondent also felt the survey would help others. By filling it out, he said, "…It is something I can pass down to other people who are going through the same experience…it might help someone else."
• Two respondents thought that the survey was "very good"; one liked that it asked about the financial burdens a cancer patient has to suffer; the other liked the focus on patient experiences in hopes that it will help others through their cancer journey. Another respondent thought the questionnaire covered "great topics".
• One respondent commented that she thought the font size was good and that the bold font and blue background made it easy to read.
• One respondent thought the survey was good, but noted that it was "very extensive and had a lot of words." The respondent felt that others with low literacy levels may struggle or have difficulty and suggested making it briefer, especially since it is a self-administered questionnaire.
• One respondent, who was diagnosed with cancer as a child, suggested that there should be more questions about the mental and social aspect of dealing with the disease, for instance, how it affects the interaction with friends and other people.
• Two respondents felt that the questionnaire should focus more on the emotional impact of cancer on your life. One commented that "the emotional aspect plays a great deal in the process of getting back into your life."

Cover Page

In general respondents seemed to feel that the cover page was not informative about the survey contents.

• Some respondents surmised from the cover that the survey was about their "experiences with cancer", "…What I was going through", or "…My cancer treatment…"
• One respondent had the impression from the cover that the questions would ask about more intimate details of her life and her cancer experience.
• One respondent said that the cover page should provide more information, and should specify how filling out this questionnaire would help other people with cancer.
• Some respondents assumed from the cover that the questionnaire would focus more on medical experiences, such as experiences with doctors, hospitals, and cancer treatment (what kind, how long it took, recovery/set-backs) while others thought the questionnaire was more about the "effects of cancer that are long-lasting" or how cancer has impacted their lives and the lives of their families.

Recommendations

Add a second sentence to the introductory paragraph on the cover page that reads, "The survey will ask about the effect of cancer on your employment, finances, health insurance coverage, and life in general."

Section 1 – Cancer History

Eleven of twenty-three respondents made a skip pattern mistake at some point in Section 1. Respondents may have had difficulty because of the large number of skips and the variety of different kinds of skip instructions (left-hand arrows, large "stop box", right-hand arrows, right-hand brackets) so early in the survey. Also, the idea of passing over more than half a section very early in the survey seemed jarring for those who were skipped out of the section at the third question.

Questions about treatment can be confusing because the word can refer either to an entire course of treatment, such as all four cycles of a chemotherapy treatment, or one particular aspect of treatment, such one dose of the chemotherapy. As well, it may be unclear to respondents whether to include in treatment other aspects of cancer care, such as biopsies, follow-up exams, and preventative measures. All of these issues appeared at various items in Section 1, which contributed further to the skip pattern problems, as described below.

Q2 – Are you currently being treated for cancer?

"Currently being treated" at Q2 refers to radiation, chemotherapy, surgery or other courses of treatment administered at the time of or shortly after diagnosis. Some respondents, who are currently being treated for cancer, were interpreting Q2 as intended.

• Two were thinking about their hormone treatment for prostate cancer. One of these respondents was also thinking about radiation.
• A third respondent was thinking of her chemotherapy.

Many others, however, did not appear to have interpreted Q2 as intended.

• One respondent whose course of chemotherapy ended in April 2010 answered "yes" at Q2 because she was including her follow-up CT scans, blood tests, and pills "to keep [her] from getting cancer."
• Another respondent who has completed his chemotherapy and radiation included his follow-ups with the oncologist and urologist as his "current treatment."
• One respondent said she considered taking "standard preventative measures" to be treatment.
• One respondent asked, "What does being treated for cancer mean?" She decided to answer "yes" at Q2, thinking about the medication she is currently taking. One other respondent was thinking of pills she takes as her "current treatment."
• One respondent said she was still in her five year follow-up and so considers herself as "currently being treated."

In some instances, respondents’ may be thinking of these more preventative measures because of the language their doctors are using.

• One respondent who has completed her chemotherapy marked "yes" at Q2 and explained that, because her cancer is slow growing, her doctor has instructed her to see him in 6 months so he can decide if she needs additional treatment.
• Similarly, another respondent marked "yes" at Q2 because her doctor told her "I am still treating you" in regard to her 6-month follow-up visits and monitoring.
• One respondent answered "no" at Q2 because she had just finished the second of four cycles of chemotherapy.

With the exception of the last respondent, the broad interpretation of "currently being treated" at Q2 led respondents to skip out of Section 1 before receiving Q5, which asks when they received their last cancer treatment (meaning, last entire course of treatment). Although we do not have direct evidence of this from the cognitive interviews, misinterpretation of "current treatment" could also lead respondents to under-report lasting effects of cancer, if they believe they have not yet experienced lasting effects because they are still currently in treatment. For the respondent who is in the middle of a four-cycle course of chemotherapy, the categories at Q5 seemed too broad (she answered "less than 12 months") and Q6 and Q7 did not make sense.

Recommendations

Consider revising Q2 to better convey the intended definition of "currently being treated." An example from the National Cancer Institute’s Adolescent and Young Adult Health Outcomes and Patient Experience (AYA HOPE) Follow-up Survey provides more detail about what to include and what to exclude. It is also much lengthier than Q2.

"What was the date of your last cancer treatment – that is, surgery, radiation, chemotherapy, bone marrow, or stem cell transplant? Please DO NOT consider a bone marrow biopsy to be a bone marrow transplant. If you are currently undergoing treatment, please indicate the current month and year."

Q3 – Is this the first time you have been treated for any type of cancer?

Of the 14 respondents who answered Q3, 10 continued to answer questions in Section 1 instead of skipping to Section 2. Most simply did not see the skip instruction. Others saw it but, in glancing ahead at the rest of the section, felt the questions applied to them and wanted to answer anyway.

• After completing Q3, one respondent read "if ever treated" at the beginning of Q4, and stated she thought she should continue on because she has been treated before. Additionally, she wanted to answer the remainder of the questions to show her cancer had come back (Q4 and Q7).
• One respondent wanted to answer Q7 in order to say that her cancer had come back.
• One respondent who had multiple cancer diagnoses thought Q3 was asking her just about her breast cancer (more recent), but didn’t understand why she couldn’t answer Q4-Q7.

Recommendations

To reinforce at Q3 that respondents need not answer any further questions in Section 1, revise the skip instruction to read, "You are currently being treated for cancer, please skip to Section 2."

Q4 – (If ever treated for cancer.) To the best of your knowledge, are you now free of cancer?

Although Dillman’s design approach uses parenthetical instructions in combination with arrows and skip instructions, the strategy does not appear to work at Q4. In part we believe this is because there is no arrow associated with the parenthetical. As noted above, at least one respondent who should have skipped to Section 2 read the parenthetical instruction and, because it appeared to apply to her (she was "ever treated for cancer"), answered it.

A separate issue with the parenthetical phrase is that it does not actually apply to all respondents who answer "no" at Q2. There are those who have been diagnosed with cancer but either have not yet begun their treatment or never got treatment. In fact, one of our respondents chose not to undergo treatment because her insurance will not cover the cost.

Recommendations

Remove the parenthetical phrase from this question. The new question should read, "To the best of your knowledge are you now free of cancer?"

To accommodate those respondents who have not been treated for their reported cancer diagnosis, add an additional response category at Q5 that reads, "I have not been treated for cancer."

Q5 – About how long ago did you receive your last cancer treatment?

Only eight respondents arrived at Q5 correctly. All of these respondents were thinking about their "primary course of treatment" or "active treatment" and mentioned when their last chemotherapy or radiation treatment occurred. These respondents noted that they were not including biopsies, follow-up visits, or medication as "treatment" but were including chemotherapy, radiation, and MRIs and CAT scans after radiation (to see if the treatment was successful).

Recommendations

As noted above, add a category "I have not been treated for cancer." Otherwise, leave Q5 as is.

Q7– What was the most recent year a doctor or health professional told you that your cancer had come back?

Respondents did not appear to have any trouble answering this question with "year." When probed, only two respondents, both of whom were diagnosed with cancer at a young age, said they preferred answering with something other than the year their cancer came back. One respondent said it was easier for him to remember his age; the other remembered what grade he was in (12^th) when he was diagnosed.

Recommendation

Leave as is and continue to test in Round 2.

Section 2 – Changes to Your Work Schedule

The majority of respondents tied most of their answers in Section 2 to their cancer, treatment and lasting effects of that treatment. As well, most seemed to understand the questions as intended. However, respondents did encounter difficulties within the section related to skip patterns and the duration questions.

Q8 – At any time since you were first diagnosed with cancer, were you working for pay at a job or business?

Sixteen of the twenty-four respondents said "yes" at Q8. These respondents all seemed to be interpreting the question as intended. Five of the eight respondents who answered "no" at Q8 seemed to have interpreted the question as intended.

• One respondent was looking for a job at the time of diagnosis, stopped after her diagnosis, and never looked for employment again.
• Several respondents were not working at the time of diagnosis (one because he was diagnosed as a teenager) and have not worked since their diagnosis.
• One respondent was retired at the time of diagnosis and has not worked since.

The other three respondents had worked since their initial cancer diagnosis but in answering the question were thinking only about their employment status at the time they had cancer or simply did not think the question applied to their unique situations.

• One respondent, who was diagnosed as a child, did not think the question applied to him because he was not working when he was diagnosed. He commented that the question "doesn’t take into account someone who was 15 at the time of diagnosis…at 15, I was going to school."
• One respondent was diagnosed 25 years ago but did not start working until 5 years after her treatment ended. She answered "no" at Q8, explaining that any work changes she might have experienced were unrelated to her cancer.
• One respondent who was diagnosed 8 years ago was not working at the time of her diagnosis and has only worked one year at a job since that time. She felt the question did not really apply to her.
• One respondent who had cancer as a child and was not working when she was diagnosed originally said "no" at Q8. However, after the interviewer asked her to explain her answer, she re-read the question and decided that she should in fact mark "yes" because she has worked since the diagnosis. She went on to comment that the question seemed to apply more to adults whose cancer might disrupt their job, than to those who were diagnosed as children.

Recommendations

To reinforce the reference period respondents should be thinking about, revise Q8 to read, "At any time from when you were first diagnosed with cancer until now, were you ever working for pay at a job or business?"

Q9 – At any time since your first cancer diagnosis, did you ask for extended sick leave, leave without pay, or a change in your hours, duties, or employment status?

Nine of the sixteen respondents who received Q9 answered "yes" to having had an employment change. Of these nine, only one had difficulty remembering if there was an employment change.

Of the seven who answered "no," most had either continued to work their regular schedules through their treatment or had enough sick leave accumulated that they would take days off here and there without changing their overall schedules. However, a few selected "no" for Q9 even if they did change their work schedules around to accommodate their treatment because they felt the arrangements were more informal than their interpretation of Q9.

• One respondent who was working part-time moved his schedule around but did not consider it a change because he continued to work part-time.
• One respondent changed to a more flexible job after he was diagnosed so that he would have an easier time working and undergoing treatment simultaneously. However, he did not include this change in the employment section.
• One respondent, who was a teacher, had to arrange for a long term substitute while she was undergoing treatment. The respondent was working from home with the substitute, once a week but was not being paid.

A few of those who said "no" at Q9 had difficulty with the word "ask."

• One respondent, who worked as a housekeeper, said that she did not have to ask for time off because she was given time off by her employers.
• One respondent was self-employed and so did not have to "ask" anyone when she wanted to change her hours.
• Another respondent worked at a nursery in a bowling alley and her boss allowed her to leave early or not come in if she was feeling sick. Her boss would "clock her out" so she would get full pay. She never had to "ask" for any employment change.

Recommendations

To accommodate respondents who did not have to "ask" for leave, as well as to better capture the experience of those who asked for but did not receive leave, consider the following revisions.

Revise Q9 to read:

At any time since your first cancer diagnosis, did you take extended sick leave, leave without pay, or a change in your hours, duties, or employment status?

Yes-Skip to Q11
No

Add a new question after Q9 to determine whether or not respondents who did not take leave actually asked for the leave:

At any time since your first cancer diagnosis, did you ask for extended sick leave, leave without pay, or a change in your hours, duties, or employment status?

Yes
No-Skip to Q32 (see recommendation at the discussion of Q33)

Revise Q10 to read, "Did you ask for or take these work changes…" This will require an instruction for those who said "no" at Q9 to skip to Q32.

Note: The following results and recommendations are based on the responses from the nine respondents who completed the remainder of the employment section and, where applicable, seven respondents who completed similar questions in the caregiver section.

In the caregiver section, a few respondents answered "yes" to the question about whether their caregivers had made employment changes but then marked "no" at all questions about the specific employment changes because their caregivers’ situation did not fit anywhere. While this did not happen in the employment section, we believe it is still worth addressing in both sections. We recommend adding a closed-ended "catch-all" question after Q31 (that asks about "less demanding job"). This new question should include a reference to "cancer, its treatment or the lasting effects of that treatment" as a reminder of the reason for the change. Furthermore, for consistency, the remaining questions in Section 2 should include that same reference. We believe this will help with some of the issues described below (e.g., forgetting to include those factors when answering about taking disability or making retirement decisions). Related to these changes, we recommend that the skips at Q9 and Q10 point to Q32 instead of Section 3. Although this did not come up in testing, it seems unlikely that respondents would think be thinking about advancements or promotions when answering Q9.

Recommendations

Insert a new question after Q31:

Did you make any other type of work arrangements because of your cancer, its treatment, or the lasting effects of that treatment?

Yes
No

Revise Q32 and Q33 to include "because of your cancer, its treatment or the lasting effects of that treatment" at the beginning of each.

See discussion at Q36 for suggested revisions to the retirement questions.

Q11 – Did you ever change from a set work schedule, where you start and end at the same time every day, to a flexible work schedule, where your start and end times vary from day-to-day?

Respondents appeared to be interpreting "flexible work schedule" as intended.

• Set hours per day and per week but not at the same time everyday;
• Being able to leave when you are tired;
• Working when you feel well enough to;
• Taking different shifts at the grocery store to accommodate treatment times.

One respondent who answered "yes" to both the flexible schedule and full-time to part-time questions was thinking of the same change, when he requested what he described as a flexible work schedule of 30 hours, reduced from his 40-hour work week.

Recommendations

Leave item wording as is, but move item in the order of employment change questions. See recommendation at Q23/Q52 below.

Q23 – Did you ever take extended paid time off from work (vacation and/or sick time)? By extended time off, we mean more than an occasional day off here and there.

Q52 – Did any of your caregivers take extended paid time off from work (vacation and/or sick time)?

Most respondents seemed to interpret "extended paid time off from work" as intended.

• One respondent included vacation time he took after receiving the diagnosis.
• One respondent took five months of sick leave from his "sick leave bank" that he had been contributing to for the past 30 years.
• One respondent took one year of leave from his job. He was including sick-leave as well as short- and long-term disability leave.
• One respondent described how her cousin took her to the beach for a month. She used two weeks of vacation time and her boss gave her two extra weeks paid.
• One respondent did not consider the time her friends took off as "extended" because it was usually just for a day. Similarly, another said that a vacation day her husband took here and there to take her to a doctor’s appointment was not extended time off.

A few respondents were confused about whether to report disability leave at this question, described in further detail at the discussion of Q33 below.

Recommendations

One source of confusion over where to include different types of employment changes may stem from the current order in which the specific changes are asked about. It may help respondents to think first about discrete and finite types of changes such as extended leave first, before tackling more amorphous changes such as flexible work schedules and working from home. As well, the proposed revision has the advantage of consistency with the order in which changes are described in Q9. We recommend the revised order for both the employment and caregiver sections.

Extended Paid time off
Unpaid time off
Switch from Part-time to full-time
Switch from full-time to part-time
Flexible work schedule
Work from home
Less demanding job

Q29 – Did you ever change to a less demanding job?

Q58 – Did any of your caregivers change to a less demanding job?

Most respondents seemed to interpret the question as intended.

• One respondent who checked "no" reported that she is an accountant and continues to perform the same type of work she did before her diagnosis. She works fewer days a week and fewer hours a day now, but does not consider this to be less demanding. To her, a less demanding job would be changing to a different position, reporting to someone else, or performing a different task.
• One respondent who answered "no" noted that she thought a less demanding job "requires adjustments, like less hours, less straining tasks, or less heavy lifting."

Ultimately, only two respondents answered "yes" to Q29 and none answered "yes" to Q58.These two struggled somewhat with whether their employment change could be considered "less demanding."

• One respondent, who answered "yes," said that she changed from working as a cashier to working in the back of the grocery store, which she considered to be an easier job. While she did not make this change because she needed something less demanding, she did make the change because of her cancer diagnosis.
• Another respondent who answered "yes" noted that although her job title remained the same, she felt the workload she was given after her diagnosis was lighter. Additionally, she no longer can get raises or promotions.

Recommendations

Leave as is.

Q33 - Did you ever leave work on disability?

Several respondents were unsure about what to include under "leave work on disability". Several respondents thought that disability and extended time off were similar, if not interchangeable.

• One respondent could not distinguish between disability and extended time off and so decided to answer "yes" at this question and at Q23 for the same employment change.
• One respondent noted that she took extended time off but only received 60% of her salary. Further probing revealed she was likely receiving disability. She answered "yes" at Q23 but "no" at Q33.
• One respondent answered "yes" to Q33 but for a disability leave that was not related to her cancer. She described the question as "very confusing" and suggested it clarify whether cancer-related disability leave should be included.
• One respondent at first thought she should report her three-month disability leave at Q23, but ultimately decided to report it at Q33 instead.

Recommendations

We would like to further discuss with the workgroup the intended meaning of "leave work on disability" before providing additional recommendations for Q33.

Q36 – Has your experience with cancer, its treatment or the lasting effects of that treatment had a mostly positive, mostly negative or an equally positive and negative effect on your retirement decision?

Respondents reported a variety of issues with this item. Two left it blank and others suggested adding a "no effect" category because they felt their retirement decisions had nothing to do with their cancer or they were not yet thinking about retirement. Additionally, respondents seemed to be answering more generally about their work experience instead of tying their retirement decision to their cancer diagnosis.

Recommendations

To help focus respondents on their retirement decision as it relates to cancer, consider the following revisions.

Skip those who say no to both Q34 and Q35 past Q36. That is, if they have not made a retirement decision, do not ask about the effect of cancer on their retirement decision.

Add "because of your cancer, its treatment, or the lasting effects of that treatment" to Q34 and Q35.

34. Because of your cancer, its treatment, or the lasting effects of that treatment, did you retire earlier than you had planned?

Yes-Go to Q36
No

35. Because of your cancer, its treatment, or the lasting effects of that treatment, did you delay your retirement beyond when you had planned?

Yes
No Go to Section 3

36. Thinking about your retirement decision, what effect has your cancer, its treatment or the lasting effects of that treatment had on it?

Mostly positive effect
Mostly negative effect
Equally positive and negative effect
Neither positive or negative effect

Section 3: Other Aspects of Work

All 16 employed respondents answered the section about other aspects of work.

Q37 – Did you ever feel that your cancer, its treatment or the lasting effects of that treatment interfered with your ability to perform any physical tasks required by your job?

Two respondents marked that they did not have to perform physical tasks for their job and eight answered "yes" to this question. Their interpretation of "physical tasks" ranged from light to strenuous, depending on the type of job they had.

• Walking around (teacher; car salesman);
• Lifting or pulling when putting away supplies or sitting for long periods (office manager);
• Standing for long periods (appliance salesman)
• Doing laundry, cleaning, making beds (housekeeper);
• Vacuuming, sterilizing toys, holding children (nursery worker);
• Lifting heavy bags, transferring passengers in wheelchairs (airline worker).

One respondent who answered "yes" was thinking of the time before her cancer diagnosis. She related that before she knew she had cancer, "I became so physically debilitated that I was unable to do a lot of the physical things I was normally able to" in her job as a teacher. Her energy returned after she finished her treatment such that she was able, once again, to perform the physical tasks required by her work. Another respondent who also considered the time before her diagnosis ultimately decided not to include it in her answer, which was "no."

Recommendations

If the workgroup is satisfied with respondents’ broad interpretation of physical tasks, leave the question as is.

Q38 – Did you ever feel that your cancer, its treatment, or the lasting effects of that treatment interfered with your ability to perform any mental tasks required by your job?

Eleven respondents answered "no" and five said "yes" to this question. Both those who did and did not report interference seemed to be interpreting "mental tasks required by your job" as intended.

• Respondents included in "mental tasks required by your job" things such as memory, composing emails, making calls to customers, maintaining "straight thoughts," "reading or doing any kind of math or computing", multitasking, getting work done on time, making judgments, and concentrating.
• Those who answered "yes" noted that their cancer and its treatment caused them to take longer to process complex tasks, become "very forgetful", be more easily distracted, and have difficulty remembering names.
• Many respondents, regardless of how they answered the question, mentioned "chemo brain" or "chemo fog", or described how chemotherapy can make you "slower" for awhile.

Recommendations

Leave question as is.

Q39 – Did you ever feel that, because of your cancer, its treatment, or the lasting effects of that treatment, you were less productive at work?

Overall, respondents appeared to understand and interpret "less productive at work" as intended. Only one respondent, with limited English fluency, did not tie her decrease in productivity to her cancer, but rather explained that she became less productive at work because of the "economy".

Some respondents acknowledged feeling less productive:

• A male respondent who is employed at a car dealership said he was productive mentally but "…couldn’t do everything the way I wanted it done…having the cars looking the way I wanted…"
• One respondent reported being less confident in her work, causing her to check her work more often which she said resulted in decreased productivity.

Others reported no change, or increased productivity:

• One respondent who answered "no" said that while she could feel the effects of her treatment, she had "not yet" noticed herself being less productive at work because of it.
• A hairdresser answered "no" because she can still mix colors and style hair.
• Two respondents who work with children, one as a teacher and the other as a day care provider, said they did not consider themselves to be "less productive" because they were still able to work with children. The teacher said that physically, she was less productive, but mentally and emotionally she pushed herself "harder than ever," and she could still "be there emotionally" for her students. The day care provider said she could not vacuum, sterilize toys, or clean up, but was still able to have contact with the children.
• One respondent, a writer, described how his experience with cancer had made him more creative and helped to improve his writing.

Recommendations

Leave question as is.

Q41 – Did you ever stay at a job in part because you were concerned about losing your health insurance?

Four of the five respondents who answered "yes" to this question did not relate their answers to their cancer.

• One respondent described an experience many years ago of keeping a job due to health issues, but not any that were related to her cancer. She went on to point out that the question does not specify that staying in one’s job because of concerns about health insurance be related to cancer.
• Another explained that she stayed at a job because she was afraid of losing her health insurance due to other health conditions unrelated to cancer.
• The third tied her answer to her responsibility toward her family, stating, "…If I don’t work, my children and my husband will not have health insurance."
• One respondent said his concerns are related to "the exorbitant cost of health care in general."

Recommendations

Add a follow-up question for those who answer "yes" at Q41 that reads:

Were you concerned about losing your health insurance because of your cancer?

Yes
No

Q42 – Has your experience with cancer, its treatment or the lasting effects of that treatment had a mostly positive, mostly negative, or an equally positive and negative effect on your work life or career?

Some respondents seemed to be answering Q42 as intended.

• The respondent who is a hairdresser felt her cancer had a positive effect on her work life because she now talks with clients about skin cancer and encourages them to have their moles looked at.
• One respondent explained that she selected "mostly negative" because she is more insecure about work now. She did not feel that her employers were as helpful as they could have been nor that they adjusted their expectations of her appropriately.

Others appeared to be answering for how cancer, its treatment or the lasting effects of that treatment impacted their life in general, not just their work.

One respondent asked why there was not a response option for cancer having neither a positive nor a negative effect. Another respondent chose not to check any of the response options but wrote "neither" below the response options.

Recommendations

To help focus respondents on their work revise the question to read:

Thinking about your work life or career, what effect has your cancer, its treatment or the lasting effects of that treatment had on it?

Mostly positive effect
Mostly negative effect
Equally positive and negative effect
Neither positive or negative effect

Q43 – Did your spouse or significant other ever stay at a job in part because he/she was concerned about losing health insurance for the family? Yes, No, Don’t have spouse/significant other, Spouse was not employed at any time after I was diagnosed

This questions makes multiple assumptions about insurance, marital, and employment status that sometimes proved challenging for respondents when thinking about their answers.

• One respondent, who had had childhood cancer hesitated and then checked, "Don’t have a spouse/significant other", explaining, "…of course I didn’t have a spouse. I was a kid!"
• Another respondent with childhood cancer remarked, "…I was only 15!"
• One respondent left the question blank because, although her husband is employed, he does not have health insurance, so she felt none of the response options applied to her.

Recommendations

Revise question to read:

Did your spouse/significant other or parent/guardian ever stay at a job in part because he/she was concerned about losing health insurance for the family?

Yes
No
Does not apply

Section 4 – Caregivers

Seven respondents reported that their caregivers made employment changes. The workgroup was concerned that respondents have in mind a broad definition of caregivers when answering the questions in this section. For the most part, we found that to be the case. Respondents included friends, neighbors, significant others/spouses, co-workers, and relatives (daughter, cousin, sister, grandson, parents). One respondent counted her doctor as a caregiver, in addition to friends and family. Another respondent was only thinking of medical professionals until he read the section introduction, at which point he revised his answer to include friends and family.

When asked to describe the type of caregiving provided, respondents mentioned bringing and cooking food, praying for them to get better, taking care of house plants, performing chores, shopping, taking children to events, keeping the respondent focused, driving them to appointments, taking notes and asking questions at appointments, visiting respondents at the hospital, taking care of them after surgery, picking up their medicine, being supportive, listening, doing heavy lifting, giving haircuts, and changing bedpans and other medical tasks (emptying drain tubes, cleaning catheters).

Q45 – Because of your cancer, its treatment or the lasting effects of that treatment, did any of your caregivers ever take extended sick leave, leave without pay, or change their hours duties, or employment status?

Three respondents reported that their caregiver experienced an employment change, but did not select any of the changes listed in the remainder of the section.

• One respondent was thinking of her daughter’s availability due to her flexible work schedule, which did not require any "drastic changes." Consequently, the specific changes described in the subsequent items in Section 4 did not apply to her situation.
• Another respondent said her spouse worked fewer hours, but did not "officially" take time off from his job.
• The husband of another respondent worked part-time for 2 months while she was in the hospital, but it was not an official change so she did not feel Q46 adequately captured his particular situation.

One respondent who was diagnosed as a child selected "no" because he did not remember whether his parents had to make employment changes at the time, and another with a child diagnosis answered "yes" but was unable to recall the details of the employment changes.

Recommendations

As in Section 2, add a question before Q58 that reads:

"Did any of your caregivers make any other type of work arrangements because of your cancer, its treatment, or the lasting effects of that treatment?"

Yes
No

Revise Q58 throughQ61 to include "because of your cancer, its treatment or the lasting effects of that treatment" at the beginning of each.

To differentiate between a true "no" answer and those who may not be aware of their caregiver’s employment situation, or may not remember because it was in the distant past or because they were children, we suggest including a "don’t know" option at Q45 (with a corresponding skip out of the section).

Section 5 – Experiences with Health Insurance

Q62 – At any time from when you were first diagnosed with cancer to now, were you covered by health insurance that paid for all or part of your medical care, tests or cancer treatment?

Most respondents had little trouble understanding or answering the health insurance question. Those who answered "yes" described a variety of health insurance situations (such as through work, a spouse/significant other/parent, or a government-funded program such as Medicaid) and, with one exception, those who answered "no" did so with little trouble. One survivor of childhood cancer was unsure at first how to answer because his medical care was covered by an NIH research grant but he has never had health insurance. He ultimately answered with "no."

Recommendations

Leave question as is.

Q63 through Q65 – Was there ever a time when health insurance refused to cover…?

There were two issues with this series of questions. First, the "no" answer could indicate either that respondents sought the benefit and were denied or that they never sought it in the first place. For example, one respondent who answered "no" at Q64 and at Q65 explained that she had never asked for a second opinion or tried to use another facility. Two respondents (one of whom is on Medicaid) explained that they have only ever seen doctors or used facilities that they know are approved by their health insurance plans, so the concept of having a "choice" about those things does not apply to them. The second issue is that some respondents who answered "yes" actually ended up receiving the benefit denied them by the health insurance company because they decided to pay for it out of pocket or they found another way to get coverage.

Recommendations

We would like to discuss these issues further with the workgroup before making a recommendation.

Q66 – Were you ever denied health insurance coverage because of your cancer.

Q67 – Were you ever denied long-term care insurance coverage because of your cancer?

Similar to the issue in Q63 through Q65, one respondent was unsure how to answer Q66 because she had not applied for any health insurance since the onset of her cancer. Some respondents also pointed out that they had never been denied long-term care insurance because they had never applied for it. In addition, some respondents did not know what long-term care insurance is.

• One respondent asked, "…Is it senior insurance?"
• Another commented, "I’m not sure which long-term health care you mean" but speculated it may be what someone has when they are older and need to be in a facility.
• One respondent said, "I don’t think I’ve ever been denied it, but I know I’ve been told that I will either have to pay a lot for insurance or it’s going to be really, really hard to find…I’m not sure I really know what you mean by long term care insurance."

Recommendations

Depending on the saliency of the issue of long term care insurance for cancer survivors, consider dropping the question.

If it is retained, consider adding a definition of long term care and a third response option, "I have not applied for long term care insurance."

Section 6 – Finances

Seventeen respondents said "no" at Q68, 5 said "yes" and one left it blank. She had cancer as a child and explained, "I wasn’t sure about borrowing money or debt; I know we scraped by for a while. But I don’t know specifics and I don’t know how to answer that."

Some respondents tied their financial difficulties directly to their cancer, its treatment or the lasting effects of that treatment.

• One respondent said "yes" at Q71 (making other kinds of financial sacrifices) because she could not afford to live on her own after paying for cancer expenses.
• One respondent was mainly thinking about money to pay for the prosthesis he received as a result of his cancer. He said that they are very expensive and at times, he has had to "let other things go… my prosthesis is the only cancer-related thing I’m worried about."

Others decided to exclude financial difficulties that arose indirectly from their cancer experience. For these respondents, the term "medical care" seemed to narrow their thinking in that regard.

• One respondent answered "no" at Q68 and at Q71 because all of her medical expenses were covered by her health insurance. When asked to explain her answer, the respondent revealed that she did have to borrow money for gas, bills, food, and other daily expenses because she was on disability leave for her cancer and only receiving 60 percent of her salary.
• One respondent who marked "no" for Q71 said he chose that answer "because it is medical care related. He explained that he has not made financial sacrifices to pay for medical care, but has made sacrifices to pay for other things because his income suffered after he retired early because of his cancer. Another respondent had a similar situation where he had no medical debt because his insurance covered everything, but he lost his job because of his cancer and could no longer pay other bills, like his mortgage.

For those whose cancer experience was in the distant past or continued over a long period of time, recalling the amount of money borrowed or debt incurred proved challenging. At least one chose to leave the amount question blank for that reason and another answered "no" at Q68 because she could not remember.

Recommendations

If the workgroup is interested in capturing the indirect as well as direct financial impact of cancer, consider refocusing this section to ask about cancer, its treatment and the lasting effects of that treatment. For example, Q68 might read:

Have you or has anyone in your family had to borrow money or go into debt because of your cancer, its treatment or the lasting effects of that treatment.

Section 7 – Medical Care

Q74 – At any time since you were first diagnosed with cancer, did any doctor ever discuss with you…

For respondents with recent or single cancer diagnoses, this question series worked well. However, some respondents with cancer diagnoses from the distant past struggle with recall. One said, "How could I know whether I talked with my doctor in detail 25 years ago?" And a respondent with multiple diagnoses said she has had so many different doctors over such a long period that she did not know how to respond. She ultimately decided to answer for the experiences that were most positive for her, choosing "discussed in detail" or "briefly discussed" for all sub-items.

Several respondents mentioned talking with social workers or counselors about their social and emotional issues, noting that doctors are not usually the ones to discuss those topics. Some respondents decided to include these other kinds of practitioners in their answers while others answered only for doctors. For example, two respondents spoke with social workers about their social and emotional issues, but answered "did not discuss" at Q74c.

Recommendations

We would like further guidance from the workgroup about whether respondents’ answers should include other healthcare practitioners before providing a recommendation.

Q75 – At any time since you were first diagnosed with cancer, were you unable to obtain medical care, tests, or treatments that you or your doctor believed were necessary?

Several respondents had trouble interpreting, "unable to obtain", commenting that it was "stiff, awkward language." For example, one respondent explained that she was able to get all the tests she needed, but could not figure out how to answer the question, focusing her frustrations on the word "unable."

Recommendations

Q75 is worded such that a "no" answer results in the confusing idea that one was "not unable" to obtain medical care. Consider revising the question to read:

At any time since you were first diagnosed, did you get all of the medical care, tests, or treatments that you and your doctor believed were necessary?

If this change is made, Q76 and Q77 would be revised to read:

Which of these are reasons you did not get all of the medical care, tests, or treatments that you or your doctor believed you needed?

Which one of these is the main reason that you did not get all of the medical care, tests, or treatments that you or your doctor believed you needed?

Section 8 – Effects of Treatment on Life in General

Q78 – Did your cancer, its treatment or the lasting effects of that treatment ever limit the kind or amount of activities you do outside of work, such as shopping, child care, exercising, studying, work around the house, and so on?

Respondents did not appear to have difficulty with Q78 and most reported some kind of limitation.

• A respondent who had osteogenic cancer as a child and is now an amputee answered "yes," saying, "have I been limited… sure I have. It’s been a lifelong experience."
• One respondent who had multiple cancer diagnoses noted that only one of his cancers limited him, describing how it leaves him feeling drained and fatigued to where he cannot do his usual exercise routine.
• One respondent reported not being able to do any heavy lifting and needing to pace himself and take breaks when performing physical activities such as doing the dishes, washing towels, or power washing. He also noted that he cannot carry heavy things for long distances, go to football games, or go for walks around the park.
• A respondent described not being able to garden and being tired from chemotherapy.

One respondent who is currently undergoing treatment said she did not feel limited at the moment, but answered "yes" at Q78 based on her assumption that soon she would be. Two older respondents reported limitations but were unsure whether they were the result of cancer or the aging process.

Q79 – How long were you limited, in weeks, months, or years, in the kind or amount of usual daily activities?

Calculating the amount of time they were limited in the activities was exceptionally challenging for many respondents.

• One respondent left this question blank, stating that the first year she could not do a lot of things, but afterwards she could. She wrote on the side of the response options, "couldn’t do 1^st year" instead of selecting an amount.
• One respondent asked, "When?...During chemo?...Here and there?...After chemo?" in the end the respondent decided to put down 8 months, the length of time she has been in treatment so far.
• Another respondent, with melanoma, remarked that she is limited in her activities every day, since she must avoid exposure to sunlight. She marked the "year" box without specifying how many.

Several respondents had difficulty with calculating the limitations because they were not continuous. All three respondents in the examples below left Q79 blank, as did others.

• One respondent who had cancer 14 years ago and had recently had a recurrence could not answer this question because she found it impossible to quantify her limitations over such a long period of time. She noted that she experienced the effects from her cancer, but that they were not "tangible limitations."
• Another respondent who had cancer as an adolescent had difficulty quantifying her answer because the effects were different at varying points in her life. She reported, "I wasn’t sure because it’s varied over the years…when I was younger it limited me most every day…since elementary school the effects have waned out, so I wasn’t really sure what to put for that one."
• One respondent wanted to answer with a percentage but couldn’t figure out how to, commenting, "the time period…I don’t understand that at all."

Recommendations

Based on this first round of testing, we might expect a significant amount of missing data on Q79. As well, it is exceptionally burdensome for respondents given the vague concept (limitation of daily activities) they are being asked to quantify.

Consider dropping the question. If the workgroup wishes to retain it, revise with response categories similar to those suggested for the employment and caregiver duration questions. We would like to further discuss this item with the workgroup before providing a specific recommendation.

Q80 – Did you ever feel that your cancer, its treatment or the lasting effects of that treatment interfered with your ability to perform any mental tasks as part of your usual daily activities?

Similar to Q38, most respondents did not have any trouble with this question and seemed to be interpreting it as intended. Most respondents did not report any interference with their mental abilities in reference to their usual daily activities.

• Eleven respondents answered "yes" to this question. Examples of how they defined the ability to perform mental tasks include decision making; trying to learn new tasks and concepts; and relying on memory (e.g., remembering to turn off the stove).
• One respondent who answered "no" said she felt "loopy and dizzy" right after surgery, but it was so short that it did not interfere with her daily activities during that period.
• Another respondent, who answered "no" said his treatment did not interfere with any tasks, but just with his ability to think and concentrate.
• One respondent reported that there has been no effect on his ability to perform mental tasks because of his cancer. In fact, just the opposite. He has been able to write several articles and a book about people’s experiences with cancer.

There may be some social desirability associated with this question and its counterpart in the employment section. Interviewers noticed that some respondents who answered "no" would, over the course of the interview, describe situations where their ability to perform mental tasks was compromised. A few respondents emphatically stated that cancer and its treatment did not interfere at all with their ability to perform their jobs. This biasing effect may disappear, however, during data collection, when respondents will be completing the survey on their own rather than in the presence of an interviewer.

Recommendations

Leave question as is.

Q81 – Have you ever needed any of the following kinds of help because of your cancer, its treatment or the lasting effects of that treatment?

Several respondents pointed out that "needing help" does not actually mean the help was received.

• One respondent explained that she needed help keeping track of her medications and figuring out her insurance, but she had to do it herself since there was no one available to help her. She answered "no" at both Q81b and Q81c.
• Another respondent related that he didn’t actually "need" his friends to come to doctor’s’ appointments with him but that he had asked them to accompany him.

Respondents who had childhood cancer also had an issue with the concept of "needing help." It was difficult for them to think of themselves as children needing "help" from their parents getting to the doctor or taking medications. In those situations, parents are not "helping" their children; they are simply performing their normal parental duties. For example, one respondent who answered no to all of the items listed in Q81a through Q81d explained that his parents took care of all of those things but that he did not consider them as having helped him because they were fulfilling their responsibilities as parents. For these kinds of respondents, "no" could either mean they did not need the help because they did it themselves or they did not need the help because their parents did it for them.

Recommendations

We would like to further discuss this item with the workgroup before providing a recommendation.

Q84 – What do you think are the chances that your cancer will come back or get worse within the next 10 years?

This question seemed odd to elderly respondents, some of whom were old enough to be more concerned about whether they would still be alive in 10 years, never mind their cancer coming back.

We also noticed that some respondents answered the question based on what their doctors or others had told them, even if they themselves disagreed. . This response pattern introduces a potential for bias, failing to capture respondents’ true anxiety levels.

• One respondent felt the chances of her cancer coming back were quite high, but checked "fairly low" because her doctor had given her that prognosis.
• One respondent marked "moderate" because it is the middle ground between what actually thinks (fairly low) and what her doctor thinks (very high).
• One respondent marked "very low" because this is what her husband, a cancer genetic specialist, has told her. She herself believes her changes are very high.

Two respondents simply did not know but could not find a place to record such an answer. One of them decided to mark "moderate" because its position at the mid-point of the list of choices was the closest she felt she could come to "don’t know."

Recommendations

Revise Q84 read:

In your own opinion, what do you think are the chances that your cancer will come back or get worse within the next 10 years?

Add a "don’t know" response category.

Q85 – Which of these, if any, have been positive things about your experiences with your cancer, its treatment, or the lasting effects of that treatment?

Most respondents enjoyed answering this question and talking about their positive experiences. One respondent left it blank, saying emphatically that "there is nothing positive about being sick." Another felt that none of the options really fit but did not want to leave the question blank.

Respondents were asked if there were any other positive things related to their cancer besides those listed. Examples of the kinds of things they added include:

• I appreciate friends more
• I can help others going through the same experience
• I learned to do things I enjoy
• I have a better attitude ("laugh at more silly things")
• I gained different perspectives on life
• I met new people (in support groups)
• I live more spontaneously now
• I appreciate life more

Recommendations

For improved data capture on a self-administered questionnaire, consider adding "yes" and "no" response boxes similar to those used in Q81. Alternatively, add a response category, "There has been nothing positive about my cancer experience."

^[1] Facebook users post links to websites, videos, photos, documents and any other content they "like" as a way of sharing their interests and activities with their friends. Facebook users can also click on the "like" button located next to content on their friends’ pages as a way of acknowledging a shared interest.

^[2] Dillman, D. A. (2009). Mail and internet survey: The tailored design method (2nd ed.). Hoboken, NJ: John Wiley and Sons.

^[3] These are preliminary findings based on examination of missing data. No articles or technical reports have yet been written about these findings.

^[4] One respondent’s literacy level was so low that the interviewer had to read the survey items to her. Even with that help she had excessive difficulty understanding many of the items and often did not provide a codeable answer. For that reason we have not included her data in the reported counts except for in selected areas.

Return To Top