MEPS Annual Methodology Report

Deliverable Number: 10.102
Contract Number: 290-2008-1004C
March 15, 2011

Submitted to:
Agency for Healthcare Research and Quality
540 Gaither Road
Rockville, Maryland 20850

Submitted by:
WESTAT
1650 Research Boulevard
Rockville, Maryland 20850-3195
301-251-1500

Table of Contents

_. Introduction
1. Sample
1.1 Sample Design and Size
1.2 Sample Delivery and Processing
2. Instrument and Materials Design
2.1 Changes to Materials and Procedures for Spring and Fall 2010
2.1.1 Transition From Paper to Electronic Format
2.1.2 Revisions to Advance Mailings
2.2 Questionnaire Changes for Spring and Fall 2010
2.3 Testing of the Questionnaire and Interviewer Management System
2.4 Instrument Evaluation Activities
3. Recruiting and Training
3.1 Recruiting for 2010
3.2 2010 Training Programs
4. Data Collection
4.1 Schedule
4.2 Operations
4.3 Data Collection Results
4.4 Authorization Form Signing Rates
4.5 Self Administered Questionnaire (SAQ) and Diabetes Care Supplement (DCS) Response Rates
5. Home Office Support of Field Activities
5.1 Preparation For Field Activities
5.2 Support During Data Collection
6. Incentive Experiment
Appendix A. Comprehensive Tables – Household Survey
Table 1-1 Initial MEPS sample size and number of NHIS PSUs, all panels
Table 1-2 Data collection periods and starting RU-level sample sizes, Spring 2003 through Fall 2010
Table 1-3 Percentage of NHIS households with partially completed interviews in Panels 3 to 15
Table 2-1 Supplements to the CAPI core questionnaire (including hard-copy materials) for 2010
Table 4-1 Data collection schedule and number of weeks per round of data collection
Table 4-2 Comparison of results of patient profile collection in 2008 through 2010
Table 4-3 MEPS HC data collection results, Panels 11 through 15
Table 4-4 Average number of contacts required to complete a round 1 interview, Panels 13-15
Table 4-5 Response rates by panel and round and for the $30 payment group in Panel 13
Table 4-6 Round 1 response rate and refusal rate for households reached and not reached in advance contact call, Panels 13-15
Table 4-7 Summary of MEPS Round 1 response, 2005-2009 panels
Table 4-8 Summary of nonresponse for Round 2, 2007-2010
Table 4-9 Summary of MEPS Round 1 response, 2005-2009 panels, by NHIS completion status
Table 4-10 Summary of MEPS Panel 15 Round 1 response rates, by sample domain by NHIS completion status
Table 4-11 Summary of MEPS round 1 results for RUs who ever refused, Panels 10-15
Table 4-12 Summary of MEPS round 1 results for RUs who were ever traced, Panels 10-15
Table 4-13 Percent of total interviews conducted on travel
Table 4-14 Timing comparison, Panels 11 through 15 (mean minutes per interview, single-session interviews)
Table 4-15 Mean round 1 interview time, in minutes, for single-session interviews, Panels 14 and 15,
by interviewer training and production groups
Table 4-16 Round 2 outcome, by Round 1 interview time (Round 1 interviews with no breaks), Panels 14 and 15
Table 4-17 Later round outcomes by ‘ever refused’ status in Round 1, Panels 14 and 15
Table 4-18 Round 2 outcome by month of Round 1 complete, Panels 14 and 15
Table 4-19 Mean contact attempts by NHIS completion status, round 1 of panels 13-15
Table 4-20 Mean days between first and final contact by NHIS completion status for round 1 of panels 13-15
Table 4-21 Signing rates for medical provider authorization forms for Panels 11 through 15
Table 4-22 Signing rates for pharmacy authorization forms for Panels 11 through 15
Table 4-23 Results of self-administered questionnaire (SAQ) collection for Panels 11 through 15
Table 4-24 Results of diabetes care supplement (DCS) collection for Panels 11 through 14
Table 5-1 Number and percent of calls to the respondent information line, 2008-2010
Table 5-2 Calls to the respondent information line, 2009 and 2010
Table A-1 Data collection periods and starting RU-level sample sizes, all panels
Table A-2 MEPS household survey data collection results, all panels
Table A-3 Signing rates for medical provider authorization forms
Table A-4 Signing rates for pharmacy authorization forms
Table A-5 Results of self-administered questionnaire (SAQ) collection
Table A-6 Results of Diabetes Care Supplement (DCS) collection
Table A-7 Calls to respondent information line

Introduction

This report documents the principal design, training and data collection activities of the Household Component of the Medical Expenditure Panel Survey for survey year 2010. These activities were conducted under Contracts 290-02-0005, awarded in July 2002, and Contract 290-2008-10004C, awarded April 30, 2008. Together, the two contracts cover MEPS Panels 8-15, with options for two additional panels.

This report covers work associated with Panel 13 Round 5, the last panel of the older contract, and Panel 14 Rounds 3 and 4 and Panel 15 Round 1, which are panels included in the new contract. It also provides a description of preparations for fielding the new panel (Panel 15) that were performed in the latter half of the prior year.

The report touches briefly on procedures and operations that remained unchanged from prior years. Its main focus is on features of the project that were new, changed, or enhanced for the 2010 year of data collection. It also presents the results of the data collection activities conducted during the year. The tables within the report document 2010 data collection results. A comprehensive set of tables showing results from prior years is included in Appendix A.

An important operational procedure was revised in 2010 that involved a transition from hard copy case materials to electronic format. This transition was motivated by the concern for security of personal identifying information (PII) associated with a case. Contacting information and record keeping logs that once appeared on hard copy were programmed into electronic versions that were accessed in the interviewer management system on the laptop.

This past year also saw an increase in evaluation activities on both the questionnaire and field procedures. The evaluation activities involved data review as well as feedback from field staff via interviewer observations, conference calls, and other formats. This information was used to modify the round 1 respondent mailing as well as make improvements to the CAPI questionnaire. Also in 2010, the incentive experiment begun in 2008 with Panel 13 was concluded in the Spring when the round 5 interviewing was complete.

Chapter 1 of the report describes the sample preparation activities. Chapters 2 through 5 discuss activities associated with the data collection for 2010 including materials development, questionnaire updates, field staff recruiting and training, data collection procedures and results, and home office processing support.

Return To Table Of Contents

1. Sample

This chapter documents the sample preparation activities associated with the fielding of the 2010 sample, which included households selected for Panel 13 Round 5, Panel 14 Round 3, and Panel 15 Round 1.

Return To Table Of Contents

1.1 Sample Design and Size

Each year MEPS draws a new household sample from among responding households in the previous year’s National Health Interview Survey (NHIS). The new MEPS sample for 2010 – Panel 15 – was selected from households that participated during the first three quarters of the NHIS in 2009, Panels 1 and 4. Panel 15 is the fourth panel using the new sample design introduced by the NHIS in 2006 and consisted of 8,968 reporting units.

The oversampling for MEPS of populations with analytical interest was modified beginning in Panel 14 to exclude the oversampling of low income households (households predicted to be below 200 percent of the poverty level). This domain was dropped because it was not providing the intended representation of low income households. Panels 14 and 15 had only three oversampled populations: Asian, Black, and Hispanic. Panel 13, also from the new NHIS sample design, retained the low income domain.

Table 1-1 shows the starting sample sizes for Panels 1 to 15 and the number of NHIS PSUs from which each panel was drawn.

Table 1-1. Initial MEPS sample size and number of NHIS PSUs, all panels

* RU: Reporting Unit

Return To Table Of Contents

Table 1-2 on the following page summarizes the combined workload for the January-June and July-December periods from spring 2003 through fall 2010. (Table A-1 in Appendix A shows the data collection periods and sample sizes for all panels and rounds.)

Across the three panels that were active during the first half of 2010, the combined workload was 23,770 RUs, the largest spring workload since 2003. For the two panels that were active during the second half of the year, the total initial workload was 13,785 RUs.

Return To Table Of Contents

1.2 Sample Delivery and Processing

The 2010 MEPS sample was received in two deliveries. The first delivery, received in September 2009, contained households sampled from the first two quarters of the 2009 NHIS. Households selected from the third quarter of the 2009 NHIS were delivered in November 2009.

Table 1-2. Data collection periods and starting RU-level sample sizes, Spring 2003 through Fall 2010

Return To Table Of Contents

As in recent years, the September sample delivery was instrumental to the project’s plan to launch interviewing for the new panel at the beginning of February. The partial file gave insight into the demographic and geographic distribution of the households in the new panel and guidance on the need for recruiting new interviewers.

As soon as the first sample delivery was received, the NHIS sample file formats were reviewed to identify any new variables or values and to make any necessary changes to the project programs that use the sample file information. With the early delivery, Westat began the standard processing through which the NHIS households are reconfigured to conform to MEPS reporting unit definitions and prepared the files needed for advance mailouts and interviewer assignments. The delivery also allowed time for checking and updating NHIS addresses to improve the quality of the initial mailouts and to identify households that have moved since the NHIS interview.

Each year, the NHIS sample includes a percentage of households classified as ‘partial completes’. Table 1-3 shows the percentage of NHIS interviews classified as "partially complete" in Panels 3 through 15. The NHIS partial completes are, as a group, more difficult to complete in MEPS than the full NHIS completes and therefore receive special monitoring. Partial completes made up 21 percent of the MEPS sample in Panel 15.

Table 1-3. Percentage of NHIS households with partially completed interviews in Panels 3 to 15

Return To Table Of Contents

2. Instrument and Materials Design

This chapter describes changes to field materials and procedures and to the computer assisted personal interviewing (CAPI) instrument for 2010 household data collection activities. The changes were implemented for Panel 13 Round 5, Panel 14 Rounds 3 and 4, and Panel 15 Rounds 1 and 2.

Return To Table Of Contents

2.1 Changes to Materials and Procedures for Spring and Fall 2010

Several changes were made for the 2010 data collection periods. The most significant involved a redesign of key field materials from hard-copy format to electronic format. In other changes, input gathered from a series of conference calls with field interviewers prompted changes to the content and sequencing of the initial respondent mailings. A number of adjustments were also made to the CAPI instrument, with several new questions added and other questions modified to clarify or simplify wording. Each of these areas of change is described further below.

Return To Table Of Contents

2.1.1 Transition From Paper to Electronic Format

The most significant redesign effort of the year was the conversion of hard-copy documents to electronic format, a change implemented to enhance the security of survey information by minimizing the number of documents used in the data collection that contained personal identifying information (PII). In the past each interviewer received a case folder that had an assignment label on the front, showing the name, address, and phone number of the respondent. Inside the case folder was a RU folder for each round; the RU folder contained a hard-copy interviewer assignment sheet, face sheet, supplemental face sheet (for Round 1 only), Advance Contact Record (Round 1), Household Relationship Summary (Rounds 2 – 5), pre-printed Authorization Forms (Rounds 2 – 5), and AF and SAQ log (Rounds 2 – 5). For 2010, the case folder was eliminated and the RU folder was redesigned to remove as much PII as possible, shifting it from hard copy to electronic format. As redesigned, the RU folder no longer includes an assignment label; it has only a mini-label showing the case ID. Where previously the interviewer referred to hard copy labels and other materials for key information needed to contact their respondents, interviewers now obtain that information from the electronic face sheet in their laptop. With the change, the number of hard copy documents enclosed in the RU folder was also reduced so that the folder contents are now limited almost entirely to documents that are to be signed or completed by the respondent, such as authorization forms and self-administered questionnaires.

The new electronic face sheet, accessed through the Interviewer Management System (IMS), combines information from two documents previously generated in hard copy: the face sheet and supplemental face sheet. Electronic face sheets for all prior rounds as well as the current round are now available to the interviewers through the IMS. To accommodate the interviewers’ need for ongoing access to some elements of respondent information in their day to day contacting work, the project now provides an interviewer notebook. Interviewers copy selected items from the electronic documents to a page in the notebook for use when contacting households. The individual pages are shredded as soon as an interview is complete.

The interviewer assignment sheet (IAS) that is used by the interviewer to track the status of the work was also converted to electronic format accessed through the IMS. Paper follow up logs for AFs, SAQs, and DCSs and the check log were combined with the IAS. These tasks are now monitored through the electronic documents.

The paper forms used to conduct the Round 1 advance contact calls were also redesigned to remove PII from the form sent to the interviewer for reference when working the case. For 2010, staff making the advance contact calls received face sheets containing the identifying information needed to contact the new Round 1 households and 3 ply response sheets for recording the outcome of the calls. The face sheets were shredded when the calls were complete and the response sheets, which contained only the call results and the RU ID, were sent to the supervisor for forwarding to the interviewers.

Return To Table Of Contents

2.1.2 Revisions to Advance Mailings

The sequence and content of the initial mailings to the Round 1 respondents were changed for 2010. This change was based on an assessment of interviewer feedback obtained in a series of conference calls that the early mailings of prior years were too bulky and gave an exaggerated perception of burden to many respondents. The two-stage mailing of prior years was replaced by a single streamlined mailing. This mailing included a letter introducing MEPS and notifying the household of their selection, a study brochure, and a record keeper for recording information about health care events prior to the Round 1 interview. The MEPS calendar, the DVD and companion booklet, the appointment postcard, the change of address card, and the $5 bill included in earlier years were eliminated from this first mailing. The packaging for the DVD and companion booklet was reduced to fit into a standard 6" by 9" envelope. Although not included in the advance mailing, the DVD and companion booklet were made available to field staff to be used on an as-needed basis to help gain cooperation. The DVD was also made available for interviewers to show to respondents on their laptop, with a new Spanish subtitle option and was added to the Participant’s Corner page of the MEPS web site. The appointment card, which was withheld from the initial mailing to the Round 1 households, was sent to respondents in all three rounds in a mailing timed to occur shortly before the interviewer’s actual contact. With this later mailing, the possibility that respondents might request an appointment occurring before the data collection period began was eliminated.

Return To Table Of Contents

2.2 Questionnaire Changes for Spring and Fall 2010

For 2010 the project introduced several new questions to address issues related to health care reform and made a number of relatively minor changes to existing questions to improve wording, reduce burden, and facilitate administration of the instrument. A major change included an update of the provider directory, used near the end of the interview to obtain full address information for providers identified by respondents during the interview. Prior year applications used a directory that had not been updated since originally compiled by the project for Panel 1.

The changes to the instrument for 2010, organized by instrument section, are summarized below.

Reenumeration

• References to the household summary (formerly printed on the hard copy face sheet) were removed and the names of members of the household were integrated into question text.

• Marital status as reported in the previous round was made into a display.

• An edit check for any RU member coded as "incorrectly listed" was created to remind the interviewer to document the reason for removing the person from MEPS data collection.

• Columns were added to the RU roster to display keyness and reference period dates for family members.

• An interviewer instruction reflecting a change in procedure for tracing foster children was added to the screen where locating information for movers is collected.

Calendar

• Additional answer categories were added to provide a broader variety of respondent record keeping materials.

Event Roster

• Interviewer instructions were modified and a new respondent show card developed to improve medical event typing.

Charge Payment

• Electronic records and patient profiles were added to the list of memory aids.

Disability Days

• A display for the number of days for an institutional care (IC) event was added and question text modified to accommodate the inclusion of time in a long-term care facility.

Health Insurance

• Question text was modified and respondent probes and interviewer instructions added to insure that all types of health insurance coverage are collected. Additional questionnaire items were added to capture information on Medicare HMO and Part D premium amounts.

Preventive Care

• Age dependent skip patterns, question text, and response categories were modified to better align cancer screening items in the MEPS instrument with comparable NHIS items.

Provider Directory

• A new directory of medical providers, with a new supporting lookup application, was introduced. The new directory was drawn from the National Provider Identification (NPI) database, with refresher updates to be made on an annual basis. Interviewer search strategies were developed to improve the quality of search results and reduce the number of potential matches.

Closing

• References to hard copy authorization form (AF) and self-administered questionnaire (SAQ) logs were deleted since tracking of forms is handled in electronic format.

• Authorization forms were no longer generated for medical providers and pharmacies located in foreign countries.

• Respondent contact information collected in the previous round was displayed.

• Text thanking respondents for their participation was revised to reflect guidance from the Office of Management and Budget (OMB) that respondent payment be referred to as a ‘gift’ for participation in the study.

Table 2-1 shows the supplements in the CAPI instrument for the rounds administered in calendar year 2010.

Table 2-1. Supplements to the CAPI core questionnaire (including hard-copy materials) for 2010

Return To Table Of Contents

2.3 Testing of the Questionnaire and Interviewer Management System

Testing of the CAPI questionnaire and supporting management system was conducted in multiple stages, with specific testing tasks coordinated across design and systems groups. Testing began early in the development cycle as programmers tested their work on a flow basis to ensure that the modifications to the instruments were in accord with the specifications developed by design staff. Once programming was completed, design and systems staff at Westat and project staff from AHRQ tested the full CAPI instrument during alpha and beta test periods. Required changes identified during testing were implemented before the CAPI instruments were ‘frozen’ approximately 6 to 8 weeks before the spring and fall data collection cycles began.

Each cycle of testing included a number of components focusing on various aspects of the instrument and supporting field management system: verification of the application against the instrument specifications, testing a variety of training scenarios to simulate data collection situations and overall usability of the instrument and supporting systems, focus on specific features such as help screen functionality and directory searches, historical testing where data entered into the revised application are compared to previously completed cases to ensure the data are captured and stored as intended, and integration testing of the CAPI application in the context of the full set of management and support systems needed during active data collection.

Two additional components were implemented with the testing for 2010 data collection: enhanced integration testing and ‘live’ testing. Both were added to test performance of the documents converted from paper to electronic format. The conversion created a number of new dependencies between data entered into CAPI during an interview and the supporting management functions on the laptop. The enhanced integration testing allowed project staff to check electronic face sheet information, test the interviewer assignment sheet, and make entries into the electronic record of calls and refusal evaluation form. Practice cases used for general alpha and beta testing provided the database for this testing. The new live testing component used information derived from actual cases to verify that all management information on the laptop was being brought forward correctly from previous rounds. This enabled project staff to verify face sheet data for more complicated but realistic scenarios such as RUs with deceased persons or with changes in family composition, to test split and merge scenarios, and to further confirm that data entered into the closing section of the instrument was successfully brought forward into the electronic interviewer assignment sheet.

Throughout the testing, issues pertaining to either the CAPI instrument or the interviewer management systems were tracked electronically, organized by panel and round, in a system that allowed monitoring of issues from initial report to final resolution.

Return To Table Of Contents

2.4 Instrument Evaluation Activities

During 2010 the project pursued several paths of activity aimed at evaluating the CAPI instrument. The efforts were intended to identify aspects of the instrument that proved confusing to respondents or interviewers or that had potential for causing error in the collection of medical event and health insurance data. Attention was also focused on possibilities for streamlining instrument content and reducing the burden of responding to the interview.

Interviewer Teleconferences. Four teleconferences were conducted between January and April 2010 with groups of interviewers chosen on the basis of their MEPS experience and the characteristics of their case loads. Three of the sessions dealt with interviewers’ perceptions of how different groups of respondents react to the content and wording of questionnaire items relating to health insurance. These sessions were motivated by concerns raised in review of recent health insurance estimates that the instrument might not be capturing a complete report of health insurance coverage for some groups of respondents. The sessions resulted in suggestions for several adjustments to the health insurance section of the instrument The fourth session focused on an evaluation of the updated provider directory using the NPI.

Observations and Feedback From Field Interviewers. Over the course of the spring rounds of data collection, the project staff increased the number of in-person interview observations. The in-person observations were supplemented with a series of audio recorded interviews that were made available for review by staff at Westat and AHRQ and by a number of one-on-one interviews conducted by senior home office staff with both new and experienced interviewers. These discussions explored the interviewers’ sense of questionnaire items that caused confusion or presented unexpected difficulty for respondents. Additional interviewer feedback on the questionnaire was collected through a mail survey sent to 21 interviewers chosen by the MEPS field managers. The survey presented screenshots of the most common paths through the CAPI questionnaire and invited interviewers to annotate the screen shots to indicate items seen as problematic for respondents or for themselves. Participants were encouraged to suggest alternative wordings.

Reports begun in prior years to track interview timings, utilization, and expenditures were continued through 2010, with reports being expanded or supplemented as needed to address specific evaluations. New reports examined interviewer use of the repeat visit function in the questionnaire, the specific questions at which events are first reported by respondents, the frequency with which the ‘calendar path’ is taken in the questionnaire, and patterns of telephone interviewing conducted across rounds of data collection for a panel. Information gathered from the instrument evaluation activities informed the CAPI questionnaire changes during the year, modifications to the interviewer training programs, and ongoing discussions with AHRQ about possibilities for future improvements for the instrument.

Return To Table Of Contents

3. Recruiting and Training

3.1 Recruiting for 2010

Recruiting for 2010 began in Fall 2009, with the first wave (2/3) of Panel 15 sample in hand. Recruiting needs were established by estimating the full workload for the new panel and adding it to the existing workloads in Panels 13 and 14. The projected total caseload in each PSU was used to calculate the number of interviewers needed. This number was compared to the number of active interviewers on staff in each PSU to determine the PSU-level staffing requirements.

For the 2010 recruiting, MEPS used the Westat web-based recruitment management system through which applicants apply online. A total of 114 interviewers were recruited, 111 of them attended training, and 108 completed the training program. With the addition of these new trainees, the project began 2010 data collection with a total of 479 interviewers. There were 26 new interviewers and 38 experienced interviewers lost to attrition during the spring interviewing rounds. An additional 18 new interviewers and 26 experienced interviewers were lost during the fall round. Total attrition for the year was 22.5 percent. This rate is comparable to the prior seven years, in which attrition rates ranged from 21 to 24 percent.

Return To Table Of Contents

3.2 2010 Training Programs

The interviewer training program for 2010 included new interviewer home study and in-person training in Los Angeles, California between January 30 and February 5, a home study for experienced interviewers prior to the start of Rounds 1/3/5 and a home study for all interviewers prior to the start of Rounds 2/4.

Home Study and In-person Training for New Interviewers

The redesigned 2009 training program for new interviewers was retained in 2010. Key elements of the revised program include expanded pre- and post-classroom home study and a seven day in-person training using a database supplemented with pre-loaded cases to support a mini-mock training format.

Based on an evaluation of the 2009 training program, several adjustments were made to the 2010 new interviewer training program including:

• Modifying the distribution schedule and content of the pre-classroom home study and expanding the time allotted for its completion.
   
• Adjusting module timings to ensure that trainees had adequate time to complete dyads during in-person training.
   
• Adjusting the content of the first mock interview to place less emphasis on instrument navigation and decreasing the number of mini-mocks presented during the seven day in-person training.
   
• Scheduling administrative training modules more effectively throughout in-person training so that key tasks were presented in shorter, more frequent sessions.
   
• Structuring evening CAPI labs to include targeted practice on concepts presented in that day’s agenda.
   
• Reducing the number of training evaluations completed by trainees, allotting more time for their completion, and encouraging a higher response rate.
   
• Expanding the post-classroom home study and accelerating its scheduled completion to reduce the interval between this and the in-person training.

All new interviewers were required to complete the self-paced pre-classroom home study which combined video, computer practice, project reading, a review of the Interviewer Field Procedures Manual, and workbook exercises. A laptop and home study package were sent to new hires on a flow basis beginning in mid-January. While plans were in place to accommodate any late hires who did not have time to receive and complete a home study before arriving at training, all new interviewers received their home study package at least one week before in-person training began.

A total of 108 trainees successfully completed the Panel 15 in-person training held at the Los Angeles Airport Marriott between January 30 and February 5, 2010. Each trainee was assigned to one of eight training communities staffed by a primary and support trainer and two classroom runners.

Prior to project-specific training, trainees new to Westat and/or survey research attended a supplemental in-person General Interviewing Training (GIT) session to provide them with instruction in basic interviewing procedures and methods for success as a professional interviewer.

The seven day project training included three full mock interviews with primary focus on the round 1 and round 3 interviews. Trainees completed full round 1, 3, and 5 interviews in dyad role plays. Multiple "mini-mock" interviews – interviews with data pre-entered to allow trainees to directly access the specific section to be addressed in a given session -- allowed for in-depth sessions on the more complex sections of the CAPI questionnaire such as reenumeration, utilization, charge payment, employment, and health insurance without necessitating the completion of a full mock interview or dyad practice. A new mini-mock was developed for the 2010 training focusing on the new provider lookup using the NPI.

To accommodate training on the new electronic case materials, interactive "contacting" modules were added to the beginning of the mocks. In addition, all mocks and dyads culminated with the completion of the electronic interviewer assignment sheet (IAS), providing trainees with a more complete "start to finish" review of the MEPS instruments.

A slight adjustment to the order of the presentations for in-person training was made midway through day five of the training program to accommodate the early departure of 21 trainees that evening due to an anticipated blizzard on the East Coast. Mock 3 and other material related to refusal conversion and various administrative tasks were moved earlier so they could be conducted in person. The trainees who departed in-person training early were given a dyad to complete at home before beginning project work.

Bilingual interviewers hired by MEPS and certified by Westat to conduct interviews in Spanish are, for the most part, trained alongside other new interviewers from their regions. Since 2008, however, the bilingual interviewers have been brought together for one full day of bilingual training during regular project training. The day’s agenda parallels that of the English-only counterparts, but all lecture, discussion, and interview practice is conducted in Spanish. A final one-half day of bilingual training is held at the conclusion of regular project training. A total of 25 bilingual interviewers successfully completed 2010 bilingual training.

The Panel 15 post-classroom home study was mailed to new interviewers approximately three weeks after in-person training. It combined questionnaire material and a review of administrative procedures. The questionnaire-related material reviewed key concepts presented in training and provided exposure to more unusual interviewing situations. The administrative procedures were selected based on field supervisor recommendations. Additionally, a series of follow-up emails related to refusal aversion/conversion strategies was sent to all interviewers from the Field Director.

Home Study for Experienced Interviewers

The 2010 home study for the spring panels and rounds for experienced interviewers followed the format of prior years. The self-paced home study addressed procedural and questionnaire updates and revisions including instruction on using the electronic formats of case materials, presented a review of the supplemental sections of the questionnaire, and included written exercises to be completed and reviewed by field supervisors. A mini-mock focusing on the new provider lookup, closing section of the CAPI questionnaire, and electronic interviewer assignment sheet (IAS) was included in the home study.

Round 2/4 Home Study

In the fall of 2010, all interviewers completed a Panel 14 Round 4 and Panel 15 Round 2 home study. Interviewers who attended the February in-person training were required to participate in a mock interview with their supervisor or quality control assistant by telephone before beginning work on their assignments. The home study featured a review of the supplemental sections of the questionnaire, enhancements to the electronic face sheet and interviewer assignment sheet (IAS), information about new procedures, and updates to the CAPI instrument. The supplemental readings from the training were added to the Interviewer Procedures Manual for future reference.

Return To Table Of Contents

4. Data Collection

This chapter summarizes results for the five rounds of data collection conducted in 2010 and provides selected comparisons to results of prior years. A number of the tables include figures from Panel 13, the panel in which the project experimented with alternative respondent payments, offering a control group the same $30 per interview as in the previous panel and alternative payment of $50 and $70 to two comparison groups. In general, the higher respondent payments resulted in higher response rates, which were achieved with fewer contacts than in prior panels or in the subsequent Panels 14 and 15, in which the respondent payment remained at $30. As a result, figures in the tables for Panel 13 introduce an element of discontinuity with the preceding and succeeding panels.

With the exception of results for Panel 13, response rates for the five rounds of data collection conducted in 2010 did not differ significantly from prior years. Overall, response rates for rounds 2 through 5 were slightly higher than those of the last several years, while the round 1 rate was lower. This decline in the round 1 response rate continued the pattern of recent years and occurred despite extensive project efforts to reverse the trend. To a greater extent than in prior years, the project used management system paradata to identify areas of concern and to guide the planning of targeted followup efforts. Senior project managers initiated an extensive new set of contacts with field managers, field supervisors, and individual interviewers to review progress, guide planning for the use of the most skilled interviewer resources, and solicit feedback on successful and unsuccessful tactics for gaining cooperation.

Return To Table Of Contents

4.1 Schedule

Table 4-1 shows the calendar dates and number of weeks per round in the standardized, "steady state" schedule for the 5 annual rounds of MEPS household data collection. This schedule, which has been in place for a number of years on the project, was followed again in 2010. Rounds 3 and 5 begin in mid-January of each year followed by a February 1 start-up for round 1. The later start of round 1 has allowed for a minimum 4-week reference period for the first round of MEPS interviews. The schedule allows a two-week interval between the end of round 1 and the start of round 2, and between the end of round 3 and the start of round 4. The fixed schedule for data collection has provided a stable reference point for planning the related activities that prepare for or immediately follow the data collection, such as the preparation of field materials for subsequent rounds, identification of the sample for the Medical Provider Component, and development of the annual estimation files.

Table 4-1. Data collection schedule and number of weeks per round of data collection

Return To Table Of Contents

4.2 Operations

Data Collection Procedures

Data collection support activities carried out from the Westat home office, such as tracking using an outside vendor, disseminating information from respondent calls to the Alex Scott line, mailing of refusal letters, and other data collection support activities were relatively unchanged from prior years. Unlike past years, however, when procedural changes to operational activities were minimal, the changes in advance mailings and the move from hard-copy to electronic format for the face sheet and interviewer assignment sheet (IAS) required a number of changes in protocols for MEPS field operations.

For 2010, the content of the round 1 advance mailing was scaled back significantly in response to concerns that the mailings gave an exaggerated perception that participation was burdensome. While this change did appear to reduce the number of comments about perceived burden stimulated by the advance mailing, it shifted some of the burden of explaining the study to the interviewer’s first contact attempt. This resulted in some respondents wanting more time to think about the decision to participate, and, for these respondents, an additional contact attempt.

With the introduction of electronic formats, preparation of hard copy case materials for interviewing, especially for round 1 households, was streamlined significantly. In the field, however, interviewers no longer had direct access to the contacting and locating information that was previously provided on hard copy materials. Planning their work schedule required use of the laptop and the interviewer notebook provided by the home office. For security reasons, interviewers were encouraged to minimize the information recorded in the notebook, limiting it to the essential items needed to locate and contact households and to record contact attempts for later entry into the electronic record of calls. This transition required time for interviewers to adjust to the new procedure and make it part of their routine interviewing practice.

Converting to the electronic Interviewer Assignment Sheet also affected the format for reporting during the interviewer’s weekly conference calls with their supervisor. To report on the status of their work during the weekly call with their supervisor, interviewers now needed to be logged on to their computer for access to the Interviewer Assignment Sheet module in the interviewer management system.

For 2010, the mailing of the pre-notification letters was delayed pending a decision on the amount of the respondent payment for the new panel. This constrained the time available for the advance contact calls even more than in other years.

Patient Profile Collection

Continuing the procedure started several years ago, pharmacy patient profiles were requested from RUs who reported using one or more of 5 pharmacy chains who have continually refused to participate in the Medical Provider Component Followup Survey. In 2009, there was a change in the pharmacy authorization form (AF) collection that included prompting for pharmacy AFs at the end of the round 2 and 4 interviews, and this changed the approach for requesting patient profiles. To reduce respondent burden and confusion, profile requests were not made for round 4 households at the end of the interview since the respondent could also be asked to sign an AF for the same pharmacy. The collection of profiles from respondents was limited to round 5 households who had completed their participation in MEPS. In September of 2010, patient profile requests were sent to 1,679 RUs with 2,503 patient provider pairs. Completed profiles were received for 16 percent of the pairs, nearly 4 percent fewer than in 2009 and 1.5 percent less than in 2008. Despite the lower rate of collection, the effort still provided patient profiles that could not be collected in the MPC through corporate contacts.

Table 4-2. Comparison of results of patient profile collection in 2008 through 2010

Return To Table Of Contents

Quality Control

Quality control measures implemented in previous panels continued to receive attention during the 2010 data collection effort. Seven full-time experienced MEPS field interviewers, including one bilingual interviewer, made validation calls by phone in English and Spanish. Field supervisors also validated some of the work in their regions – especially the work of new interviewers. Cases without phone numbers or cases that were difficult to reach by phone were either validated in person or by mail. About 20 percent of the sample was pre-selected for validation and at least 15 percent of each interviewer’s case assignment was validated to ensure that the interview took place and appropriate procedures were followed. In addition, supervisors had the option to select one interviewer from the region in each data collection cycle (spring and fall) for 100 percent validation. All travelling interviewers had cases validated in the areas they worked on travel status in addition to cases they completed in their home PSUs. As in prior years, all interviews completed in less than 30 minutes were validated. The problems found in interviews of less than 30 minutes were comparable in frequency and type to those found in the validation of interviews greater than 30 minutes. For interviews of less than 30 minutes, some respondents told the validator that the interview took from 45 minutes to an hour, but many respondents were not certain about the interview duration. No falsifications were found in the interviews of less than 30 minutes. All new interviewers were observed in person at least once during their first year of interviewing. No interviewers were released as a result of an observation, although most received feedback on ways to improve specific interviewing skills.

Security Incidents

To comply with the requirement of reporting incidents involving loss or theft of hardcopy materials with respondent PII or laptops, field staff continued to use an automated loss reporting system to report incidents. As before, reported incidents were subsequently tracked through the use of a documentation log which was provided to AHRQ whenever an entry or update to an incident occurred. A security incident report was also filed for each incident with the Westat IRB.

There were 16 incidents of lost/stolen hardcopy and laptops reported in 2010. In 12 incidents the hard copy was recovered. In 5 of these cases it was necessary to inform the respondent and a certified letter was sent. One incident involved the loss or theft of a laptop that was not recovered. Since laptops are full disk encrypted, respondent identity was not at risk and in this instance respondent notification was not required.

Return To Table Of Contents

4.3 Data Collection Results

Table 4-3 provides an overview of the data collection results, showing sample sizes, average interviewer hours per completed interview, and response rates for Panels 11 through 15. (Table A-2 in Appendix A shows the data collection results for all panels.) Although Panel 14 Rounds 3 and 4 showed a drop in response rates compared to Panel 13 (the panel with the payment experiment), the rates were higher than they had been in several prior panels. Panel 15 Round 1 had a lower response rate than Panel 14 Round 1, but the response rate for Panel 15 Round 2 was slightly higher than the rate achieved in Panel 14 Round 2.

With the exception of Panel 13 Round 5, the hours per complete for all rounds conducted in 2010 were higher than in the prior year. The increase was seen particularly in the round 1 cases which although below the high of 14.2 hours per complete in Panel 12, nevertheless required nearly a full hour more per complete than in Panel 14. For round 2, the hours per complete was 8.9, an increase from the 8.3 hours per complete the year before, but similar to the 9 hours per complete achieved in round 2 of Panels 12 and 13. The hours per complete in both Rounds 3 and 4 of Panel 14 increased slightly in 2010 though they remained below the hours per complete in Panel 12. As shown in Table 4-4, the increase in hours per complete was reflected in a similar increase in the number of contacts needed to complete a case, which increased substantially from Panel 14.

Table 4-3. MEPS HC data collection results, Panels 11 through 15

Return To Table Of Contents

Table 4-4. Average number of contacts required to complete a round 1 interview, Panels 13-15

Return To Table Of Contents

The response rates for Panel 14 Rounds 3 and 4 and Panel 15 Rounds 1 and 2 were lower than the comparable rounds of Panel 13. However, the respondent payment experiment in Panel 13 had a positive impact on the overall response rates, with two thirds of the panel receiving a payment amount that was higher than the amount paid in Panels 14 and 15. Table 4-5 shows response rates by round for Panels 10 through 15 and for the $30 payment group in Panel 13. This table provides a more relevant base for comparison. The table shows that the Panel 15 Round 1 response rate (73.5%) and the Panel 13 Round 1 response rate (73.5%) for the $30 payment group were the same. With the exception of Panel 14 Round 3, all other rounds conducted in 2010 had higher response rates than the comparable round in the $30 payment group of Panel 13. The Panel 14 Round 3 response rate (94.9%) was only .1 percent lower than Panel 13 Round 3, $30 payment group (95.0%).

Table 4-5. Response rates by panel and round and for the $30 payment group in Panel 13

Return To Table Of Contents

After the pre-notification mailing to round 1 households takes place, interviewers call the households to confirm they received the mailing and answer questions about the study. This procedure helps pave the way for the first in-person contact and provides an early opportunity to identify households that have moved or are reluctant to respond. Movers and reluctant households receive special handling at the start of the round and are usually assigned to interviewers who are skilled in tracking or refusal aversion/conversion. Table 4-6 shows the percent of households reached on advance contact and the final round 1 response rates and refusal rates for households reached and not reached. The relatively short interval does not allow enough time to reach all households. As a result, the percent reached tends to reflect the households who were easier to contact and thus more likely to be cooperative. Among those who were reached, the response rates, across all three panels shown in the table, range from 11 to 15 percent higher than for those who were not reached. In addition, refusal rates were 6 to 8 percentage points higher for those not reached than for those reached in the advance calls. The advance contact effort, though limited in the number of households reached because of the short amount of time available to make calls, is an important first step toward gaining cooperation from a new panel of households.

Tables 4-7 through 4-12 address additional factors affecting the 2010 data collection. As a group, they reflect the increasing difficulty the project has experienced in gaining and maintaining respondent cooperation.

Table 4-7 summarizes components of nonresponse associated with the round 1 households contacted for the panels begun in calendar years 2006 through 2010. As the table shows, the components of nonresponse other than refusals – the ‘not located’ and ‘other’ – have remained relatively stable over the years. What stands out in the table is that the 2.6 percent decline in response rate over the period shown is matched by a 2.6 percent increase in the refusal rate over that period. This suggests that the observed decline in MEPS response rates is primarily associated with the increasing unwillingness of respondents to participate rather than with increasing difficulty in locating sampled households or in making contact with the residents at the sampled households.

Table 4-6. Round 1 response rate and refusal rate for households reached and not reached in advance contact call, Panels 13-15

Return To Table Of Contents

Table 4-7. Summary of MEPS Round 1 response, 2006-2010 panels

Return To Table Of Contents

Table 4-8 shows the components of nonresponse for round 2. Here again, Panel 15 shows a small increase in the percentage of refusals, although the response rate improved slightly from Panel 14. The percent not located was the lowest it has been in recent panels.

Table 4-8. Summary of nonresponse for Round 2, 2007-2010

NOTE: Includes only households in sample originally provided from NHIS.

Return To Table Of Contents

Table 4-9 shows the round 1 results by NHIS completion status. The proportion of NHIS partial completes in 2010 was the lowest it has been since 2007. The difference in response rate between the NHIS completes and partial completes, however, remained in the same 15-16 percent range as in most previous years.

Table 4-9. Summary of MEPS Round 1 response, 2006-2010 panels, by NHIS completion status

Return To Table Of Contents

Table 4-10 breaks out response information for the NHIS completes and partial completes by sample domain categories. The results for 2010 are similar to those of prior years. Overall, the largest component of the sample, the white/other domain, had the lowest response rate, followed in order by the Asian, Hispanic, and Black domains. Households in the Black domain participated at high rates, both in the NHIS partials category and the NHIS complete category. In fact, the response rate among Black households in the partial complete group (74.2 percent) was nearly a percentage point higher than the overall round 1 response rate (73.5). The white/other domain shows the greatest difference in response rate between the NHIS completes and partial completes – a difference of 20 percentage points, nearly double the differential shown for the other domains. As with the overall sample, the bulk of the nonresponse in the white/other domain is refusals, reaching 42.1 percent among the partial completes. Another distinction between the NHIS completes and partial completes is in the percent of households not located. The limited information on household composition provided for the NHIS partial completes makes tracing them more difficult.

Table 4-10. Summary of MEPS Panel 15 Round 1 response rates, by sample domain by NHIS completion status

Return To Table Of Contents

NOTE: Includes reporting units added to sample as "splits" and "students" from original NHIS households, which were given the same ‘complete’ or ‘partial complete’ designation as the original household.

The difference in the response rates and refusal rates among the NHIS partial completes between 2009 and 2010 is large. The response rate for partial completes in 2009 for white/other households was 60.0 and the response rate in 2010 for this group is 7 percent less ( 53.1 percent). The same difference exists for the refusal rate in these years. The refusal rate in 2010 is 42.1 percent compared to 35.2 percent in 2009. Just over half of the white/other households in the NHIS partial complete category participated and just under half refused. The white/other domain makes up nearly half of the sample; its response rate has a large impact on the overall rate.

Table 4-11 summarizes the results of refusal conversion efforts by panel. With the exception of Panel 13 (with the incentive experiment), the table shows a steady increase, from 21.6 percent in Panel 10 to 29.4 percent in Panel 15, in the percent of households that refuse at least once during the round 1 field period. The pattern for final refusals is similar, increasing from 16.6 percent in Panel 10 to 21 percent in Panel 15. Although the conversion rate for Panel 15 (26.6 percent) was 1.5 percent higher than for Panel 14 (25.1 percent), the ever refused rate was nearly two percent higher in Panel 15 (29.4 compared to 27.6).

Table 4-11. Summary of MEPS round 1 results for RUs who ever refused, Panels 10-15

Return To Table Of Contents

Table 4-12 shows results of locating efforts for households that required tracking during the round 1 field period by panel. The percent requiring tracing in Panel 15 (16.7 percent) was nearly two percent higher than in Panel 14 (14.9 percent) and a higher percent of those traced were not located. The tracing rate and not located rate in Panel 15 was more similar to the experience in Panel 12 than to Panels 13 or 14.

Table 4-12. Summary of MEPS round 1 results for RUs who were ever traced, Panels 10-15

Return To Table Of Contents

Highly skilled interviewers travel to PSUs with high refusal rates or lack of adequate local interviewing staff in an effort to raise response rates. These efforts are concentrated toward the end of the data collection period. Table 4-13 shows the percent of cases completed on travel status during the spring data collection rounds in 2008 through 2010. For the three spring panels combined, the reduction in the percent of cases completed on travel status from 2008 to 2009 held pretty closely in 2010, increasing only slightly from 13.3 to 13.7. Within round 1 alone, the increase in the percent of completes obtained on travel status was slightly greater, from 16.9 percent in Panel 14 to 18.1 percent in Panel 15. For all three years, the table also shows that the round 1 households account for a higher percentage of the total interviews completed on travel status than rounds 3 and 5.

Table 4-13. Percent of total interviews conducted on travel

Return To Table Of Contents

Tables 4-14 through 4-16 examine the length of interview in comparison to prior panels for round 1 interviews.

Table 4-14 shows the mean interview length for single session interviews for all 5 rounds since Panel 11, the last panel that conducted interviews in the DOS-based application. Starting in Panel 12 with the new windows-based application, the length of interview increased substantially . In the panels since that initial transition, however, the mean length has continued to decrease. The panels and rounds conducted in 2010, with the exception of Panel 13 Round 5, have a mean interview length that is within 3 to 5 minutes of the Panel 11 experience.

Table 4-14. Timing comparison, Panels 11 through 15 (mean minutes per interview, single-session interviews)

Return To Table Of Contents

Table 4-15 looks at the mean round 1 interview length for single session interviews by interviewer training group and level of production. As expected, experienced interviewers had lower mean times than new interviewers and the more productive interviewers had lower mean times than the less productive. The substantial difference in mean length between the production groups of new interviewers suggests the level of familiarization and learning that must occur as new interviewers learn to perform their job under actual field conditions.

Table 4-15. Mean round 1 interview time, in minutes, for single-session interviews, Panels 14 and 15, by interviewer training and production groups

Return To Table Of Contents

Table 4-16 shows, for the major outcome categories of round 2, the mean interview time for the round 1 interviews completed in a single session. For both Panels 14 and 15, households that did not respond in round 2 had shorter round 1 administration times, than those who did respond, which would seem to indicate that the length of the round 1 interview had little impact on later round participation rates.

Table 4-16. Round 2 outcome, by Round 1 interview time (Round 1 interviews with no breaks), Panels 14 and 15

Return To Table Of Contents

Table 4-17 shows the round 2 and round 3 response rates for households that cooperated in round 1 with no reported refusal and those that cooperated only after having refused at least once. The table shows a noticeable differential in the round 2 response rates: a difference of 12 percent in Panel 14 and 8 percent in Panel 15. This suggests that there is some ‘carryover’ from the initial decision to refuse in round 1 to the participation decision in later rounds, even when that initial decision is reversed for the round 1 interview. The Round 3 rates shown in the table for Panel 14 – a difference of 3 percent -- suggests that the impact of the round 1 refusal lessens in later rounds.

Table 4-17. Later round outcomes by ‘ever refused’ status in Round 1, Panels 14 and 15

Return To Table Of Contents

Table 4-18 shows the round 2 response rates by the month in which the round 1 interviews were completed. Both panels show the same pattern of gradual decline in response rate as the field period continues, with the lowest response rate among those households completed during the last month of Round 1. It seems likely that many of these late cooperators were completed late in the field period because they were "difficult" in some respect – hard to locate, hard to find at home, or reluctant to participate. These types of difficulty likely persisted to some extent in round 2.

Table 4-18. Round 2 outcome by month of Round 1 complete, Panels 14 and 15

Return To Table Of Contents

Table 4-19 shows mean contact attempts by mode and NHIS completion status for all cases in round 1 of Panels 13 through 15. This table shows that the NHIS partial complete cases required more contacts than the NHIS completes across all three panels. The partial completes are often more reluctant to participate and more are not located. Both of these result statuses require more effort to bring to a final resolution. The table also indicates that the mean number of contact attempts has been increasing since Panel 13. Although the response rate is trending downward across panels, the level of effort required to obtain the final response rate has been increasing.

In addition to requiring more contacts to close out a case, the NHIS partial completes also take more elapsed calendar time to close out. Table 4-20 shows the mean number of days between first contact and final contact by NHIS completion status for round 1 of Panels 13 through 15. The NHIS partial completes took an average of 16 to 18 more days to complete than the NHIS completes. As expected, the table also shows that the number of days between first and final contacts has been increasing across the last 3 panels.

Table 4-19. Mean contact attempts by NHIS completion status, round 1 of panels 13-15

Return To Table Of Contents

Table 4-20. Mean days between first and final contact by NHIS completion status for round 1 of panels 13-15

Summary

Overall, results of the 2010 rounds of data collection were similar to those of most other recent years: the project was successful in maintaining response rate levels for rounds 2 through 5, but, despite an increase in level of effort and the introduction of a number of initiatives to boost participation rates, saw further erosion of the round 1 response rate. The decline in the round 1 response rate was seen particularly within the white/other domain, which makes up almost half of the sample. The NHIS partial completes remained a source of difficulty, with a response rate 15 percent lower than among the NHIS full completes. The associated increase in nonresponse was concentrated primarily in refusals, with other categories of nonresponse remaining relatively even with prior years and accounting for 5 to 6 percent of the net sample. The level of effort expended in work to build higher response rates, as measured in terms of interviewer hours expended and number of contact attempts per case, increased.

Return To Table Of Contents

4.4 Authorization Form Signing Rates

Medical provider authorization form signing rates are shown in Table 4-21 for Panels 11 through 15. (Table A-3 in Appendix A shows the signing rates for all panels and rounds to date.) Panel 14 Round 3 signing rates increased slightly from Panel 13 Round 3, although in Round 4 the signing rate was slightly lower than in the previous Panel. Signing rates for Panel 15 Rounds 1 and 2 dropped but remained above the rates from Panel 12.

Table 4-22 shows signing rates for pharmacy authorization forms for Panels 11 through 15 (Table A-4 in Appendix A shows the signing rates for all panels and rounds to date.) Beginning in 2009, pharmacy authorization forms were collected for rounds 2 through 5; previously they were collected only in rounds 3 and 5. The signing rate for Panel 13 Round 5 was 70.2 percent, the lowest rate shown in the table. The trend toward lower compliance in Panel 13 caused some concern that asking for authorization forms in each round was perceived as additional burden. This trend has continued in Panel 14; the signing rates for Panel 14 Rounds 3 and 4 were slightly higher than the rates for Panel 13, but remained lower than the Round 5 rates from Panels 11 and 12.

Table 4-21. Signing rates for medical provider authorization forms for Panels 11 through 15

Return To Table Of Contents

Table 4-22. Signing rates for pharmacy authorization forms for Panels 11 through 15

Return To Table Of Contents

4.5 Self Administered Questionnaire (SAQ) and Diabetes Care Supplement (DCS) Response Rates

Self-Administered Questionnaire (SAQ) data collection begins in rounds 2 and 4 of a panel, with followup for nonresponse in rounds 3 and 5. Table 4-23 shows both the round-specific response rate and the combined rate after the followup round is completed. (Table A-5 in Appendix A shows the results of the SAQ collection for all applicable panels and rounds to date.) The response rate in Panel 15 Round 2 dropped slightly to 85.4 percent from the Panel 14 Round 2 rate of 86.5. The combined response rate for the first full year of SAQ collection in Panel 14 (2009), 92.6 percent, was a slight decline from the rate achieved in Panel 13 in 2008, but is higher than the combined rate for previous years.

Table 4-23. Results of self-administered questionnaire (SAQ) collection for Panels 11 through 15

Return To Table Of Contents

The response rates for the Diabetes Care Supplement (DCS) for Panels 11 through 14 are shown in Table 4-24. (Table A-6 in Appendix A shows the results of Diabetes Care supplement (DCS) collection for all applicable panels and rounds to date.) Since the DCS is collected only during rounds 3 and 5, with no followup in the subsequent round, efforts to gain a high response rate are limited to the one round in which the DCS is requested. The DCS rates in the table include the results of an additional followup effort conducted by telephone toward the end of the field period.

In 2009, the Panel 12 Round 5 and Panel 13 Round 3 rates were the highest achieved since Panel 7; in Panel 13 Round 5 and Panel 14 Round 3, the response rates returned to rates more typical of recent years.

Table 4-24. Results of diabetes care supplement (DCS) collection for Panels 11 through 14

Return To Table Of Contents

5. Home Office Support of Field Activities

The home office supports the data collection effort in several important ways. Prior to the start of the data collection rounds, the home office prepares and ships hard copy case materials and material supply boxes to the field staff. The home office also responds to the toll free respondent information line and disseminates information from the calls to the field. During data collection, the home office supports the creation of new split and student RUs, manages the interviewer validation effort, receipts case materials and scans authorization forms, manages a home office tracking effort, generates and distributes reports for monitoring field production, and provides technical support to the field staff through the MEPS CAPI Hotline.

Return To Table Of Contents

5.1 Preparation For Field Activities

As each wave of cases became available for fielding, clerical staff created RU folders which contained hard copy materials associated with the case such as authorization forms, response sheets from advance contact, follow up SAQs, and mini labels that are used to identify hard copy documents for the case. In 2010, with the transition to electronic format for most of the interviewing materials that contain personally identifiable information (PII), case identification information that had been in hard copy format is now available to the interviewer on the laptop and this significantly streamlined the assignment preparation task.

The cases were shipped to the supervisors one to two weeks prior to the start of each round so they could be assigned and sent to the interviewers in time to start interviewing. At the same time, interviewers received the CAPI software for the upcoming rounds by way of a CD update as well as the home study training package to prepare them for interviewing in the upcoming rounds. Field interviewers and supervisors also received a replenishment of supplies at the start of the round and ordered additional supplies from the home office as needed.

Home office staff also managed the advance mailings and provided postal return information to the field from letters returned to the home office as undeliverable. Advance mailings were prepared and mailed by a mailing vendor. The mailing vendor standardized the addresses and submitted them to the National Change of Address (NCOA) database to obtain the most current addresses for mailing.

Return To Table Of Contents

5.2 Support During Data Collection

During data collection the home office manages a variety of tasks that support the field effort. The home office responds to the toll-free respondent information line and relays information from respondent calls to the field supervisors. The field supervisor must contact the respondent and inform the home office about the results of the call within 24 hours of being notified. Table 5-1 shows the number and percent of calls made to the respondent information line in the Spring and Fall rounds of 2008 - 2010. The respondent information line received far fewer calls for Panel 15 Round 1 (2.7 percent) than for Panel 14 Round 1 (4.6 percent) and Panel 13 Round 1 (6.0 percent). This may be because respondents received just one mailing before the data collection period began, rather than two, and didn’t perceive the study as burdensome as in the past. Also, the advance contact calling period was truncated because the pre-notification mailing went out to Round 1 respondents later than in the past in anticipation of an increase in the respondent incentive amount. However, the combined calls from Panel 14 Round 3 and Panel 13 Round 5 in 2010 (3.5 percent) was higher than in the two prior years.

Table 5-1. Number and percent of calls to the respondent information line, 2008-2010

Return To Table Of Contents

Table 5-2 shows the number and types of calls received during 2009 and 2010. (Table A-8 in Appendix A shows the number and types of calls from 2000 through 2009.) There was an increase in the number of calls to make an interview appointment in all rounds of 2010 as compared to 2009. As described in Chapter 2, several changes were made to the advance mailing protocols in 2010 as a result of feedback from the interviewers. One of the changes was the inclusion of an appointment postcard in all of the advance mailings, which are the mailings that take place during the field effort, about two weeks before interviewer contact. Respondents are given a choice of mailing the appointment postcard or calling the respondent information line to request an appointment, and the inclusion of the post card in the later mail out encouraged more respondents to call for appointments.

Home office staff monitored production and provided reports and feedback (such as CAPI interviews conducted in less than 30 minutes) to field managers and supervisors for review and follow-up. Reports generated from the BFOS management system that show weekly and cumulative figures on field production, response rate, and costs were sent to AHRQ on a weekly basis. The home office printed validation abstracts, which contain information from the interview, and sent them to the quality control assistants for validation calls. Home office staff also printed and distributed split and student processing reports that provide information for conducting interviews with a split or student RU. Refusal letter requests and requests for locating information from an outside tracking service were also managed at the home office.

Contents of completed case folders sent to the home office from the field were reviewed and recorded in the receipt system. Authorization forms were edited for completeness and scanned into an image database. Problems with authorization forms are documented and feedback is sent to the field supervisor to review with the interviewer. The receipt department also tracked interview dates and notified the field if the case materials for a completed interview had not arrived within 2 weeks of the interview date. SAQ and DCS questionnaires were also receipted and prepared for coding. Supply requests from the field were emailed to the MEPS supply center at the home office where they were filled.

The MEPS CAPI Hotline provided technical support for field interviewing activities during 2010. Hotline staff is available 7 days a week to help field staff resolve CAPI, Field Management System, transmission, laptop, and modem problems. The CAPI Hotline serves as a focal point for tracking and shipping all field laptops, maintaining systems for monitoring field laptop assignment, and coordinating laptop repair.

Table 5-2. Calls to the respondent information line, 2009 and 2010

Return To Table Of Contents

6. Incentive Experiment

To learn more about this experiment, refer to Respondent Payment Experiment – Results from Panel 13.

Return To Table Of Contents

Appendix A. Comprehensive Tables – Household Survey

Table A-1. Data collection periods and starting RU-level sample sizes, all panels

Return To Table Of Contents

Table A-2. MEPS household survey data collection results, all panels

Return To Table Of Contents

Table A-3. Signing rates for medical provider authorization forms

Return To Table Of Contents

Table A-4. Signing rates for pharmacy authorization forms

Return To Table Of Contents

Table A-5. Results of self-administered questionnaire (SAQ) collection

*Totals represent combined collection of the SAQ and the parent-administered questionnaire (PAQ).

Return To Table Of Contents

Table A-6. Results of Diabetes Care Supplement (DCS) collection*

*Tables represent combined DCS/proxy DCS collection.

Return To Table Of Contents

Table A-7. Calls to respondent information line

Return To Table Of Contents