MEPS Annual Methodology Report 2021

Deliverable Number: 121C.102
Contract Number: 75Q80120D00024
June 15, 2022

Authors
Westat
Westat Reference Number: 2-7-634
Draft

Submitted to:
Agency for Healthcare Research and Quality
Center for Financing, Access, and Cost Trends
560 Fishers Lane
Rockville, MD 20850

Submitted by:
Westat
An Employee-Owned Research Corporation^®
1600 Research Boulevard
Rockville, Maryland 20850-3129
(301) 251-1500

Table of Contents

Introduction
1 Sample
1.1 Sample Composition
1.2 Sample Delivery and Processing
2 Instrument and Materials Design
2.1 Introduction
2.2 Changes to CAPI Instrument for 2021
2.3 Testing of the Questionnaire and Interviewer Management System
2.4 Changes to Materials and Procedures for 2021
3 Recruiting and Training
3.1 Field Interviewer Recruiting for 2021
3.2 2021 Interviewer Training
3.2.1 Experienced Interviewer Training
3.2.2 Continuing Education for All Interviewers
4 Data Collection
4.1 Data Collection Procedures
4.2 Data Collection Results: Interviewing
4.3 Data Collection Results: Authorization Form Signing Rates
4.4 Data Collection Results: Self-Administered Questionnaire (SAQ), Diabetes Care Supplement (DCS), and Collection Rates
4.5 Social Determinants of Health Self-Administered Questionnaire (SDOH SAQ): Methods and Results
4.6 Quality Control
4.7 Security Incidents
5 Home Office Support of Field Activities
5.1 Preparation for Field Activities
5.2 Support During Data Collection
6 Data Processing and Data Delivery
6.1 Processing to Support Data Delivery
6.1.1 Schedules for Data Delivery
6.1.2 Data Quality Control System
6.1.3 Transformation
6.1.4 TeleForm/Data Editing of Scanned Forms
6.1.5 Coding
6.2 Data Delivery
6.2.1 Variable Construction
6.2.2 File Deliveries
Appendix A Comprehensive Tables – Household Survey
Table 1-1 Initial MEPS sample size (RUs) and number of NHIS PSUs, all Panels
Table 1-2 Data collection periods and starting RU-level sample sizes, spring 2017 through fall 2021
Table 1-3 Percentage of NHIS households with partially completed interviews in Panels 4 to 26
Table 1-4 Distribution of Panel 26 sampled RUs by sample domain
Table 2-1 Supplements to the CAPI core questionnaire (including hard-copy materials) for 2021
Table 3-1 Staffing for spring field period, 2017�2021
Table 3-2 Spring attrition rate among new and experienced interviewers, 2017�2021
Table 3-3 Fall attrition rate among new and experienced interviewers, 2017�2021
Table 3-4 Annual attrition rate among new and experienced interviewers, 2017�2021
Table 4-1 Data collection schedule and number of weeks per round of data collection, 2021
Table 4-2 Case potential categories for classifying and prioritizing case work, spring 2021
Table 4-3 MEPS-HC data collection results, Panels 20 through 26
Table 4-4 Response rates by data collection year, 2012-2021
Table 4-5 Summary of MEPS Round 1 response and nonresponse, 2016-2021 panels
Table 4-6 Summary of MEPS Round 1 response, 2016-2021 panels, by NHIS completion status
Table 4-7 Summary of MEPS Panel 26 Round 1 response rates, by sample domain by NHIS completion status
Table 4-8 Summary of MEPS Round 1 results for RUs who ever refused, Panels 20-26
Table 4-9 Summary of MEPS Round 1 results for RUs who were ever traced, Panels 20-26
Table 4-10 Interview timing comparison, Panels 20 through 26 (mean minutes per interview, single-session interviews)
Table 4-11 Mean contact attempts by NHIS completion status, Round 1 of Panels 24-26
Table 4-12 Signing rates for medical provider authorization forms for Panels 19 through 26
Table 4-13 Signing rates for pharmacy authorization forms for Panels 19 through 26
Table 4-14 Results of Self-Administered Questionnaire (SAQ) collection for Panels 20 through 26
Table 4-15 Results of Diabetes Care Supplement (DCS) collection for Panels 18 through 25
Table 4-16 Impact of fall follow-up on total completed SDOH SAQs
Table 5-1 Number and percent of respondents who called the respondent information line, 2017-2021
Table 5-2 Calls to the respondent information line, 2020 and 2021
Table 6-1 2021 cases with comments or data check issues
Table 6-2 Total number of comments by category
Table A-1 Data collection periods and starting RU-level sample sizes, all Panels
Table A-2 MEPS household survey data collection results, all Panels
Table A-3 Response rates by data collection year
Table A-4 Summary of MEPS Round 1 response and non-response
Table A-5 Summary of Round 1 response by NHIS completion status
Table A-6 Summary of MEPS Round 1 results for all RUs who ever refused
Table A-7 Summary of MEPS Round 1 results for RUs who were ever traced, Panels 15-26
Table A-8 Interview timing comparison (mean minutes per interview, single-session interviews)
Table A-9 Mean contact attempts by NHIS completion status, Round 1
Table A-10 Signing rates for medical provider authorization forms
Table A-11 Signing rates for pharmacy authorization forms
Table A-12 Results of Self-Administered Questionnaire (SAQ) collection
Table A-13 Results of Diabetes Care Supplement (DCS) collection*
Table A-14 Results of patient profile collection
Table A-15 Calls to respondent information line
Table A-16 Files delivered during 2021
Figure 4-1 SDOH contact mode determination flowchart
Figure 4-2 Communication protocol for SDOH contact modes
Figure 4-3 Response rate by contact mode and interview number
Figure 6-1 Blaise to Dex transformation

Introduction

The Household Component of the Medical Expenditure Panel Survey (MEPS-HC, Contract 290- 2016-00004I, awarded July 1, 2016, and Contract 75Q80120D00024, awarded July 13, 2020) is the central component of the long-term research effort sponsored by the Agency for Healthcare Research and Quality (AHRQ) to provide timely and accurate data on access to, use of, and payments for healthcare services by the U.S. civilian non-institutionalized population. The project has been in operation since 1996, each year producing a series of annual estimates of health insurance coverage, healthcare utilization, and healthcare expenditures. This report documents the principal design, training, data collection, and data processing activities of the MEPS-HC for survey year 2021.

Data are collected for the MEPS-HC through a series of overlapping household Panels. Each year a new Panel is enrolled for a series of five in-person interviews conducted over a 2½-year period.

Panels 23 and 24, however, have been extended to nine interviews conducted over 4½ years, as described in the section below on changes due to COVID-19. This report describes work performed for all of the Panels active during calendar year 2021. Data collection operations in 2021 were for Panel 23, Rounds 7 and 8; Panel 24, Rounds 5 and 6; Panel 25, Rounds 3 and 4; and Panel 26, Rounds 1 and 2. Data processing activity focused on delivery of full year utilization and expenditure files for calendar year 2019.

The report touches lightly on procedures and operations that remained unchanged from prior years, focusing primarily on results of the 2021 operations and features of the project that were new, changed, or enhanced for 2021. Tables in the body of the text highlight 2021 results, with limited comparison to prior years. A set of tables showing data collection results over the history of the project is included in the Appendix.

Chapter 1 of the report describes the 2021 sample and activities associated with preparing the sample for fielding. Chapters 2 through 5 discuss activities associated with the data collection for 2021: updates to the survey questionnaire and field procedures; field staff recruiting and training; data collection operations and results; and home office support of field activities. Chapter 6 describes data processing and data delivery activities.

Return To Table Of Contents

Changes Due to COVID-19

All MEPS Household Component (MEPS-HC) face-to-face interviewing ceased on March 17, 2020, due to the impact of COVID-19 on American life. Data collection switched to the telephone mode, and in 2021 a mix of in-person and telephone interviewing was used, depending on the level of the COVID-19 pandemic. In-person data collection was impacted by the COVID-19 much more in the spring rounds than in the fall rounds. In the spring, Round 1 had 71.7 percent of interviews conducted by telephone, Round 3 had 96.4 percent, and Round 5 had 98.2 percent. In the fall, 33.8 percent of Round 2 and 39.7 percent of Round 4 were conducted by telephone. (In a typical year, around 5 percent to 8 percent of interviews are conducted by telephone, mostly student interviews and Round 5 interviews.)

MEPS-HC continued several modifications to project systems, processes, and procedures begun in 2020 to respond to the pandemic. Please see the 2020 methodology report for additional details:

Extension of Panels 23 and 24. Anticipating potential negative impacts of the COVID-19 pandemic on response rates and the number of households that would be included in 2020 and 2021 data and beyond, a decision was made to extend Panel 23 and Panel 24 through nine rounds. The extended Panel rounds have been conducted primarily by telephone, with limited in-person interviewing conducted when safe for hard-to-reach or hearing-impaired respondents.

Enhancing the Quality of Telephone Interviewing. MEPS provided respondents with a website for show cards and other documents that interviewers would normally present in-person on paper to respondents. Interviewers requested that respondents refer to the online show cards in answering each item or read the show cards out loud, mirroring the in-person protocol. Interviewers received headsets and telephone interviewing protocols and training, including data quality protocols specific to each round of data collection.

Maximizing Response Rates. The project developed and sent COVID-19-specific letters and postcards tailored for each Panel and round to notify households that the study was ongoing and to expect telephone outreach. The project also added efforts to increase return of hard-copy materials, particularly authorization forms (AFs), including a formal protocol for reminder calls, re-mailing unreturned AFs, and a modified in-person protocol for the retrieval of completed AFs.

CAPI Instrument Changes. To more fully capture telehealth events, the project added an event type for telehealth events, broadened the text in the provider probes section to prompt respondents to include telehealth events, and adjusted the wording in other corresponding sections to accommodate care received via telehealth. Wording exclusive to in-person visits (i.e. “seen,” “in person”) was updated to allow for the inclusion of telehealth visits. A section was added to capture delays in care due to the COVID-19 pandemic, referring back to the start of the pandemic for the spring 2021 interview and back to the date of the spring interview for the fall 2021 interview. A question was added for the fall interview about whether household members had received a COVID-19 vaccine.

In-Person Data Collection. COVID-19 in-person mitigation protocols were developed and distributed to interviewers who were authorized to conduct in-person interviewing. Interviewers received training on use of personal protective equipment (PPE) and COVID-19 mitigation. Using Westat’s COVID Dashboard for Household Surveys, MEPS monitored conditions for safe in-person interviewing and AF collection. In-person efforts led to both in-person interviews and enhanced the ability to make telephone interview appointments.

Return To Table Of Contents

1. Sample

Each year a new, nationally representative sample for the Medical Expenditure Panel Survey Household Component (MEPS-HC) is drawn from among households responding to the previous year’s National Health Interview Survey (NHIS). Up until 2020, households in a new Panel participated in a series of five interviews that collect data covering two full calendar years. For each calendar year the sample respondents from two Panels�one completing its first year in the study (Round 3) and one completing its second year (Round 5)�have been combined for analysis purposes, resulting in a series of annual estimation files. Beginning in 2020, with the onset of the COVID-19 pandemic, there were concerns of declining response rates as well as challenges in recruiting respondents by telephone for Panel 25. These concerns continued for Panel 26 in 2021. The extension of Panel 23 for a third year of data collection has helped to maintain the ongoing sample.

The sample for Panel 26 was selected from among households responding to the NHIS in the preceding year where the NHIS sample was based on the NHIS sample design initially implemented in 2016 (as were Panels 22-25). Specifically, the MEPS household sample was randomly selected from among those that participated in the NHIS during the first three quarters of 2020 and who had been assigned to NHIS Panels 1 and 3, the NHIS Panels designated for MEPS. However, due to the pandemic, households from Panels 2 and 4 were also selected for the MEPS sample.

This chapter describes the 2021 MEPS sample drawn from 2020 NHIS responding households as well as steps taken to prepare the new sample for fielding.

Return To Table Of Contents

1.1 Sample Composition

Table 1-1 shows the starting sample sizes in terms of the number of reporting units (RUs) for all MEPS Panels through Panel 26 and the number of MEPS primary sampling units (PSUs) from which each Panel was drawn. Note that the change in the number of PSUs for Panel 12 reflects the redesign of the NHIS sample implemented in 2006 (thus affecting MEPS in 2007), following the 2000 decennial census. The number of PSUs for Panel 26 is based on the number of PSUs associated with MEPS after the 2016 NHIS sample redesign, the fifth such MEPS Panel under this design. The reduction in the number of PSUs after Panel 22 stemmed from further modifications to the NHIS design. The MEPS sample units presented are RUs, each of which represents a set of related persons living together within the same NHIS-responding household selected for MEPS participation. Related members of the NHIS households sampled for MEPS who move as a unit during the MEPS data collection period (as well as separate individuals) form new RUs for interviewing purposes. Each new RU is followed over the course of the five MEPS data collection rounds and interviewed at their new address.

* RUs: Reporting units; PSUs: Primary sampling units.

Return To Table Of Contents

MEPS data collection is conducted in two main fielding periods each year. Typically, during the January-June period, Round 1 of the new Panel and Rounds 3 and 5 of the two continuing Panels are fielded, with the Panel in Round 5 retiring at mid-year. Normally, during the July-December period, Round 2 of the new Panel and Round 4 of the remaining continuing Panel are fielded.

However, with a third Panel added for the first time in 2020, a Round 6 for Panel 23 was also fielded in the fall data collection period. It should be noted that, because Round 5 of Panel 23 collected MEPS data only through December 31, 2019, the reference period for Round 6 covered from the date of interview back to January 1, 2020. As a result, the reference periods for Round 6 were typically much longer than the usual reference period for a fall interview. Table 1-2 summarizes the combined workload for the January-June and July-December periods from spring 2017 through fall 2021.

Over the years shown in Table 1-2, the combined spring and fall workload has ranged from a low of 37,305 in 2019 to a high of 44,466 in 2021. Typically, the interviewing workload during the spring field period, when three Panels are active, is substantially larger than during the fall, when there are only two. In 2021, there were four active Panels in both the spring and fall field periods. The spring field period still had more cases, with 25,126 cases fielded, an increase from the low number of cases fielded in spring 2020, while the fall workload had 19,340 RUs, the highest of the 5 years shown.

* RU-level sample size for this table derived from field management system counts and operational reports detailing fielded sample.

Return To Table Of Contents

Each new MEPS Panel includes some oversampling of population groups of particular analytic interest. Since 2010 (Panel 15), the set of sample domains has included oversamples of Asians, Blacks, and Hispanics. All households set aside in the NHIS for MEPS that have at least one household member in any of these three categories (Asian, Black, or Hispanic) are included in the MEPS sample with certainty. “White and other race” households have been partitioned into two sample domains and subsampled at varying rates across the years. These domains reflect whether an NHIS responding household characterized as “White or other race” provided “complete” information at the household level for the NHIS or if only “partially complete” information was provided.

As background, the partitioning of the “White, Other” domain into these two domains began in 2011 (Panel 16). The partial completes were sampled at a lower rate than the full completes in order to lessen the impact on the field effort resulting from the difficulty of gaining the cooperation of these households. The last two columns in Table 1-3 show the subsampling rates for the two groups since Panel 16. The partial completes in the “White, Other” domain have been subsampled at rates ranging from a low of 40 percent (Panel 17) to a high of 53 percent (Panel 20).

* The figures in the second column of the table are the proportion of partial completes in the total delivered sample, after subsampling. The figures in the third and fourth columns are subsampling rates applied to the two White/Other subdomains in Panels 16 through 22.

Return To Table Of Contents

The Panel 26 sample distribution by domain, as detailed in Table 1-4, was impacted by the pandemic and overall NHIS response rates. This reduces our ability to oversample households in minority domains, altering the proportional composition from previous years. This has implications for weighting and response rates as households in minority domains exhibit higher levels of response in comparison to the other group.

Return To Table Of Contents

1.2 Sample Delivery and Processing

The 2021 MEPS sample was received from AHRQ and NCHS in three deliveries. The first delivery, containing households sampled from the first quarter of the 2020 NHIS, was received on October 2, 2020. MEPS did not receive a sample delivery from the 2020 NHIS second quarter due to the pandemic. Households selected from the third quarter of the NHIS were delivered on December 2, 2020. To achieve a sample size comparable to previous years, households were selected from Panels 2 and 4, Quarter 1 of the 2020 NHIS. These Panels are not normally incorporated for MEPS use, and these households were added later into the field period on February 9, 2021, as a supplemental third wave.

The October delivery of the first majority of the new sample is instrumental to the project’s schedule for launching interviewing each year in early January. The partial file gives insight into the demographic and geographic distribution of the households in the new Panel. This information, when combined with information on older Panels continuing in the new year, guides project decisions on the number and location of new interviewers to recruit.

Upon receipt of the first portion of the 2021 sample, project staff also reviewed the NHIS sample file formats to identify any new variables or values and to make any necessary changes to the project programs that use the sample file information. Following this initial review, staff proceeded with the standard processing through which the NHIS households are reconfigured to conform to MEPS reporting unit definitions and prepared the files needed for advance mailouts and interviewer assignments. The early sample delivery also allows time for checking and updating NHIS addresses to improve the quality of the initial mailouts and to identify households that have moved since the NHIS interview.

Return To Table Of Contents

2. Instrument and Materials Design

2.1 Introduction

Each year, the project makes a number of changes to the instrument used to collect MEPS-HC data, as well as to the field procedures followed by the interviewers who collect the data. The notable changes made for 2021 are detailed in this chapter.

Return To Table Of Contents

2.2 Changes to the CAPI Instrument for 2021

The MEPS-HC computer-assisted personal interviewing (CAPI) instrument was modernized as part of a technology upgrade launched in spring 2018. For each data collection cycle since then, AHRQ and Westat have worked together to define a set of modifications to the CAPI instrument. Some modifications are new items or new sections, whereas others are updates or fixes to existing items.

In addition to substantive modifications during 2021, the CAPI instrument was “genericized” to accommodate the extended rounds added because of the COVID-19 pandemic. Genericizing was necessary to successfully administer round 6 through 9 interviews planned for 2021, 2022, and 2023, as well as increased flexibility, in case the MEPS Panel design needs to be extended or revised in future cycles. Instead of referring to the round number directly (such as 1, 2, or 3) in CAPI specifications and programming, all Panel rounds were assigned to one of the following four general types:

• First (first MEPS round)
• Within (round within a single calendar year)
• Cross (round that crosses two calendar years)
• Final (final MEPS round)

While genericizing did not change which questions were administered in MEPS interviews, it required significant design, programming, and testing effort.

For 2021, there were a few other global changes across the CAPI instrument:

• For single-person households, revised question text to refer to “you” instead of household-level phrases such as “this household,” “each household member,” or “anyone in this household.” This occurred in questions across multiple sections, including Health Status (HE), Provider Probes (PP), Provider Roster (PV), Health Insurance (HX), and Closing (CL).
• Show card call-outs on CAPI screens were color coordinated to match the printed tabs on the physical show card. This change was intended to assist field interviewers and respondents to effectively use the show cards to answer MEPS questions.

Section-specific changes for the 2021 data collection period, both spring and fall, are summarized below.

Calendar (CA). In response to feedback from experienced MEPS interviewers, the CA30 Records Grid was revised to add a column for the MEPS Record Keeper. This makes it easier to track which respondents used the MEPS Record Keeper tool and to encourage respondents to start using it if they are not already doing so.

Provider Probes (PP). A new show card (PP-16) was added at the final PP items PP160 and PP320, with similar content as the Records Job Aid. Given that the PP question series already includes numerous show cards, including this content as an additional show card is less burdensome for interviewers and respondents than referring to materials outside the show card binder.

Telehealth (TH). Due to the pandemic, there was a global shift to increased provision of health care via telehealth. To capture this type of care, a new “telehealth” (TH) event type was added in spring 2021. For health care identified as a telehealth event, a new TH utilization section was developed. The TH section closely mirrored the MV and OP utilization sections, although two new items were added specifically for TH events. The first item collects the mode of telehealth visit; asking whether it was via phone, video or some other way (TH10). The second item asks whether the provider (or the place where the provider works) is owned or operated by a hospital (TH60). Additionally, existing PP questions about home health were revised from asking about “care received at home” to “care received from someone who visited your home” to ensure this was not confused with telehealth care. Throughout other instrument sections, wording exclusive to in-person visits (e.g., “seen,” “in person”) was also updated to allow for the inclusion of telehealth visits.

Provider Specialty and Types. In spring 2021, show cards were added for the items in the utilization sections that collect the doctor’s specialty (MV20, OP20, TH30) and type of medical provider seen (MV30, OP30, TH40). This change was made to reduce respondent cognitive burden and help cue recall.

COVID-19 (CV). In response to the COVID-19 pandemic, a new section was added in spring 2021 to collect information about delays in care due to the pandemic, including medical care, dental care, and prescription medicines. This section was continued in fall 2021, with the addition of questions about COVID-19 vaccination.

Health Insurance (HX) and Related Sections. Throughout the Health Insurance section, items that referred to state-specific Medicaid names were modified to additionally refer to “Medicaid.” This included revisions to question texts, fills, and context headers. This change was made to help cue respondents who may not be familiar with the name of their state-specific Medicaid program but are familiar with the Federal term “Medicaid.”

Supplements to the CAPI Instrument

Table 2-1 shows the supplements for the rounds administered in calendar year 2021. The major change for 2021 was the introduction of a new “Social and Health Experiences” self-administered questionnaire (SAQ), known internally as the Social Determinants of Health (SDOH) SAQ. For more information about the SDOH SAQ, please refer to Chapter 4, Section 4.5.

Return To Table Of Contents

2.3 Testing of the Questionnaire and Interviewer Management System

Testing for the spring 2021 (Rounds 1/3/5/7) instrument was conducted between September and December 2020. Testing for the fall 2021 (Rounds 2/4/6/8) instrument was conducted between March and June 2021. Since 2018, many of the testing approaches and procedures used for the technical upgrade have been continued or adapted to maintain a comprehensive testing plan that supports the ongoing instrument development schedule.

CAPI instrument development and testing included multiple programming/testing iterations that each lasted several weeks. Testing was conducted by a mix of corporate testers, MEPS project staff, and trained programming staff. Project and systems staff performed all testing in close coordination with the design team. For each of the spring and fall instruments, AHRQ received an alpha delivery and conducted its own testing. The following month, AHRQ received a beta delivery and conducted additional testing.

The testing ensured that CAPI followed the design as intended and assessed whether the layout of the overall screen for a given question, and across questions, consistently met the requirements designed to minimize measurement error. Feature testing thoroughly tested all new features against specifications, including wording, text fills, legal and illegal responses, boundary conditions, and skip patterns. Testers validated every possible variation allowed by the specifications.

Both scripted and free-form testing were used throughout the development and testing process. A full suite of scripted test cases was defined by the design staff and analytic leads at Westat and is updated each cycle. These scripted test cases represent approximately 80 percent of the cases fielded, including common paths through the CAPI instrument across all Panel rounds. The test script suite was executed through alpha and beta for the spring and fall testing cycles.

In contrast, free-form testing focused on design changes in the current instrument build and ensured that any reported instrument bugs had been fixed. Free-form testing was also utilized to ensure the stability of the CAPI data model and to evaluate the stored data in new or unusual situations. Testers routinely pushed array limits, used back-up, changed answers, and used break-off and restart cases to challenge performance boundaries.

Additional testing components, including enhanced integration testing and ad hoc/free-form testing, were also conducted. The enhanced integration testing allowed project staff to check electronic Face Sheet information, test the Interviewer Assignment Sheet (IAS), and make entries into the electronic record of calls and refusal evaluation form. The ad hoc testing component used information derived from actual cases to verify that all management information was brought forward correctly from previous rounds. Using actual case data also allowed staff to check uncommon paths through the MEPS instrument so that specific changes to the questionnaire could be thoroughly tested.

Return To Table Of Contents

2.4 Changes to Materials and Procedures for 2021

The manuals and the materials for the 2021 field effort were updated as needed to reflect changes to the questionnaire and management systems. Below is a description of the key changes to the materials and procedures.

Instructional Manuals

The field interviewer procedures manual was updated to address changes in field procedures and updates to the Interviewer Management System (IMS). In the fall of 2021, a PDF of the Field Interviewer Procedures Manual was added to the MEPS laptop to give interviewers access to a searchable electronic version of the manual.

Electronic Materials

To help prepare for upcoming interviews, the electronic face sheet in the IMS provides interviewers with information needed to contact their assigned households and familiarize themselves with the composition of the household and relevant details about their prior history with the survey. In 2021, the face sheet was expanded to show information for up to eight rounds. The Policy Booklet section was removed from the face sheet since all follow-up efforts were completed for that task at the end of 2020. A flag to indicate if there was an outstanding SDOH (Social and Health Experiences SAQ) from the previous round to collect was added to the face sheet for the fall 2021 data collection.

The IMS also contains an RU Information module for documenting operational information to help the next round’s interviewer effectively work each case, an RU Contact module for reporting address and telephone number changes identified prior to the CAPI interview, and the Interviewer Assignment Sheet (IAS), which supports follow-up for AFs and SAQs not completed at the time of the interview. A section was added to the IAS in 2021 to display paper Social and Health Experiences SAQ (SDOH SAQ) requests from CAPI.

New to interviewers in 2021 was the deployment of iPhones with mFOS, or the mobile field operating system. The mFOS application gives interviewers access to some of the same information and capabilities available in the IMS. They can view face sheets for their assigned cases, report address and telephone number changes, and enter electronic records of calls (EROCs) on the iPhone. Data from the iPhone and the IMS are synced upon transmission from the laptop. The mobile phones also offer navigation, email, and other administrative applications useful to interviewers. The phone also serves as a hot spot to allow for internet connectivity and data transmission.

Advance Contact and Other Case Materials

All respondent letters, monthly planners, and self-administered questionnaires were updated with the appropriate year references, and the Income Job Aid was updated with 2018 data. Further, the Monthly Planner, MEPS brochure, and the advance mailing envelope were redesigned with a more modern font and elements that provided a more cohesive look for the advance mailings.

The MEPSDocs.org website was repurposed for telephone interviewing, providing respondents with access to respondent cooperation and record keeping materials, and the show cards, in both English and Spanish.

In fall 2021, the AHRQ signature on letters was updated to that of Joel Cohen, Director of AHRQ’s Center for Financing, Access and Cost Trends.

Return To Table Of Contents

3. Recruiting and Training

3.1 Field Interviewer Recruiting for 2021

Overview. For spring 2021 data collection, MEPS attempted to recruit approximately 140 new interviewers to join the team of approximately 270 interviews who were active on MEPS at the start of the 2021 data collection in early January. Our goal was to increase the team and to start spring data collection with about 400 interviewers.

To put the recruiting and attrition numbers into perspective, Table 3-1 summarizes the MEPS spring data collection staffing for the period of 2017 � 2021.

Spring 2021 Attrition Staffing - *Note that the total of 143 includes the 36 Interviewers who were not trained until mid-June in order to shore up fall staffing.

Return To Table Of Contents

Recruiting Goals. Based on a projected sample size of approximately 24,200 RUs across the four Panels to be fielded for spring 2021 and the likely number of experienced MEPS interviewers available at the end of fall 2020 data collection (about 260), including a MEPS travel team of 10 to 12 members, Westat estimated needing to recruit approximately 140 new interviewers for the standard staffing model. The goal was to start data collection with approximately 400 interviewers actively working during the spring 2021 data collection period.

Westat uses the Field Interviewer Recruitment Module (FIRM) software designed to manage the data collector recruiting process. This system works in conjunction with BrassRing, an online application system used to collect, track, and manage applications for all positions at Westat. The BrassRing system collects applications from both external (new to Westat) and internal (current or former Westat field data collectors) applicants.

The main recruiting of new field interviewers for 2021 began in October 2020 and continued into early January 2021. Due to a relatively low number of applicants, MEPS developed a national telephone/ traveling data collector position to expand opportunities to a national audience. The interviewing in spring 2021 was heavily weighted toward telephone interviewing, so this new position seemed worth trying. Because there was a higher level of attrition during the virtual new hire training, it was decided that MEPS would hold an attrition training to supplement the staff available to move forward into fall data collection in July. Recruitment for the attrition training began in early April and ended in early June 2021.

Recruiting Outcomes. During the main recruiting period, 146 candidates accepted job offers, of whom 121 started training and 111 finished. With the addition of these new trainees, the project began 2021 data collection with a total of 383 interviewers. During the attrition training period, the goal was to add 70 more interviewers. However, only 36 candidates accepted job offers and all 36 completed attrition training.

Interviewer Attrition During 2021 Data Collection. During the spring data collection, 62 new interviewers and 33 experienced interviewers were lost to attrition. An additional 30 new interviewers and 27 experienced interviewers were lost during the fall round. Total attrition for the year was 35.4 percent, the highest attrition level MEPS has experienced in the past 5 years. In looking forward to 2022, MEPS plans to expand the interviewing staff so that we may begin data collection with at least 400 interviewers. The breakdown of interviewer attrition is shown in Tables 3-2, 3-3, and 3-4.

Table 3-2 shows the overall attrition rate during the spring data collection period from 2017 through 2021. Note that although the total spring 2021 attrition rate of 22.6 percent was not quite as high as the spring 2020 attrition rate of 23.8 percent, but it was close. This high attrition rate in 2021 seems to have been exacerbated by a virtual new hire training process that made it much easier for a new hire to quit, as evidenced by a 40.8 percent spring new hire attrition rate.

Return To Table Of Contents

Table 3-3 shows the overall attrition rate during the fall data collection period from 2017 through 2021. Note that the total fall 2021 attrition rate was 17.1 percent, the highest fall attrition rate in the past 5 years. Only the fall 2017 attrition rate was in double digits, and that was because some MEPS PSUs were retired due to changes in the new sampling frame.

Return To Table Of Contents

Table 3-4 shows the annual attrition rate across new and experienced interviewers from 2017 � 2021. The annual attrition rate for 2021 was 35.4 percent, the highest rate in the past 5 years. The extremely high rate of attrition among new hires can be attributed in large part to the continuation of the pandemic conditions, namely, a reliance on a high proportion of the interviewing being done by telephone. As noted above, the virtual training format seems to have made it much easier for new hires to quit mid-training.

Return To Table Of Contents

3.2 2021 Interviewer Training

The overall structure for training new interviewers in 2021 was a departure from prior years in order to remotely administer the training due to the COVID-19 pandemic. It began with a home study, followed by a remote training conducted over Zoom for government in early February 2021, and ending with completion of a two-part, post-classroom home study component. An attrition training was also conducted in June 2021 with minor alterations made through the experience gained during the prior session.

Pre-Training Activities. This package included a project laptop and an interactive self-paced workbook with exercises and online modules including videos and quizzes administered through Westat’s Learning Management System (LMS). The LMS generated regular reports, allowing home office and field management staff to monitor the completion of each trainee’s home study. New hires received their home study package early enough to complete the package before the remote training, but not so early that their introduction to important study concepts and project terminology would degrade before the remote training. The attrition training added additional practice with the Zoom platform prior to the remote training.

Remote Training. The usual 7½-day training format for in-person training was extended by 1 day, to accommodate the completion of both synchronous and asynchronous content. During the June attrition training, trainees were given the weekend off to attend to asynchronous content that had not been completed and address personal needs that were impacted by the remote approach. Any synchronous content accommodated trainees from the East Coast to the West Coast; therefore, the training day hours were from 12 PM through 5:30 PM EST for synchronous content.

Training sessions used a “block” approach to the training, with each training day consisting of a block of synchronous training and a block of asynchronous training. Trainees had synchronous training for some portion of each training day. Trainees completed required asynchronous blocks prior to the corresponding synchronous blocks.

For the 8½ days of project-specific training, each trainee was assigned to one of seven training classrooms (three for the June attrition training) staffed by a primary and support trainer, one or two classroom runners, and a Zoom host. The selection of trainers for the 2021 new hire trainings was based on several criteria including experience training with the CAPI instrument, overall project knowledge, and prior training experience. Prior to remote training, all training and support staff received a training on the remote platform; the associated technologies; and the content, activities, and procedures associated with remote training.

The training sessions used a variety of formats for presenting material, including lecture, question-and-answer interactions, written exercises, group discussion of problems and resolutions, and activities in which trainees were required to seek answers by consulting project resource materials. In addition, full and “mini” mock interviews (or “mocks”) and dyad role-plays were used throughout the training, and were central to training on both the mechanics and substance of the CAPI instrument.

Mocks are scripted interviews usually led by a classroom trainer who serves as both trainer and “respondent” while trainees take turns as the interviewer. Full mocks present the entire interview from Re-enumeration through Closing, while a “mini” mock relies on preloaded data to allow the training to begin at the desired questionnaire section. For the remote training, the mocks were delivered in one of three ways: demonstration, simulation, and teleconference.

Mock 1 (Round 1) was demonstrated in a synchronous session, with trainers displaying the CAPI screens and trainees reading the questions from the screen and calling out the appropriate keyboard response to the questions.

Mock 2 (Round 3) was posted on the LMS as an interactive CAPI simulation, with respondent answers coded into the simulation. Although the simulation looked and behaved like the CAPI instrument, corrective feedback was given immediately when the trainee coded incorrectly.

Mock 3 (Round 5) was administered via teleconference call led by an experienced trainer with additional support for troubleshooting. Teleconference allowed for additional hands-on CAPI practice for trainees and gave the trainer the opportunity to evaluate trainee performance.

Mini-mocks and materials on the IMS were presented in one of three modes: synchronous training in the virtual classroom, CAPI simulation hosted on the LMS, and independent practice from hard-copy materials to allow for hands-on CAPI/IMS practice.

Dyads paired trainees in a virtual breakout room to conduct an interview with one trainee playing the role of interviewer, and the other using a script to play the respondent. Each dyad pair was observed by a dyad observer, either a field supervisor or other training staff. Dyads are an effective tool for reinforcing questionnaire concepts and building interviewer confidence in administering the instrument. They also provide trainers with an opportunity to assess each trainee’s interviewing skills and mastery of the questionnaire application.

The remote training component maintained the emphasis on interviewer behaviors and interviewing techniques that facilitate complete and accurate reporting. Trainers were instructed to reinforce good interviewing behaviors during mock interviews. Good interviewing behaviors include reading questions verbatim, training respondents to use records to aid recall, actively engaging respondents in the use of show cards, and using active listening and probing skills. Trainers called attention to instances in which interviewers demonstrated such behaviors. To enhance trainee awareness of behaviors that affect data quality, dyad scripts included instructions to take a “time-out” at certain items in the interview to highlight relevant data quality issues.

In the past, scripted lab material had been provided to trainers and trainees for in-person lab practice. Often, trainees who wanted additional CAPI practice would take the scripts with them to work on independently. For the remote training, Westat offered some hard-copy scripted materials to all trainees as required independent practice.

1. Westat offered “office hours” for trainees to connect by video with experienced MEPS staff who could answer questions and address concerns.
   
   
2. Similar to in-person labs, Westat had a sign-up method (CVENT) for trainees to attend sessions for targeted review of concepts. Westat had the trainee share the screen for trainers to watch. Since the majority of help for trainees during the remote session labs was one-on-one practice, rather than using scripted materials, trainers spoke with the training team lead as well as the trainee themselves to get a feel for where extra practice was needed. The trainer then customized the one-on-one instruction to meet the needs of the trainee.
   
   
3. When a trainer or field management staff identified a trainee as needing one-on-one help, a member of the training floater team was assigned to work with the trainee.

One hundred eleven new hires successfully completed the main training, and 36 successfully completed the attrition training.

Bilingual training followed a similar format to in-person training. Bilingual trainees participated in a 4-hour block of training on the last half-day of training. Trainees completed a Round 3 dyad in Spanish. The same format for dyads used in the main training was applied to bilingual training. Trainees divided into breakout rooms to complete the dyad with training staff visiting the breakout rooms to ensure good interviewing behaviors and an understanding of the CAPI instrument. Additionally, trainees used the breakout room approach to practice refusal conversion in Spanish. Eight new interviewers successfully completed 2021 bilingual training and four new interviewers completed the bilingual attrition training.

Post-Remote Training Activities. The post-classroom home study was administered in two parts for the main training and combined into one part for the attrition training (to allow trainees to complete the home study prior to launch of the fall rounds). The first component was distributed on the last day of remote training, and new interviewers had to have successfully completed it before beginning fieldwork. It contained an interactive exercise in BFOS Secure Messaging (BSM) and completion of a mini-mock with a proxy respondent.

The home study also included a memo from the Field Director reviewing their tasks in preparation to interview, and it provided an “early work period” documentation form to assist them in setting up a work plan with their supervisor and completing tasks in a timely manner. At the same time, all field supervisors received a memo from the Field Director outlining their role in the post-classroom training through the setting of clear expectations, support, and ongoing training to their interviewers.

In addition to the home study, field supervisors engaged in additional post-training activities with new hires. New hires sat in on the report call of an experienced field interviewer and also reviewed assigned cases to report to their supervisor the best contact strategy for each. Field managers and field supervisors coordinated and implemented a mentoring/buddy plan that paired new hires with experienced FIs.

The new interviewers received the second component of the post-classroom home study about 6 weeks after the remote training. This component included both hard-copy materials as well as modules in the electronic LMS. This last component provided interviewers with additional training on respondent cooperation and participation in record-keeping activities. It also provided training on several important Re-enumeration topics and student Rus, and it reinforced interviewer practices related to collecting quality data.

Return To Table Of Contents

3.2.1 Experienced Interviewer Training

Spring 2021 Round 1/3/5/7 Home Study. The Round 1/3/5/7 home study in December 2020 followed established formats but was expanded to accommodate the introduction of the MEPS iPhone and mFOS (mobile Field Operating System), the extension of the rounds, telephone interviewing procedures, COVID-19 Mitigation protocols, the SDOH SAQ, and the new telehealth event type. The 3-hour self-paced program contained an instructional memo, independent CAPI practice, iPhone training, and a quiz.

In-Person Refresher Training. Due to the COVID-19 pandemic, the refresher training scheduled for August 2021 was canceled.

Return To Table Of Contents

3.2.2 Continuing Education for All Interviewers

Fall 2021 Round 2/4/6/8 Home Study. The Round 2/4/6/8 home study in July 2021 followed established formats. The 1½-hour self-paced program contained an instructional memo, example materials, and a quiz. Topics included the extension of the rounds in response to the COVID-19 pandemic, additional training on telephone interviewing and the use of the telephone-interviewing website for respondents, COVID mitigation protocols, and follow-up cost-sharing document collection. New interviewers hired in the spring were required to complete a mock interview with their supervisor, field manager, or designated senior interviewer before beginning the fall rounds of data collection.

Weekly Newsletter. In 2021, MEPS continued offering its field interviewer newsletter in a weekly format. The newsletter allows for additional training opportunities in a concise format and the ability to deliver content as needed to the field. Topics include CAPI questionnaire topics, procedural content, and answers to field interviewer questions.

Return To Table Of Contents

4. Data Collection

This chapter describes the MEPS-HC data collection operations and provides selected results for the eight rounds of MEPS-HC interviewing conducted in 2021. Selected comparisons to results of prior years are also presented. Tables showing results for all years of the study are provided in the Appendix.

Return To Table Of Contents

4.1 Data Collection Procedures

MEPS data collection management relies on a set of interrelated systems and procedures designed to accomplish three goals: efficiency, data quality, and cost containment. The systems include the Basic Field Operating System (BFOS), which facilitates case management through case assignment, case status and hours reporting, data quality reporting, and interviewer efficiency. Related systems include the computer-assisted recorded interview (CARI) system and the MEPS supervisor dashboard, which was placed into production in 2018. The CARI system allows for review of recordings for selected interview items to assist in the assessment of interviewer performance and question assessment. The MEPS supervisor dashboard provides views into daily and weekly management tasks related to the tracking of hours per complete, key alerts from casework in the field, the management of weekly production goals, and a number of metrics designed to facilitate weekly field calls with interviewers regarding hours worked, production, and interview quality. These tools, along with the implementation of models designed to identify cases with a higher propensity for completion, as well as on-hold procedures designed to prevent the overwork of cases in the field, form a comprehensive framework for the management of MEPS data collection.

Due to the ongoing COVID-19 pandemic, the procedures followed in the 2021 data collection differed greatly than those of years prior to 2020.

As in prior years, respondent contact materials provided respondents with the link to the MEPS website (www.meps.ahrq.gov); a toll-free number to Alex Scott, a study representative at Westat; and the link to the Westat website (www.westat.com). Calls received from the Alex Scott line were logged into the call-tracking system and the appropriate supervisor notified so that he/she could take the proper course of action.

The advance contact calls to Panel 26 Round 1 households were made by a subset of the experienced MEPS interviewers.

Typically, for Round 1 households, interviewers are instructed, with few exceptions, to make initial contact with the household in-person. For later rounds, interviewers are allowed to make initial contacts to set appointments by telephone, so long as the household had been cooperative in prior rounds. In response to COVID-19, all in-person interviewing ceased on March 17, 2020, and all contacts and interviews were conducted over the telephone. Prior to 2020, interviews conducted on the telephone represented only 5-8 percent of interviews. Procedures for telephone and text contacts were developed and implemented in 2020. These were adjusted in 2021 to instruct in-person contact where community spread was below a reasonable threshold. After initial contact, an in-person interview proceeded as scheduled when both parties agreed; otherwise, a telephone interview was scheduled.

Procedures for collecting the medical and pharmacy authorization forms for the Medical Provider Component (MPC) and self-administered questionnaires (SAQs) underwent significant changes due to the pandemic. In 2020, interviewers mailed authorization forms to respondents and had them return them to the home office via business reply envelope (BRE). After the in-person interview, the forms were generated and mailed by the interviewer from home shortly after the interview was completed, along with a BRE and the incentive check. The interviewer made a phone call to follow up within several days.

In 2021, the protocols from fall 2020 were expanded to address the steep decline in returned signed authorization forms experienced in 2020, including instituting a procedure for interviewers to place up to three reminder calls to ensure AFs were completed and returned or ready for pickup.

MEPS also continued the 2020 practice of contactless AF pickup instituted in the fall field period. MEPS continued a re-mail effort started in late fall 2020, mailing new sets of AFs to RUs where AFs were expected but not received. This was paired with the reminder calls for RUs with a larger number of AFs or hospital visits.

MEPS field managers, field directors, and the task leader for field operations continued to manage the field data collection in collaboration with the field supervisors, reinforcing the importance of balancing data quality with production and cost goals across regions. Field staff referred to this collaborative effort as the “No Region Left Behind” approach.

Throughout the year Westat continued to review data for all respondents reported to have been institutionalized in order to identify any individuals who might have been inappropriately classified and, as a result, treated as out of scope for MEPS data collection.

Data Collection Schedule. The sequence for beginning the spring rounds of data collection, most recently adjusted in 2014, was maintained for the spring round of 2021. Data collection began with Round 5, followed by Round 3, and then Round 1. For the Round 1 respondents, the later starting date allowed several additional weeks of elapsed time in which respondents could experience healthcare events to report in their Round 1 interview, with these additional events giving them a more realistic understanding of what to expect in the subsequent rounds of the study. To maintain the highest levels of quality of MEPS data, a decision was made to extend Panels 23 and 24 to nine rounds; therefore, there was no exit round in 2021.

The field period dates for the eight rounds conducted in 2021 are shown in Table 4-1.

Return To Table Of Contents

Data Quality (DQ) Monitoring. The MEPS DQ field monitoring system and procedures allowed supervisors and field managers to identify interviewers whose work deviated from quality standards and who might need additional coaching on methods for getting respondents to more completely report their healthcare events. CARI review was further integrated into weekly monitoring activities with supervisors listening to portions of roughly 1,000 interviews per field period. These reviews were used to reinforce positive interviewing behaviors and techniques; in addition, listening to CARI gave field supervisors direct exposure to interviewing behaviors that needed to be addressed. In some cases, CARI recording results were such that interviewers were instructed to stop working until they could receive some re-training, including administering a practice interview to their field supervisor. This effort was supported by DQ alerts built into the supervisor dashboard to identify possible DQ issues related to record use and event entry. Supervisors investigated these issues and retrained when necessary.

Case Potential Listing. The project continued the use of a model predicting a completed interview from a given case (“propensity to complete”) relative to other pending cases in a region. The model is designed to identify cases with a high likelihood of completion at that point in the field period relative to other pending cases. The model is dynamic and is updated weekly based on the specific conditions for pending cases at that time. The model was tested in 2019 to determine if updates were necessary to better fit the data; however, the existing model remains well-suited to current interview conditions and remains in effect even for telephone interviews.

Information from this model is integrated into BFOS (the system used for case management), providing propensity to complete as part of a comprehensive view of a case for a given week. Supervisors were to instruct interviewers�in the absence of other field information that would dictate otherwise�to attempt these cases during the next production week. Table 4-2 illustrates the potential categories used to classify cases on a weekly basis to promote field efficiency.

Return To Table Of Contents

4.2 Data Collection Results: Interviewing

Table 4-3 provides an overview of the data collection results for Panels 20 through 26, showing sample sizes, average interviewer hours per completed interview, and response rates. Table 4-4 shows the final response rates a second time, reformatted to facilitate by-round comparisons across Panels and years. Both tables display the additional Round 7 and 8 data new for 2021.

Return To Table Of Contents

Return To Table Of Contents

Return To Table Of Contents

Of the data collection rounds conducted in 2021, the response rates for most rounds showed a similar pattern to 2020, a decline when compared to the rates from 2018 and 2019. With the shift to telephone data collection continuing throughout 2021, the Round 1 response rate was seriously impacted.

Other rounds suffered similar relative declines based on the lack of in-person opportunity. Because of this decline, a decision was made to extend Panels 23 and 24 to nine rounds to maintain the sample.

While not as extensive as in 2020, hours per complete across each Panel/round were lower than pre-pandemic years due to the large amount of telephone work. The biggest impact was seen in Round 1, where the average was 11.1 hours per complete compared with 11.7 hours (over the prior 4 years prior to the pandemic).

Components of Response and Nonresponse

Table 4-5 summarizes components of nonresponse associated with the Round 1 households by Panel beginning in 2016. As the table shows, prior to 2020 the components of nonresponse other than refusals�the “not located” and “out of scope” categories�remained relatively stable; however, in 2021, the “other nonresponse” and “not located” categories showed a significant increase. While there have been relative decreases in these categories in 2021, they remain elevated in comparison to 2019 and earlier. The larger year-to-year changes are reflected in the percentage of refusals, whereby increases and decreases in the percentage of refusals align closely with corresponding decreases and increases in the completion rate.

Return To Table Of Contents

Tables 4-6 through 4-13 summarize results for additional aspects of the 2021 data collection. Because Round 1 is the most difficult of all the rounds, the presentation focuses primarily on Panel 26, Round 1.

Note: Figures shown are based on original NHIS sample and exclude RUs added to the sample as “splits” and “students.”

Return To Table Of Contents

NHIS Completion Status

Each year the MEPS sample includes a number of households classified in the NHIS as “partial completes,” in which the interviewer was able to complete part, but not all, of the full NHIS interview. Given the NHIS redesign implemented in 2018, the partial completes included in the 20 MEPS sample included some cases that completed only the roster module of the NHIS. The MEPS experience has been that for many of these NHIS cases, the difficulty experienced by the NHIS interviewer carries over to the MEPS interview: the MEPS response rate for the NHIS partial completes is substantially lower than for the NHIS completes. As noted in Chapter 1, for the 2021 sample, AHRQ repeated the step taken since 2012 of sampling the NHIS partial completes in the “White/other” category at a lower rate than the NHIS completes.

The upper portion of Table 4-6 shows the proportion of partial completes in the sample over recent years. Across all domains, the proportion of the 2021 sample classified as partial complete was significantly lower than all the previous years shown on the table. The lower portion of the table shows the persistent and substantial difference in response rate between these two components of the sample. Among the cases originally delivered from the NHIS (that is, with new reporting units discovered during the MEPS interviewing excluded from the counts), the response rate for the NHIS partial completes has been around 12 percentage points fewer or less than that for the NHIS completes. In 2020, that difference jumped up to 16.7 percentage points, and there is a 19-point difference in 2021. In 2021, the proportion of partial completes is higher than 2020’s was, at 14.7 percent, which is closer to the 2019 level.

Sample Domain

Table 4-7 breaks out response information for the NHIS completes and partial completes by sample domain categories, including the veterans domain introduced in Panel 24. Table 4-7, unlike Table 4-6, does include reporting units added to the sample during Round 1 data collection; it shows the differential in response rates between the NHIS partial completes and full completes persisting across all of the domains. The difference across the full 2021 sample was 18.2 percentage points, with NHIS partial completes responding at a lower rate in all domains. Within the individual domains the difference between the response rate for the NHIS completes and the NHIS partials was greatest for the White/other domain � 22 percentage points.

Note: Includes reporting units added to sample as “splits” and “students” from original NHIS households, which were given the same “complete” or “partial complete” designation as the original household.

Return To Table Of Contents

Refusals and Refusal Conversion

Table 4-8 summarizes the results of refusal conversion efforts by Panel. The rate of “ever refused” for RUs in Panel 26 increased to its highest level to 5.6 percent. The percentage of converted RUs for Round 1 rebounded slightly to 19.3 percent of cases converted from a low of 12.3 percent in 2020.

Return To Table Of Contents

Tracing and Locating

Table 4-9 shows results of locating efforts for households that required tracking during the Round 1 field period by Panel. The percent of households that required some tracing in 2021 (11.3%) dropped 0.4 percent from 2020; the final rate of households that were not located after tracing efforts was its highest point since 2015, although the 2021 “not located” rate was still within the range of 3.0-4.3 percent for the 7-year period shown in the table.

Return To Table Of Contents

Interview Length

Table 4-10 shows the mean length (in minutes) for interviews conducted without interruption in a single session in Panels 20-26. Starting in 2020, with the pandemic shutdown, everything moved to telephone interviews; in 2021, a large number of interviews were still conducted by telephone, which took longer as interviewers had to read the show cards aloud, thus adding time to the interview.

Return To Table Of Contents

Mean Contact Attempts Per Case

Table 4-11 shows mean contact attempts, by mode and NHIS completion status, for all cases in Round 1 of Panels 24-26. Overall, the number of contacts required per case in Panel 26 remains elevated compared to 2019 and earlier. The 2021 number of contacts is one attempt lower than in 2020. This increase is chiefly attributed to the challenges incurred by the COVID-19 pandemic and the shift to telephone interviewing. As in prior years, in Panel 26 the NHIS partial complete cases required substantially greater effort than the NHIS completes, roughly 0.8 additional in-person contacts per household.

Return To Table Of Contents

4.3 Data Collection Results: Authorization Form Signing Rates

During the Respondent Forms section of the MEPS CAPI interview, interviewers are prompted to ask respondents to sign the authorization forms (AFs) needed to conduct the MPC of MEPS. Authorization forms are requested for each unique person-provider pairing identified during the interviews as a source of care to a key member of the household. Medical provider AFs are requested for physicians seen in an office-based setting; for inpatient, outpatient, or emergency room care received in a hospital; for care received from a home health agency; for telehealth; and for certain stays in long-term care institutions. Pharmacy AFs are requested for each pharmacy from which a household member obtained prescription medicines.

Table 4-12 shows round-by-round signing rates for the medical provider AFs for Panels 19 through 26. Signing rates increased from 2020 but remained lower than in previous years.

Return To Table Of Contents

Calculation of the round-by-round collection rate for the medical provider authorization forms is based on all forms requested during a round. The rates calculated for Rounds 2-8 include forms fielded but not signed in an earlier round (nonresponse). Included as well were forms that were fielded in an earlier round and signed but rendered obsolete because the person had another health event with the provider after the date on which the original form was signed.

Table 4-13 shows signing rates for pharmacy authorization forms for Panels 19 through 26. Pharmacy authorization forms are requested in Rounds 2 through 9, with follow-up for nonresponse in subsequent rounds similar to that for medical provider authorization forms. The decline in signing rates for 2020 can be attributed to the move to telephone interviewing as a result of the COVID-19 pandemic. As with the medical provider authorizations forms, there was a slight increase in signing rate in 2021, but it remained lower than years prior to 2020.

Return To Table Of Contents

4.4 Data Collection Results: Self-Administered Questionnaire (SAQ), Diabetes Care Supplement (DCS), and Collection Rates

Self-administered questionnaires (SAQs) are requested from key adult household members in Rounds 2 and 4. Forms that are not collected in Rounds 2 and 4 are requested again in Rounds 3 and 5. In fall 2021, SAQs were requested from Panel 24 Round 8 and Panel 24 Round 6 respondents as well. Table 4-14 shows both the round-specific response rates and the combined rates after the follow-up round was completed. See Section 4.5 for information about the Social Determinants of Health Self-Administered Questionnaire (SDOH SAQ) methodology and results.

Response rates have been declining over time, however. Notably, 2020 saw a significant drop in response rate as a result of telephone interviewing due to COVID-19. There was a slight increase for initial requests in 2021, but it remained relatively low. Overall procedures for the distribution and collection of hard-copy materials have not changed with the exception of additional concentrated follow-up. Additional evaluation is underway to understand and attempt to improve the hard-copy collection rates.

Return To Table Of Contents

In Rounds 3 and 5, key adult household members who are reported as having been diagnosed with diabetes were asked to complete a short SAQ, the Diabetes Care Supplement (DCS). Forms not completed for pickup at the time of the interviewer’s visit were followed up by telephone in the latter stages of Rounds 3 and 5, but unlike the SAQ, there was no follow-up in the subsequent round for forms not collected in the round when first requested. Response rates for the DCS for Panels 18 through 25 are shown in Table 4-15. Completion rates for the DCS showed a modest but relatively steady decline over time. 2021 experienced a noticeable drop, both in requests and response rate.

Return To Table Of Contents

4.5 Social Determinants of Health Self-Administered Questionnaire (SDOH SAQ): Methods and Results

There is an increasing policy and research focus in the United States on determinants of health other than use of healthcare services. The Social Determinants of Health (SDOH) study was developed so that AHRQ could create a robust and readily available database. This data would contribute to the larger understanding of trends, and provide health system leaders, policymakers, researchers, and other stakeholders with data and information to improve health care quality and population health outcomes.

Social and behavioral factors play important roles in physical and mental health, though they have not been traditionally taken into account in the health care system. Some provider groups are now collecting information on these social and behavioral factors in order to understand determinants of health and design appropriate interventions. Adding measures of social and behavioral determinants of health to the MEPS will allow researchers to investigate the relationship between these measures and measures of healthcare use and health expenditures in a nationally representative sample of adults. The SDOH SAQ is a short supplemental survey about these social, environmental, and behavioral influences on health.

In developing this SAQ, AHRQ consulted with several experts in the area and used their expertise to identify questions that had already been tested and widely accepted. There was a total of 55 items in the SDOH supplement, all drawn from Federal surveys or from instruments that had been carefully validated. It was estimated that the survey would take 7 minutes to complete.

All cohorts of the MEPS Panel were included in this study�some were in their first round of data collection, whereas others were in later rounds. In addition to the MEPS household respondent, most other adults ages 18+ in the household were eligible to participate.

In spring 2021, the questions in the CAPI instrument for round 1, 3, 5, and 7 interviews identified RU members who were eligible to participate in the SDOH survey. This included most RU members age 18 or older or in age category 4, which includes people with an estimated age of 16 to 23 years old. This selection process resulted over 33,451 individuals across more than 19,300 households being invited to participate in SDOH.

A multimode web and paper approach was primarily selected for the SDOH SAQ to further protect respondents’ privacy due to the sensitive nature of some of the items, especially those about adverse childhood experiences. Web completion was the main mode with paper offered to those with barriers to internet access.

Contact Modes. During the spring 2021 CAPI interview, the respondent was asked to provide contact information for each RU member who was eligible to complete the SDOH SAQ. The CAPI interviewer requested each eligible person’s cell phone number, permission to send short message service (SMS) text messages, email address, first and last name if an alias was provided in the re-enumeration interview section, and the person’s preferred language for communications about the SDOH SAQ (i.e., English or Spanish). Depending on the information provided, we assigned each person to receive their survey invitation and reminders using that mode or combination of modes. Based on the information collected, each eligible person was assigned to one of five contact modes and protocols for receiving information about the SDOH SAQ, as displayed in Figure 4-1.

Figure 4-1. SDOH contact mode determination flowchart

If both an email and permission to text were provided, SDOH SAQ respondents would receive both an email and a SMS text message each time we prompted for survey completion. SDOH SAQ respondents for whom the respondent provided only SMS or email information received electronic invitations and reminders via that one contact mode. If the CAPI respondent did not provide either cell phone or email information, then the respondent was asked if that person had access to the internet. If the respondent indicated the person had the ability to access the internet, that person was assigned to receive a USPS mailing containing the study URL and a unique login. Interviewers provided a paper copy of the SDOH SAQ to the eligible persons whom the household respondent said could not or would not use the internet to complete a survey.

While all other supplements have been administered by paper, this approach was minimized for the SDOH SAQ due to the sensitive nature of some of the questions, such as domestic violence, and the possibility that one household member might see the responses of another member. Dashboard alerts enabled field supervisors and project management staff to monitor the degree to which interviewers were providing paper questionnaires to household members. Supervisors counseled outliers on the standard protocol for presenting paper as a last resort. The goal was to try to direct approximately 80 percent of potential SDOH SAQ respondents to the web. A $20 token of appreciation was offered for each RU member who completed the SDOH survey, regardless of the person’s contact mode or survey completion mode.

Communication Protocol. The SDOH survey communication protocol included several scheduled contacts, which ceased after a selected person submitted a completed web survey or the MEPS receipt staff received a completed paper survey. Figure 4-2 displays the schedule and types of communications for each contact mode.

Figure 4-2. Communication protocol for SDOH contact modes

For persons for whom SMS and email information was available, email was sent in the morning, and SMS in the evening on days 1, 3, and 7. Nonrespondents for the SDOH survey received a postcard reminder with the URL and PIN on day 14, and then if there was still no response, a paper version of the questionnaire was mailed on Day 28. A similar approach was followed if only SMS or only email was provided.

For those who had no email or SMS recorded but had internet access, we mailed a push to web letter on day 1, a postcard reminder on day 7, a reminder letter on day 14, and a final mailing with the paper questionnaire on day 28. If a paper copy of the questionnaire had been sent via FedEx after a telephone interview or left in the household during an in-person interview, a replacement paper questionnaire was sent on day 28.

Reduced staffing at Westat’s offices affected the schedule for mailings related to the survey. The campus was closed except for essential services, so mailings were prepared and sent out 3 days per week rather than 5 days per week. This had a small impact on the number of days between scheduled reminder mailings.

Contact Mode Assignment Results and Spring 2021 SDOH SAQ Response Rates. Figure 4-3 displays the differing response rates broken down by contact mode and interview number. Overall, we were able to collect both a cell phone number (with permission to send SMS messages) and an email for 47 percent of the sample. The longer a household had been participating in MEPS rounds, the more likely the respondent was to provide both pieces of information. Either email only or SMS only were collected for an additional quarter of the sample. There were few differences by Panel round. Only 2 percent of participants had no email or SMS provided, but the household respondent indicated that they had internet access. Finally, 24 percent of the SDOH SAQ sample received only a paper questionnaire. Those in their first round of interviewing were significantly more likely to be assigned to a paper version of the questionnaire than those in their last round. This may have been because they were less familiar with MEPS so the respondents were more reluctant to provide contact information, especially for other household members. Overall, we achieved a 66 percent conditional response rate (using the AAPOR 3 response rate). Response rates increased with each round of the interview, with the SDOH SAQ respondents belonging to the first round less likely to respond than those in their last round.

Figure 4-3. Response rate by contact mode and interview number

* p<.0001 � Participants contacted by SMS+Email, Email only, and SMS only were significantly more likely to complete the SDOH than those only receiving the Paper questionnaire.

** p<.0005 � Participants contacted by Letter w/URL were significantly less likely to complete the SDOH than those only receiving the Paper questionnaire.

A nearly 80 percent response rate was achieved when the CAPI respondent provided a person’s SMS and email information, highest among the Panels in later rounds. Response rates were lower when we had only one method of contact. Email performed better than SMS (66.2% vs. 55.6%), and less than a 40 percent response rate was achieved for the small number of SDOH respondents who were only sent USPS mailings. Finally, for cases where the interviewer either left behind a paper questionnaire or shipped it to the household via FedEx, we achieved only a 45 percent response rate, which is much lower than the usual MEPS SAQ.

Nonrespondent Follow-up During Fall 2021 and Overall Response Rate. Paper-only nonresponse follow-up took place during the fall data collection cycle in July through December 2021. The follow-up phase did not include any additional household members. We followed up with 8,379 people who were eligible to complete SDOH SAQ during the spring data collection but who did not do so.

For each SDOH SAQ nonrespondent, the interviewer either distributed a paper questionnaire during an in-person interview or shipped a paper questionnaire to the household via FedEx after a telephone interview. Respondents were offered a $20 per person, post-collection incentive, just as was done during the spring cycle. During the fall, there were 2,856 completed paper SDOH SAQs, which was a 34 percent completion rate. Table 4-16 displays the total number and percentage of SDOH SAQs completed during the spring versus fall cycles, and the impact of the fall follow-up on the overall response rate for the SDOH supplemental survey.

* Of the 8,379 eligible SDOH SAQs distributed in the fall, the response rate was 34 percent.

Return To Table Of Contents

4.6 Quality Control

Interviewer performance was monitored through validation case review using GPS, CARI, and telephone interviews. The purpose of validation was to verify that the correct individual was contacted for the interview and that the interview was conducted according to MEPS-approved procedures.

Generally, all completed cases were validated by first examining the GPS data stored and encrypted on the laptop. Then, if the case could not be properly validated due to missing data or the GPS information could not be verified to show the interviewer at the respondent address or another documented location at the time of the interview, the case was then reviewed in the CARI system.

However, beginning in mid-March of 2020 and continuing throughout much of 2021, the majority of cases were completed by telephone due to the COVID-19 pandemic. Therefore, GPS data could not be relied on as heavily for validation and CARI review became the main mode of validation in 2021. If a case could not be validated in CARI due to poor quality or missing CARI data, the case was referred for telephone validation. All interviews completed in less than 30 minutes were referred for telephone validation.

In the spring 2021 rounds, 18,037 cases (93 percent of the completed cases) were validated: 84.1 percent of the cases were validated using CARI and only 2.6 percent of cases were validated with GPS due to minimal in-person interviews. In spring 2021,. a minimum of 41 percent of an interviewer’s completes were validated, with an average of 75 percent of each interviewer’s completes validated. In the fall 2021 rounds, 16,144 cases (98.3 percent of the completed cases) were validated: 82.2 percent of cases were validated by CARI and telephone and 15.8 percent were validated with GPS due to increased in-person activity. Staff validated an average of 89 percent of an interviewer’s completes.

In addition to validating cases, MEPS field supervisors and managers typically conduct observations as part of a comprehensive mentoring process. Generally, MEPS uses technical solutions in place of in-person observations; however, there are specific needs met by specialized observation. As much as possible, observations are conducted in the early weeks of data collection so that problems can be detected and corrected as quickly as possible and interviewers are given feedback on ways to improve specific interviewing skills. While CARI offers a high-quality portal for evaluating interviewers on question administration, observations are still a critical tool, particularly of newly hired staff. Compared with the observation process, CARI and other report mechanisms do not allow for assessment of the full range of interviewer skills, including respondent contact, trip planning, gaining cooperation, and interviewer-respondent interactions. In addition, the observer serves as an on-site resource in situations where remedial training is necessary. Observation forms are processed and reviewed at the home office to determine the need for individual and field-wide follow-up on specific skills. In 2021, in-person observations were suspended due to the COVID-19 pandemic. For quality control, additional CARI related to data quality and follow-up with field interviewers regarding COVID-19 specific procedures were effective alternatives.

Return To Table Of Contents

4.7 Security Incidents

To comply with the requirement of reporting incidents involving loss or theft of hard-copy materials with respondent personally identifiable information (PII) or laptops, field staff continued to use an automated loss reporting system (ILRS) to report incidents. Incidents were investigated, updates were sent to AHRQ and MEPS staff who received the initial automated ILRS notification, and results were recorded in an annual MEPS PII Log. A security incident report was submitted to the Westat IRB for each confirmed incident.

A total of 25 incidents of lost or stolen laptops/iPhones or hard-copy PII were reported in 2021. Of those reported incidents, two involved MEPS laptops and iPhones that were reported stolen or lost. In one case, the laptop and iPhone were eventually returned by the interviewer; in the other case, the Portland, Oregon, Police Department never recovered the laptop and iPhone that were stolen from a rental car when a MEPS interviewer was on a production travel trip. The password-protected laptops were shut down at the time of the loss. Since MEPS laptops are full disc encrypted, respondent identity was not at risk.

Twenty-three incidents reported suspected or confirmed loss of hard-copy materials with respondent PII loss or breach of confidentiality. Thirteen of the 23 reported hard-copy losses were located: hard-copy documents in nine incidents were intact and uncompromised; in the other four incidents, the hard-copy documents were intact but there had been a breach of confidentiality. Following extensive searches, no documents were recovered in the other 10 reported losses. Included among the PII hard-copy losses were AFs, SAQs, Preventive Care Self-Administered Questionnaires (PSAQs), Diabetes Care Supplements, and SDOH SAQs. All households with PII loss were notified. The AHRQ Information Security Manager alerted the HHS Privacy Incident Response Team (PIRT) of all MEPS-reported PII incidents.

Return To Table Of Contents

5. Home Office Support of Field Activities

The home office supports the data collection effort in several important ways. One phase of activity supports the launch of each new round of data collection; another phase supports the field operation while data collection is in progress. These two phases of activity are described in this chapter.

Return To Table Of Contents

5.1 Preparation for Field Activities

Hard-copy materials were assembled prior to data collection for cases being fielded in Rounds 2 through 8. Clerical staff created an RU folder for each case being fielded and inserted any authorization forms and SAQs that were printed for the case. There are no hard-copy case materials generated for Round 1 cases so RU folders were not created prior to data collection for Round 1 cases.

Supervisors received a Supervisor Assignment Log listing all of the cases to be released in their region for each wave of cases. For the first wave of each round, supervisors used this log to assign cases to their interviewers. They entered the ID of the interviewer to be assigned each case and sent the log back to the home office. Home office staff then shipped the RU folders directly to the interviewers for the first wave and to regional clerks to distribute to the FIs in later waves. A file with the assignments was also sent to programming staff to make the electronic assignments in the BFOS field management system.

For later waves, the prepared RU folders were sent to the field supervisors, who made the electronic assignments in their Supervisor Management System (SMS) and shipped the hard-copy materials to their interviewers.

Prior to the start of data collection for each period, interviewers connected remotely to the home office to download the CAPI software update for the upcoming rounds and received a home study training package to prepare them for interviewing. Field interviewers also received a replenishment of supplies at the start of the rounds.

Advance mailings to all respondent households were prepared and mailed by the home office staff. Addresses were first standardized and sent through the National Change of Address (NCOA) database to obtain the most current addresses for mailing. Any mail returned as undeliverable was recorded and the appropriate supervisor was notified. Requests to re-mail the Round 1 advance package to households who reported not receiving it were prepared and mailed by home office staff.

Return To Table Of Contents

5.2 Support During Data Collection

Respondent Contacts. Respondent contacts are an important component of home office support for the MEPS data collection effort. Printed materials mailed to respondents contain an email address and toll-free telephone number that respondents can use to contact the project with questions, with requests to make or to cancel interview appointments, or to decline participation in the study. Home office staff received and initiated the response to all respondent contacts. They forward information received from respondent calls to the field supervisors, who initiated the appropriate follow-up and informed the home office of the results of their follow-up within 24 hours of notification. Table 5-1 shows the number and percent of RUs who made calls to the respondent hotline in the spring and fall rounds of 2017�2021. There was a significantly higher percentage of calls to the hotline in both spring and fall 2020. In spring 2021, the percentage of calls to the hotline was more in line with years prior to 2020. The percentage of calls dropped in fall 2021 compared to fall 2020 but was still higher than in previous years.

Return To Table Of Contents

Table 5-2 shows the number and types of calls received on the respondent hotline during 2020 and 2021. As in prior years, a substantial portion of the Round 1 calls was from refusals, with a much smaller proportion of refusals and a higher proportion of appointment requests in the later rounds.

Return To Table Of Contents

Monitoring Production. Home office staff monitored production, cost, and data quality, and they provided reports and feedback to field managers and supervisors for review and follow-up. Each week they generated and distributed reports to AHRQ showing weekly and cumulative figures on field production, response rate, and costs.

Home Office Support. Refusal letters were generated and mailed by home office staff as requested by the field. Home office staff also responded to supply requests from the field, replenishing interviewer and supervisor stocks of materials as needed.

Receipt Control. As interviewers completed cases, they transmitted the data electronically and shipped the case folders containing any hard-copy documents to the home office receipt operation. Interviewers shipped all material containing PII via Federal Express, which facilitates tracking of late or lost shipments. When preparing a shipment to the home office receipt department, interviewers used the Ship to Receipt module to indicate exactly what materials were included in the package and recorded the FedEx tracking number. This information was sent directly to the receipt control system so it was known what materials were expected. For interviews completed by phone due to the COVID-19 pandemic and for which contactless pickup of hard-copy documents could not be arranged, interviewers provided a BRE for the respondent to send their documents directly to the home office. Contents of the cases received at the home office were reviewed and recorded in the receipt system. Authorization forms were edited for completeness and scanned into an image database. When a problem was found in an authorization form, the problem was documented and feedback was sent to the field supervisor to review with the interviewer. All self-administered questionnaires, including SAQs/PSAQs, and DCSs, were receipted and sent out for TeleForm scanning.

Helpdesk Support. The MEPS CAPI Helpdesk continued to provide technical support for field interviewing activities during 2021. Helpdesk staff were available 7 days a week to help field staff resolve CAPI, Field Management System, transmission, laptop, and iPhone problems. Incoming calls were documented for follow-up as needed to resolve individual issues and to identify issues reported by multiple interviewers. The CAPI Helpdesk served as the coordinating point for tracking and shipping all field laptops, monitoring field laptop assignment, and coordinating laptop and phone repairs.

Return To Table Of Contents

6. Data Processing and Data Delivery

This chapter briefly describes the activities that supported Westat’s data delivery work during the year and identifies the principal files related to data year 2018 delivered in 2021.

Return To Table Of Contents

6.1 Processing to Support Data Delivery

6.1.1 Schedules for Data Delivery

Adhering to the schedule for delivery of the key MEPS public use files is of paramount importance to the project. Throughout 2021, data processing activities to support the major file deliveries for the year proceeded simultaneously along several different delivery paths, with activity focused separately on each of the Panels for the annual Full Year Files. As in past years, the project used a set of comprehensive data delivery schedules to guide management of the effort. The schedules integrate key dates for the data collection, data capture, coding, editing and imputation, weights construction, and documentation production tasks. These schedules provide a framework for assessing the potential impact of proposed changes at the start of each processing cycle and for coordinating the succession of processes that comprise the delivery effort.

Return To Table Of Contents

6.1.2 Data Quality Control System

The data quality control (DQC) system consists of both a consolidated database that preserves data as returned from the field, and a DQC-specific database that shows the current values of data following any required updates. DQC technicians access the data through a secure portal.

Technicians review and edit the data using the Blaise database model that is used in the field for data collection. All DQC work occurs at a “case” level. The DQC system automatically creates a unique “issue” for each instance of text entered as a comment and includes the comment category selected by the field interviewer associated with the text entry. As cases are loaded into DQC, each comment and category is checked by a Natural Language Processing (NLP) algorithm that identifies the most likely category. During processing, data technicians have the opportunity to accept or update this category. Technicians then follow standardized procedures for data review and editing based on the comment category.

The DQC system also runs a series of programmatic checks and assigns a new “issue” for each instance that triggers a consistency or edit check. These checks are designed to ensure that data changed during editing conform fully to the rules of the CAPI instrument before the data are released. During spring 2021, 14.2 percent of cases received from the field included a comment (Table 6-1). Cases with any issue, a field comment, or a consistency check totaled 29.9 percent. For fall 2021, 12.7 percent of cases received from the field included a comment while cases with any issue totaled 25.7 percent.

Return To Table Of Contents

Return To Table Of Contents

Field interviewers must select one of 10 categories for each comment text string; after selecting a category, CAPI provides category-specific guidance on information to include in the comment (e.g., RU member name, event date, etc.). They receive training to help identify the most meaningful category and avoid over-use of the category “Other.” Table 6-2 shows the number of comments made in each category as assigned by the NLP algorithm and confirmed by the data technicians.

Return To Table Of Contents

6.1.3 Transformation

Transformation is the process of extracting data from the Blaise data models optimized for data collection and writing them to the data exchange format (Dex) required by the data delivery teams. The transformation has two logical activities: First is transforming the structure of the data from data collection to Dex and then transforming the format of the data from Blaise to Oracle. The resulting data, now stored in Oracle using the Dex structure, serves as input to the analytic editing, variable construction, public use files (PUFs), and other file deliveries. The goal is to dislocate the delivery activities as little as possible in order to provide data of the highest quality as efficiently as possible.

As shown in Figure 6-1, data transformation has four distinct layers. The metadata layer contains all the variable definitions�including names, tables, or segments or blocks�and transformation logic, sometimes known as plain-language transformation specifications. The analytic group leads at Westat are typically responsible for the metadata and the transformation logic.

Return To Table Of Contents

Figure 6-1. Blaise to Dex transformation

Based on the metadata, two specifications are developed. The first describes the Dex structure using a formal schema, which is expressed as a set of SQL statements to create the empty Oracle Dex database. The second specification is the detailed transformation specification. Each variable is assigned to a set of similar variables called a transformation class. A unique transformation class is defined by the information needed to specify the transformation. For instance, some variables simply need to be copied to an appropriate location in the Dex. These are known as passthrough variables and belong to the Passthrough class. Code All That Apply variables are transformed based on the value selected by the interviewer, so the specification requires an additional Dex variable for each possible value. Code All That Apply is another transformation class. All of the classes are developed through discussions with AHRQ and are sent to AHRQ for approval.

The third layer is the transformation (or programming) layer. Using the specifications just described, the data are read from the Blaise database in the data collection structure, the transformation logic is applied, and a data file for each Dex table is written. The Dex tables are generally identical to the legacy Cheshire segments, such as BASE, HOME, or PERS. This set of intermediate data files is known as pre-Dex and has the same structure as the Dex database, but all files are in the Blaise format. Next, the format is transformed from the Blaise format to Oracle, writing to the Single- Round Database (SRD). The single-round structure is necessary because the data collection instrument does not contain all data for all rounds for a given case; rather, only the data required to field the case in that specific round are included. The SRD data are then merged into the existing data, yielding a cumulative Multi-Round Database (MRD).

The final layer relates the different databases to selected key deliverables. This layer is intentionally general. For example, while the MRD is the source for the PUF deliveries, there are many additional steps to edit the data, construct variables, and deliver a data file and codebook.

Return To Table Of Contents

6.1.4 TeleForm/Data Editing of Scanned Forms

TeleForm, a commercial off-the-shelf (COTS) software system for intelligent data capture and image processing, was used in 2021 to capture data collected in the DCS and the SAQ. TeleForm software reads the form image files and extracts data according to the project specifications. Supporting software checks the data for conformity with project specifications and flags data values that violate the validation rules for review and resolution.

As SAQs evolve to be multimode (web and paper), we will update this section to discuss data harmonization and web data collection.

Return To Table Of Contents

6.1.5 Coding

Coding refers to the process of converting data items collected in text format to pre-specified numeric codes. The plan for the 2021 coding effort (for items collected during the calendar years 2019 and early 2020) was described in Deliverables 20.506, .507, and .508. For the MEPS-HC, five types of information require coding:

• Medical conditions;
  
  
• Prescribed medicines;
  
  
• Source of payment for medical events and prescriptions;
  
  
• Industry/Occupation; and
  
  
• Geographic identifiers.

Condition and Prescribed Medicine Coding

In 2021, coding was performed on the conditions and prescribed medicine text strings reported by household respondents for calendar year 2020. An automated system enabled coders to easily search for and assign the appropriate ICD-10-CM code (for conditions) or Generic Product Identifier (GPI) code (for medicines). The system supports the verifier’s review of all codes and, as needed, correction of the coder’s initial decision. For the prescribed medicine coding, a pharmacist provided a further review of text strings questioned by the verifier, uncodable text strings, foreign medicines, and compound drugs. All coding actions are tracked in the system and error rates calculated weekly. Both the condition and prescribed medicine coding efforts were staffed by three coders.

During the 2021 coding cycle, coding managers continued to refine a number of new and revised procedures and processes implemented for the coding of 2018 data in 2019. These revisions were a result of many months of collaboration between AHRQ and Westat in evaluating all aspects of the coding processes for household reported conditions, prescribed medicines, and sources of payment, including updating and maintaining the authority tables and the development of tools and resource documents to facilitate the execution of these tasks. Also in 2019, Westat deployed a new web-based coding system for condition and prescribed medicine coding to replace the Access database previously used. The new system better supports downstream-processing activities and aligns with other web-based systems used across other components of MEPS. All aspects of coding work are supported by a number of scheduled quality control checks before, during, and after each coding cycle.

In 2021, medical conditions were coded to include the greatest specificity indicated by the text string. The fully specified ICD-10 code is needed to accurately match to the CCS. A total of 2,750 unique strings were manually coded and the authority table was constructed with AHRQ-approved code assignments. This represented a 73 percent reduction in the number of strings needing manual review due to the implementation of a condition pick list and search tool integrated into the CAPI instrument. The overall error rate for coders was 1 percent, below the contractual error rate goal of 2 percent.

Prescription medicine text strings for data year 2021 were coded to the set of GPI codes, associated with the Master Drug Data Base (MDDB) maintained by Medi-Span, a part of Wolters Kluwer. The codes characterize medicines by therapeutic class, form, and dosage. To augment the assignment of codes to less specified and ambiguous text strings, AHRQ developed procedures for assigning partial GPI codes and higher-level drug categories that were implemented in 2017 and continued through subsequent coding cycles. AHRQ also developed a set of exact and inexact matching programs to reduce the number of prescribed medicine strings sent for manual coding. Westat’s implementation of these matching programs reduces the number of prescribed medicine text strings sent for manual coding by approximately 40 percent each year. The matching programs are reviewed and approved each year. A total of 9,413 strings were manually coded from 2021 data. In a process similar to condition text strings, the prescription medicine text strings undergo two rounds of unduplication to identify the unique strings to be coded. AHRQ’s exact and inexact matching programs are then run to further reduce the number of strings to be coded. The overall coding error rate (across all coders) was 1 percent, 1 percent lower than the contractual goal of 2 percent. As with conditions, all prescription text strings/codes were reviewed by a verifier, with additional review of selected strings provided by a pharmacist.

Source of Payment Coding

Source of payment information (SOP) is collected in both the household and the medical provider components. In the HC charge payment section of the CAPI instrument, the names of the sources of payment are collected in three places: when the bill was paid by a source identified in response to a direct question about payment (REIMNAM); when the bill was sent to a source other than the respondent and the respondent names that source (WHOBILL#); and in response to a question about a direct payment source for prescription medicines (SRCNAME). The responses are coded to one of the sources of payment options in which healthcare expenditures are reported in the MEPS PUFs. These payment sources include:

• Out of pocket;
  
  
• Medicare;
  
  
• Medicaid;
  
  
• Private health insurance;
  
  
• Veterans Administration, including CHAMPVA;
  
  
• Tricare or CHAMPUS;
  
  
• Other Federal;
  
  
• Other state and local;
  
  
• Workers’ compensation;
  
  
• Indian Health Service;
  
  
• Contractual allowance; and
  
  
• Charity or free care.

The SOP Coding Guidelines is a manual updated each year before the start of the annual coding cycle, submitted for AHRQ approval, and distributed to the coders. Health insurance show cards and data from the health insurance planfill file for CAPI are available to coders as resource materials. Since the Medical Provider Component (MPC) of MEPS uses the same set of source of payment codes as the Household Component, coding rules and decisions are coordinated with the MPC contractor (RTI) to ensure consistency in the coding. Before the start of the coding cycle, Westat compares RTI’s authority tables with its own to identify any inconsistencies. AHRQ adjudicates these to ensure the authority tables from each contractor are aligned.

Each year, the source of payment text strings extracted from the reference year data is matched to a historical file of previously coded SOP text strings to create a file of matched strings with suggested or “matched” codes. These match-coded strings are reviewed by coders and verified or modified as needed. This review is required because insurance companies change their product lines and coverage offerings very frequently, and as a result, the source of payment code for a given text string (e.g., the name of an insurance company or plan) can change from year to year. For example, from one year to the next an insurer or insurance product may participate in or drop out of state exchanges; may offer Medicare Part D or dental or vision insurance or may drop it; may add Medicare Advantage plans in addition to Medicaid HMOs; or may gain or lose state contracts as Medicaid service providers. As a result of these changes, the appropriate code for a company or specific plan may also change from year to year. Strings that do not match to a string in the history table are researched and have an appropriate SOP code assigned by coding staff.

SOP coding during 2021 was for the payment sources reported for 2020 events. For cases when the bill was paid by a source identified in response to a direct question about payment (REIMNAM), a total of 756 previously coded sources of payment text strings were reviewed and updated as needed. After unduplication of the strings reported for 2020, coders reviewed and coded 2,064 strings. If the bill was sent to a source other than the respondent and the respondent names that source (WHOBILL#), coders reviewed and coded 3,145 strings. For text strings reported as direct payers for prescription medicine (SRCNAME), 506 new text strings were reviewed and coded by coders.

Industry and Occupation Coding

Industry and Occupation coding is performed for MEPS by the Census Bureau using the Census Bureau’s Demographic Surveys Division’s (DSD’s) computer-assisted industry and occupation (I&O) codes, which can be cross-walked to the 2007 North American Industrial Classification (NAIC) coding system, and the 2010 Standard Occupational Classifications (SOC). The codes characterize the jobs reported by household respondents and are released annually on the FY JOBS file. During 2021, 12,756 jobs were coded for the 2020 JOBS file.

During the 2021 coding cycle, AHRQ again expanded the scope of work to include coding data year 2020 text strings to multiple versions of the NAICS and SOCS; specifically, the data runs included 2007 NAICS and 2000 SOCS, 2012 NAICS and 2010 SOCS, and 2017 NAICS and 2018 SOCS.

Additionally, AHRQ requested that Westat have the 17,605 text strings for data year 2009 coded to:

• 2007 NAICS and 2010 SOCS
  
  
• 2017 NAICS and 2018 SOCS

This was a one-time request.

GEO Coding

The Westat Geographic Information Systems (GIS) division GEO-codes household addresses, assigning the latitude and longitude coordinates, as well as other variables such as county and state Federal Information Processing Standards (FIPS) codes, Metropolitan Statistical Area (MSA) status, Designated Market Area, Census Place, and county. RU-level data are expanded to the person level and delivered to AHRQ as part of the set of “Master Files” sent yearly. These data are not included in a PUF, but some variables are used for the FY weights processing.

During the calendar year 2021 coding cycle, 20,258 unique address records for full-year reporting units were processed.

Return To Table Of Contents

6.2 Data Delivery

The primary objective of MEPS is to produce a series of data files for public release each calendar year. The inter-round processing, editing, and variable construction tasks all serve to prepare these PUFs. Each file addresses one or more aspects of the U.S. civilian non-institutional population’s access to, use of, and payments for healthcare.

The Oracle system has a separate database for each data year. This is a recent departure from having individual databases for each Panel/year combination. The goal of this is to make data processing more streamlined, and this was necessitated by the extension Panels 23 and 24 to collect data through nine rounds.

Due to the pandemic, Panels 23 and 24 are being extended through Round 9. The MEPS 2020 database contains Panels 23 through 25, and the MEPS 2021 database contains Panels 23 through 26. The remainder of this section focuses on the 2021 database.

After the data are in the Oracle delivery database, each analytical team performs basic edit checks on the data to begin the process. These edits ensure the data conform to the CAPI instrument’s flow as well as to AHRQ’s analytical needs. These edits can be run in SAS, using SAS datasets extracted from the delivery database, or in SQL directly on the delivery database. Problems identified through the basic edits process may require updates to the data. If updating is required, these updates may be accomplished in one of two ways:

1. Programmatic updates can correct problems affecting a large volume of cases that fail a basic edit.
   
   
2. Manual updates can be set up with audit trails maintained to correct data anomalies.

Once all the edits have been completed for an analytical team, and QC frequencies and univariates have been approved, notification is sent to all other analytical teams so that work can be coordinated in those areas.

Return To Table Of Contents

6.2.1 Variable Construction

Analytical groups at AHRQ work with Westat analysts to define the variables of interest for inclusion on the PUF and other key data deliveries. Variables are named according to standard naming conventions, and once the list is approved, descriptive specifications are written to define each variable and to provide detailed information for programming.

Specifications are written at two levels. The high-level specification is a descriptive specification intended to document the concept of the variable and provide high-level information regarding the variable construction requirements. The detailed-level specifications contain the details required to develop programming code for building the variables. Specifications are written and sent to AHRQ for approval. Once approval is received for the specification, program development can proceed for that variable.

Specifications guide programming development, and once programs have been written, code reviews compare newly developed code against specifications to identify problems in either code or specifications. This program development process includes a number of steps and checkpoints to ensure that all new programs meet all specification requirements:

1. Review approved high- and detailed-level specifications.
   
   
2. Write programs for each specification using SAS or SQL.
   
   
3. Test all programmed code for accuracy.
   
   
4. Conduct detailed code reviews to review specifications and code.
   
   
5. Test code on SAS production files or Oracle database without committing.
   
   
6. Construct variables either in SAS (and either load variables to Oracle or continue development in SAS, depending on the file) or directly in the Oracle production database.
   
   
7. Review frequencies and cross-tabulations for accuracy.

This model is followed for the development of all new programs required for data delivery. For mature programs that are re-used in subsequent deliveries with only minor modifications, the process is appropriately streamlined to ensure both accuracy and efficiency on all programs.

Return To Table Of Contents

6.2.2 File Deliveries

Public Use File Deliveries

The principal files delivered during calendar year 2021 are listed below:

• Full Year 2019 Use and Insurance File;
  
  
• Full Year 2019 Use and Expenditure File;
  
  
• Full Year 2019 Expenditure Event Files for events included in the MPC data collection including hospital inpatient, outpatient, and emergency room events, office-based physician visits, and home health agency events;
  
  
• Full Year 2019 Expenditure Event files for events not included in the MPC data collection, including dental events, office-based non-physician events, and other medical expenses;
  
  
• Full Year 2019 Prescribed Medicines Expenditure File;
  
  
• Full Year 2019 Medical Conditions File;
  
  
• Full Year 2019 Jobs File;
  
  
• Full Year 2019 Appendix to MEPS Event Files; and
  
  
• 2019 Person Round Plan File.

Ancillary File Deliveries

In addition to the principal data files delivered for public release each year, the project also produces a number of ancillary files for delivery to AHRQ. These include an extensive series of person- and family-level weights, “raw” data files reflecting MEPS data at intermediate stages of capture and editing, and files generated at the end of each round or as needed to support analysis of both substantive and methodological topics. A comprehensive list of the files delivered during 2021 appears in the Appendix.

Medical Provider Component (MPC) Files

During each year’s processing cycle, Westat also creates files for the MPC contractor and, in turn, receives data files back from the MPC. As in prior years, Westat provided sample files for the MPC in three waves, with the first two waves delivered while HC data collection was still in progress. In preparing the sample files to be delivered in 2021 for MPC collection of data about 2020 health events, Westat again applied the program developed in 2014 for de-duplicating the sample of providers. This process, developed in consultation with AHRQ, was designed to reduce the number of duplicate providers reported from the household data collection.

Early in 2021, following completion of MPC data collection and processing for 2020 events, Westat received the files containing data collected in the MPC with linkages to matching events collected in the MPC with events collected in the HC. In processing at Westat, matched events from the MPC served as the primary source for imputing expenditure variables for the 2019 events. A similar file of prescribed medicines was also delivered to support matching and imputation of expenditures for the prescribed medicines at AHRQ. Timely and well-coordinated data handoffs between Westat and the MPC are critical to the timely delivery of the full year expenditure files. With each additional year of interaction and cooperation, the handoffs between the MPC and HC have gone more and more smoothly.

Return To Table Of Contents

Appendix A
Comprehensive Tables – Household Survey

* RU-level sample size for this table derived from field management system counts and operational reports detailing fielded sample.

Return To Table Of Contents

* Figures in the table are weighted to reflect results of the interim nonresponse subsampling procedure implemented in the first round of Panel 16.

Return To Table Of Contents

Return To Table Of Contents

Return To Table Of Contents

Note: Figures shown are based on original NHIS sample and exclude reporting units added to the sample as “splits” and “students.”

Return To Table Of Contents

Return To Table Of Contents

Return To Table Of Contents

Return To Table Of Contents

Return To Table Of Contents

Return To Table Of Contents