Correlation Analysis - Correlations can be utilized to test the relationship between two survey items. The strength of the correlation, or relationship, is given by the correlation coefficient which values can range from –1.00 to +1.00. A correlation coefficient of 0 implies there is no relationship between the variables. As the correlation coefficient increases, so does the strength of the relationship between the two variables.

Database – a database file is a collection of confidential member-level information provided by the client for a particular TMG project. This data usually consists of a member’s name, address, phone number, plan type, age, gender, and additional variables for use in data analysis for the final report. A random, stratified, or other type of draw is performed on the database, and the members in the final sample are contacted for survey completion (attempts can be made by mail, phone, or Internet).

Demographic Categoriesare the grouping of respondents by age, gender, race, etc. The Myers Group will often collapse several of the respondent characteristic variables into fewer segments than those defined by the survey. The consolidation of these categories with small samples allows for more valid between-group statistical comparisons.

Disposition (Disposition Category) is the final status assignment given to a respondent within the sample. The category signifies both the survey administration protocol used to complete the survey (M=mail, and T=phone) and the status of the record (M10= mail complete, T22= phone, language barrier). All record code assignments of “10” are considered valid responses.

Global Proportions are a graphical presentation of the percentage of members who responded to each response choice, organized by composite category and the attributes contained within each.

HEDIS® (Health Plan Employer Data and Information Set) is the most widely used set of standardized performance measures designed to ensure that purchasers and consumers have the information they need to reliably compare the performance of managed healthcare plans. It is part of an integrated system to establish accountability in managed care across the nation. The performance measures in HEDIS are related to many significant public health issues such as cancer, heart disease, smoking, asthma and diabetes. HEDIS also includes a standardized survey of consumers' experiences that evaluates plan performance in areas such as customer service, access to care and claims processing. HEDIS is sponsored, supported and maintained by NCQA.
(http://www.ncqa.org/programs/hedis/index.htm)

NCQA (National Committee for Quality Assurance) is an independent, 501(c)(3) not-for-profit organization committed to assessing, reporting on, and improving the quality of care provided by organizing delivery systems. NCQA is governed by a Board of Directors that includes employers, consumer and labor representatives, health plans, quality experts, regulators, and representatives from the field of organized medicine. (www.ncqa.org)

Question Summaries (a.k.a. Frequency Distributions) are the proportion of respondents that fall into each response category for all questions. For most reports a section entitled Question Summaries is included. This section includes: Question category, Question number, Question verbiage, Valid number of responses, and Response options. Other options that may be included are Summary Rates, Trend data, Benchmark scores, and Significance testing.

Rangeis the percentage point difference between Summary Rate percentages for two or more segment groups within one population sample. The larger the number, the greater the difference in Summary Rates between segment groups for any given item.

Rating Questionsuse an 11-point scale with “0” representing the worst rating and “10” representing the best rating.

Raw Data File (Member-Level Data File) is either an Excel or SPSS data file that includes a project’s data before any statistical analysis has been applied, and does not include any member identifying variables. A data file may include the survey ID, individual ID, disposition code (mail, phone, internet), individual question responses coded numerically, open-ended question text responses, and additional variables such as region, clinic, provider, plan type (HMO, POS, PPO), or disease type. These additional variables are provided by the client in their original database sample, thereby allowing the respondent data to be matched with these variables in the raw data file.

Regression Analysis- Opportunity Analysis can also use Regression Analysis to identify Key Drivers. Regression estimates are measures of the relationship between composite scores and Overall Satisfaction. Regression Analysis takes into consideration all of the interrelationships between attribute/composites when determining the strength of the relationship between attribute/composites and Overall Satisfaction. The numbers reported next to each significant composite are Beta coefficients. The higher the Beta coefficient, the larger the effect the composite has on overall satisfaction.

Response Rateis only calculated for those respondents who were eligible and able to respond. According to NCQA protocol, ineligible members include those who are deceased, do not meet the eligible population criteria, have a language barrier, or are either mentally or physically incapacitated. Non-respondents include those members who have refused to participate in the survey, could not be reached due to a bad address or telephone number, or members that reached a maximum attempt threshold and were unable to be contacted during the survey time period.

Rounding of Numerical and Percentage Data Typically, when percentages are calculated in our report applications, all decimal places are computed, but only the first decimal place is actually shown. As such, adding rounded single-digit decimals may not equal 100%. If the same figures were taken out an additional decimal place, however, they would add to exactly 100%. Through consultation with a number of our clients, The Myers Group has determined that using a single decimal place in the reporting of percentages provides an adequate level of detail. Finally, when rounding, TMG employs the standard practice of rounding down any number from 1 to 4, and rounding up any number from 5 to 9.

Sampling Error can be thought of as the extent to which survey results may differ from what would be obtained if every eligible member in the sample had been surveyed. The size of such error depends largely on the percentage distributions (i.e., the number of respondents selecting each answer category) and the number of members surveyed. The more disproportionate the percentage distributions or the larger the sample size, the smaller the error will be.

The following tables may be used in estimating approximate sampling error. The first table shows the range (plus or minus the figure shown) within which the population percentage could be expected to lay 95 out of 100 times a sample of that size and percentage distribution would be selected. The second table shows the range (plus or minus the figure shown) within which the population percentage could be expected to lay 90 out of 100 times a sample of that size and percentage distribution would be selected.

The sampling error table is used in the following manner. Assume that “overall satisfaction with the health plan” received a Summary Rate Score of 70% from a sample of 500 valid responses. For a 95% confidence interval, look at the first table where the sample size of 500 intersects the percentage distribution of 70/30. The margin of error for this sample size is four percentage points (4%). Therefore, on average, in 95 out of 100 similar samples, the 95% confidence interval (e.g., 66% to 74%) will span the true unknown population percentage.

Statistical Significance is not necessarily related to the amount one score is higher or lower than another score. The number of respondents and the percentage distribution of response options also influence statistical significance. One score is statistically different from another score when the difference between the scores is more than would be expected by sampling error (margin of error). Statistical significance is the likelihood that conclusions resulting from a sample also hold true for the population from which the sample was taken. For example, if the difference between a plan’s overall rating scores for two consecutive years is statistically significant at the .05 level, you can be 95% confident that the difference between the two scores would also be observed if all members were surveyed for both years.

Summary Rate Score represents the percentage of respondents who chose the most favorable response option(s). For example, one question’s Summary Rate Score is computed using the following proportion:

Survey Administration Protocol describes the process in which the data was collected for the survey.

Trend Comparisons show how your health plan’s current year composite, attribute, and rating Summary Rate Scores compare to your scores from previous years.

Z-Test is a statistical inference test used to determine if the difference between two proportions (Summary Rates) is large enough to be statistically significant. Statistical testing is done to draw conclusions about differences in proportions between a sample and a set constant (e.g., a national benchmark) or between different samples (e.g., a Summary Rate for this year versus a Summary Rate for last year). When checking for significant differences between proportions various conditions must be met: The sample must be from a simple random sample and the population from which the sample was drawn must have a normal variance (bell-shaped curve). If it is not known that the population has a normal variance, it suffices to have a sufficiently large sample.