Methods Notes

Below is a brief description of the methods used in this report. For more detail, please see the Technical Appendix on our website (www.hqontario.ca).

The findings presented in this report are based on analysis of administrative and survey data, as well as on stories from patients and caregivers with lived experience.

Results were obtained from the following data providers: the Institute for Clinical Evaluative Sciences (ICES), the Ontario Ministry of Health and Long-Term Care and Social Science Research Solutions.

The following data sources were used:

• 2013 Commonwealth Fund International Health Policy Survey of the General Public (Commonwealth Fund Survey)
• 2014/15 Health Care Experience Survey (HCES)
• National Ambulatory Care Reporting System (NACRS)
• Registered Persons Database (RPDB)

Significance testing was not performed for administrative data.

Health Care Experience Survey data significance testing

Confidence intervals around each result were calculated at the 95% confidence level. Results from the HCES state an increase/decrease or higher/lower result only when the 95% confidence intervals of the results do not overlap (i.e., when the differences in the results are statistically significant).

Commonwealth Fund survey data significance testing

Social Sciences Research Solutions conducted statistical analyses to compare responses across countries and provinces within Canada. For provincial comparisons, statistical tests were conducted to compare each province’s responses to those of every other province and to Canada as a whole. Ontario’s results were also compared to other countries. Significance was assessed based on a P-value of less than 0.05, meaning that there was less than a 5% probability that the difference was due to chance rather than real differences in respondents’ experiences.

Emergency department (ED) use was based on all unplanned ED visits by Ontarians, as identified in the NACRS.  These visits and the population identified served as the basis for the rest of the indicators analyzed using administrative databases.

Timely access to emergency department care is important to patients, health care providers and the public. Measuring the use of hospital emergency departments and the length of patients’ visits, as well as changes in these measures over time, provides information not only about care within hospitals but also about how well other parts of the health system are working. For these reasons, this report focuses on who accesses emergency departments, how long they wait to see a physician and how long their visits take. Where possible, changes over time are reported.

Key measures of emergency department performance in this report include:

• 90^th percentile emergency department length of stay
• 90^th percentile emergency department wait time to physician initial assessment
• 90^th percentile emergency wait time for inpatient bed for admitted patients

In addition, information about Ontarians’ recent use of emergency departments and their perception of the care they received while there is provided using survey data. Specifically:

• The percentage of adults (aged 16 and older) in Ontario who visited an emergency department because they were sick or had a health-related problem in the previous 12 months (HCES)
• The percentage of adults (aged 16 and older) in Ontario who reported receiving excellent/very good/good or fair/poor medical care in the emergency department (HCES)
• The percentage of adults who reported that the last time they went to a hospital emergency department it was for a condition that they thought could have been treated by the doctors or staff at the place where they usually get medical care if they had been available (Commonwealth Fund Survey)

To better understand how different populations utilize and experience care at emergency departments, utilization and wait times were examined using select stratifications such as age group, sex, rural/urban location of the patient and hospital, neighbourhood income, immigration status, hospital type and patient group.

Time to physician initial assessment, wait time for an inpatient bed for admitted patients and emergency department length of stay are measured in this report at the 90th percentile – the amount of time within which nine out of 10 patients saw a doctor or completed their visit. The 90th percentile indicator was chosen because it represents the maximum wait to see a doctor or length of stay for the vast majority – 90% – of patients.

For the purposes of the emergency department wait time indicators used in this report, patients are divided into three groups according to Canadian Triage and Acuity Scale (CTAS) scores and visit disposition status as found in the NACRS. The three patient groups are as follows:

• “High-acuity discharged” includes patients with CTAS scores of 1, 2 or 3 and visit disposition of 01, 04-05 or 08-15 (discharged, transferred)
• “Low-acuity discharged” includes patients with CTAS scores of 4 or 5 and visit disposition of 01, 04-05 or 08-15 (discharged, transferred)
• “Admitted” includes patients with any CTAS score (CTAS 1-5), and patients missing a CTAS score, who have a visit disposition of 06 (admitted into the reporting facility as an inpatient to critical care unit or operating room directly from the ambulatory care visit functional centre) or 07 (admitted into the reporting facility as an inpatient to another unit of the reporting facility directly from the ambulatory care visit functional centre)

Data from the 2014/15 Health Care Experience Survey were weighted to reflect the design characteristics of the survey and the population of Ontario. Urban/rural status is defined using Statistics Canada’s Statistical Area Classification. Respondents who answered “don’t know” or who “refused” are excluded from results.

Data from the 2013 Commonwealth Fund International Health Policy Survey of the General Public are based on participants 18 years of age and older who were interviewed by telephone (land line or cellphone) between March and June 2013. In Canada, 5,412 respondents were surveyed; the Ontario population was oversampled (1,543 respondents) to enable the calculation of provincial estimates from the survey.

For a full list of data sources and methods as they relate to each indicator, please refer to the Technical Appendix.

There are certain limitations of the analysis that should be considered when interpreting the results. Some of the limitations are specific to the data source, the indicator and the methodology used to calculate it. For further details, please see the individual Indicator Templates in the online Technical Appendix.