Ten Answers Every Child Welfare Agency Should Provide

A university-child welfare agency partnership between the Factor-Inwentash Faculty of Social Work at the University of Toronto and Highland Shores Children’s Aid (Highland Shores), a child welfare agency in Ontario, allowed for the identification and examination of ten questions to which every child welfare organization should know the answers. Using data primarily from the Ontario Child Abuse and Neglect Data System (OCANDS), members of the partnership were able to answer these key questions about the children and families served by Highland Shores and the services provided to children and families. The Ontario child welfare sector has experienced challenges in utilizing existing data sources to inform practice and policy. The results of this partnership illustrate how administrative data can be used to answer relevant, field-driven questions. Ultimately, the Barbara Fallon1, Mark Kartusch2, Joanne Filippelli3, Nico Trocmé4, Tara Black5, Parlin Chan6, Praveen Sawh7, Nicolette Joh-Carnella8 1 Professor and Chair in Child Welfare, Factor-Inwentash Faculty of Social Work, University of Toronto, Toronto, Ontario, Canada. 2 Executive Director, Catholic Children’s Aid Society of Toronto, Toronto, Ontario, Canada. 3 Research Associate, Factor-Inwentash Faculty of Social Work, University of Toronto, Toronto, Ontario, Canada. 4 Professor and Philip Fisher Chair in Social Work, McGill University, Montreal, Quebec, Canada . 5 Assistant Professor, Factor-Inwentash Faculty of Social Work, University of Toronto, Toronto, Ontario, Canada. 6 Programmer, Ontario Child Abuse and Neglect Data System (OCANDS), Factor-Inwentash Faculty of Social Work, University of Toronto, Toronto, Ontario, Canada. 7 Senior Programmer, Ontario Child Abuse and Neglect Data System (OCANDS), Factor-Inwentash Faculty of Social Work, University of Toronto, Toronto, Ontario, Canada. 8 Research Assistant, Factor-Inwentash Faculty of Social Work, University of Toronto, Toronto, Ontario, Canada. Corresponding author: Barbara Fallon, Professor and Chair in Child Welfare, Factor-Inwentash Faculty of Social Work, University of Toronto, 246 Bloor Street West, Toronto, ON, Canada M6S 3W6, barbara.fallon@utoronto.ca

As in other jurisdictions in Canada, the dual mandate of Ontario's child welfare organizations is to promote the safety and well-being of children served (Trocmé, Kyte, Sinha, & Fallon, 2014). Each day, child welfare workers assess concerns reported, provide in-home services, and in very rare cases, place children out-of-home. Despite decades of significant policy changes in Ontario, including efforts to strengthen accountability to funders, communities, and families (Commission to Promote Sustainable Child Welfare, 2012), there is minimal understanding of child welfare service trajectories and the impact of these services on children and families (Fallon, Filippelli, Black, Trocmé, & Esposito, 2017). This lack of understanding is a significant barrier to accountability, transparency, and responsive practice and policies. This brief report provides ten questions that are informative at an agency level.
The Ontario child welfare sector has experienced numerous challenges in utilizing existing data sources for daily operations management and the evaluation of practice and policies (Fallon et al., 2017). Child welfare organizations typically do not have the resources, research, and analytic capacity to analyze administrative and census data Fallon et al., 2017;Trocmé, Roy, & Esposito, 2016). Notwithstanding these challenges, there is great promise in utilizing existing administrative data to better understand child welfare services and their impact (Drake & Jonson-Reid, 1999;Fallon et al., 2017).
A university-child welfare agency partnership between the Factor-Inwentash Faculty of Social Work at the University of Toronto and Highland Shores Children's Aid (Highland Shores), one of 49 child welfare organizations in Ontario, afforded a unique opportunity to identify ten fundamental questions to which every child welfare organization should know the answer (see Table 1). The final ten answers/questions were established though continual discussion and feedback from the agency, the research team, and the OCANDS programmers. These questions reflect different points along the service continuum, from the initial investigation to out-of-home placement. The answers were derived by combining existing sources of non-identifying, aggregate data from the Ontario Child Abuse and Neglect Data System (OCANDS), the Census, and the Ontario Incidence Study of Reported Child Abuse and Neglect 2013 (OIS-2013) . OCANDS data are case-level administrative data that are extracted from different information systems, mapped and harmonized. OCANDS allows for the construction of entry and exit cohorts. This report illustrates how university-child welfare partnerships are integral to utilizing and harnessing the potential of existing data sources that can increase understanding of services and outcomes for vulnerable children and their families.

From Questions to Answers: Enhancing Understanding and Accountability
Highland Shores Answers Every Child Welfare Agency Should Provide (Table 1) emphasize how formal university-child welfare partnerships are critical to both advancing knowledge and enhancing agency accountability. These ten answers provide basic and timely information about the work of Highland Shores to child welfare professionals, policymakers, and their community. For instance, Highland Shores posted these answers on their organization's website.
The Ontario Child Abuse and Neglect Data System (OCANDS) was primarily used to answer the ten questions. OCANDS is the first data system in Ontario to track child welfareinvolved children and families (Fallon et al., 2017). OCANDS is a child-specific, event-level, longitudinal database that corresponds to the child welfare service continuum. OCANDS data can be used to respond to administrative or practice questions (Fallon et al., 2017). Each participating agency can access information about its service performance on available measures, along with a comparison to provincial norms on OCANDS' web-based reporting tool. The ten answers that originated from this partnership were replicated for other OCANDS participating agencies and can be accessed through OCANDS' dynamic reporting tool.
Each of the ten questions and answers are summarized below.

The Question
The Answer The Methodology Used Why is it Important?  Each year, about 5.2% of children come to the attention of Highland Shores for a concern about their wellbeing or safety that requires assessment.
First, the approximate number of investigated children was calculated by multiplying the number of families investigated by a correction factor of 1.6 (the average number of children investigated per family by child welfare agencies in Ontario in 2013) . The calculated number of investigated children was then divided by the child population and multiplied by 100 to derive the proportion of children assessed.
This is useful to measure community need and agency practice. Comparisons between agencies can illuminate differences in these areas.
3. How many families are assessed for a concern each year?
Each year, approximately 1,228 families are assessed for a concern about their children The total number of investigations closed was divided by the number of fiscal years used in the calculation to get an annual estimate.
Understanding the number of families investigated is helpful to measure the volume of work and calculate other measures, including recurrence.
4. How many families after assessment are provided with ongoing child welfare services each year?
About 640 families (or 2.7% of the child population of Highland Shores' catchment area) are provided with ongoing services after assessment each year.
The number of cases closed at ongoing services was divided by the number of fiscal years included in the calculation to derive an annual estimate. Multiplying this estimate by a correction factor of 1.6 (the average number of children per family investigated by child welfare agencies in Ontario in 2013) , dividing by the child population, and multiplying by 100 gave the proportion of the population provided with ongoing child welfare services.
This shows the volume of families who move beyond investigation to service provision. The agency can make both historical comparisons and comparisons with other agencies. This measure also provides the basis for the calculation of the OCANDSgenerated provincial and publically reported service performance indicator related to recurrence within 12 months following the closure of a case at ongoing child welfare services. 6. How many families return to our agency after receiving ongoing child welfare services?
After receiving ongoing services from Highland Shores, 23% of families return within 12 months.
The number of cases that received ongoing services and then were reopened within 12 months was divided by the total number of cases that received ongoing services in the fiscal year and multiplied by 100.
Knowing the number of cases that reopen after receiving ongoing services sheds light on whether cases were closed prematurely or whether the service provided was effective.

How many families after assessment are provided with ongoing child welfare services each year?
The volume of families who move beyond investigation is an important measure with respect to needs of the population and agency practice. Approximately 640 families (2.7% of the Highland Shores child population) are provided with ongoing services after assessment each year. Differences between agencies in the proportion of cases that remain open beyond the investigative phase could be the result of the needs of the population, agency practice, and/or other factors.

Why do families return to our agency after their investigation file has been closed?
Child welfare organizations in Ontario have identified understanding recurrence (having contact with the child welfare system after file closure) as a key priority (Fallon et al., 2017). According to analyses of OCANDS-generated system metrics, rates of recurrence vary considerably between agencies (Fallon et al., 2016(Fallon et al., , 2017. Higher rates of recurrence have been associated with organizations serving a higher proportion of individuals with lower income, a greater proportion of the Indigenous population, and a greater proportion of lone parent families (Fallon et al., 2016).
In order to better understand recurrence, an urgent-chronic investigative framework was applied to cases closed after an investigation (Fallon et al., 2017). This framework categorizes investigations as urgent protection where a child's safety is the overriding concern, or chronic need, where the focus of concern is on the impact of family dysfunction on child well-being (Trocmé et al., 2014). After receiving investigations, families return to Highland Shores for an urgent need in approximately 3.9% of cases and for a chronic need in 14.2% of cases. This framework was applied to several other child welfare organizations in Ontario through another formal university-child welfare agency partnership (Fallon et al., 2017). Similar to Highland Shores, investigations at these agencies classified as having chronic needs are more likely to return to the attention of child welfare authorities (Fallon et al., 2017).

How many families return to our agency after receiving ongoing child welfare services?
Approximately 23% of families receiving ongoing child welfare services from Highland Shores come back in contact with the organization within 12 months of case closure. This recurrence metric assists agencies in better understanding and determining whether reopened cases were prematurely closed and whether service was effective.
The partnership conducted a file review on a subset of cases to explore the reasons why investigations classified as urgent recurred as urgent. A key finding of the file review was that, although the urgent designation for the investigation was applicable and appropriate, these investigations occurred within the context of chronic family dysfunction and challenges that can threaten child well-being without adequate intervention (Filippelli, Kartusch, Fallon, Trocmé, & Cascone, 2018).

How many children do we place in out-of-home care?
Highland Shores places approximately 3% of all children investigated (less than 1% of the service population). This measure permits the examination of trends over time and provides a metric for comparisons across organizations. This measure importantly underscores that 97% of the families that Highland Shores serves are not placed into outof-home care. Just as the decision to open a case is fraught with complexity, determining whether a child should come into care is arguably the most critical decision made by a child welfare worker.

How long do these children remain in the care of our agency
Within 36 months, 91% of children placed into out-of-home care are discharged. Time to discharge is a proxy for permanency, and understanding the time to discharge for children is important for facilitating discussions about the reasons children remain in care

What is the average number of days that children spend in out-of-home care?
This measure is another proxy for permanency. Using an entry cohort to ensure valid comparisons, the average number of days in care for children discharged within 36 months at Highland Shores is 241. The construction of entry cohorts allow us to track the trajectories of children which can be difficult using administrative data.

Do children placed in care stay in the same placement during their time in care?
About 65% of children stay in the same placement while in care. Approximately 20% of children move once, 6% of children move twice, and 9% of children move three times or more. This measure of placement stability is important as it identifies the proportions of move frequencies, which can assist in identifying factors that influence placement stability.

Implications: Data as the Cornerstone of University-Child Welfare Partnerships and Accountability
The work of this university-child welfare agency partnership presented in this paper illustrates how field-driven, administrative and practice questions can act as catalysts to harness the potential of existing administrative data. The Ten Answers Every Child Welfare Agency Should Provide are an example of how data are critical to aligning child welfare services with identified needs and to facilitating agency accountability, transparency, and responsive practices and policies. Analyses such as the Ten Answers represent an important step in combining and further developing collaborations to strengthen the knowledge mobilization components of University-Child Welfare sector initiatives in order to develop capacity towards sustainability of research expertise in the child welfare sector in Ontario. It also provides opportunities for greater integration of research in child welfare practice and policy.