Training Evaluation

American Humane has extensive experience in developing and conducting training evaluations and providing technical assistance to child welfare agencies in conducting evaluations. The following are some of these projects.

CALSWEC

CALSWEC Interdisciplinary Training Evaluation

This evaluation monitored the effectiveness of training design and delivery, pre- and post- testing to assess the effectiveness of training in promoting new learning, and follow-up to assess transfer of learning to practice and attitude change with regard to interdisciplinary concurrent planning.

Colorado

The Colorado Department of Human Services

The Colorado Department of Human Services contracted with American Humane to conduct a five-year evaluation of child welfare training that included:

• Competency-based needs assessments for caseworkers, case aides, supervisors, county directors, child welfare administrators, and kinship providers to help set training priorities and guide curriculum development.
• Development of a strategic plan for training evaluation with input from county and state staff and training vendors to guide decision making about evaluation of training, including what training projects should be evaluated and at what level.
• Assessment of trainee satisfaction with training and self-assessment of learning in key competency areas addressed in the course.
• Knowledge exams, keyed to course competencies, and developed through a rigorous process of test construction and item analysis.
• Skills-based trainee assessment for Colorado's advanced sexual abuse interviewing training.

Illinois

Illinois Child Endangerment Risk Assessment Protocol

This project involved the development of a safety assessment and planning instrument and protocol, development of a related curriculum, training of trainers, delivery of distance learning training, and development and validation of a test to certify mastery of the safety assessment protocol. Approximately 6,000 public and private contract agency child protection workers received competency-based training and certification testing in response to a legislative mandate. The development of the test involved specification of item content and format, several levels of content review and validation, and the use of the Rasch model for scaling, item analysis, reliability estimation, and construct validity evidence. American Humane also conducted an evaluation of the total safety protocol, which compared the short-term rate of recurrence (confirmed within 60 days of initial report) for the year before implementation of the protocol with the rate two years after implementation.

Northern California

Northern California Training Academy at UC Davis:

This project involved the development of a computerized database to store and retrieve evaluation results efficiently, item analysis of the questionnaire used to measure learning, and analysis of the effectiveness of the core program in addressing the needs of caseworker and supervisor trainees.

Pennsylvania

Pennsylvania Training Evaluation Project

American Humane conducted a comprehensive evaluation of the entire Pennsylvania Child Welfare training program, including the governance, structure, and operation of the program; the processes for developing courses, ensuring quality and relevance of content, and supporting continued trainer competence; and the current and potential capability to evaluate effectiveness of the training delivered by the program. American Humane is currently working with Pennsylvania in their efforts to strengthen their competency-based training program based on American Humane's recommendations in the evaluation report. These efforts include developing a skills-based curriculum for new workers and supervisors, developing sophisticated evaluation methods to measure skill acquisition and transfer of learning, and building an infrastructure to support evaluation and outcome measurement of the training program.

Project CAN DO

American Humane evaluated the effectiveness of a cross-systems training program designed to link mental health aspects of post-legal adoption to services provided by county caseworkers and to increase family involvement in the planning and delivery of services. American Humane also conducted a case study of a successful interdisciplinary collaboration around post-legal adoption services provision. The goal of the case study was to develop a model for interdisciplinary collaboration that could provide the basis for curriculum development and further training.