Document
Appendix Ii
ICR 201502-0970-010 · OMB 0970-0456 · Object 56537701.
Document [pdf]
Download: pdf | txt
January 27, 2015 Family Connection Cross-site Evaluation Survey [CLUSTER NAME] Project Leader Survey [PROJECT NAME] SURVEY LEAD-IN AND INSTRUCTIONS PAGE Thank you for taking the time to complete this survey about the Family Connection – [CLUSTER NAME] project, also known as the [INSERT PROJECT NAME HERE]. Your input is very valuable to the cross-site evaluation as we work toward understanding and synthesizing data on how grantees implement services and the impact of the services on the organization, community, and children and families. This survey is about the [INSERT PROJECT NAME HERE]. Please answer all questions in regard to your involvement with [INSERT PROJECT NAME HERE]. The “grantee organization” is the recipient of Family Connection Federal funding. The grantee organization is [INSERT GRANTEE ORGANIZATION NAME HERE]. Your survey response is confidential and will not be shared with other members of the project team, project partners, the Children’s Bureau, or other Federal partners. All data collected through this survey will be aggregated for analysis and reporting purposes; information that identifies individual participants will not be used. The survey will take approximately 45 minutes to complete. For your convenience, you can enter and exit the survey as many times as you like to answer the survey items. Your response is not final until you select “Submit” at the end of the survey. To begin the survey now, please click here: CONTINUE Note: ___________ = pre-filled information and/or an action conducted by the web survey system (i.e., no action required by the survey respondent). OMB NO: TBD EXPIRATION DATE: TBD Burden Statement: Public reporting burden for this collection of information is estimated to average .75 hours per respondent. These estimates include the time for reviewing instructions, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Project Leadership 1 January 27, 2015 Evaluation ID: Respondent: Project Role: Organization: __________ ____________________ __________ __________ Family Connection Cross-site Evaluation Survey [CLUSTER NAME] Project Leader Survey A. BACKGROUND The following questions ask about your background with the project and the grantee organization. The “grantee organization” is the recipient of Family Connection Federal funding. The grantee organization is [INSERT GRANTEE ORGANIZATION NAME HERE]. 1. Please verify your primary role on the Family Connection – [CLUSTER NAME] project. (Check one) a. Executive Leaders (e.g., President, CEO) b. Project Director c. Other Project Leaders (e.g., Deputy Project Director, Program or Project Manager, service provider Supervisor) d. Service Provider (e.g., advocate, case manager, case worker, counselor, educator, facilitator, nurse, mentor, therapist, etc.) e. Public Child Welfare Partner representative f. Community Partner representative g. Lead Evaluator/Principal Investigator h. Evaluation Team Member i. Other (specify): _______________________________ 2. How long have you been in this role for the Family Connection project? a. Less than 1 year b. 1+ to 2 years c. 2+ to 3 years 3. How long have you worked for your current organization? a. Less than 1 year b. 1+ to 2 years c. 3+ to 5 years d. 5+ to 10 years e. Over 10 years Project Leadership 2 January 27, 2015 B. PROJECT PLANNING AND READINESS The next few questions ask you to reflect upon the grantee organization’s planning and readiness for implementing the Family Connection project. The “grantee organization” is the recipient of Family Connection Federal funding. The grantee organization is [INSERT GRANTEE ORGANIZATION NAME HERE]. 1. Were you involved in planning the Family Connection project (e.g., writing the grantee’s funding application, planning during the initial start-up period, etc.) a. Yes b. No [If NO, skip to Section C]. 2. What are the top three reasons the grantee organization sought to incorporate Family Connectionfunded services into your practice? a. Model/services already a practice within the organization (e.g., staff members already trained) b. Model/services complement existing practices within organization c. Model/services a good “fit” for the population served (e.g., cultural fit, etc.) d. Model/services a good “fit” for the organization (e.g., matched philosophy) e. Model/services an opportunity to increase the organization’s capacity to serve families f. Benefits of the model/services described in academic or other literature g. Key partner(s) had experience with the model h. Local community assessments indicated need for model/services i. Other (specify): _______________________________ 3. Do you know what other project staff members and partners were engaged in planning the Family Connection project (e.g., writing and/or otherwise contributing to the grantee’s funding application, planning during the initial start-up period, etc.)? a. Yes b. No [If NO, skip to B5] 4. What other project staff members and partners were engaged in planning the Family Connection project? (Check all that apply) a. Executive Leaders (e.g., President, CEO) b. Project Director c. Other Project Leaders (e.g., Deputy Project Director, Program or Project Manager, service provider Supervisor, etc.) d. Service Provider (e.g., advocate, case manager, case worker, counselor, educator, facilitator, nurse, mentor, therapist, etc.) e. Public Child Welfare Partner representative f. Community Partner representative g. Lead Evaluator/Principal Investigator h. Evaluation Team Member(s) i. Prior Family Connection grantee(s) Project Leadership 3 January 27, 2015 j. Other (specify): _______________________________ 5. To what degree was the grantee organization ready for the Family Connection project? Readiness may include, but not be limited to: project leaders and service providers with the appropriate knowledge, skills and abilities to implement services; support from project partners; training, coaching and supervision for project leaders, service providers, and partners; leaders to oversee work; administrative support; space; technology; etc. a. Very ready b. For the most part ready c. Somewhat ready d. Slightly ready e. Not at all ready 6. To what degree was the broader child welfare system (e.g., other public and private child welfare agencies who serve children at risk of or who are removed from the home, and their partner organizations) ready for the Family Connection project? Readiness may include, but not be limited to: knowledge of the funding opportunity, knowledge of the grantee organization’s project, staff members or other resources available to the project, providing a Memorandum of Understanding (MOU) or letter of support to the funding application, etc. a. Very ready b. For the most part ready c. Somewhat ready d. Slightly ready e. Not at all ready [If SLIGHTLY OR NOT AT ALL READY, skip to B8] 7. Please list the top three factors that contributed to the child welfare system’s readiness: Factor 1: _______________________________________________________________________ Factor 2: _______________________________________________________________________ Factor 3: _______________________________________________________________________ 8. Please list the top three factors that prevented the child welfare system from being ready: Factor 1: _______________________________________________________________________ Factor 2: _______________________________________________________________________ Factor 3: _______________________________________________________________________ 9. How has the Family Connection project affected child welfare practice in the community? (Check all that apply) a. Improved support and advocacy for the Family Connection project’s services from individuals b. Improved support and advocacy for the Family Connection project’s services from organizations c. Improved service planning for families in the child welfare system d. Filled service gaps in the system e. Other (specify): ___________________________________________ Project Leadership 4 January 27, 2015 C. STAFFING PATTERNS, SELECTION, TRAINING, AND COACHING The following questions address staffing patterns, selection, training and coaching, and supervision of staff members working on the Family Connection project. 1. What was the initial staffing strategy for program services? (Check all that apply) a. Draw from existing staff members to assume Family Connection tasks/deliver services b. Hire new staff members to manage and/or deliver Family Connection services c. Contract with external organizations to deliver Family Connection services d. Other (specify): ______________________________________________ e. Do not know [If DO NOT KNOW, skip to C4]. 2. How easy or challenging was it to fully staff the project within the 90-day planning period? Fully staff may include shifting responsibilities for existing staff members, hiring new staff members, or contracting with external organizations. a. Very challenging b. Challenging c. Neither challenging nor easy d. Easy e. Very easy f. Do not know [If NEITHER CHALLENGING NOR EASY, EASY, VERY EASY, OR DO NOT KNOW, skip to C4]. 3. To what extent did challenges with initial staffing delay project implementation? a. Greatly delayed project implementation b. Delayed project implementation c. Somewhat delayed project implementation d. Only slightly delayed project implementation e. Did not delay project implementation f. Do not know 4. Has the staffing plan for Family Connection services been modified since funding was received? a. Yes b. No c. Do not know [If NO or DO NOT KNOW, skip to C6]. 5. How have staffing plans for Family Connection services been modified? (Check all that apply) a. Increased number of program staff members b. Decreased number of program staff members c. Modified program staffing to reflect service population d. Changed how program positions were funded e. Modified roles and responsibilities of program staff members f. Increased number of evaluation staff members Project Leadership 5 January 27, 2015 g. Decreased number of evaluation staff members h. Other (specify): ______________________________________________ 6. Has the Family Connection project experienced turnover in staff members from the grantee organization (including project leaders and service providers) during the grant period? a. Yes, there has been substantial turnover (50% or more) b. Yes, there has been moderate turnover (25% to 49%) c. Yes, there has been minimal turnover (less than 25%) d. No, there has been no turnover e. Do not know [If NO or DO NOT KNOW, skip to C8]. 7. What are the top three reasons for staff member turnover? a. Advancement within organization b. Reassignment within organization c. Left organization for employment elsewhere d. Left organization or was terminated due to performance issues e. Left organization due to retirement f. Left organization for other reasons (e.g., health issues, childcare, eldercare, moved out of area, etc.) g. Other (specify): ________________________________________ __________ 8. To what extent has staff member turnover impacted project implementation? g. Greatly impacted project implementation h. Impacted project implementation i. Somewhat impacted project implementation j. Only slightly impacted project implementation k. Did not impact project implementation [If DID NOT IMPACT, skip to C10]. 9. What are the top three ways that staff member turnover has impacted implementation? a. Additional time needed to train new staff members b. Additional refresher training for existing staff members c. Decreased or inconsistent quality of service provision d. Decreased client referrals e. Decreased client retention f. Lost knowledge from departing staff members g. Negative impact on evaluation data collection timeliness/quality h. Raised concerns among staff members regarding the ability to implement the project i. Staff members overloaded j. Other (specify): ______________________________________________ 10. Who receives training on Family Connection project services? This may include general project orientation, training on the overall service model, or training on specific components and services of the service model. (Check all that apply) a. Executive Leaders (e.g., President, CEO) Project Leadership 6 January 27, 2015 b. Project Director c. Other Project Leaders (e.g., Deputy Project Director, Program or Project Manager, service provider Supervisor, etc.) d. Service Provider (e.g., advocate, case manager, case worker, counselor, educator, facilitator, nurse, mentor, therapist, etc.) e. Public Child Welfare Partner representative f. Community Partner representative g. Lead Evaluator/Principal Investigator h. Evaluation Team Member(s) i. Other (specify): ______________________________________________ j. Do not know [IF DO NOT KNOW, skip to C13]. 11. What training is provided to develop knowledge of the Family Connection project’s service model? (Check all that apply) a. General orientation to the project (e.g., project goals and objectives) b. Training on the project’s service model c. Supplemental topics related to direct services d. Other (specify): ______________________________________________ 12. What training is provided to develop application of skills related to the Family Connection project’s service model? (Check all that apply) a. Presentations or trainings by in-house agency staff members b. Presentations or trainings by external trainers/experts c. Role play d. Service observation e. Other (specify): ______________________________________________ 13. Have service providers received coaching or supervision on the Family Connection project’s service model? a. Yes, at least weekly b. Yes, at least monthly c. Yes, at least quarterly d. No coaching or supervision received 14. What top three experience, skills, and personal characteristics are required to be a successful Family Connection project team member? a. Ability to collaborate b. Ability to engage individual clients and/or families in services c. Ability to work in a team environment d. Effective communication skills e. Experience working with families f. Interpersonal skills g. Leadership skills h. Passion for serving families i. Positive role model (for staff members, families) j. Persistence Project Leadership 7 January 27, 2015 k. Knowledge of resources (in the community and within the organization) l. Other (specify): _________________________________________________ __________ 15. Who has received training regarding the evaluation component of the Family Connection project? (Check all that apply) a. Executive Leaders (e.g., President, CEO) b. Project Director c. Other Project Leaders (e.g., Deputy Project Director, Program or Project Manager, service provider Supervisor, etc.) d. Service Provider (e.g., advocate, case manager, case worker, counselor, educator, facilitator, nurse, mentor, therapist, etc.) e. Public Child Welfare Partner representative f. Community Partner representative g. Lead Evaluator/Principal Investigator h. Evaluation Team Member(s) i. Other (specify): ______________________________________________ D. DESCRIPTION OF PARENTS, CHILDREN, AND FAMILIES / TAILORING SERVICES The following questions ask about the families and children served by the Family Connection project. 1. Has the project served the number of children and families the organization expected to serve? a. Exceeding service expectations (i.e., serving more children and families than anticipated) b. Meeting service expectations (i.e., on track with the number of children and families served) c. Partially meeting service expectations (i.e., behind projections for number of children and families served) d. Do not know 2. How easy or challenging has it been for the project to engage clients in services? This may include, but not be limited to initially engaging clients via referrals and recruitment, and keeping clients engaged in services. g. Very challenging h. Challenging i. Neither challenging nor easy j. Easy k. Very easy l. Do not know [If NEITHER CHALLENGING NOR EASY, EASY, VERY EASY, OR DO NOT KNOW, skip to D4]. 3. What are the top three factors that have made it difficult to engage clients in services? a. Basic needs of individual client and/or family (e.g., food, housing) take priority over other project services b. Eligibility criteria (e.g., changes in eligibility criteria) c. Client resistance to receiving services d. Client resistance to other family member involvement in services / treatment e. Drug dependency/substance abuse of client and/or family members Project Leadership 8 January 27, 2015 f. g. h. i. j. k. l. m. n. o. p. Heath issues of clients – specifically aging caregivers Heath issues of clients – other health issues Lack of transitional housing Lack of support system (e.g., friends, family in area) Lack of transportation to services Language barriers (e.g., primary language not read or spoken by service providers) Low literacy level of clients and/or family members Mental health issues of client Referral process (e.g., changes in process, challenges of process) Transient/highly mobile population Other (specify): __________________________________________ 4. Has the project modified its service population for the Family Connection project from the population described in the Federal funding application? This may or may not have involved a request to submit a change in scope to the Children’s Bureau. a. Yes b. No c. Do not know [If NO or DO NOT KNOW, skip to D6]. 5. Why did the project modify its service population (e.g., eligibility criteria, expanding or centralizing geographic reach, etc.)? __________________________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ 6. Since the beginning of the project, has Family Connection service delivery changed to meet cultural or other needs of the service population? a. Yes b. No c. Do not know If NO or DO NOT KNOW, skip to Section E. 7. What are the top three ways the project has changed Family Connection service delivery? a. Provided expanded service hours b. Provided services in child or family home c. Provided services in different languages d. Provided transportation to families e. Included practices in service delivery that are responsive to culture and/or other unique population characteristics (e.g., substance users, remote rural locations, ethnicity) f. Altered written materials to address literacy levels g. Spent time addressing basic needs h. Spent time locating families for services i. Other (specify): ___________________________________________ Project Leadership 9 January 27, 2015 E. PROJECT SERVICE MODEL / IMPLEMENTING KEY SERVICES These questions ask about how children and families are referred or selected to receive Family Connection services, the flow of services they receive, and implementation of Family Connection and other key services. 1. Has the eligibility criteria for services changed since the beginning of the Family Connection project? a. Yes b. No c. Do not know [If NO or DO NOT KNOW, skip to E3]. 2. If Yes, please explain: __________ 3. Has the referral process for services changed since the beginning of the Family Connection project? a. Yes b. No c. Do not know [If NO or DO NOT KNOW, skip to E5]. 4. If Yes, please explain: __________ [GRANTEE-SPECIFIC ITEMS FOR THIS SECTION BEGIN HERE] Note: Include drop down lists of services for each grantee from their evaluation semi-annual report that apply to adult, child, and family services. 5. What top three services offered through the Family Connection project have been hardest to implement? 6. What top three services offered through the Family Connection project have been easiest to implement? Adult-Level Services a. Grantee-specific service #1 b. Grantee-specific service #2 Child-Level Services c. Grantee-specific service #3 d. Grantee-specific service #4 Family-Level Services e. Grantee-specific service #5 f. Grantee-specific service #6 Project Leadership 10 January 27, 2015 5. What are the top three strategies the project has used to engage adults, children, and families in services? a. Address basic needs and provide resources b. Assist with family finding c. Be flexible in planning meetings and other activities d. Develop relationship with family/establish rapport and trust e. Empower the family (e.g., balance family and service provider participation; allow family to lead the process) f. Explain process to families/ prepare family for services g. Explain role of facilitator / coordinator h. Focus on the unique needs of each family i. Include children in the process j. Treat families with respect k. Other (specify): ______________________________________ 6. In what percent of cases do domestic violence issues surface in service delivery? Note: Familyfinding FGDM cluster-specific question. a. 76% to 100% b. 51% to 75% c. 26% to 50% d. 0% to 25% e. Do not know [If NO or DO NOT KNOW, skip to Section F]. 7. What are the top three strategies the project has used to address domestic violence in service delivery? Note: Family-finding FGDM cluster-specific question. a. Address safety concerns and issues during FGDM meeting b. Collaborate with partners to address domestic violence issues/ refer to other services as needed c. Ensure support person is present for victim d. Ensure a safe environment in the meeting/follow “no contact” order e. Make alternative arrangement to include input from both sides f. Provide domestic violence advocacy g. Provide domestic violence information/materials to families about h. Provide staff training in domestic violence i. Screen family for domestic violence issues pre-meeting j. Other (specify): ______________________________________________ 8. What are the top three essential ingredients or requirements for an effective FGDM meeting? Note: Family-finding FGDM cluster-specific question. a. Buy-in from meeting participants b. Clear expectations for service providers c. Clear meeting purpose d. Effective coordinator/facilitator e. Effective communication during meeting f. Family-driven process g. Family members at the meeting h. Preparing for meeting Project Leadership 11 January 27, 2015 i. j. k. l. Private family time Safe environment Family activities, goals, and timelines developed during the meeting Other (specify): ______________________________________________ F. LEADERSHIP The following questions ask about project leaders’ involvement in the Family Connection project. Leaders include Executive Leaders (e.g., President, CEO), the Project Director, and other project leaders (e.g., Deputy Project Director, Program or Project Manager, service provider Supervisor) involved in the Family Connection project. 1. Besides yourself, how many leaders who are involved in the Family Connection project support the project? a. All leaders are supportive (100%) b. Most leaders are supportive (about 75%) c. Some leaders are supportive (about 50%) d. A few leaders are supportive (about 25%) e. No other leaders are supportive (0%) 2. What are the top three activities leaders, including yourself, have engaged in to support the Family Connection project? a. Advocate/act as spokesperson for the project b. Assist/work with staff members to implement the project c. Involve executive leaders (e.g., Board of Directors) d. Prioritize training on project-related services and models e. Share information with staff members regarding project implementation and outcomes f. Support the evaluation g. Other (specify): ______________________________________________ 3. Is there a process for regular communication regarding successes and concerns about the Family Connection project among leaders, service providers, and the evaluation team? a. Yes b. No c. Do not know [If NO or DO NOT KNOW, skip to Section G] 4. What are the top three communication processes regarding the Family Connection project? a. Regularly scheduled project staff meetings b. Regularly scheduled project leadership meetings c. Written updates from leaders to service providers d. Written updates from evaluators to leaders e. Written updates from evaluators to service providers f. Other __________ Project Leadership 12 January 27, 2015 G. COLLABORATION These next few questions ask about collaborative relationships with project partners. 1. Have there been any changes in project partners (e.g., public child welfare agency, external service provider, community organization, evaluation team, etc.) and their roles on the Family Connection project? a. Yes b. No c. Do not know [If NO or DO NOT KNOW, skip to G3]. 2. If YES, what change(s) have occurred? (Check all that apply) a. Added one or more partners b. No longer working with one or more partners c. Partner(s) assumed additional responsibilities/tasks on the project d. Partner(s) reduced responsibilities/tasks on the project e. Other (specify): ___________________________________________ 3. Have proposed relationships with partners worked out as originally planned? a. Yes b. No [If YES, skip to G5]. 4. If No, why not? (Check all that apply) a. Partner(s) did not fulfill roles and responsibilities b. Lack of partner support for project services and activities c. Collaboration mechanisms were ineffective or not used d. Poor communication e. Other (specify): ___________________________________________ 5. How frequently does the Family Connection project share successes and concerns with project partners? a. At least daily b. At least weekly c. At least monthly d. At least quarterly e. At least semi-annually f. At least annually g. Never h. Do not know 6. How frequently do project partners share concerns, issues and feedback with the Family Connection project? a. At least weekly b. At least monthly Project Leadership 13 January 27, 2015 c. d. e. f. g. At least quarterly At least semi-annually At least annually Never Do not know 7. What are the top three communication mechanisms between the grantee organization and project partners? a. Advisory board b. Case-specific meetings/conference calls c. Community meetings d. E-mail memos and updates e. General project meetings/conference calls – Regularly scheduled f. General project meetings/conference calls – As needed g. Reports (e.g., progress reports, evaluation findings, etc.) h. Surveys i. Informal channels j. Other (specify): ___________________________________________ 8. Has feedback from project partners led to changes in project services, processes, evaluation activities, etc.? a. Yes b. No [If NO, skip to G10]. 9. Please describe the feedback and the change it facilitated: Feedback Example Resulting Change 1. 2. 3. 10. What are the top three benefits the project has experienced through its work with partners? a. Addressed service gaps/ expands services able to provide b. Created a team mentality c. Expanded knowledge base d. Facilitated data sharing e. Improved ability to collect evaluation data f. Improved ability to achieve desired adult, child and family-level outcomes g. Improved ability to address systemic issues/conflicts h. Improved ability to collect evaluation data i. Improved service coordination/facilitates linking families to services j. Increased family engagement in services k. Increased sense of support among families Project Leadership 14 January 27, 2015 l. m. n. o. p. q. r. Increased understanding of project services by partners Increased understanding of partners by project Shared risk and responsibility among partners Source of referrals Strengthened collaboration Created sustainability opportunities Other (specify): ___________________________________________ 11. What are the top three challenges the project has experienced while working with partners? a. Change/loss of project partner b. Change in partner leadership c. Changes required in service approaches/processes/procedures d. Differing communication styles e. High staff turnover in partner agency f. Lack of partner support g. Limited partner staff availability h. Substantial staff training and orientation requirements i. Other (specify): ___________________________________________ 12. What are the top three ways the Family Connection project has impacted the grantee organization’s relationship with project partners? a. Expanded awareness of the grantee organization’s own strengths and weaknesses per project partner feedback b. Improved reputation of grantee organization among project partners c. Increased awareness of project services among project partners d. Increased contacts between grantee organization and project partners e. Increased/improved case coordination f. Increased understanding of child welfare system by project partners g. Other (specify): ___________________________________________ 13. What are the top three strategies for developing and sustaining a successful partnership? a. Co-locate staff members from grantee organization and/or project partners b. Engage in ongoing, open communication (including one-on-one personal contact) e. Enter formal agreement (e.g., Memorandum of Understanding, contract/subcontract) d. Identify common goals e. Share information about organizations f. Share information about project (e.g., activities, findings, results) g. Ensure that partners understand roles and responsibilities on the project (e.g., partner trainings) h. Other (specify): ___________________________________________ H. QUALITY ASSURANCE The following questions address quality assurance processes for the Family Connection project. 1. Beyond fidelity measures that assess specific services, practices, or the service model, what other strategies has the grantee organization developed for continuous quality improvement in the Family Connection project? (Check all that apply) Project Leadership 15 January 27, 2015 a. b. c. d. e. f. Training for grantee organization staff members Training for project partners, including service providers employed by partners Technical assistance resources provided (e.g., print materials, websites, blogs, etc.) Individual guidance (e.g., mentoring, shadowing) for service providers Other (specify): ___________________________________________ Do not know [If DO NOT KNOW, skip to H3]. 2. To what extent have these strategies been effective? a. Very effective b. Effective c. Somewhat effective d. Minimally effective e. Not at all effective f. Do not know 3. Have new policies and procedures been developed in the grantee organization as a result of the Family Connection project? a. Yes b. No c. Do not know [If NO or DO NOT KNOW, skip to Section I]. 4. What new policies and procedures were developed? (Check all that apply) a. New service delivery policies/practices created b. Existing service delivery policies/practices modified c. New job descriptions created d. Existing job descriptions modified e. New staff member performance criteria created f. Existing staff member performance criteria modified g. New staff training and technical assistance practices or materials created h. Existing staff training and technical assistance practices or materials modified i. Original project eligibility criteria modified j. Original project recruitment/enrollment practices modified k. Original project referral practices modified l. Other (specify): ___________________________________________ 5. Please provide a brief description of the policy/procedural changes that were made. __________________________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ Project Leadership 16 January 27, 2015 I. LOCAL EVALUATION The following questions ask about your role in the local evaluation of the Family Connection project, and how evaluation results are shared with project team members and partners. 1. Do you have a role in the local evaluation (e.g., evaluation planning and design, report review and feedback)? a. Yes b. No [If NO, skip to I3]. 2. What is your role in the local evaluation? (Check all that apply) a. Evaluation planning and design b. Instrument selection and design c. Collect data from clients and/or families d. Provide own data from evaluation activities (e.g., training or collaboration surveys) e. Data entry / processing f. Data analysis g. Evaluation report review and feedback h. Other (specify): ___________________________________________ 3. Are process and/or outcome evaluation information (e.g., progress, results) shared with others involved in the Family Connection project (e.g., project leaders, service providers, public child welfare partner, community partner, etc.)? a. Yes b. No c. Do not know d. Have not completed analysis yet [If NO, DO NOT KNOW, or HAVE NOT COMPLETED ANALYSIS, skip to I6]. 4. How are process and/or outcome evaluation results shared? (Check all that apply) a. Written reports, memos, or other documents b. In-person presentations (during meetings) c. Virtual presentations (via webinars) d. Other (specify): ___________________________________________ e. Do not know 5. How often are process and/or outcome evaluation results information shared? (Check all that apply) a. Weekly b. Monthly c. Quarterly d. Semi-annually e. Annually f. Other (specify): ___________________________________________ g. Do not know Project Leadership 17 January 27, 2015 6. How frequently are process and/or outcome evaluation results used to guide decisions about the Family Connection project? a. Always b. Frequently c. Occasionally d. Rarely e. Never f. Do not know [If NEVER or DO NOT KNOW, skip to SECTION J]. 7. How have process and/or outcome evaluation results been used by the Family Connection project? Please provide an example of how process and/or outcome evaluation results were used in relation to the following aspects of the project: a. Service population (e.g., understanding service needs, strategies for recruitment or retention, etc.): ______________________________________________________________________________ b. Service delivery (e.g., type of service, method of providing services, frequency of providing services, duration of one or multiple sessions of service provision, etc.): ______________________________________________________________________________ c. Service provider (e.g., efficacy of provider-client match, adequacy of staff member training or supervision, etc.): ______________________________________________________________________________ d. Outcomes (e.g., effectiveness of instruments and methods to measure outcomes, expectations to achieve outcomes, etc.): ______________________________________________________________________________ e. Other (e.g., dissemination to community, support for funding opportunities): ______________________________________________________________________________ J. SUSTAINABILITY The next few questions focus on project sustainability. 1. Is the grantee organization planning to sustain Family Connection-funded services? a. Yes, we are planning to sustain all services b. Yes, we are planning to sustain most services c. Yes, we are planning to sustain a few services d. No, we are not planning to sustain services e. Do not know [If NO or DO NOT KNOW, skip to Section K]. Project Leadership 18 January 27, 2015 Note: Services listed in the following question are placeholders. The actual survey will include dropdown lists of services for each grantee from their evaluation semi-annual report that apply to adult, child, and family services. 2. At this time, what are the top three services offered through the Family Connection project that are priorities to sustain? Adult-Level Services a. Grantee-specific service #1 b. Grantee-specific service #2 Child-Level Services c. Grantee-specific service #3 d. Grantee-specific service #4 Family-Level Services e. Grantee-specific service #5 f. Grantee-specific service #6 2. How is the grantee organization planning to sustain services? (Check all that apply) a. Current department will continue to cover costs b. Another department within the organization will provide funding c. Another organization will provide funding d. Grantee organization will integrate sustainable practices into current work e. Grantee organization will implement policy to enforce current practices/services f. Other (specify): ___________________________________________ 3. To what degree has the cost study been helpful in sustainability planning? a. Very helpful b. Helpful c. Somewhat helpful d. Minimally helpful e. Not at all helpful 4. How has the cost study assisted in sustainability planning? Example #1: _______________________________________________________________________ Example #2: _______________________________________________________________________ Example #3: _______________________________________________________________________ L. FACILITATORS, BARRIERS, AND LESSONS LEARNED This section asks about facilitators and barriers to project implementation, and lessons learned to pass on to grantees implementing similar projects. Facilitators are people, policies and procedures, actions, circumstances, and other factors that help something run more smoothly and effectively Project Leadership 19 January 27, 2015 and/or bring about an outcome. Barriers make processes and the achievement of outcomes difficult or impossible. 1. What have been the top three facilitators of project implementation? a. Celebratory events b. Communication c. Developmental approach taken d. Evaluation plan e. Fidelity to services f. Project leadership support g. Staff member capacity (enough staff members) to implement project h. Staff member selection i. Staff member training (initial and ongoing) j. Support from public child welfare partner k. Support from community partners l. Sustainability incorporated into planning and implementation m. Other (specify): ___________________________________________ 2. What have been the top three challenges to project implementation? a. Initial training insufficient for service provision b. Insufficient number of referrals/difficulty reaching targeted enrollment level c. Lack of or insufficient incentives for clients to engage in services (e.g., monetary, transportation) d. Lack of support from child welfare partner e. Lack of support from community partners f. Need for more physical space g. Refinements to service model and/or individual services h. Processes and procedures are not clear to staff members and/or partners i. Processes and procedures are not followed by staff members and/or partners j. Staff member recruitment k. Staff member retention l. Time needed for staff members to understand the model and provide services m. Too many cases n. Start-up delays o. Other (specify): ___________________________________________ 3. What top three strategies were used to overcome implementation challenges? a. Improved initial training (e.g., content, type, frequency) b. Improved ongoing training (e.g., content, type, frequency) c. Increased availability of resource materials d. Improved referral system e. Strengthened communication with partners f. Co-located staff g. Other (specify): ___________________________________________ 4. What advice would you give someone implementing a similar project about how to achieve positive outcomes – your “lessons learned”? Project Leadership 20 January 27, 2015 a. In regard to serving adults, children, and families? ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ b. In regard to evaluating project services? ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ c. In regard to collaborating with partners? ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ 5. Is there anything else you would like to add? __________________________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ Thank you for your survey response! Project Leadership 21 January 27, 2015 Family Connection Cross-site Evaluation Survey [CLUSTER NAME] Service Provider Survey [PROJECT NAME] SURVEY LEAD-IN AND INSTRUCTIONS PAGE Thank you for taking the time to complete this survey about the Family Connection – [CLUSTER NAME] project, also known as the [INSERT PROJECT NAME HERE]. Your input is very valuable to the cross-site evaluation as we work toward understanding and synthesizing data on how grantees implement services and the impact of the services on the organization, community, and children and families. This survey is about the [INSERT PROJECT NAME HERE]. Please answer all questions in regard to your involvement with [INSERT PROJECT NAME HERE]. The “grantee organization” is the recipient of Family Connection Federal funding. The grantee organization is [INSERT GRANTEE ORGANIZATION NAME HERE]. Your survey response is confidential and will not be shared with other members of the project team, project partners, the Children’s Bureau, or other Federal partners. All data collected through this survey will be aggregated for analysis and reporting purposes ; information that identifies individual participants will not be used. The survey will take approximately 30 minutes to complete. For your convenience, you can enter and exit the survey as many times as you like to answer the survey items. Your response is not final until you select “Submit” at the end of the survey. To begin the survey now, please click here: CONTINUE Note: ___________ = pre-filled information and/or an action conducted by the web survey system (i.e., no action required by the survey respondent). OMB NO: TBD EXPIRATION DATE: TBD Burden Statement: Public reporting burden for this collection of information is estimated to average .5 hours per respondent. These estimates include the time for reviewing instructions, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Service Provider 1 January 27, 2015 Evaluation ID: __________ Respondent: ____________________ Project Role: __________ Organization: __________ Family Connection Cross-site Evaluation Survey [CLUSTER NAME] Service Provider Survey A. BACKGROUND The following questions ask about your background with the project and the grantee organization. The “grantee organization” is the recipient of Family Connection Federal funding. The grantee organization is [INSERT GRANTEE ORGANIZATION NAME HERE]. 1. Please verify your primary role on the Family Connection – [CLUSTER NAME] project. (Check one) a. Executive Leaders (e.g., President, CEO) b. Project Director c. Other Project Leaders (e.g., Deputy Project Director, Program or Project Manager, service provider Supervisor) d. Service Provider (e.g., advocate, case manager, case worker, counselor, educator, facilitator, nurse, mentor, therapist, etc.) e. Public Child Welfare Partner representative f. Community Partner representative g. Lead Evaluator/Principal Investigator h. Evaluation Team Member i. Other (specify): _______________________________ 2. How long have you been in this role for the Family Connection project? a. Less than 1 year b. 1+ to 2 years c. 2+ to 3 years 3. How long have you worked for your current organization? a. Less than 1 year b. 1+ to 2 years c. 3+ to 5 years d. 5+ to 10 years e. Over 10 years B. STAFFING PATTERNS, SELECTION, TRAINING, AND COACHING The following questions address staffing patterns, selection, training and coaching, and supervision of staff members working on the Family Connection project. Service Provider 2 January 27, 2015 1. What top three experiences, skills, and personal characteristics are required to be a successful Family Connection project service provider? a. Ability to collaborate b. Ability to engage individual clients and/or families in services c. Ability to work in a team environment d. Effective communication skills e. Experience working with families f. Interpersonal skills g. Leadership skills h. Passion for serving families i. Positive role model (for staff members, families) j. Persistence k. Knowledge of resources (in the community and within the organization) l. Other (specify)_____________________________________________ 2. What training have you received to develop knowledge of the Family Connection project’s service model? (Check all that apply) a. General orientation to the project (e.g., project goals and objectives) b. Training on the project’s service model c. Supplemental topics related to direct services d. Other (specify): ______________________________________________ 3. What training have you received to apply skills with clients related to the Family Connection project’s service model? (Check all that apply) a. Presentations or trainings by in-house agency staff members b. Presentations or trainings by external trainers/experts c. Role play d. Service observation e. Other (specify): ______________________________________________ 4. Have you received coaching or supervision on the Family Connection project’s service model? a. Yes, at least weekly b. Yes, at least monthly c. Yes, at least quarterly d. No coaching or supervision received 5. Are your roles and responsibilities regarding the Family Connection project clear to you? a. Very clear b. Clear c. Somewhat clear d. Only slightly clear e. Not at all clear 6. How easy or challenging was it to fully staff the project within the 90-day planning period? Fully staff may include shifting responsibilities for existing staff members, hiring new staff members, or contracting with external organizations. a. Very challenging b. Challenging Service Provider 3 January 27, 2015 c. d. e. f. Neither challenging nor easy Easy Very easy Do not know [If NEITHER CHALLENGING NOR EASY, EASY, VERY EASY, OR DO NOT KNOW, skip to B8]. 7. To what extent did challenges with initial staffing delay project implementation? a. Greatly delayed project implementation b. Delayed project implementation c. Somewhat delayed project implementation d. Only slightly delayed project implementation e. Did not delay project implementation f. Do not know 8. Has the Family Connection project experienced turnover in staff members from the grantee organization (including project leaders and service providers) during the grant period? a. Yes, there has been substantial turnover (50% or more) b. Yes, there has been moderate turnover (25% to 49%) c. Yes, there has been minimal turnover (less than 25%) d. No, there has been no turnover e. Do not know If NO or DO NOT KNOW, skip to Section C]. 9. What are the top three reasons for staff member turnover? a. Advancement within organization b. Reassignment within organization c. Left organization for employment elsewhere d. Left organization or was terminated due to performance issues e. Left organization due to retirement f. Left organization for other reasons (e.g., health issues, childcare, eldercare, moved out of area, etc.) g. Other (specify): __________________________________________________ 10. To what extent has staff member turnover impacted project implementation? g. Greatly impacted project implementation h. Impacted project implementation i. Somewhat impacted project implementation j. Only slightly impacted project implementation k. Did not impact project implementation [If NOT AT ALL, skip to Section C]. 11. What are the top three ways that staff member turnover has impacted implementation? a. Additional time needed to train new staff members b. Additional refresher training for existing staff members c. Decreased or inconsistent quality of service provision Service Provider 4 January 27, 2015 d. e. f. g. h. i. j. Decreased client referrals Decreased client retention Lost knowledge from departing staff members Negative impact on evaluation data collection timeliness/quality Raised concerns among staff members regarding the ability to implement the project Staff members overloaded Other (specify): ______________________________________________ C. DESCRIPTION OF PARENTS, CHILDREN, AND FAMILIES / TAILORING SERVICES The following questions ask about the families and children served by the Family Connection project. 1. How easy or challenging has it been for the project to engage clients in services? This may include, but not be limited to initially engaging clients via referrals and recruitment, and keeping clients engaged in services. g. Very challenging h. Challenging i. Neither challenging nor easy j. Easy k. Very easy l. Do not know [If NEITHER CHALLENGING NOR EASY, EASY, VERY EASY, OR DO NOT KNOW, skip to C3]. 2. What are the top three factors that have made it difficult to engage clients in services? a. Basic needs of individual client and/or family (e.g., food, housing) take priority over other project services b. Eligibility criteria (e.g., changes in eligibility criteria) c. Client resistance to receiving services d. Client resistance to other family member involvement in services / treatment e. Drug dependency/substance abuse of client and/or family members f. Heath issues of clients – specifically aging caregivers g. Heath issues of clients – other health issues h. Lack of transitional housing i. Lack of support system (e.g., friends, family in area) j. Lack of transportation to services k. Language barriers (e.g., primary language not read or spoken by service providers) l. Low literacy level of clients and/or family members m. Mental health issues of client n. Referral process (e.g., changes in process, challenges of process) o. Transient/highly mobile population p. Other (specify): __________________________________________ 3. Since the beginning of the project, has Family Connection service delivery changed to meet cultural or other needs of the service population? a. Yes b. No Service Provider 5 January 27, 2015 c. Do not know [If NO or DO NOT KNOW, skip to Section D]. 4. What are the top three ways the project has changed Family Connection service delivery? a. Provided expanded service hours b. Provided services in child or family home c. Provided services in different languages d. Provided transportation to families e. Included practices in service delivery that are responsive to culture and/or other unique population characteristics (e.g., substance users, remote rural locations, ethnicity) f. Altered written materials to address literacy levels g. Spent time addressing basic needs h. Spent time locating families for services i. Other (specify): ___________________________________________ D. PROJECT SERVICE MODEL / IMPLEMENTING KEY SERVICES These questions ask about how children and families are referred or selected to receive Family Connection services, the flow of services they receive, and implementation of Family Connection and other key services. 1. Has the eligibility criteria for services changed since the beginning of the Family Connection project? a. Yes b. No c. Do not know [If NO or DO NOT KNOW, skip to D3]. 2. If Yes, please explain: ___________________________________________ 3. Has the referral process for services changed since the beginning of the Family Connection project? a. Yes b. No c. Do not know [If NO or DO NOT KNOW, skip to C5]. 4. If Yes, please explain: ___________________________________________ Note: Services listed in the following question are placeholders. The actual survey will include dropdown lists of services for each grantee from their evaluation semi-annual report that apply to adult, child, and family services. Service Provider 6 January 27, 2015 5. What top three services provided through the Family Connection project have been hardest to implement? 6. What top three services provided through the Family Connection project have been easiest to implement? Adult-Level Services a. Grantee-specific service #1 b. Grantee-specific service #2 Child-Level Services c. Grantee-specific service #3 d. Grantee-specific service #4 Family-Level Services e. Grantee-specific service #5 f. Grantee-specific service #6 5. What top three strategies has the project used to engage adults, children, and families in services? a. Address basic needs and provide resources b. Assist with family finding c. Be flexible in planning meetings and other activities d. Develop relationship with family/establish rapport and trust e. Empower the family (e.g., balance family and service provider participation; allow family to lead the process) f. Explain process to families/prepare family for service g. Explain role of facilitator/coordinator h. Focus on the unique needs of each family i. Include children in the process j. Treat families with respect k. Other (specify): ______________________________________ 6. In what percent of cases do domestic violence issues surface in service delivery? Note: Familyfinding FGDM cluster-specific question. Will not be asked of Kinship Navigator-TANF or Residential Family Treatment clusters. a. 76% to 100% b. 51% to 75% c. 26% to 50% d. 0% to 25% e. Do not know [If NO or DO NOT KNOW, skip to D8]. 7. What are the top three strategies the project has used to address domestic violence in service delivery? Note: Family-finding FGDM cluster-specific question. Will not be asked of Kinship Navigator-TANF or Residential Family Treatment clusters. a. Address safety concerns and issues during FGDM meeting Service Provider 7 January 27, 2015 b. c. d. e. f. g. h. i. j. Collaborate with partners to address domestic violence issues/ refer to other services as needed Ensure support person is present for victim Ensure a safe environment in the meeting/follow “no contact” order Make alternative arrangement to include input from both sides Provide domestic violence advocacy Provide domestic violence information/materials to families about Provide staff training in domestic violence Screen family for domestic violence issues pre-meeting Other (specify): ______________________________________________ 8. What are the top three essential ingredients or requirements for an effective FGDM meeting? Note: Family-finding FGDM cluster-specific question. Will not be asked of Kinship Navigator-TANF or Residential Family Treatment clusters. a. Buy-in from meeting participants b. Clear expectations for service providers c. Clear meeting purpose d. Effective coordinator/facilitator e. Effective communication during meeting f. Family-driven process g. Family members at the meeting h. Preparing for meeting i. Private family time j. Safe environment k. Family activities, goals, and timelines developed during the meeting l. Other (specify): ______________________________________________ E. LEADERSHIP The following questions ask about leaders’ involvement in the Family Connection project from the grantee organization. Leaders include Executive Leaders (e.g., President, CEO), the Project Director, and other project leaders (e.g., Deputy Project Director, Program or Project Manager, service provider Supervisor) involved in the Family Connection project. 1. How many leaders who are involved in the Family Connection project support the project? a. All leaders are supportive (100%) b. Most leaders are supportive (about 75%) c. Some leaders are supportive (about 50%) d. A few leaders are supportive (about 25%) e. No leaders are supportive (0%) [If NO LEADERS ARE SUPPORTIVE, skip to E3]. 2. What are the top three activities leaders have engaged in to support the Family Connection project? a. Advocate/act as spokesperson for the project b. Assist/work with staff members to implement the project c. Involve executive leaders (e.g., Board of Directors) d. Prioritize training on project-related services and models Service Provider 8 January 27, 2015 e. Share information with staff members regarding project implementation and outcomes f. Support the evaluation g. Other (specify): ______________________________________________ 3. Is there a process for regular communication regarding successes and concerns about the Family Connection project among leaders, service providers, and the evaluation team? a. Yes b. No c. Do not know [If NO or DO NOT KNOW, skip to Section F]. 4. What are the top three communication processes regarding the Family Connection project? a. Regularly scheduled project staff meetings b. Regularly scheduled project leadership meetings c. Written updates from leaders to service providers d. Written updates from evaluators to leaders e. Written updates from evaluators to service providers f. Other __________ F. COLLABORATION These next few questions ask about collaborative relationships with project partners. 1. Have there been any changes in project partners (e.g., public child welfare agency, external service provider, community organization, evaluation team, etc.) and their roles on the Family Connection project? a. Yes b. No c. Do not know [If NO or DO NOT KNOW, skip to F3]. 2. If YES, what change(s) have occurred? (Check all that apply) a. Added one or more partners b. No longer working with one or more partners c. Partner(s) assumed additional responsibilities/tasks on the project d. Partner(s) reduced responsibilities/tasks on the project e. Other (specify): ___________________________________________ 3. What are the top three communication mechanisms between the grantee organization and project partners? a. Advisory board b. Case-specific meetings/conference calls c. Community meetings d. E-mail memos and updates e. General project meetings/conference calls – Regularly scheduled Service Provider 9 January 27, 2015 f. g. h. i. j. k. General project meetings/conference calls – As needed Reports (e.g., progress reports, evaluation findings, etc.) Surveys Informal channels Other (specify): ___________________________________________ Do not know 4. What are the top three benefits the project has experienced through its work with partners? a. Addressed service gaps/expands services able to provide b. Created a team mentality c. Expanded knowledge base d. Facilitated data sharing e. Improved ability to achieve desired adult, child and family-level outcomes f. Improved ability to address systemic issues/conflicts g. Improved ability to collect evaluation data h. Improved service coordination/facilitates linking families to services i. Increased family engagement in services j. Increased sense of support among families k. Increased understanding of project services by partners l. Increased understanding of partners by project m. Shared risk and responsibility among partners n. Source of referrals o. Strengthened collaboration p. Created sustainability opportunities q. Other (specify): ___________________________________________ 5. What are the top three challenges the project has experienced while working with partners? a. Change/loss of project partner b. Change in partner leadership c. Changes required in service approaches/processes/procedures d. Differing communication styles e. High staff turnover in partner agency f. Lack of partner support g. Limited partner staff availability h. Substantial staff training and orientation requirements i. Other (specify): ___________________________________________ 6. What are the top three ways the Family Connection project has impacted the grantee organization’s relationship with project partners? a. Expanded awareness of the grantee organization’s own strengths and weaknesses per project partner feedback b. Improved reputation of grantee organization among project partners c. Increased awareness of project services among project partners d. Increased contacts between grantee organization and project partners e. Increased/improved case coordination f. Increased understanding of child welfare system by project partners g. Other (specify): ___________________________________________ Service Provider 10 January 27, 2015 7. What are the top three strategies for developing and sustaining a successful partnership? a. Co-locate staff members from grantee organization and/or project partners b. Engage in ongoing, open communication (including one-on-one personal contact) c. Enter formal agreement (e.g., Memorandum of Understanding, contract/subcontract) d. Identify common goals e. Share information about organizations f. Share information about project (e.g., activities, findings, results) g. Ensure that partners understand roles and responsibilities on the project (e.g., partner trainings) h. Other (specify): ___________________________________________ G. QUALITY ASSURANCE The following questions address quality assurance processes for the Family Connection project. 1. Beyond fidelity measures that assess specific services, practices, or the service model, what other strategies has the grantee organization developed for continuous quality improvement in the Family Connection project? (Check all that apply) a. Training for grantee organization staff members b. Training for project partners, including service providers employed by partners c. Technical assistance resources provided (e.g., print materials, websites, blogs, etc.) d. Individual guidance (e.g., mentoring, shadowing) for service providers e. Other (specify): ___________________________________________ f. Do not know [If DO NOT KNOW, skip to G3]. 2. To what extent have these strategies been effective? a. Very effective b. Effective c. Somewhat effective d. Minimally effective e. Not at all effective f. Do not know 3. Have new policies and procedures been developed in the grantee organization as a result of the Family Connection project? a. Yes b. No c. Do not know [If NO or DO NOT KNOW, skip to Section H]. Service Provider 11 January 27, 2015 4. What new policies and procedures were developed? (Check all that apply) a. New service delivery policies/practices created b. Existing service delivery policies/practices modified c. New job descriptions created d. Existing job descriptions modified e. New staff member performance criteria created f. Existing staff member performance criteria modified g. New staff training and technical assistance practices or materials created h. Existing staff training and technical assistance practices or materials modified i. Original project eligibility criteria modified j. Original project recruitment/enrollment practices modified k. Original project referral practices modified l. Other (specify): ___________________________________________ 5. Please provide a brief description of the policy/procedural changes that were made. __________________________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ H. LOCAL EVALUATION The following questions ask about your role in the local evaluation of the Family Connection project, and how evaluation results are shared with project team members and partners. 1. Do you have a role in the local evaluation (e.g., collect data from clients, provide data from evaluation activities that you participate in, such as completing training assessments, participate in evaluation-related meetings)? a. Yes b. No [If NO skip to H3]. 2. What is your role in the local evaluation? (Check all that apply) a. Evaluation planning and design b. Instrument selection and design c. Collect data from clients and families d. Provide own data from evaluation activities (e.g., training or collaboration surveys) e. Provide data on fidelity f. Data entry/processing g. Data analysis h. Evaluation report review and feedback i. Other (specify): ___________________________________________ 3. Has evaluation information (e.g., progress, results) been shared with you? a. Yes b. No c. Do not know Service Provider 12 January 27, 2015 [If NO or DO NOT KNOW skip to H6]. 4. How has evaluation information been shared? (Check all that apply) a. Written reports, memos, or other documents b. In-person presentations (during meetings) c. Virtual presentations (via webinars) d. Other (specify): ___________________________________________ e. Do not know 5. How often is evaluation information shared? (Check all that apply) a. Weekly b. Monthly c. Quarterly d. Semi-annually e. Annually f. Other (specify): ___________________________________________ g. Do not know 6. How frequently are data from the local evaluation used to guide decisions about implementing and improving the Family Connection project? a. Always b. Frequently c. Occasionally d. Rarely e. Never f. Do not know [If NEVER or DO NOT KNOW, skip to SECTION I]. 7. How have process and/or outcome evaluation results been used by the Family Connection project? Please provide an example of how process and/or outcome evaluation results were used in relation to the following aspects of the project: a. Service population (e.g., understanding service needs, strategies for recruitment or retention, etc.): ______________________________________________________________________________ b. Service delivery (e.g., type of service, method of providing services, frequency of providing services, duration of one or multiple sessions of service provision, etc.): ______________________________________________________________________________ c. Service provider (e.g., efficacy of provider-client match, adequacy of staff member training or supervision, etc.): ______________________________________________________________________________ d. Outcomes (e.g., effectiveness of instruments and methods to measure outcomes, expectations to achieve outcomes, etc.): Service Provider 13 January 27, 2015 ______________________________________________________________________________ e. Other (e.g., dissemination to community, support for funding opportunities): ______________________________________________________________________________ I. FACILITATORS, BARRIERS, AND LESSONS LEARNED This section asks about facilitators and barriers to project implementation, and lessons learned to pass on to grantees implementing similar projects. Facilitators are people, policies and procedures, actions, circumstances, and other factors that help something run more smoothly and effectively and/or bring about an outcome. Barriers make processes and the achievement of outcomes difficult or impossible. 1. What have been the top three facilitators of project implementation? a. Celebratory events b. Communication c. Developmental approach taken d. Evaluation plan e. Fidelity to services f. Project leadership support g. Staff member capacity (enough staff members) to implement project h. Staff member selection i. Staff member training (initial and ongoing) j. Support from public child welfare partner k. Support from community partners l. Sustainability incorporated into planning and implementation m. Other (specify): ___________________________________________ 2. What have been the top three challenges to project implementation? a. Initial training insufficient for service provision b. Insufficient number of referrals/difficulty reaching targeted enrollment level c. Lack of or insufficient incentives for clients to engage in services (e.g., monetary, transportation) d. Lack of support from child welfare partner e. Lack of support from community partners f. Need for more physical space g. Refinements to service model and/or individual services h. Processes and procedures are not clear to staff members and/or partners i. Processes and procedures are not followed by staff members and/or partners j. Staff member recruitment k. Staff member retention l. Time needed for staff members to understand the model and provide services m. Too many cases n. Start-up delays o. Other (specify): ___________________________________________ 3. What top three strategies were used to overcome implementation challenges? a. Improved initial training (e.g., content, type, frequency) Service Provider 14 January 27, 2015 b. c. d. e. f. g. Improved ongoing training (e.g., content, type, frequency) Increased availability of resource materials Improved referral system Strengthened communication with partners Co-located staff Other (specify): ___________________________________________ 4. What advice would you give someone implementing a similar project about how to achieve positive outcomes – your “lessons learned”? a. In regard to serving adults, children, and families? ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ b. In regard to collaborating with partners? ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ 5. Is there anything else you would like to add? __________________________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ Thank you for your survey response! Service Provider 15 January 27, 2015 Family Connection Cross-site Evaluation Survey [CLUSTER NAME] Evaluation Team Survey [PROJECT NAME] SURVEY LEAD-IN AND INSTRUCTIONS PAGE Thank you for taking the time to complete this survey about the Family Connection – [CLUSTER NAME] project, also known as the [INSERT PROJECT NAME HERE]. Your input is very valuable to the cross-site evaluation as we work toward understanding and synthesizing data on how grantees implement services and the impact of the services on the organization, community, and children and families. This survey is about the [INSERT PROJECT NAME HERE]. Please answer all questions in regard to your involvement with [INSERT PROJECT NAME HERE]. The “grantee organization” is the recipient of Family Connection Federal funding. The grantee organization is [INSERT GRANTEE ORGANIZATION NAME HERE]. Your survey response is confidential and will not be shared with other members of the project team, project partners, the Children’s Bureau, or other Federal partners. All data collected through this survey will be aggregated for analysis and reporting purposes; information that identifies individual participants will not be used. The survey will take approximately 45 minutes to complete. For your convenience, you can enter and exit the survey as many times as you like to answer the survey items. Your response is not final until you select “Submit” at the end of the survey. To begin the survey now, please click here: CONTINUE Note: ___________ = pre-filled information and/or an action conducted by the web survey system (i.e., no action required by the survey respondent). OMB NO: TBD EXPIRATION DATE: TBD Burden Statement: Public reporting burden for this collection of information is estimated to average .75 hours per respondent. These estimates include the time for reviewing instructions, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Evaluation Team 1 January 27, 2015 Evaluation ID: Respondent: Project Role: Organization: __________ ____________________ __________ __________ Family Connection Cross-site Evaluation Survey [CLUSTER NAME] Evaluation Team Survey A. BACKGROUND The following questions ask about your background with the project and the grantee organization. The “grantee organization” is the recipient of Family Connection Federal funding. The grantee organization is [INSERT GRANTEE ORGANIZATION NAME HERE]. 1. Please verify your primary role on the Family Connection – [CLUSTER NAME] project. (Check one) a. Executive Leaders (e.g., President, CEO) b. Project Director c. Other Project Leaders (e.g., Deputy Project Director, Program or Project Manager, service provider Supervisor) d. Service Provider (e.g., advocate, case manager, case worker, counselor, educator, facilitator, nurse, mentor, therapist, etc.) e. Public Child Welfare Partner representative f. Community Partner representative g. Lead Evaluator/Principal Investigator h. Evaluation Team Member i. Other (specify): _______________________________ Note: JBA will develop a list of each survey recipient and their role, which will determine which survey they receive. The above question will be kept as a logical “opener” to the survey, but not open specific documents. 2. How long have you been in this role for the Family Connection project? a. Less than 1 year b. 1+ to 2 years c. 2+ to 3 years B. PROJECT COLLABORATION These questions ask about collaborative relationships with project leaders, service providers, and partners. Leaders includes Executive Leaders (e.g., President, CEO), the Project Director, and other project leaders (e.g., Deputy Project Director, Program or Project Manager, service provider Supervisor) involved in the Family Connection project. Service providers may be employed by the grantee organization or a project partner. Project partners include public child welfare agencies and other community partners (e.g., private, contracted provider of child welfare services). Evaluation Team 2 January 27, 2015 1. Please characterize the level of collaboration between the evaluation team and project leaders. Collaboration refers to the cooperative way that two or more parties work together toward a shared goal. Collaborative relationships often involve: defined roles; sharing ideas, information, and resources; frequent communication; shared decision-making; and mutual trust. a. Very collaborative b. Collaborative c. Somewhat collaborative d. Minimally collaborative e. Not collaborative at all f. Do not know 2. Please characterize the level of collaboration between the evaluation team and service providers. a. Very collaborative b. Collaborative c. Somewhat collaborative d. Minimally collaborative e. Not collaborative at all f. Do not know 3. How involved have project leaders been in planning the evaluation (e.g., providing feedback in early discussions regarding evaluation design and data collection)? g. Very involved h. Involved i. Somewhat involved j. Minimally involved k. Not involved at all l. Do not know 4. How involved have project leaders been in implementing the evaluation (e.g., attending and supporting evaluation training, supporting program staff members as data collectors as needed, providing secondary data as needed, being engaged in evaluation updates and receptive to evaluation reports). a. Very involved b. Involved c. Somewhat involved d. Minimally involved e. Not involved at all f. Do not know 5. How involved have service providers been in implementing the evaluation? Service providers may be employed by the grantee organization or a project partner. a. Very involved b. Involved c. Somewhat involved d. Minimally involved e. Not involved at all f. Do not know Evaluation Team 3 January 27, 2015 6. Does the public child welfare agency – if different from the grantee organization – have any of the following roles in the evaluation? (Check all that apply). a. Input on evaluation planning and design b. Input on Instrument selection and design c. Collect primary data (e.g., administer surveys, conduct interviews, etc.) d. Provide secondary / administrative data (e.g., SACWIS) e. Evaluation data entry f. Evaluation data analysis and/or reporting g. Review, provides feedback on evaluation reports by evaluation team h. Public child welfare agency is the grantee organization i. Other (specify): ___________________________________________ 7. Do community partners have any of the following roles in the evaluation? (Check all that apply). a. Input on evaluation planning and design b. Input on Instrument selection and design c. Collect primary data (e.g., administer surveys, conduct interviews, etc.) d. Provide secondary / administrative data (e.g., SACWIS) e. Evaluation data entry f. Evaluation data analysis and/or reporting g. Review, provides feedback on evaluation reports by evaluation team h. Other (specify): ___________________________________________ C. PROJECT IMPLEMENTATION The following questions ask about project implementation. These questions address project readiness and start up, service provision, and key implementation drivers. Project Readiness and Start-Up 1. Were you a project evaluator during the project’s initial 3-month start-up period (October 2012 to December 2012)? a. Yes b. No [If NO, skip to C5]. 2. Were you involved in planning the evaluation of the Family Connection project (e.g., writing the evaluation portion of the grantee’s funding application, planning during the initial start-up period, etc.) a. Yes b. No If NO, skip to C5]. 3. To what degree was the grantee organization ready for the Family Connection project? Readiness may include, but not be limited to: project leaders and service providers with the appropriate Evaluation Team 4 January 27, 2015 knowledge, skills and abilities to implement services; support from project partners; training, coaching and supervision for project leaders, service providers, and partners; leaders to oversee work; administrative support; space; technology; etc. Your answer may be based on evaluation data, your observations and interactions with the grantee organization, etc. a. Very ready b. For the most part ready c. Somewhat ready d. Slightly ready e. Not at all ready f. Do not know 4. To what degree was the broader child welfare system (e.g., other public and private child welfare agencies who serve children at risk of or who are removed from the home, and their partner organizations) ready to engage with the Family Connection project? Readiness may include, but not be limited to: knowledge of the funding opportunity, knowledge of the grantee organization’s project, staff members or other resources available to the project, providing a Memorandum of Understanding (MOU) or letter of support to the funding application, etc. a. Very ready b. For the most part ready c. Somewhat ready d. Slightly ready e. Not at all ready f. Do not know Service Provision 5. Has the project served the number of children and families the organization expected to serve? a. Exceeding service expectations (i.e., serving more children and families than anticipated) b. Meeting service expectations (i.e., on track with the number of children and families served) c. Partially meeting service expectations (i.e., behind projections for number of children and families served) 6. How easy or challenging has it been for the project to engage clients in services? This may include, but not be limited to initially engaging clients via referrals and recruitment, and keeping clients engaged in services. a. Very challenging b. Challenging c. Neither challenging nor easy d. Easy e. Very easy f. Do not know [If NEITHER CHALLENGING NOR EASY, EASY, VERY EASY, OR DO NOT KNOW, skip to C8]. 7. What are the top three factors that have made it difficult to engage clients in services? a. Basic needs of individual client and/or family (e.g., food, housing) take priority over other project services b. Eligibility criteria (e.g., changes in eligibility criteria) Evaluation Team 5 January 27, 2015 c. d. e. f. g. h. i. j. k. l. m. n. o. p. Client resistance to receiving services Client resistance to other family member involvement in services / treatment Drug dependency/substance abuse of client and/or family members Heath issues of clients – specifically aging caregivers Heath issues of clients – other health issues Lack of transitional housing Lack of support system (e.g., friends, family in area) Lack of transportation to services Language barriers (e.g., primary language not read or spoken by service providers) Low literacy level of clients and/or family members Mental health issues of client Referral process (e.g., changes in process, challenges of process) Transient/highly mobile population Other (specify): __________________________________________ 8. Does the population receiving services match the population the project originally intended to serve? a. Yes b. No c. Do not know [If YES or DO NOT KNOW, skip to C10]. 9. How does the actual service population differ from the population originally intended to be served (e.g., gender, age, family relationship, geographic location, etc.)? __________________________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ 10. Has the project modified its service population for the Family Connection project from the population described in the Federal funding application? This may or may not have involved a request to submit a change in scope to the Children’s Bureau. a. Yes b. No c. Do not know [If NO or DO NOT KNOW, skip to C12]. 11. Why did the project modify its service population (e.g., eligibility criteria, expanding or centralizing geographic reach, etc.)? __________________________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ Evaluation Team 6 January 27, 2015 Implementation Drivers 12. How easy or challenging was it to fully staff the project within the 90-day planning period? Fully staff may include shifting responsibilities for existing staff members, hiring new staff members, or contracting with external organizations. a. Very challenging b. Challenging c. Neither challenging nor easy d. Easy e. Very easy f. Do not know [If NEITHER CHALLENGING NOR EASY, EASY, VERY EASY, OR DO NOT KNOW, skip to C14]. 13. To what extent did challenges with initial staffing delay project implementation? a. Greatly delayed project implementation b. Delayed project implementation c. Somewhat delayed project implementation d. Only slightly delayed project implementation e. Did not delay project implementation f. Do not know 14. Has the Family Connection project experienced turnover in staff members from the grantee organization (including project leaders and service providers) during the grant period? a. Yes, there has been substantial turnover (50% or more) b. Yes, there has been moderate turnover (25% to 49%) c. Yes, there has been minimal turnover (less than 25%) d. No, there has been no turnover e. Do not know [If NO or DO NOT KNOW, skip to C17]. 15. To what extent has staff member turnover impacted project evaluation? g. Greatly impacted project evaluation h. Impacted project evaluation i. Somewhat impacted project evaluation j. Only slightly impacted project evaluation k. Did not impact project evaluation [If DID NOT IMPACT EVALUATION, skip to C17]. 16. Please list up to three examples in which staff member turnover has impacted project evaluation. Example 1: ________________________________________________________________________ Example 2: ________________________________________________________________________ Example 3: ________________________________________________________________________ Evaluation Team 7 January 27, 2015 17. Who receives training on Family Connection project services? This may include general project orientation, training on the overall service model, or training on specific components and services of the service model. (Check all that apply) a. Executive Leaders (e.g., President, CEO) b. Project Director c. Other Project Leaders (e.g., Deputy Project Director, Program or Project Manager, service provider Supervisor, etc.) d. Service Provider (e.g., advocate, case manager, case worker, counselor, educator, facilitator, nurse, mentor, therapist, etc.) e. Public Child Welfare Partner representative f. Community Partner representative g. Lead Evaluator/Principal Investigator h. Evaluation Team Member(s) i. Other (specify): ______________________________________________ 18. Do service providers receive regular coaching or supervision on the Family Connection project’s service model? a. Yes, at least weekly b. Yes, at least monthly c. Yes, at least quarterly d. No coaching or supervision received e. Do not know 19. Has training, coaching, and/or service provider supervision promoted fidelity to the service model? Your answer may be based on evaluation data, your observations and interactions with the grantee organization, etc. a. Greatly promoted fidelity b. Promoted fidelity c. Somewhat promoted fidelity d. Only slightly promoted fidelity e. Did not promote fidelity f. Do not know [If DO NOT KNOW, skip to C21]. 20. How has training, coaching, and/or service provider supervision promoted or not promoted fidelity to the service model? __________________________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ 21. Who receives training regarding the evaluation component of the Family Connection project? (Check all that apply) a. Executive Leaders (e.g., President, CEO) b. Project Director c. Other Project Leaders (e.g., Deputy Project Director, Program or Project Manager, service provider Supervisor, etc.) Evaluation Team 8 January 27, 2015 d. Service Provider (e.g., advocate, case manager, case worker, counselor, educator, facilitator, nurse, mentor, therapist, etc.) e. Public Child Welfare Partner representative f. Community Partner representative g. Lead Evaluator/Principal Investigator h. Evaluation Team Member(s) i. Other (specify): ______________________________________________ 22. Have project leaders facilitated the evaluation of the project? a. Very much so b. For the most part c. Somewhat d. Only slightly e. Not at all f. Do not know [If NOT AT ALL or DO NOT KNOW, skip to C24]. 23. How have project leaders facilitated the evaluation of the project? __________________________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ 24. Have project leaders been a challenge to the evaluation of the project? a. Very much so b. For the most part c. Somewhat d. Only slightly e. Not at all f. Do not know [If NOT AT ALL or DO NOT KNOW, skip to C26]. 25. How have project leaders been a challenge to the evaluation of the project? __________________________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ 26. Has the grantee’s relationship with the public child welfare agency – if different from the grantee organization – facilitated the evaluation of the project? a. Greatly facilitated the evaluation b. Facilitated the evaluation c. Somewhat facilitated the evaluation d. Only slightly facilitated the evaluation e. Did not facilitate the evaluation f. Do not know g. Public child welfare agency is the grantee organization Evaluation Team 9 January 27, 2015 [If DID NOT FACILITATE THE EVALUATION, DO NOT KNOW, or PUBLIC CHILD WELFARE AGENCY IS THE GRANTEE ORGANIZATION, skip to C30]. 27. How has the grantee’s relationship with the public child welfare agency facilitated the evaluation of the project? __________________________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ 28. Has the grantee’s relationship with the public child welfare agency challenged the evaluation of the project? a. Greatly challenged the evaluation b. Challenged the evaluation c. Somewhat challenged the evaluation d. Only slightly challenged the evaluation e. Did not challenge the evaluation f. Do not know [If DID NOT CHALLENGE THE EVALUATION or DO NOT KNOW, skip to Section D]. 29. How has the grantee’s relationship with the public child welfare agency challenged the evaluation of the project? __________________________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ 30. Has the grantee’s relationship with other community partners facilitated the evaluation of the project? a. Greatly facilitated the evaluation b. Facilitated the evaluation c. Somewhat facilitated the evaluation d. Only slightly facilitated the evaluation e. Did not facilitate the evaluation f. Do not know [If DID NOT FACILITATE THE EVALUATION or DO NOT KNOW, skip to C32]. 31. How has the grantee’s relationship with other community collaborators facilitated the evaluation of the project? __________________________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ 32. Has the grantee’s relationship with other community partners challenged the evaluation of the project? a. Greatly challenged the evaluation b. Challenged the evaluation c. Somewhat challenged the evaluation Evaluation Team 10 January 27, 2015 d. Only slightly challenged the evaluation e. Did not challenge the evaluation f. Do not know [If DID NOT CHALLENGE THE EVALUATION or DO NOT KNOW, skip to Section D]. 33. How has the grantee’s relationship with the public child welfare agency and/or other community collaborators challenged the evaluation of the project? __________________________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ D. QUALITY ASSURANCE / DECISION SUPPORT DATA SYSTEMS The following questions ask about fidelity assessments of project services or the service model, and the use of evaluation results in the project. Fidelity Assessments 1. Does the local evaluation include fidelity assessments of project services or the service model in general? a. Yes b. No c. Do not know [If NO or DO NOT KNOW, skip to D8]. 2. Are fidelity results shared with others involved in the Family Connection project (e.g., project leaders, service providers, public child welfare partner, community partner, etc.)? a. Yes b. No c. Do not know d. Have not completed fidelity assessments yet [If NO, DO NOT KNOW, or HAVE NOT COMPLETED ASSESSMENTS, skip to D8]. 3. Who are fidelity results shared with? (Check all that apply) a. Executive Leaders (e.g., President, CEO) b. Project Director c. Other Project Leaders (e.g., Deputy Project Director, Program or Project Manager, service provider Supervisor) d. Service Provider (e.g., advocate, case manager, case worker, counselor, educator, facilitator, nurse, mentor, therapist, etc.) e. Public Child Welfare Partner representative f. Community Partner representative g. Other (specify): _______________________________ Evaluation Team 11 January 27, 2015 4. How are fidelity results shared? (Check all that apply) a. Written briefs or reports b. Oral briefings (during meetings) c. Presentations / webinars d. Other (specify): ___________________________________________ e. Do not know 5. How often are fidelity results shared? (Check all that apply) a. Weekly b. Monthly c. Quarterly d. Semi-annually e. Annually f. Other (specify): ___________________________________________ g. Do not know 6. Have fidelity results been used by the Family Connection project? a. Yes b. No c. Do not know [If NO or DO NOT KNOW, skip to D7]. 7. How have fidelity results been used by the Family Connection project? Example 1: ________________________________________________________________________ Example 2: ________________________________________________________________________ Example 3: ________________________________________________________________________ Local Evaluation Results 8. Are process and/or outcome results shared with others involved in the Family Connection project (e.g., project leaders, service providers, public child welfare partner, community partner, etc.)? a. Yes b. No c. Do not know d. Have not completed analysis yet [If NO, DO NOT KNOW, or HAVE NOT COMPLETE ANALYSIS, skip to Section E]. 9. Who are process and/or outcome evaluation results shared with? (check all that apply) a. Executive Leaders (e.g., President, CEO) b. Project Director c. Other Project Leaders (e.g., Deputy Project Director, Program or Project Manager, service provider Supervisor) d. Service Provider (e.g., advocate, case manager, case worker, counselor, educator, facilitator, nurse, mentor, therapist, etc.) e. Public Child Welfare Partner representative f. Community Partner representative Evaluation Team 12 January 27, 2015 g. Other (specify): _______________________________ 10. How are process and/or outcome evaluation results shared? (Check all that apply) a. Written reports, memos, or other documents b. In-person presentations (during meetings) c. Virtual presentations (via webinars) d. Other (specify): ___________________________________________ e. Do not know 11. How often are process and/or outcome evaluation results shared? (Check all that apply) a. Weekly b. Monthly c. Quarterly d. Semi-annually e. Annually f. Other (specify): ___________________________________________ g. Do not know 12. Have process and/or outcome results been used by the Family Connection project? a. Yes b. No c. Do not know [If NO or DO NOT KNOW, skip to Section E]. 13. How have process and/or outcome evaluation results been used by the Family Connection project? Please provide an example of how process and/or outcome evaluation results were used in relation to the following aspects of the project: a. Service population (e.g., understanding service needs, strategies for recruitment or retention, etc.): ______________________________________________________________________________ b. Service delivery (e.g., type of service, method of providing services, frequency of providing services, duration of one or multiple sessions of service provision, etc.): ______________________________________________________________________________ c. Service provider (e.g., efficacy of provider-client match, adequacy of staff member training or supervision, etc.): ______________________________________________________________________________ d. Outcomes (e.g., effectiveness of instruments and methods to measure outcomes, expectations to achieve outcomes, etc.): ______________________________________________________________________________ e. Other (e.g., dissemination to community, support for funding opportunities): ______________________________________________________________________________ Evaluation Team 13 January 27, 2015 E. EVALUATION DATA MANAGEMENT The following questions ask about data management tools and processes for the project’s local evaluation. “Data management system” may include, but not be limited to Microsoft products such as Excel or Access, evaluator or grantee-developed systems, vendor supplied systems, etc. Grantees may have more than one data management system for the local evaluation, and different people involved in the Family Connection project may have different levels of access to these systems. 1. Was a new data management system or systems for evaluation data developed specifically for the Family Connection project, or was an existing one modified for the project? a. New system(s) developed for project b. Existing system(s) modified for project c. Combination of new and updated existing system(s) used for project d. Other (specify): _______________________________ e. Do not know [If DO NOT KNOW, skip to E3]. 2. Who was primarily responsible for developing and/or modifying the data management system or systems? (check all that apply) a. Grantee organization administrative support b. Grantee organization MIS / data administrator c. Grantee organization IT support d. Evaluation team member e. External vendor f. Other (specify): _______________________________ g. Do not know 3. Who has access to the data management system or systems? Different people involved in the Family Connection project may have different levels of access to these systems. (check all that apply) a. Executive Leaders (e.g., President, CEO) b. Project Director c. Other Project Leaders (e.g., Deputy Project Director, Program or Project Manager, service provider Supervisor) d. Service Provider (e.g., advocate, case manager, case worker, counselor, educator, facilitator, nurse, mentor, therapist, etc.) e. Grantee organization administrative support f. Grantee organization MIS / data administrator g. Grantee organization IT support h. Lead Evaluator/Principal Investigator i. Evaluation Team Member j. Other (specify): _______________________________ 4. Did the evaluation team provide initial training to those with system(s) access? a. Yes b. No Evaluation Team 14 January 27, 2015 c. Do not know 5. Has the evaluation team provided ongoing training and/or coaching to those with system(s) access? a. Yes b. No c. Do not know 6. Does the evaluation team conduct quality review processes with the data management system or systems, such as checking for completeness, timeliness, and accuracy? a. Yes, daily b. Yes, weekly c. Yes, monthly d. Yes, bi-monthly e. Yes, quarterly f. No processes are conducted g. Do not know 7. Will the data management system be sustained by the grantee organization after Federal Family Connection project funding concludes? a. Yes, the entire system will be sustained b. Yes, parts of the system will be sustained c. No, the system will not be sustained d. Do not know [IF NO or DO NOT KNOW, skip to E9]. 8. Who is most likely to maintain the system? a. Grantee organization administrative support b. Grantee organization MIS / data administrator c. Grantee organization IT support d. Evaluator from Family Connection project e. Another evaluator f. Other (specify): _______________________________ g. Do not know 9. What are the top three ways having a data management system has impacted the Family Connection project? Factor 1: __________________________________________________________________________ Factor 2: __________________________________________________________________________ Factor 3: __________________________________________________________________________ F. EVALUATION PROCESSES AND LESSONS LEARNED These questions ask about areas in which the evaluation team would have liked to change the local evaluation, areas in which the evaluation was changed, and facilitators and barriers to the local evaluation. Evaluation Team 15 January 27, 2015 a. Scope of evaluation (number and/or breadth of activities) b. Evaluation design c. Evaluation variables d. Primary data sources (e.g., evaluation instruments) e. Secondary data sources (e.g., SACWIS, local administrative databases) f. Evaluation population (e.g., target population, sample size) g. Evaluation recruitment and retention processes h. Evaluation data collection processes (e.g., incorporated qualitative interviews) i. Data management j. Analysis plan k. Quality assurance processes for the evaluation l. Scope and/or frequency of sharing results with grantee m. Interactions with project leaders n. Interactions with project staff members o. Other (please describe) 1. In what areas would the evaluation team have liked to change the local evaluation? (Check the top three areas. You may not check any options if there were no areas the team wanted to change.) 2. In what areas did the evaluation team actually make changes? (Check all that apply.) How? ____________________ How? ____________________ How? ____________________ How? ____________________ How? ____________________ How? ____________________ How? ____________________ How? ____________________ How? ____________________ How? ____________________ How? ____________________ How? ____________________ How? ____________________ How? ____________________ How? ____________________ How? ____________________ How? ____________________ How? ____________________ How? ____________________ How? ____________________ How? ____________________ How? ____________________ How? ____________________ How? ____________________ How? ____________________ How? ____________________ How? ____________________ How? ____________________ How? ____________________ How? ____________________ 3. Please list the top three factors that have facilitated the evaluation: Factor 1: __________________________________________________________________________ Factor 2: __________________________________________________________________________ Factor 3: __________________________________________________________________________ Evaluation Team 16 January 27, 2015 4. Please list the top three factors that have challenged the evaluation: Factor 1: __________________________________________________________________________ Factor 2: __________________________________________________________________________ Factor 3: __________________________________________________________________________ 5. Is there anything else you would like to add? __________________________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ Thank you for your survey response! Evaluation Team 17 January 27, 2015 Family Connection Cross-site Evaluation Survey [CLUSTER NAME] Public Child Welfare Agency Survey [PROJECT NAME] SURVEY LEAD-IN AND INSTRUCTIONS PAGE Thank you for taking the time to complete this survey about the Family Connection – [CLUSTER NAME] project, also known as the [INSERT PROJECT NAME HERE]. Your input is very valuable to the cross-site evaluation as we work toward understanding and synthesizing data on how grantees implement services and the impact of the services on the organization, community, and children and families. This survey is about the [INSERT PROJECT NAME HERE]. Please answer all questions in regard to your involvement with [INSERT PROJECT NAME HERE]. The “grantee organization” is the recipient of Family Connection Federal funding. The grantee organization is [INSERT GRANTEE ORGANIZATION NAME HERE]. Your survey response is confidential and will not be shared with other members of the project team, project partners, the Children’s Bureau, or other Federal partners. All data collected through this survey will be aggregated for analysis and reporting purposes; information that identifies individual participants will not be used. The survey will take approximately 15 minutes to complete. For your convenience, you can enter and exit the survey as many times as you like to answer the survey items. Your response is not final until you select “Submit” at the end of the survey. To begin the survey now, please click here: CONTINUE Note: ___________ = pre-filled information and/or an action conducted by the web survey system (i.e., no action required by the survey respondent). OMB NO: TBD EXPIRATION DATE: TBD Burden Statement: Public reporting burden for this collection of information is estimated to average .75 hours per respondent. These estimates include the time for reviewing instructions, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Public Child Welfare Agency 1 January 27, 2015 Evaluation ID: Respondent: Project Role: Organization: __________ ____________________ __________ __________ Family Connection Cross-site Evaluation Survey [CLUSTER NAME] Public Child Welfare Agency A. BACKGROUND The following questions ask about your background with the project and the grantee organization. The “grantee organization” is the recipient of Family Connection Federal funding. The grantee organization is [INSERT GRANTEE ORGANIZATION NAME HERE]. 1. Please verify your primary role on the Family Connection – [CLUSTER NAME] project. (Check one) a. Executive Leaders (e.g., President, CEO) b. Project Director c. Other Project Leaders (e.g., Deputy Project Director, Program or Project Manager, service provider Supervisor) d. Service Provider (e.g., advocate, case manager, case worker, counselor, educator, facilitator, nurse, mentor, therapist, etc.) e. Public Child Welfare Partner representative f. Community Partner representative g. Lead Evaluator/Principal Investigator h. Evaluation Team Member i. Other (specify): _______________________________ 2. How long have you been in this role for the Family Connection project? a. Less than 1 year b. 1+ to 2 years c. 2+ to 3 years 3. How long have you worked for your current organization? a. Less than 1 year b. 1+ to 2 years c. 3+ to 5 years d. 5+ to 10 years e. Over 10 years B. PUBLIC CHILD WELFARE AGENCY RELATIONSHIP WITH GRANTEE ORGANIZATION The following questions ask about the public child welfare agency’s prior relationship with the Family Connection grantee organization. Public Child Welfare Agency 2 January 27, 2015 The “grantee organization” is the recipient of Family Connection Federal funding. The grantee organization is [INSERT GRANTEE ORGANIZATION NAME HERE]. 1. Did the public child welfare agency work with the grantee organization prior to the Family Connection project? a. Yes b. No c. Do not know [If NO or DO NOT KNOW, skip to Section C]. 2. How would you describe the prior collaborative relationship/partnership? a. Very collaborative b. Collaborative c. Somewhat collaborative d. Minimally collaborative e. Not at all collaborative C. FAMILY CONNECTION PROJECT INVOLVEMENT The following questions ask about the public child welfare agency’s current involvement with the grantee organization’s Family Connection project. The “grantee organization” is the recipient of Family Connection Federal funding. The grantee organization is [INSERT GRANTEE ORGANIZATION NAME HERE]. 1. Does the public child welfare agency refer families to the Family Connection project? a. Yes b. No c. Do not know [If NO or DO NOT KNOW, skip to C7]. 2. Are there eligibility criteria or a selection process to refer families to the Family Connection project? a. Yes b. No c. Do not know [If NO or DO NOT KNOW, skip to C7]. 3. Have there been changes to the referral process since the start of the Family Connection project? a. Yes, there have been major changes b. Yes, there have been minor changes c. No, there have been no changes d. Do not know [If NO or DO NOT KNOW, skip to C5]. Public Child Welfare Agency 3 January 27, 2015 4. Please briefly describe the changes to the referral process. __________________________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ 5. Have there been challenges to the public child welfare agency’s referral process? a. Yes b. No c. Do not know [If NO or DO NOT KNOW, skip to C7]. 6. What challenges has the public child welfare agency experienced with the referral process? (check all that apply) a. Grantee organization project staff turnover b. Public child welfare agency staff turnover c. Incomplete referral information d. Lengthy waiting period from referral to service e. Adult client or family eligible, but not ready or willing to engage in service f. Other (specify): ______________________________________________ 7. How else is the child welfare agency involved with the Family Connection project? (check all that apply) a. Grantee organization staff member(s) co-located at public child welfare agency b. Public child welfare agency staff member(s) co-located at grantee organization c. Participates in Family Connection project advisory group d. Participates in Family Connection project management and/or team meetings e. Provides funding f. Provides direct service provision to Family Connection project clients g. Other (specify): ______________________________________________ D. FAMILY CONNECTION PROJECT COMMUNICATION The following questions ask about the public child welfare agency’s communication processes with the grantee organization. The “grantee organization” is the recipient of Family Connection Federal funding. The grantee organization is [INSERT GRANTEE ORGANIZATION NAME HERE]. 1. How clear are the roles and responsibilities between the public child welfare agency and the Family Connection project? a. Very clear b. For the most part clear c. Somewhat clear d. Slightly clear e. Not at all clear f. Do not know Public Child Welfare Agency 4 January 27, 2015 [If VERY CLEAR, skip to D3]. 2. How can clarity in roles and responsibilities be improved? __________________________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ 3. Is there a process for regular communication regarding successes and concerns about the Family Connection project between the public child welfare agency and the grantee organization? a. Yes b. No c. Do not know [If NO or DO NOT KNOW, skip to D5] 4. What communication mechanisms does the public child welfare agency experience as part of the Family Connection project? (check all that apply) a. Advisory board membership b. Case-specific meetings/conference calls c. Community meetings d. E-mail memos and updates e. General project meetings/conference calls – Regularly scheduled f. General project meetings/conference calls – As needed g. Reports (e.g., progress reports, evaluation findings, etc.) h. Surveys i. Informal channels j. Other (specify): ___________________________________________ 5. Is there enough communication between the Family Connection project and the public child welfare agency to keep you well-informed about the project? a. Yes b. No [If YES, skip to D7]. 6. How can communication be improved? __________________________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ 7. Did the public child welfare agency have an opportunity to provide its perspective in planning the Family Connection project? This may include input provided during the grant application process or during the 3-month start-up period. a. Yes b. No c. Do not know [If NO or DO NOT KNOW, skip to D9]. Public Child Welfare Agency 5 January 27, 2015 8. Through what channels did the public child welfare agency provide its perspective in planning the Family Connection project? (check all that apply) a. Advisory board meetings b. Community meetings c. Family Connection project planning meetings d. One-on-one telephone calls with project planners e. E-mails with project planners f. Surveys g. Informal channels h. Other (specify): ___________________________________________ 9. Has the public child welfare agency had an opportunity to provide ongoing feedback to the Family Connection project? a. Yes b. No c. Do not know [If NO or DO NOT KNOW, skip to D11]. 10. Through what channels has the public child welfare agency provided ongoing feedback to the Family Connection project? (check all that apply) a. Advisory board meetings b. Case-specific meetings/conference calls c. Community meetings d. General project meetings/conference calls – Regularly scheduled e. General project meetings/conference calls – As needed f. Feedback on reports (e.g., progress reports, evaluation findings, etc.) g. One-on-one telephone calls with project managers and/or staff members h. E-mails with project managers and/or staff members i. Surveys j. Informal channels k. Other (specify): ___________________________________________ 11. Has the Family Connection project changed the public child welfare agency’s relationship with the grantee organization? a. Relationship is much more positive b. Relationship is more positive c. No evident change d. Relationship is more negative e. Relationship is much more negative f. Do not know [If DO NOT KNOW, skip to Section E]. Public Child Welfare Agency 6 January 27, 2015 12. What factors have contributed to positive aspects of the relationship? __________________________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ 13. What challenges has the public child welfare agency experienced with the grantee? __________________________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ 14. How have the challenges been addressed? __________________________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ E. LOCAL EVALUATION The following questions ask about your role in the local evaluation of the Family Connection project, and how evaluation results are shared with project team members and partners. 1. Do you have a role in the local evaluation (e.g., e.g., provide administrative data to evaluation team, provide data from evaluation activities that you participate in, such as completing a collaboration survey, participate in evaluation-related meetings)? a. Yes b. No [If NO skip to E3]. 2. What is your role in the local evaluation? (Check all that apply) a. Input on evaluation planning and design b. Input on instrument selection and design c. Collect primary data (e.g., administer surveys, conduct interviews, etc.) d. Provide secondary/administrative child welfare data (e.g., SACWIS) e. Provide own data from evaluation activities (e.g., training or collaboration surveys) f. Evaluation data entry g. Evaluation data analysis and/or reporting h. Evaluation report review and feedback i. Other (specify): ___________________________________________ 3. Has evaluation information (e.g., progress, results) been shared with the public child welfare agency? a. Yes b. No c. Do not know [If NO or DO NOT KNOW skip to D6]. Public Child Welfare Agency 7 January 27, 2015 4. How has evaluation information been shared? (Check all that apply) a. Written reports, memos, or other documents b. In-person presentations (during meetings) c. Virtual presentations (via webinars) d. Other (specify): ___________________________________________ e. Do not know 5. How often is evaluation information shared? (Check all that apply) a. Weekly b. Monthly c. Quarterly d. Semi-annually e. Annually f. Do not know g. Other (specify): ___________________________________________ h. Do not know 6. Have evaluation results from the Family Connection project been used by the public child welfare agency? a. Yes b. No c. Do not know [If NO or DO NOT KNOW, skip to SECTION F]. 7. Please provide up to three examples of how Family Connection evaluation data were used by the public child welfare agency: Example 1: ________________________________________________________________________ Example 2: ________________________________________________________________________ Example 3: ________________________________________________________________________ F. PROJECT IMPACT The following questions ask about your perception of the Family Connection project’s achievements, how the project has influenced the public child welfare agency, and how the project has affected the broader child welfare system. 1. Has the Family Connection project influenced the larger child welfare system? a. Very positively b. Positively c. No evident influence d. Negatively e. Very negatively f. Do not know [If DO NOT KNOW, skip to F3]. Public Child Welfare Agency 8 January 27, 2015 2. How has the Family Connection project influenced the larger child welfare system? a. Fills a service gap b. Increased awareness of Family Connection services among child welfare providers c. Increased communication and coordination among child welfare providers d. Increased family engagement and involvement in services e. Increased permanent placements f. Influenced child welfare social workers in regard to Family Connection services g. Reduced the number of child removals h. Reduced the number of foster care entries i. Reduced time spent in foster care j. Other (specify): ___________________________________________ 3. To what extent has the public child welfare agency integrated elements of the Family Connection project into its own service delivery system? a. To a great extent b. Somewhat c. Very little d. Not at all e. Do not know [If NOT AT ALL or DO NOT KNOW, skip to F5]. 4. What are the project’s most important accomplishments? Accomplishment 1: __________________________________________________________________ Accomplishment 2: __________________________________________________________________ Accomplishment 3: __________________________________________________________________ 5. Are there benefits to developing a partnership between the public child welfare agency and private/not-for-profit agencies? a. Yes b. No [If NO, skip to F7]. 6. Please list up to three benefits of developing a partnership between the public child welfare agency and private/not-for-profit agencies. Benefit 1: _________________________________________________________________________ Benefit 2: _________________________________________________________________________ Benefit 3: _________________________________________________________________________ 7. How can collaboration between public child welfare agencies and private/not-for-profit agencies be strengthened? _________________________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ Public Child Welfare Agency 9 January 27, 2015 8. Is there anything else you would like to add? __________________________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ Thank you for your survey response! Public Child Welfare Agency 10 January 27, 2015 Family Connection Cross-site Evaluation Survey [CLUSTER NAME] Community Partner Survey [PROJECT NAME] SURVEY LEAD-IN AND INSTRUCTIONS PAGE Thank you for taking the time to complete this survey about the Family Connection – [CLUSTER NAME] project, also known as the [INSERT PROJECT NAME HERE]. Your input is very valuable to the cross-site evaluation as we work toward understanding and synthesizing data on how grantees implement services and the impact of the services on the organization, community, and children and families. This survey is about the [INSERT PROJECT NAME HERE]. Please answer all questions in regard to your involvement with [INSERT PROJECT NAME HERE]. The “grantee organization” is the recipient of Family Connection Federal funding. The grantee organization is [INSERT GRANTEE ORGANIZATION NAME HERE]. Your survey response is confidential and will not be shared with other members of the project team, project partners, the Children’s Bureau, or other Federal partners. All data collected through this survey will be aggregated for analysis and reporting purposes; information that identifies individual participants will not be used. The survey will take approximately 15 minutes to complete. For your convenience, you can enter and exit the survey as many times as you like to answer the survey items. Your response is not final until you select “Submit” at the end of the survey. To begin the survey now, please click here: CONTINUE Note: ___________ = pre-filled information and/or an action conducted by the web survey system (i.e., no action required by the survey respondent). OMB NO: TBD EXPIRATION DATE: TBD Burden Statement: Public reporting burden for this collection of information is estimated to average .75 hours per respondent. These estimates include the time for reviewing instructions, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Community Partner 1 January 27, 2015 Evaluation ID: Respondent: Project Role: Organization: __________ ____________________ __________ __________ Family Connection Cross-site Evaluation Survey [CLUSTER NAME] Community Partner Survey A. BACKGROUND The following questions ask about your background with the project and the grantee organization. The “grantee organization” is the recipient of Family Connection Federal funding. The grantee organization is [INSERT GRANTEE ORGANIZATION NAME HERE]. 1. Please verify your primary role on the Family Connection – [CLUSTER NAME] project. (Check one) a. Executive Leaders (e.g., President, CEO) b. Project Director c. Other Project Leaders (e.g., Deputy Project Director, Program or Project Manager, service provider Supervisor) d. Service Provider (e.g., advocate, case manager, case worker, counselor, educator, facilitator, nurse, mentor, therapist, etc.) e. Public Child Welfare Partner representative f. Community Partner representative g. Lead Evaluator/Principal Investigator h. Evaluation Team Member i. Other (specify): _______________________________ 2. How long have you been in this role for the Family Connection project? a. Less than 1 year b. 1+ to 2 years c. 2+ to 3 years 3. How long have you worked for your current organization? a. Less than 1 year b. 1+ to 2 years c. 3+ to 5 years d. 5+ to 10 years e. Over 10 years B. COMMUNITY PARTNER RELATIONSHIP WITH GRANTEE ORGANIZATION The following questions ask about the community partner’s prior relationship with the Family Connection grantee organization. Community Partner 2 January 27, 2015 The “grantee organization” is the recipient of Family Connection Federal funding. The grantee organization is [INSERT GRANTEE ORGANIZATION NAME HERE]. 1. Did the community partner work with the grantee organization prior to the Family Connection project? a. Yes b. No c. Do not know [If NO or DO NOT KNOW, skip to Section C]. 2. How would you describe the prior collaborative relationship/partnership? a. Very collaborative b. Collaborative c. Somewhat collaborative d. Minimally collaborative e. Not at all collaborative C. FAMILY CONNECTION PROJECT INVOLVEMENT The following questions ask about the community partner’s current involvement with the grantee organization’s Family Connection project. The “grantee organization” is the recipient of Family Connection Federal funding. The grantee organization is [INSERT GRANTEE ORGANIZATION NAME HERE]. 1. Does the community partner refer families to the Family Connection project? a. Yes b. No c. Do not know [If NO or DO NOT KNOW, skip to C7]. 2. Are there eligibility criteria or a selection process to refer families to the Family Connection project? a. Yes b. No c. Do not know [If NO or DO NOT KNOW, skip to C7]. 3. Have there been changes to the referral process since the start of the Family Connection project? a. Yes, there have been major changes b. Yes, there have been minor changes c. No, there have been no changes d. Do not know [If NO or DO NOT KNOW, skip to C5]. Community Partner 3 January 27, 2015 4. Please briefly describe the changes to the referral process. __________________________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ 5. Have there been challenges to the community partner’s referral process? a. Yes b. No c. Do not know [If NO or DO NOT KNOW, skip to C7]. 6. What challenges has the community partner experienced with the referral process? (check all that apply) a. Grantee organization project staff turnover b. Community partner staff turnover c. Incomplete referral information d. Lengthy waiting period from referral to service e. Adult client or family eligible, but not ready or willing to engage in service f. Other (specify): ______________________________________________ 7. How else is the community partner involved with the Family Connection project? (check all that apply) a. Grantee organization staff member(s) co-located at community partner b. Community partner staff member(s) co-located at grantee organization c. Participates in Family Connection project advisory group d. Participates in Family Connection project management and/or team meetings e. Provides funding f. Provides direct service provision to Family Connection project clients g. Other (specify): ______________________________________________ D. FAMILY CONNECTION PROJECT COMMUNICATION The following questions ask about the community partner’s communication processes with the grantee organization. The “grantee organization” is the recipient of Family Connection Federal funding. The grantee organization is [INSERT GRANTEE ORGANIZATION NAME HERE]. 1. How clear are the roles and responsibilities between the community partner and the Family Connection project? a. Very clear b. For the most part clear c. Somewhat clear d. Slightly clear e. Not at all clear Community Partner 4 January 27, 2015 f. Do not know [If VERY CLEAR, skip to D3]. 2. How can clarity in roles and responsibilities be improved? __________________________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ 3. Is there a process for regular communication regarding successes and concerns about the Family Connection project between the community partner and the grantee organization? a. Yes b. No c. Do not know [If NO or DO NOT KNOW, skip to D5] 4. What communication mechanisms does the community partner experience as part of the Family Connection project? (check all that apply) a. Advisory board membership b. Case-specific meetings/conference calls c. Community meetings d. E-mail memos and updates e. General project meetings/conference calls – Regularly scheduled f. General project meetings/conference calls – As needed g. Reports (e.g., progress reports, evaluation findings, etc.) h. Surveys i. Informal channels j. Other (specify): ___________________________________________ 5. Is there enough communication between the Family Connection project and the community partner to keep you well-informed about the project? a. Yes b. No [If YES, skip to D7]. 6. How can communication be improved? __________________________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ 7. Did the community partner have an opportunity to provide its perspective in planning the Family Connection project? This may include input provided during the grant application process or during the 3-month start-up period. a. Yes b. No c. Do not know Community Partner 5 January 27, 2015 [If NO or DO NOT KNOW, skip to D9]. 8. Through what channels did the community partner provide its perspective in planning the Family Connection project? (check all that apply) a. Advisory board meetings b. Community meetings c. Family Connection project planning meetings d. One-on-one telephone calls with project planners e. E-mails with project planners f. Surveys g. Informal channels h. Other (specify): ___________________________________________ 9. Has the community partner had an opportunity to provide ongoing feedback to the Family Connection project? a. Yes b. No c. Do not know [If NO or DO NOT KNOW, skip to D11]. 10. Through what channels has the community partner provided ongoing feedback to the Family Connection project? (check all that apply) a. Advisory board meetings b. Case-specific meetings/conference calls c. Community meetings d. General project meetings/conference calls – Regularly scheduled e. General project meetings/conference calls – As needed f. Feedback on reports (e.g., progress reports, evaluation findings, etc.) g. One-on-one telephone calls with project managers and/or staff members h. E-mails with project managers and/or staff members i. Surveys j. Informal channels k. Other (specify): ___________________________________________ 11. Has the Family Connection project changed the community partner’s relationship with the grantee organization? a. Relationship is much more positive b. Relationship is more positive c. No evident change d. Relationship is more negative e. Relationship is much more negative f. Do not know [If DO NOT KNOW, skip to Section E]. 12. What factors have contributed to positive aspects of the relationship? Community Partner 6 January 27, 2015 __________________________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ 13. What challenges has the community partner experienced with the grantee? __________________________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ 14. How have the challenges been addressed? __________________________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ E. LOCAL EVALUATION The following questions ask about your role in the local evaluation of the Family Connection project, and how evaluation results are shared with project team members and partners. 1. Do you have a role in the local evaluation (e.g., provide administrative data to evaluation team, provide data from evaluation activities that you participate in, such as completing a collaboration survey, participate in evaluation-related meetings)? a. Yes b. No [If NO skip to E3]. 2. What is your role in the local evaluation? (Check all that apply) a. Input on evaluation planning and design b. Input on instrument selection and design c. Collect primary data (e.g., administer surveys, conduct interviews, etc.) d. Provide secondary/administrative child welfare data (e.g., SACWIS) e. Provide own data from evaluation activities (e.g., training or collaboration surveys) f. Evaluation data entry g. Evaluation data analysis and/or reporting h. Evaluation report review and feedback i. Other (specify): ___________________________________________ 3. Has evaluation information (e.g., progress, results) been shared with the community partner? a. Yes b. No c. Do not know [If NO or DO NOT KNOW skip to D6]. 4. How has evaluation information been shared? (Check all that apply) a. Written reports, memos, or other documents Community Partner 7 January 27, 2015 b. c. d. e. In-person presentations (during meetings) Virtual presentations (via webinars) Other (specify): ___________________________________________ Do not know 5. How often is evaluation information shared? (Check all that apply) a. Weekly b. Monthly c. Quarterly d. Semi-annually e. Annually f. Do not know g. Other (specify): ___________________________________________ h. Do not know 6. Have evaluation results from the Family Connection project been used by the community partner? a. Yes b. No c. Do not know [If NO or DO NOT KNOW, skip to SECTION F]. 7. Please provide up to three examples of how Family Connection evaluation data were used by the community partner: Example 1: ________________________________________________________________________ Example 2: ________________________________________________________________________ Example 3: ________________________________________________________________________ F. PROJECT IMPACT The following questions ask about your perception of the Family Connection project’s achievements, how the project has influenced the public child welfare agency, and how the project has affected the broader child welfare system. 1. Has the Family Connection project influenced the larger child welfare system? a. Very positively b. Positively c. No evident influence d. Negatively e. Very negatively f. Do not know [If DO NOT KNOW, skip to F3]. 2. How has the Family Connection project influenced the larger child welfare system? a. Fills a service gap b. Increased awareness of Family Connection services among child welfare providers Community Partner 8 January 27, 2015 c. d. e. f. g. h. i. j. Increased communication and coordination among child welfare providers Increased family engagement and involvement in services Increased permanent placements Influenced child welfare social workers in regard to Family Connection services Reduced the number of child removals Reduced the number of foster care entries Reduced time spent in foster care Other (specify): ___________________________________________ 3. To what extent has the community partner integrated elements of the Family Connection project into its own service delivery system? a. To a great extent b. Somewhat c. Very little d. Not at all e. Do not know [If NOT AT ALL or DO NOT KNOW, skip to F5]. 4. What are the project’s most important accomplishments? Accomplishment 1: __________________________________________________________________ Accomplishment 2: __________________________________________________________________ Accomplishment 3: __________________________________________________________________ 5. Are there benefits to developing partnerships between public child welfare agencies and private / not-for-profit agencies? a. Yes b. No [If NO, skip to F7]. 6. Please list up to three benefits of developing partnerships between public child welfare agencies and private / not-for-profit agencies. Benefit 1: _________________________________________________________________________ Benefit 2: _________________________________________________________________________ Benefit 3: _________________________________________________________________________ 7. How can collaboration between public child welfare agencies and private/not-for-profit agencies be strengthened? _________________________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ 8. Is there anything else you would like to add? __________________________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ Community Partner 9 January 27, 2015 Thank you for your survey response! Community Partner 10
| File Type | application/pdf |
| File Title | Appendix Ii |
| File Modified | 2015-01-29 |
| File Created | 2015-01-28 |