The State Child Welfare Information Systems Project Manager’s Guide is a reference for individuals involved with the development and management of State child welfare automation projects. For the purposes of this document, the Project Manager (PM) refers to the individual who has overall responsibility for managing the State child welfare case management information system development and/or maintenance project. Each section of the document highlights issues and challenges that project staff may encounter during different phases of the Systems Development Life Cycle (SDLC). Child welfare information system projects are generally large, complex and expensive undertakings for public sector agencies; therefore, it is prudent for States engaged in IT projects to share their experiences. The information, illustrations, and tips contained in the Guide were derived from numerous projects implemented across the United States since the early 1990’s.

The Planning section (Section 1) describes activities that assist in defining the over-all scope of the project. This decision will determine the framework for the remainder of the project as well as the procurement and contract negotiation processes.

Section 3 identifies technical issues that must be considered over the life of the project, such as the identification of business rules which must be translated into software requirements and managing the conversion of existing data from legacy computer systems.

Managing project risk is discussed in Section 4 and includes suggestions for managing the unexpected situations that arise during the life of any project. The business of child welfare is dynamic and constantly changing. Project staff should anticipate and attempt to understand how the business will affect the project and how the project will ultimately change the business and the agency.

Section 5 identifies and describes critical documents and statistical reports that are required by the Department of Health and Human Services, Administration for Families and Children.

The final section contains suggestions and tips from State child welfare information systems projects that have been implemented successfully.

Please note that links to websites included in this document are subject to change and will be updated periodically.

I. Introduction

In their efforts to provide effective and efficient child welfare services, States continue to improve and enhance the automated systems supporting child welfare practices, and most States strive to meet the requirements of a Statewide Automated Child Welfare Information System (SACWIS). Whether a State builds a SACWIS or a non-SACWIS application, the U. S. Department of Health and Human Services (HHS) will provide funding for approved information systems development and operation.

Development efforts across the States vary dramatically and include disparate initiatives such as adding new functionality to meet specific SACWIS requirements, moving from a client-server application to a web-based application, and making modest enhancements to the existing system in response to needs identified by system users. While the development work differs, States embrace a System Development Lifecycle (SDLC) methodology that includes gathering stakeholder needs, developing user requirements, system design, system development, testing, implementation, and maintenance.

The large numbers of activities associated with the lifecycle of child welfare information systems projects are both challenging and complicated. According to the Standish Group’s 2004 CHAOS Report, only 34% of all information technology (IT) projects are successful (delivered on time, on budget, with required features and functions); while 51% are challenged (late, over budget, and/or with fewer than the required features and functions); and 15% fail (canceled prior to completion or delivered and never used).

This State Child Welfare Information Systems Project Manager’s Guide identifies, describes, and offers practical guidance to project managers to assist in effectively addressing the challenges associated with a child welfare information systems project. The Guide covers a broad range of topics at a high level and offers references to existing resource materials and training courses that offer more in-depth information on many topics.

II. Planning

Effective planning may be the most important task performed by project managers. This section discusses project definition, project planning, Federal approval, the definition of resource needs, and the procurement of services.

A. Project Definition

Successful planning begins with a clear definition of the project. To achieve this goal, business and technical drivers, project outcomes, and business objectives for the SACWIS project should be defined and understood. During the project definition phase, States should assign and engage a project manager and the executive sponsor and identify other important stakeholders, including key senior managers, business unit leaders, and other internal and external stakeholders who will help define the scope of the project and influence project success.

TIPS:

Identify and adopt industry standards that will be used to develop processes and documentation that will be used for the project. Creating and adopting consistent, repeatable practices and standards for all lifecycle activities - including project planning, definition, execution, risk management, and closure - can assist project managers in meeting the demand for IT-enabled child welfare information system business and technical capabilities. A disciplined approach can dramatically increase the likelihood of project success.

The project manager should identify potential stakeholders early in the project definition phase to ensure that the appropriate user community, including business, program, and technical representatives, are involved in project definition, execution, communication, and funding activities. The stakeholder identification process should include role definition, time commitments, and the stakeholders’ areas of expertise as they relate to the project.

Assemble a project steering committee that consists of stakeholders.

Create formal brainstorming sessions with the executive sponsors and other key agency personnel to identify guiding principles for the project, identify mandatory functionality, and outline any known project constraints, including budget and technical constraints.

The first phase of a child welfare SDLC consists of gathering, assessing, and prioritizing of stakeholder needs and should involve many different stakeholders. A formal “statement of needs” document should be created out of this process and then referenced throughout the development effort. A useful needs statement focuses on desired outcomes rather than technical requirements, incorporates both short- and long-term goals for the requested solution, and defines the criteria on which a State bases its acceptance or rejection.

The next step in the project definition phase is developing user requirements which transform business objectives into statements that describe a system condition or capability.

TIPS:

Taking adequate time to define the business or service requirements pays off in the long run. The cost of making changes increases as the project moves through the SDLC. For example, adding a new requirement after system development and testing is significantly more expensive than making the change to the user requirements at the beginning. Changing a requirements document may take minutes or at most hours, while changing the system design document, software code, test scripts, and user manuals could take days, weeks or months.

Involving multiple stakeholders in a structured requirements assessment process facilitates the development of an accurate and comprehensive statement of needs. In general, it is better to cast a wide net, include as many stakeholders as possible, and then prioritize needs.

When defining requirements, it is important to limit the scope to articulating business objectives and avoid making decisions about how to meet those objectives.

ADDITIONAL RESOURCES:

An online course about gathering stakeholder needs and developing user requirements: Getting IT Right and Keeping It Right: Processes, Strategies, and Tools to Develop and Manage Requirements: http://www.state-itc.org/cw_resources.html

Webcast of the 2006 Child Welfare Regional Training: Getting IT Right and Keeping It Right: Processes, Strategies, and Tools to Develop and Manage Requirements: http://www.state-itc.org/rtc2006/recordings/index.html.

B. Planning and Approval

After defining the scope of the project, States must identify a path to achieve their objectives and obtain Federal approval for their approach if they desire the Federal government to participate in funding the planning, development, and operation of their system. This effort includes analyzing alternatives, choosing an approach to obtain the desired functionality, developing a business case, and receiving Federal approval to move forward.

This process presents the opportunity to evaluate market trends such as technical innovations, to evaluate the competitive environment, and to include qualitative and quantitative benefits of the proposed system. The business case should include project cost estimates, a cost-benefit analysis, and a discussion regarding alignment of strategy with any existing or applicable standards.

A critical deliverable in this stage is creating the business case for the project, which will serve as a key investment decision driver for the project’s executive sponsor and the Administration for Children and Families (ACF). The business case describes the justification for the project in terms of value to be added to the business as a result of the project outcomes. The business case should also incorporate any specific goals of the agency and benefits of the proposed system to clients, caseworkers, or other agency staff.

Planning and approval require cooperation and involvement of several different agency executives, including the Commissioner/Secretary, the Chief Information Officer (CIO), and the Chief Financial Officer (CFO).

TIPS:

Procurement planning is the process of identifying which project needs can be met best by procuring products or services outside the project’s organization or agency. It involves considering whether to procure, how to procure, what to procure, how much to procure, and when to procure.

Obtaining detailed information during the planning stage helps in subsequent phases of the project. The information collected helps to determine the State agency’s strategic direction and can be used in developing the Implementation Advance Planning Document (IAPD) for Federal approval. The estimated, detailed project cost should incorporate a lifecycle perspective and include costs for system development, implementation, and the estimated cost of maintaining the system over time.

Identify and define metrics for measuring project benchmarks as early as possible. A Cost-Benefit Analysis (CBA) is a Federal requirement of an approved State SACWIS project. During the planning stage, the initial analysis and quantification of project benefits capturing financial and nonfinancial impacts of the project are critical to meeting future deliverables and expectations.

Identify your State’s technical standards or preferences (if any) and the child welfare policy and practice standards. Create a checklist of desired functionality that can be included in an eventual Request for Proposals. The Children’s Bureau encourages States to allow vendors to propose solutions that meet a State’s defined policy, practice and standards without locking the vendors into a specific approach, transfer, new development or COTS. Presenting a strategic business case supported the State’s problem statement, policy/practice requirements and State standards will assist in gaining Federal ACF project approval.

ADDITIONAL RESOURCES:

“Advance Planning Document – 101” video recording of presentation by Tom Wetterhan, ACF Division of State Systems, at the 2006 Child Welfare National Training Conference: http://www.state-itc.org/Webinars_Webcasts/Navigation/Resource_Events/NTC/NTC2006_accessible_index_webcasts.html

“Cost-Benefit Analysis – 101” video recording of presentation by Tom Wetterhan, ACF Division of State Systems, at the 2006 Child Welfare National Training Conference: http://www.state-itc.org/Webinars_Webcasts/Navigation/Resource_Events/NTC/NTC2006_accessible_index_webcasts.html

Online courses on the Advance Planning Document (APD) process and CBA: http://www.state-itc.org/cw_resources.html

Presentations on the APD and CBA processes from the 2006 National Child Welfare Training conference: http://www.state‑itc.org/ntc2006/accessible/ntc2006_accessible_index.html

C. Resource Planning

Resource planning involves determining the type, quantity, and quality of physical resources (people, equipment, materials) necessary to perform project activities. The objective of this step is to identify high-level resource requirements for the composition of the project. Adequate resources must be available to complete the project requirements, which will help to fully determine the demand for specific skills and the resources needed to successfully complete the project.

The following table is an example of a project management resource planning or forecasting framework, which may be used to identify both roles and estimated time required on a child welfare information systems project. Once completed, the resource plan can be used to determine whether internal resources are available or external sources will need to be engaged.

Estimated Resources Required for SACWIS Project

Resource (Name or Number)	Role	Estimated Time Required on Project
Child Welfare Division Administrator or Commissioner	Executive Sponsor	5%
One	Project Manager	100%
One	Program Manager	100%
One	Technical Manager	100%
Executive Sponsor, Vendor Management, Project Manager, Chief Technology Officer	Steering Committee	5%
Other Department or External Stakeholders	Business Partners	10%
Multiple	Child Welfare Business Analysts	100%
One	Administrative Assistant	100%
Multiple	State Technical Support including Database Administrator	30%
Multiple for Requirements Gathering and User Acceptance Testing	Subject Matter Experts	30%
Multiple	Implementation Support, including Application Trainers, Help Line	100%
One	Project Financial Accountant	35%
Legal Office	Contract Negotiation, other legal issues	5%

ADDITIONAL RESOURCES:

Webcasts and materials from the “Resource Estimation and Allocation” and “Driving Quality” presentations at the 6^th Annual National Child Welfare IT Managers’ Meeting: http://www.state-itc.org/Webinars_Webcasts/Navigation/Resource_Events/NTC/NTC2007_accessible_index_webcasts.html

D. Developing the Request for Proposal

Typically, the procurement vehicle to acquire vendor services to develop and implement a child welfare information system is a Request for Proposal (RFP). Although often time-consuming and labor-intensive, the RFP continues to provide States with the most straightforward process for the acquisition of large-scale IT purchases. States need to consider several important factors in developing an RFP. Perhaps the most critical factor is ensuring that the RFP is a complete, precise, and well-written document that explicitly articulates the agency’s business needs, standards and problem definition, avoiding the description of a specific solution. Also, keep in mind that any proposal leading to a contract should be submitted to ACF for prior written approval before the State can proceed with publishing the RFP. Please see the APD regulations at 45 CFR 95 Subpart F for additional guidance.

This task can be pursued in a variety of ways. If the expertise and resources exist within the State organization, then developing the RFP documents can be accomplished internally. If expertise and/or resources are limited, but well-established procurement processes exist, the organization may be served better by outsourcing RFP development.

When developing the RFP, the organization should focus primarily on defining the problems, the requirements and any State or Federal standards in order to provide the vendors with the information necessary to propose various solutions. ACYF-CB-PI-07-10 states, “In order to promote maximum practical open and free competition, ACYF encourages States to create planning and procurement documents that clearly define its business problems that must be solved and/or business opportunities that could be realized by using the sought after information technology. States should focus on defining detailed business and technical requirements, and specify the parameters and limitations within which States must operate. By conveying this type of information in their procurement documents, States enable potential bidders to formulate and propose a variety of solutions targeted to resolve the specified business problems and/or provide the specified business opportunities referenced in the States' procurement documents within the parameters articulated by the States.”

The RFP includes two primary topics: the Statement of Work (SOW) and the instructions to bidders (vendors). The SOW includes information such as the work to be performed by the vendor, when work is due, how the work will be measured for acceptance, and the working relationship between the vendor and your organization. The SOW becomes a major part of the eventual contract between the vendor and the organization. The instructions to bidders, the other primary topic of an RFP, provide vendors with the information they need to follow in creating and submitting a bid proposal.

Successful RFPs are those documents that do the following:

Contain the input of relevant stakeholders.
Provide a well-conceived vision of the client’s desired outcome and business needs.
Provide detailed information that will result in the best solutions from the vendors at the best available price.
Define the criteria on which the State bases its acceptance or rejection of the vendor’s proposal.
Contain information on the categories that each proposal will be evaluated against and the process that your organization will follow in selecting the best candidate vendor.
Include reasonable terms and conditions, which will increase interest and result in more vendor responses.

TIPS:

Choosing the right procurement model for your organization is one of the most important tasks an organization can undertake. Why? Because the right solution for your organization will have a profound impact on controlling the schedule, costs, and quality of a project.

There is a direct relationship between the completeness of your RFP and the completeness of the bid proposal received. Unclear business requirements create risk for the vendor, and risk can drive up the cost of the bid and result in a less than satisfactory response.

An early discussion and decision regarding project space should occur during development of the RFP. The State must decide whether to assume responsibility for the entire office space, office equipment, project staff (including the vendor) computers, project file servers, related software and licenses, and telecommunication and connectivity needs or to include these items as vendor requirements.

Regardless of which entity bears the burden, the cost of space, hardware, software, and personnel must be factored into the entire project lifecycle. While the child welfare information system project is just beginning, the project manager must anticipate the time when the system becomes fully operational. This effort will help the State decide who assumes responsibility for project space and equipment. Will an offsite project location be needed for the future application maintenance staff? Will maintenance be assumed by State staff or be outsourced to a vendor? Will the State provide necessary software and hardware? Generally State governments have the capacity to secure these support resources at a lower cost than a vendor. This strategy can make a significant difference in the vendor’s cost proposal.

When it comes to the quality of work from a vendor, States typically get what they ask for in the procurement documents. To elicit what you want, ask for it in the RFP! Specify any existing development standards, usability requirements, testing guidelines, and Quality Assurance (QA) processes that may be in place, and require the vendor to adhere to these standards. If none exist, require vendors to discuss their existing procedures or proposed processes for the project in their response.

ADDITIONAL RESOURCES:

An online course, Public Sector IT Procurement, is located at: http://www.state-itc.org/cw_resources.html

E. Procurement Process and Selecting a Vendor

The underlying principle for the procurement process is to ensure that free and open competition occurs. Selecting a vendor can be a painstaking chore; however, the time invested in selecting the most qualified candidate can become one of the best investments that a project manager can make. Each State must follow its own clearly articulated RFP process. Within the existing framework, the success or failure of an RFP depends on a number of key success factors such as those described in this section.

1. Establishing an Effective Evaluation Team

Putting together the right team is one of the initial and most important steps in the RFP process. Drawing on the appropriate individuals that represent key stakeholder groups helps establish buy-in of key stakeholders, especially front-line workers or other users of the system. Additionally, large information systems are complex as are the issues related to the acquisition of any new IT architecture. To address this complexity, the ideal RFP team would include a procurement specialist, a policy representative, an end user, and an IT business analyst. Depending on the nature of the need and potential solutions, other key stakeholders need to be involved at different points in the process, including legal counsel, budget representatives, and more specialized IT staff, such as a security officer, systems architect, and database administrator.

The following are other considerations for the RFP team:

· One or more team members should be effective leaders and possess strong program management skills.

· Team members should be multifunctional and have abilities to write, analyze, and communicate.

· Team members should have experience in RFP development and IT development.

· Team members should possess relevant program knowledge and technical knowledge from all the programs affected by the procurement.

· Adequate training in writing successful RFPs and in evaluating responses is a must for team members.

· To avoid time delays, develop a list of alternates to stand in for members who cannot attend meetings.

· Develop a preferred method of communication; many States are effectively using a shared workspace that includes project documents, schedule, and online discussion capabilities.

2. Developing a Procurement Management Plan

Once the team is established and trained, the first order of business is creating a procurement management plan that describes all the phases and activities involved in the procurement process. The plan keeps the RFP team focused and prevents critical items from being overlooked and deliverable dates from being missed. Having the team develop the plan as one of its first activities engenders ownership among team members and helps identify the full breadth of necessary activities.

A project communication plan should be developed and implemented very early in the procurement process. Communication planning involves determining the information and communications needs of the stakeholders: who needs what information, when they will need it, and how it will be given to them. Also, a risk management process can be implemented during the procurement stage to facilitate the identification, escalation, and mitigation strategies relating to potential procurement risks. A risk management approach will assist in keeping the procurement on time and within budget. Both plans should be part of the project management portfolio and revised on an ongoing basis.

3. Establishing an Online RFP Documentation Library

Developing a library of governance documents enables team members to review, create, upload, check out, and modify documents. These documents may be contained within an agency’s Project Management Office, and the library may retain some or all of the following resources:

· RFPs and responses

· Requests for Information (RFIs) and responses

· Project Plan Templates and copies of plans from past IT projects

· Proposal Evaluation Criteria Templates

· Procurement training courses

· Blanket Purchase Agreements

· Statement of Needs

4. Determining Processes for Vendor Questions

Typically, once the RFP is released, vendors may request clarification of the information contained in any part or section within the document. The procedures for submitting questions should be included in the general information section of the RFP. The State is generally obligated to uniformly acknowledge and communicate any clarifications or changing requirements to all vendors who may be interested in bidding. Many States have implemented web-based procurement systems enabling the State to post responses whereby the vendors can browse the question and answer section and check whether any requirements have been amended since the RFP was released.

5. Planning a Bidders’ Conference

The State should determine whether a bidders’ conference will bring added value to either the State or to the vendors. It is common practice to invite only vendors who have submitted an “intent to bid” by the prescribed timeline to a bidders’ conference. However, a bidders’ conference may not prove to be of significant value because vendors are generally guarded about asking questions that may reveal a company strategy or a perceived competitive advantage.

6. Evaluation

States typically use a prescribed evaluation methodology that combines several elements, including cost, quality, references, and meeting performance bond and other threshold organizational requirements. While cost is a major factor in determining the eventual award, it is a best practice to withhold the bidders’ cost data from the evaluation team until all other scoring is complete. Because each member of the evaluation team typically scores the entire response, scoring of each bid for the business, technical, staffing, and quality requirements should be the primary priority. Once completed and the individual scores finalized and documented, then the cost information is provided to the evaluation team. The value analysis is the final step in completing the bid evaluation process and developing the recommendation to executive management.

The following recommendations can improve evaluations:

· Seek out referrals from current and past projects from the child welfare arena as well as other health and human services references for projects of similar size and complexity.

· Evaluate contractors’ technical and management processes, including their development and usability design methodologies, to ensure compliance with State and agency policies and procedures.

· Evaluate applicants’ organizational capacity. Purchasing a product or product license and IT services from a small firm can be risky for large, multiyear projects.

· Carefully consider the intellectual property tradeoffs of any IT investment and have a thorough understanding of Federal regulations regarding intellectual property rights and software ownership. Purchasing a COTS product may enable a State to achieve its business/service objectives more quickly. The State purchases a license to use the COTS, but the underlying code belongs to the company. Building processes on software licensed from a vendor is commonplace in State government, especially with the use of desktop applications; however, States need to be aware of their short- and long-term dependence on the product and the company that supports that product. Conversely, building custom applications or modifying applications available in the public domain can be costly and time-consuming.

· Refer to the ACF Program Instruction ACYF-CB-PI-07-10 for guidelines related to commercial-off-the shelf (COTS) software products: http://www.acf.hhs.gov/programs/cb/laws_policies/policy/pi/2007/pi0710.htm

· For products, consider having applicants conduct demonstrations.

· Oral briefings from the top few applicants offer an excellent opportunity to clarify unclear aspects of the proposal, get a better perspective about the key personnel included in the proposal, and test the knowledge of the key personnel.

· Not all procurements result in an award. If the proposals do not meet the business/service needs, then a new approach or a new request with a different scope may be necessary.

With all approvals in place, an “Intent to Award” document is sent to the selected vendor, and all other bidders are notified that a vendor has been selected. The letter of intent is a nonbinding document confirming the State’s desire to negotiate with the vendor for products and/or services. Standard State procurement rules provide a “grace period” whereby non-selected vendors can provide notification that they protest the award. There may be differing processes, procedures, and timeframes unique to your State so it is best to work directly with your procurement office on these issues. However, once the award is final, any bidder may request a debriefing meeting to discuss if and how they could improve future responses.

7. Best Value

The best value concept has taken hold in many States as a better alternative than best price, particularly in the area of IT services. Unlike best price, which means the lowest price at which a State can purchase goods or services, best value connotes a process for selecting the most advantageous solution by evaluating and comparing all relevant factors in addition to price.

Under this paradigm, a winning proposal may entail a higher price but provide greater quality and benefits for the State. Best value factors may include long-term project benefits, cost avoidance, increased productivity, maintenance and replacement costs, cost versus technical superiority tradeoffs, vendor support, and user satisfaction.

The following list describes several issues to consider when implementing a best-value approach:

· Evaluating best-value bids is more complicated than evaluating low-cost bids. Decision makers must make thoughtful decisions about the relative weight of different evaluation criteria. For example, what percentage of the overall evaluation score will the State allocate to cost of the solution, corporate qualifications, the technical approach, and understanding the business/service need?

· States need to make sure the data used to evaluate factors are reliable. Unless States clearly and carefully articulate evaluation standards, it can be easy to make subjective judgments. For example, do the number of hours spent on a project and the tasks within that project provide a more accurate accounting of the level of effort than the number of individuals and whether they are working full-time or part-time?

· States need to communicate the relevant factors that make up the evaluation criteria for a best-value bid to the vendor.

· The review team needs to be scrupulous in its use and documentation of rating factors. Inconsistent rating factors could lead to poor procurement choices and potential legal challenges that drive up costs and move projects off schedule.

8. Avoiding Common Mistakes

Many of the following tips are adapted from Winning E-Learning Proposals: The Art of Development and Delivery (Karl J. Kapp, J. Ross Publishing, 2003). Avoiding these common mistakes can help States effectively procure the products and services necessary to meet their business/service objectives.

· Poorly written or illogical content. Despite their enormous importance, RFPs are notoriously poorly written. Vendors are more likely to bid on an RFP that is well written. In addition to standard writing procedures, such as using a technical editor, writers of RFPs may want to include diagrams, examples, and reference additional, available documents such as an agency’s strategic plan, descriptions of the existing technical environment, and a synthesis of the stakeholder needs.

· Providing too little detail. Vendors cannot help meet business/service needs or solve problems if there is too little information about the current business process, technological infrastructure, or proposed budget. However, States should avoid providing unnecessary detail in the content of the proposal regarding solution design. Sharing information about the current technical environment helps vendors understand the gap between current operations and their proposed solution.

· Lack of imagination. The RFP process is a good time to brainstorm internally and think outside the box. As a Service-Oriented Architecture (SOA) approach gains hold among States, more RFPs are calling for web-based services that can be used across the enterprise by multiple programs. Additionally, the advent of framework technology creates new possibilities to leverage a single software product across multiple programs. Looking to the experience and solutions used in other industries with similar functions may produce new, innovative ideas.

· Poorly scoped. Poorly scoped RFPs typically overstate or understate the business/service needs and the level of effort. Discussing lessons learned with other States can help a State avoid this problem.

· Failing to account for business needs. If the RFP does not clearly include the statement of business needs and the desired outcomes, the quality of all other aspects of the RFP process does not matter. If vendors know the business needs driving your RFP, they can leverage their knowledge to help identify a solution.

· Avoiding the “cone of silence.” Many States maintain strict requirements related to interaction between State personnel and vendors, particularly during the RFP process. Vendors possess great knowledge of industry practices and technological capabilities that can be valuable to the RFP team in understanding the external environment and identifying potential options. Finding acceptable ways to get this information from vendors could significantly benefit the project. Options for interaction include issuing an RFI, issuing a draft RFP for comment, holding a pre-bid conference, and providing time for “discovery” for vendors to interview and observe State and local staff so that they better understand the business processes and challenges.

· Process timeline challenges. States spend a significant amount of time developing requests for proposals, including what seems like a logical and reasonable timeline for each stage of the RFP process. A too aggressive schedule may limit qualified vendors’ ability to evaluate and prepare a response. Conversely, if the schedule is too long, the resources a vendor had planned to deploy may no longer be available. Be diligent in sticking to your procurement process timeline, but also make sure that prospective vendors have ample time to develop the best response possible to meet your business needs.

F. Understanding SACWIS IT Acquisition Requirements

The SACWIS project manager is responsible for managing, monitoring, and approving related IT costs associated with the State’s SACWIS project and understanding the Federal requirements and thresholds pertaining to the project IT equipment and services to be acquired. The Project Manager should work closely with your agency’s administrative and financial representatives, who collaborate on developing and maintaining the Annual IAPD.

ACF provides guidance to the States (and territories) regarding the regulatory requirements for SACWIS related State information technology and services acquisitions. To support this process ACF has developed an “acquisition checklist” which provides an optional tool for States to use to provide assurances that an acquisition of automated data processing equipment and/or services complies with all Federal regulations and policies. The Federal Department(s), in accordance with the regulations at 45 CFR 95.611, may grant an exemption from prior approval for an acquisition document based on a State’s favorable responses to this checklist. This checklist may be used for Requests for Proposal, Requests for Quote, Invitations to Bid, or similar State and Territory acquisition documents; however it may not be submitted for contracts or Advance Planning Documents that require Federal prior approval.

ADDITIONAL RESOURCES:

· The Information Memorandum associated with the IT acquisition checklist is IM-05-02, which can be accessed at: http://www.acf.hhs.gov/programs/cb/laws_policies/policy/im/2005/im0502.htm

· Clarifying information regarding the use of State master contracts to acquire State information technology products and services can be found in Information Memorandum IM-05-04, which can be accessed at: http://www.acf.hhs.gov/programs/cb/laws_policies/policy/im/2005/im0504.htm

G. Contract Negotiation

Once the RFP responses have been evaluated and a potential vendor selected, there are several important considerations and steps that remain before the project can officially begin:

· State procurement rules and requirements must be identified, resolved, and satisfied.

· ACF must be in agreement that the completed procurement process and potential intent to award meet the principles of a fair and open competition.

· The contract must be negotiated and agreement reached.

· State and Federal approvals of the contract must be secured before it can be executed and signed leading up to the project start.

ACF, as your funding partner, must approve the contract. The “project contract” may include several artifacts, such as a final agreed upon and detailed SOW and deliverables, a detailed project plan, a negotiated and accepted final cost proposal, an instrument containing the State’s and the vendor’s legal obligations and limitations, detailed project processes and associated remedy agreements.

The project manager needs to be proactive in participating in the contract negotiation to ensure that the agency’s business needs are well represented. The project manager generally will coordinate the negotiation meetings, communicate negotiation status to the executive sponsor, promote any issues for resolution, and make sure the State’s key negotiators (office of legal representatives and the SACWIS project legal, program, and technical representatives) are:

· Familiar with State and agency procurement rules, as well as any expectations regarding their roles and responsibilities.

· Fully aware of their individual areas of expertise and responsibility pertaining to the requirements, the SOW, the terms and conditions in the RFP, and the corresponding vendor responses to (and negotiation of) those requirements.

· Prepared to support the project manager in making final trade-off or other negotiation decisions.

TIPS:

The project manager and a representative from the agency’s legal office are key participants in the contract negotiation process. This tandem must be keenly aware of the State’s legal requirements, contractual language and terminology, required terms and conditions, and the process for Best and Final Offer (BAFO) negotiations.

Prior to signing the contract, review all the requirements contained in the RFP SOW with the candidate vendor. This review provides a forum and opportunity to eliminate the trap of “this is what I said, but this is what I really meant.” It is critical to eliminate as many ambiguities as possible and to clarify expectations. This process also begins to establish the necessary partnership between the organization’s contract administrator, the State’s project manager, and the vendor’s key staff.

A good negotiation team should have three goals in mind:

To reach satisfying agreements for both parties
To reach agreements efficiently
To conclude negotiations on a positive note

H. Project Start-Up

During this phase, project managers begin the day-to-day project management tasks and the responsibility of vendor contract administration. Put simply, contract administration is managing the project lifecycle relationship with the vendor. Contract administration includes monitoring and enforcing the processes, procedures, schedules, deliverables, and responsibilities negotiated within the contract. It also includes ensuring compliance with the terms and conditions and documenting and agreeing on any changes that may arise during its implementation or execution. It can be summarized as the process of efficiently managing the execution of the contract to maximize financial and operational performance and to minimize risk to both the agency and the vendor.

TIPS:

Timely and thorough completion of the procurement processes contributes to a positive project initiation. Unforeseen delays introduce additional stress and strain in meeting stringent project timelines and deliverables, implementation schedules, and end-user/stakeholder project acceptance.

The experience of some State child welfare information system projects supports a practice of co-locating an integrated project team composed of State and vendor staff. Co-location provides an immediate opportunity to create an important professional partnership between State and vendor staff, supports ease of communication, builds process familiarity, opens up issue resolution, provides the capacity for knowledge transfer, and removes external distractions.

Make sure that all vendor key staff (as agreed to in the contract) are on site and ready to begin work immediately because time equals money!

Holding a project kick-off meeting with key internal and external stakeholders can help create excitement about the project, manage expectations, and introduce the vendor.

ADDITIONAL RESOURCES:

Overview of Project Start-Up, Project Management Methodology, State of Kansas:

www.da.ks.gov/kito/Rel23/4startup.doc.

III. Technical Considerations

A. Requirements Definition

Regardless of the eventual proposed solution to the State’s business requirements s(transfer, build, or COTS), the project team should focus on the needs of the end users, State and Federal business requirements in developing system requirements. Before any development or customization occurs, the project manager needs to institute a process to gather and prioritize stakeholder (including end users) needs and identify system constraints.

The most common requirements-related problems are:

Stakeholder problems and requirements are not understood.
Stakeholder problems and requirements are not fully documented.
Agreement has not been reached on the validity or the content of the requirements.
Changes to an agreed-to requirements document are not controlled and communicated.
The project has document version control problems and poor communication.

TIPS:

Before developing an RFP, take the time to fully understand the unique needs and problems of all the stakeholders which you must satisfy. Once user needs, problems and State standards are documented, continue to communicate with vendors so that they can propose the solutions that will best meet the users’ needs (new development, transfer or COTS).

Use of focus groups, interviews and requirement identification sessions are good mechanisms to communicate, understand, and document stakeholder needs.

Poorly defined requirements result in serious problems later on. Therefore, define your stakeholder problems and needs in the requirements definition phase of the software development process. Many software products are developed with weak, outdated, or nonexistent technical or functional requirements. This situation can have a devastating effect on a project’s costs, schedule, quality, and customer satisfaction.

The product requirements document represents the foundation from which the product’s solution is developed. The time and energy invested in ensuring a thorough review by the affected stakeholders may be the best investment that can be made throughout the SDLC.

ADDITIONAL RESOURCES:

Getting IT Right the First Time and Keeping It Right: Processes, Strategies, and Tools to Develop and Manage User Requirements, regional training sponsored by the U.S. Department of Health and Human Services, Administration for Children and Families, Children’s Bureau:

§ Materials: http://www.state-itc.org/Webinars_Webcasts/Navigation/Resource_Events/RTC/RTC2006_accessible_Presentation_webcasts.html

§ Webcast: http://www.state-itc.org/rtc2006/recordings/index.html

§ Online course: http://www.state-itc.org/cw_resources.html

B. Data and Organizational Readiness

Many States cite the following “lesson learned” from implementing their child welfare information systems: data readiness, data conversion, and data clean-up take much longer than estimated. In light of this challenge, States should consider conducting a data research effort to provide a foundation for developing a conversion plan and related documents, including a communication plan, a data conversion guide, a conversion work plan, and conversion worksheets. To mitigate risk and delay, these products should be developed early in the project timeline, even before an implementation vendor is chosen.

A conversion plan sets out a blueprint for conversion activities with the associated timeframes reflected in the project work plan. The conversion activities of the project work plan represent the steps involved in the conversion effort and the projected start and stop dates for those steps. The project plan should be updated with additional details and tasks as they are added to conversion activities.

The following deliverables are representative of conversion planning:

· Conversion Plan. This plan includes the following: a business case for conversion planning, conversion objectives and scope, an overview of the conversion phases, conversion methodology, and a conversion work plan.

· Conversion Specifications Document. This document includes data cross-references (e.g., data dictionary elements, edits/conversion rules); data extract and load sequences for current system(s); manual conversion of data collected; and conversion specifications for automated and manual system(s). Program specifications should include detailed program functions, requisite input data and output data, and the detailed processing that will be required for each program.

· Automated Conversion Software. A State may be able to employ automated system conversion software to support conversion. States need to appropriately train staff on the use of any conversion software and institute quality control checks.

· Manual Conversion Software. Many conversions require a manual process using common desktop software applications. The process for using manual conversion software needs to be well documented and all participants appropriately trained. A QA check of this effort is critical.

· Conversion System Test Results Report. This document records the results of the System Test Phase, including expected and actual results, issues, issue resolutions, and changes needed to the conversion processes as a result of System Test.

· Conversion Phase Signoff. This signoff (by the project manager) signifies the acceptance of the results of the data conversion and the completion of conversion activities.

States should also develop a communications plan that includes messages to promote the benefits and normalize the workload impacts that the new child welfare information system will introduce to the end users. Good communication lays the foundation for positive change, prepares the end users to participate productively, and establishes buy-in and cooperation. If end users can understand the long-term benefits to their work, then they will be more inclined to accept the burdens related to the comprehensive and complex data conversion activities.

TIPS:

An excellent communication medium for providing timely information to the end users and other stakeholders is a project website where project goals, schedules, data readiness expectations, data conversion processes, training, and other pertinent information can be instantly published, updated, and accessed.

Use regional/field staff in meetings and planning sessions to help identify local practices and workflow and to establish buy-in. Focus groups can be a great source for understanding and doing both user and task analysis.

C. New System Implementation

A successful implementation requires careful planning that mitigates risk and focuses on the needs of system users. States are strongly encouraged to use a phased implementation strategy by piloting their new system to demonstrate that the new system design works and meets the organization’s business and technical requirements. A successful pilot reduces the organization’s risk during full-scale statewide deployment.

Moving beyond the pilot phase requires a carefully defined, communicated, and rigorous statewide implementation plan. Plan elements may include:

· A schedule that allows for the up-front development of system modifications, conversion routines, and interface programs.

· A phased implementation approach that rolls out system implementation across logical geographic boundaries, allowing for training and onsite support to be targeted and manageable.

· A training plan for end users, help-desk staff, and technical staff supporting the system.

· Activities that take advantage of lessons learned (technical, training, programmatic) from pilot implementation.

· Deployment of implementation teams to train system users and provide onsite technical assistance.

· Identification of key end-user partners who are well respected and leaders in statewide child welfare organizations. These early adapters and implementers of the SACWIS application can positively influence others’ participation.

· System performance monitoring tools, metrics, and reports to track system performance as new users are added.

· On-call schedule for key technical staff to address system implementation issues that may occur during roll-out.

TIPS:

Understand your project’s impacts on other information systems, particularly those impacted by the mandatory SACWIS interfaces:

· Communicate with other IT teams regularly and consider integrating key managers across affected IT systems into a larger project steering committee.

· Plan careful coordination among project teams and between system users and developers.

· Develop a master schedule of system releases across affected systems.

· Designate a project integration manager.

· Know the impacts of the project under development on current business. For example, the agency is planning an email upgrade at the same time as the SACWIS implementation.

· Understand hardware, software, or communications/network connectivity requirements of the new system.

· Communicate infrastructure standards and guide infrastructure acquisition to ensure system compatibility and performance to counties in a State-managed, county-administered governance structure.

Understand the landscape:

· Carefully assess readiness of the central office (agency) and local offices.

· Conduct site reviews.

· Know whether legislation, policy, or procedures changes will be required. Start early - get a handle on what changes are “must haves” versus “nice to haves.”

· Identify and understand any upcoming legislative initiatives and procedural hurdles.

· Develop needed policies and procedures prior to application design and development for inclusion in the new system.

· Work with stakeholders, including court monitors and private agencies, to ensure that new policies and procedures are followed after system implementation.

Assemble the right implementation team:

· People who are willing and able to handle multiple tasks simultaneously and can work under pressure to meet critical deadlines.

· Technical experts who understand the technical aspects of the new development (even better if they know child welfare business models)

· Trainers, who can establish a timeline for training, prepare course materials and curriculum, organize training facilities, and help ensure that users are prepared when the system is installed.

· Consider involving trainers in application design sessions and as application testers.

· Expert or “power” users of the existing system.

· Experts in the business areas, including policy experts, financial staff, private provider staff, and data analysis/reporting staff.

· Generalists who can fill in where needed and integrate the efforts of the other team members.

· Include technical architecture, network, and database administration representation on the implementation team.

· Avoid using subject matter experts on the team who are assigned day-to-day casework.

· Good testers for quick-fix items.

· Onsite support during implementation.

Establish good communication with system stakeholders:

· Plan briefings to senior/executive level staff to introduce the project, identify the project’s guiding principles, establish a steering committee, and hold regular meetings to update project status.

· Plan a kick-off meeting that articulates project goals, objectives, tasks, timelines, responsible parties, and communicates benefits by stakeholder group. The kick-off is a great opportunity to manage expectations, introduce vendor partner(s), and allow senior agency staff to send a positive message to the stakeholders.

· Disseminate information to users via formal letters, memos, system news, websites, project networks, and emails.

· Consider giving problem agencies or offices special attention with phone calls, onsite orientation, and onsite technical assistance during implementation.

· Convene advisory groups, pilot users, and user groups.

· Ensure that the help desk is adequately prepared and staffed during the initial phase of implementation.

· Assign a manager to handle communications and cultural change.

· Ensure that the needs of county and private agency staff performing case management activities are address and adequately considered in all phases of the project.

D. Transitioning to Maintenance

The transition from development to production maintenance status occurs once the State rolls out equipment and infrastructure. Once the pilot site(s) implementation is completed, the application has transitioned to maintenance mode.. This transition needs to be carefully monitored and can be difficult and challenging as several events are occurring simultaneously. Statewide implementation is proceeding, system changes and enhancements are occurring, production batch processing is running and end users are expecting problem resolution and operational support and assistance. Thus it is critical that States carefully manage all of the concurrent development, implementation and production maintenance activities to avoid potential disaster.

Prior to the maintenance phase and during development it is important to focus significant energy and resources on designing and executing a comprehensive organizational change management plan. Putting a solid framework in place early in the project will assist in managing the overlapping development, implementation and maintenance activities, encourage early adoption of the new system and, engender enthusiasm as opposed to anxiety among end users.

When the SACWIS has been deployed to the first set of production users d the agency is faced with a new and robust system to maintain. Make sure the help desk for the system is ready to answer questions concerning the new system features. In addition, an ongoing training plan will be required because new end users will need training, and trainers will need to add and maintain training material covering the unending “system enhancements” to their regular course curriculum.

Similar scope expansion is required for technical support. The staff providing system maintenance and support must, rely on good system documentation to troubleshoot and fix system problems. It is strongly recommended that an organization develop system documentation standards so the support team can be assured that all relevant information is available for each system component and that this information can be accessed easily and updated as needed.

TIPS:

Ask, answer, and act on these critical questions:

· Has a maintenance staffing plan, including a system maintenance manager and support team, been put in place?

· Does the IT organization have the requisite staff with the necessary skills and competencies to maintain the system?

· When is the operative time to train and are there adequate resources available for the IT organization to assume system maintenance?

· Has a CBA been conducted that enables executive management to decide on whether to assume maintenance with internal IT staff or to outsource?

· Have the short-term and long-term resources necessary to maintain the system been obtained?

· Is the necessary technical infrastructure in place to support the system?

· Has the necessary knowledge transfer been completed?

· Is high-quality system documentation in place?

· Are end-user support systems in place? If so, have system support staff been trained to handle requests regarding the system?

· Has a process for the identification, prioritization, development, testing, and implementation of system enhancements been established?

· Is an effective training plan in place?

· Is there a formal process in place to provide feedback to the user community on the status of performance and enhancements?

· Is this system part of the State’s continuity of operations plan?

· Have the annual Federal APD and cost allocation plan updates been completed?

· Has a consensus-driven maintenance plan been developed?

Establish a cross-functional management team to guide maintenance activities, consisting of the following:

· Agency Chief Executive Officer (CEO) or Executive Sponsor

· CFO

· SACWIS Accountant

· SACWIS Functional Manager

· SACWIS Business Analyst(s)

· SACWIS Training Manager

· CIO

· Technical Application Development Manager

· Technical Infrastructure/Operations Support Manager

· Representatives from the user community, including private agency staff

Ensure that the following activities are addressed in the implementation contract or State implementation plan in order to expedite a seamless transition from development to maintenance:

· Include transition activities, intellectual property, and software in the requirements/contract for development.

· Ensure that the code is structured, well documented, and conforms to industry or State application standards.

· Eliminate system “bugs” to the fullest extent possible before the transition to maintenance team.

· The creation and existence of a “requirements traceability matrix” will provide an invaluable reference resource to IT staff assigned to the ongoing system maintenance responsibility.

· Ensure that all final acceptance criteria have been met.

· To the fullest extent possible, include maintenance staff in user requirements definition, general and technical design reviews, and system testing.

· Hold knowledge transfer sessions to review system documentation, including: Federal SACWIS requirements, Adoption and Foster Care Analysis and Reporting System (AFCARS), National Child Abuse and Neglect Data System (NCANDS), National Youth in Transition Database (NYTD) and other Federal reporting requirements, user requirements, the system general design document, the system technical design document, test plans and results, user manual and other user training materials, online help, and batch interface or operational run schedules.

· Review help-desk reports and error reports.

· Review the inventory of enhancements in progress or pending.

· Maintenance staff should go through end-user training and observe users in the field.

· Use job shadowing for all scheduled runs necessary for system operation and maintenance.

· Verify that the document library is complete and minimally includes:

§ All pertinent notes and documents from user requirements definition sessions

§ The definitive user requirements documentation

§ All general and technical design documentation

§ Batch operations user guide, including “run” schedules—daily, monthly, quarterly, and annually

· Verify that system support facilities and processes are in place and functional, including:

§ End-user help desk. Incident reporting, tracking, and resolution.

§ Application development testing and QA. Unit, integration, and user acceptance processes.

§ Change management. Prioritization, defect correction, and system enhancements.

§ Technical support. Batch operations, system access administration, database administration, back up, recovery and business resumption.

Include transition activities, intellectual property, and software in the vendor contract, as applicable.

E. Cost Allocation

At the point that any portion of a State’s SACWIS is considered to be operational (application or infrastructure), the State must begin allocating the cost of the system to agency programs that benefit from having the system in place.

To find information regarding a comprehensive overview of existing cost allocation requirements related to SACWIS operational costs, review Program Instruction ACYF-CB-PI-01-05. It explains those policies and reiterates the need to allocate SACWIS operational costs to the appropriate benefiting programs once any portion of the system or supporting network becomes operational.

TIPS:

Include a financial management representative as part of the team to assist in the transition to maintenance activities.

To facilitate cost allocation development and obtain approvals, frequently communicate with your Federal ACF Regional office.. Scheduling regular “touch-base” phone calls and sending draft documents in advance of “final” versions will expedite the approval process.

ADDITIONAL RESOURCES:

· Program Instruction ACYF-CB-PI-01-05 can be found at: http://www.acf.hhs.gov/programs/cb/laws_policies/policy/pi/2001/pi0105.htm

· A Cost Allocations Methodologies Toolkit is available on the Federal Guidance page of the DSS website at: http://www.acf.hhs.gov/programs/cb/systems/sacwis/federal.htm

· Presentation: “SACWIS Operational Costs -
Cost Allocation Considerations,” SACWIS Managers’ Conference Call, March 2004, posted on the National Resource Center for Child Welfare Data and Technology website at: http://www.nrccwdt.org/resources/sacwis_archive/presentations/sacwis_pres1.html

IV.Program Management Considerations

A. Risk Management

Risk management is defined as identifying a concern before it becomes a crisis or threat to the project. Risk management activities include assessing situations that have a potential adverse impact on the success of the project and taking steps to mitigate the potential risk. In some situations, the project manager can only reduce the consequences of the risk that cannot be avoided. Consequently, the goal of risk management is improving the likelihood of achieving the project’s goals and objectives by peeking over the horizon at the danger that may be looming. To assess the impact of a particular event or circumstance on project success, it is necessary to understand the goals and objectives of the project. The relationship between risk events can be mapped to which project objective(s) may be most adversely impacted. Risks that do not jeopardize project objectives can be eliminated from consideration or categorized as low risk, for which no actions may be required. Any risk with a high or medium probability of occurring should have a risk mitigation strategy identified. Planning precedes the accomplishment of work. Therefore, a high-level risk management plan should be developed to define the process that the SACWIS project will use for risk management and outline risk mitigation strategies.

1. Risk Identification

Risks must be identified before they can be managed. Risk identification consists of determining which risks are likely to affect the project and documenting the characteristics of each. Risk identification needs to be an ongoing part of the risk management process, as new risks are identified and existing risks are further clarified. Each risk should be assigned to an owner who is responsible for mitigation of the risk. Several techniques are available for identifying risks:

· Use of risk checklists

· Interviews with stakeholders

· Working group sessions

· Periodic meetings for an ongoing risk dialogue

· Surveys - electronic or paper

Risk categories are established to ensure that risks have been evaluated in all key areas impacting a given project. Risk categories also assist with the management and ownership/assignment of risks within each category. A typical SACWIS project may identify several categories of risks underneath the three primary project roles: Management, Technical, and Functional.

2. Risk Assessment

Risk assessment and analysis consists of converting risk data into risk decision-making information. Assessment ensures that the team is working on the right tasks. Risk analysis determines the relative priority and severity of identified risks to ensure that the project team and the stakeholders are taking appropriate actions to mitigate the most severe risks.

Risk analysis also consists of evaluating and defining the risk exposure (RE) associated with each identified risk. RE is computed using two factors: the probability (P) of a risk occurring and the impact (I) of that risk to project success based on the severity of the event.

Numerical values are assigned to both P and I, which are multiplied to determine the numeric RE for each risk. Assigning a numerical range also can be extrapolated to High, Medium, or Low, which equate to the risk score. Applying this commonly used methodology and evaluation for the relative ranking of risks ensures a standard and consistent approach for the relative ranking of risks. This ranking determines what, where, and when the project team and stakeholders should focus their risk mitigation efforts.

3. Risk Evaluation

Once an assessment or analysis has been made of the risks associated with the project, the risks are ranked according to severity and classification. Finally, a risk profile is developed, which documents the outcome of the risk assessment and analysis process and identifies risks that require management.

The risk management plan defines the SACWIS risk mitigation strategies that the project will use to manage risks. The following strategies can be used in risk mitigation:

· Acceptance. Consciously choose to live with the risk consequences. This applies only if the impact of the risk event is acceptable.

· Avoidance. Eliminate the risk - usually by changing the scope of objectives of the project.

· Protection. Create a backup or contingency plan to lessen the impact of the risk event.

· Reduction. Reduce either the P or I of the risk. This is the most common mitigation strategy.

· Research. Gather more information about the risk and/or possible mitigation approaches.

· Risk Reserve. Provide a contingency in the plan as a margin for error - most commonly used to mitigate schedule- and budget-related risks.

· Transfer. Shift the risk to another person, group, or organization. (However, ultimately the responsibility remains with the project team/stakeholders.)

4. Risk Mitigation Plan

This is the development of the risk mitigation plan to reduce, contain, and control project risk cost-effectively. Developing risk mitigation strategies is normally the responsibility of the SACWIS project management team with input from stakeholders, such as the business units that are affected by the risk. However, the SACWIS Project Director is responsible for ensuring that risk mitigation tasks are incorporated into the project work plan and carried out according to the plan by the assigned parties.

A Risk Repository (manual or electronic) facilitates the tracking, monitoring, communication, and reporting of up-to-date risk information scores, the owner of the mitigation strategy, and status. Regardless of the type of repository, elements that are often tracked relative to risk are:

· Description of the risk

· Date identified

· Who identified (role or name)

· Category of risk

· P score

· I score

· RE score

· Current status

· Risk owner

· Mitigation strategy

· Action plan

5. Risk Control

This process consists of regularly reviewing the risk profile; monitoring whether new risks emerge or existing risks escalate in severity to keep the risk mitigation plan current; and keeping the project sponsor, the Steering Committee, and other key stakeholders apprised of risk status and change.

The project manager needs to understand that a principal goal of the project’s risk management process should be to protect the agency or organization and its ability to perform their mission. Once that is understood, then it is also easy to understand that exercising ongoing control and review of the risk profile is critical to the agency and the SACWIS project’s success.

TIPS:

Risks are inherent to every project. Dealing successfully with risk requires that risk management be incorporated as an ongoing, iterative project process, rather than a one-time activity.

Assessing and managing risk increase the probability of project success.

ADDITIONAL RESOURCES:

An online course: “Risk Management for Child Welfare Information Systems” at: http://www.state-itc.org/cw_resources.html

“Project Management for the Project Life Cycle,” Regional Child Welfare Training Conference 2004 at: http://www.state-itc.org/rtc2004/accessible/rtc2004_accessible_index.html

B. Project Change Management

During the design, development, and implementation phases of a project, modifications to the documentation and application software system frequently become necessary. Change management provides the ability to control the scope of a project; associate the anticipated changes with a Level of Effort (LOE) (i.e., hours needed to prioritize, analyze, design, develop, test, and implement the proposed changes); and create a chain of command for authorizing changes.

Change request implementation involves identifying, analyzing, documenting, and prioritizing business needs. Analysis includes impact analysis, risk analysis, LOE, and an implementation plan. Prioritizing ranks the change requests based on need and LOE.

Effective change management is initiated and administered throughout all phases of the project to maintain system software and documentation accurately and to control changes. The change control process is used to request changes to materials that have already been marked “final” or for enhancements that are outside the original project scope.

Generally, three conditions may initiate a change request:

· Changes/additions in scope, requirements, or design requested by State project or vendor staff:

§ Changes in software or documentation in development.

§ Submission of a problem report by a member of the project team, containing either a suggested change or enhancement to the system software and documentation.

§ Management decision to implement enhancements.

· Changes/additions in scope, requirements, or design initiated by the State (pilot, counties, or other end-user stakeholders) prior to and during the implementation phase:

§ Changes in software or documentation currently within the system.

§ Submission of a problem report by the pilot, county, other end-user stakeholders or a member of the project team, containing either a suggested change or enhancement to system software and/or a different business process flow that requires additional system modifications or functionality.

· Changes in software or documentation in the production environment:

§ Changes requested in maintenance activities

§ Changes requested by project staff or end users

§ Changes mandated by changes in State or Federal legislation

The primary roles of the SACWIS project manager are implementing a project change management process and then monitoring its application and consistent use throughout the SACWIS project lifecycle. Achieving success over the life of the project largely depends on the definition, implementation, and enforcement of a process to follow for making changes to the project specifications and functionality.

TIPS:

Even the best-written product specifications require some changes as the product is being built, tested, and implemented.

A change management or change control process should be followed to ensure that the following occur:

· Only necessary changes are made to the product.

· Changes are communicated to all.

· Changes are implemented in an orderly fashion.

A financial management representative should be included in the change management process to assist in identifying a new or existing SACWIS project funding source for changes or enhancements if they are outside the scope of the initial SACWIS contract.

A strictly enforced change control process is essential for a well-managed software development process.

C. Organizational Change Management

Child welfare information systems are complex, automated case management applications that can have profound effects on all levels of the organization when implemented. A common lesson learned by many States is that managing the organizational process changes resulting from SACWIS implementation is just as important as managing the automation project itself.

It is crucial for the project manager to fully understand the impact that system implementation will have on the delivery of client services, caseworkers, operational support staff, and agency business partners. The day-to-day management of the project can be organized in many different ways. Some of the most successful projects have included roles such as a cultural change project manager and a program project manager from the Child Protective Services (CPS) program area. These individuals are part of the project management team and help communicate project status and issues to other agency departments to keep them actively engaged as the project progresses. Program project managers also can become one of the best advocates for the new system if they have the ability to travel to different locales and demonstrate the new system.

Communication about the project that occurs early and often with the user community establishes a foundation for positive change and helps prepare staff for the new system. If users can understand “what’s in it for me,” they will more readily accept the inevitable changes in their work processes required by the new system. Developing a structured communication process and plan to keep the entire organization informed about the status of the SACWIS project is a good way to keep a handle on change management issues.

TIPS:

Consider establishing a project website that can be viewed by all agency staff. Publish information regarding project timelines and schedules, descriptions of proposed system functionality, and information about project staff.

Identify and use as many communication methods as possible, including formal newsletters, memos, system announcements, websites, bulletin boards, and emails. There can never be too much communication with your users.

Implement user groups consisting of front-line caseworkers and support staff to engage them in project meetings. This approach promotes two-way communication between project staff and users. These forums also help identify variations in local casework practice that must be addressed during application design or in preparation for system implementation.

ADDITIONAL RESOURCES:

· Change Management in Child Welfare, Child Welfare National Training Conference 2004: http://www.state-itc.org/ntc2004/accessible/NTC2004_accessible_index.html.

· Managing the Alignment of Technology with Practice, Child Welfare National Training Conference 2005: http://www.state-itc.org/Webinars_Webcasts/Navigation/Resource_Events/NTC/NTC2005_accessible_index_webcasts.html

· Managing Change Inside the Agency: Making Our Own Processes Better: http://www.state-itc.org/Webinars_Webcasts/RTC_Events.html

V. Working With Federal Government Partners

The Department of Health and Human Services is a partner in the development, implementation, and maintenance of a State’s child welfare information system. This section outlines the APD requirements, which enable States to receive Federal funding to support the development and maintenance of their child welfare information system. This section also outlines the role that HHS plays in monitoring the system and the quality of data reporting to HHS.

A. Advance Planning Document

The following is an excerpt from the ACF website and is a high-level explanation of the SACWIS APD requirement and purpose.

“States are encouraged to develop SACWIS systems. States electing to develop such systems with Federal financial participation (FFP) come under the existing Federal review and approval processes, initiated and updated by Advance Planning Documents (APDs) submitted to ACF.”

In summary, the primary purposes of an APD are to:

· Request FFP.

· Present a business case and plan to support Federal expenditures for IT acquisitions.

· Demonstrate that all appropriate IT planning has been considered.

· Establish that sufficient resources are allocated and disciplined processes are in place to achieve project objectives.

Two major types of APDs must be developed and submitted to ACF for approval prior to the start of an intended SACWIS project. The first is a Planning APD (PAPD) consisting of a written plan of action that requests FFP to determine the need for, feasibility, and cost factors of data processing services or equipment acquisition. The second is an IAPD focusing on a written plan of action that supports the request of FFP to acquire the proposed data processing services or equipment.

Two types of APDUs are relevant to the lifecycle of a SACWIS project:

Annual APD Update. Used for providing the official project status updates, requesting continued project funding, and reporting post-implementation costs and benefits. Once a SACWIS project has begun, an Annual APD Update must be submitted to ACF. The Annual APD Update serves multiple purposes, including:

§ Requesting continued approval of project activities and funding for planning and implementation

§ Informing the Federal government of project status

§ Updating project-related information

· As-Needed APD Update. Used to report significant changes to the project approach, procurement, methodology, schedule, or costs. This type of APD should also be submitted when one or more critical milestones are missed.

The SACWIS project manager may not be the author of these documents. Other IT or agency staff may be the author of these documents depending on how the State agency is structured. Regardless, the project manager must be integrally involved in the development and review of these documents and will most likely spend a lot of time providing information to the person assigned these responsibilities.

TIP:

Work closely with your Federal analyst to make sure that the necessary information is timely, thorough, and accurate. As the SACWIS project manager, you are responsible for understanding the importance and significance of the APD process, as well as the expectations of ACF. Embrace the APD process as an opportunity to establish a positive and trusting working relationship with your Federal partner. Relationship management is an important part of the SACWIS experience, and it is a good place to start in order to avoid surprises and keep your project on track.

Take steps to ensure that project budget in the approved APD is consistent with the fiscal claims submit to HHS.

ADDITIONAL RESOURCE:

An online course sponsored by HHS, APD 101, can be found at:

http://www.state-itc.org/cw_resources.html

B. SACWIS Assessment Review

1. Process

SACWIS does not have any specific certification requirements; however, 90 system functional requirements are used as the basis for the SACWIS review. Once a system is operational, ACF will conduct and report the results of a SACWIS Assessment Review (SAR). The purpose of these reviews is ensuring that all aspects of the project, as described in the approved APD, have been completed adequately and conform to applicable regulations and policies. Either the State or ACF may initiate these reviews; however, ACF reserves the right to initiate a SACWIS Assessment Review at any time in the lifecycle of a system.

A SACWIS SAR is based on the following:

· The requirements of law

· Implementing regulations

· SACWIS and HHS Action Transmittals (ATs) and Program Instructions (PIs)

· The State’s approved APD, State contract documents, and any additional policy guidance or conditions provided to the State

After a State’s child welfare automated system is operational for approximately one year, or if the State requests a review, the DSS conducts a review to assess the system’s functionality against:

· SACWIS functionality requirements specified in 45CFR 1355.53 (b)

· The functionality described in the State’s IAPD

· SACWIS policy documents

The Division of State Systems (DSS) typically conducts a week-long onsite review, which includes a system walkthrough and interviews with local and State-level users. Their assessment is based on the degree the system is used to support case workers, not the number of fancy features. The onsite review provides DSS with an in-depth look at the system, as well as an opportunity to gauge the extent to which the system does the following:

· Implements SACWIS-required functionality

· Supports child welfare services business practices throughout the State

· Is accepted and used by front-line workers, supervisors, managers, and others in functional positions that were identified as system users, including private agency staff conducting SACWIS related functions

While some preliminary findings are presented to the State during the review’s exit conference, a detailed exception report is generated, providing the State with all the review team’s findings. The review process can be finalized only when the State has either modified the system or created an acceptable corrective action plans to thoroughly address the issues.

The timing and manner in which SARs are conducted is based on available staff and resources; not all SARs will be conducted on site, nor will they necessarily be conducted at the time the system becomes operational.

Generally, ACF will not conduct the Assessment Review until, at a minimum, the following conditions are met:

· Thirty percent of the total foster care and adoption caseloads (State and Federal) have been converted to the fully functional SACWIS system and are maintained under it.

· One or more county/district offices are fully operational.

TIP:

States that use contractor assistance in developing their system should not link final acceptance or payment to an ACF review. Instead, States are encouraged to base contractor payments on task-specific deliverables and system acceptance on demonstrations and system acceptance tests.

2. Preparing for the SACWIS Review

In preparation for a SAR, ACF will generally make every effort (considering resource availability) to conduct a technical assistance consultation with the State. This discussion may occur during or immediately after pilot implementation. The State and ACF should consider this technical assistance consultation as an opportunity to estimate system conformance with SACWIS requirements, specify a timeframe for resolution of obvious (i.e., highly visible) issues, and discuss when the SAR should be conducted.

To prepare for the review and provide DSS with background information, States will complete a SACWIS Assessment Review Guide (SARGe), which is typically submitted at least 6 weeks prior to the onsite review. In addition, States should submit system documentation, such as user’s guides or training materials, to supplement the information contained in the review guide. By reviewing the information provided through the completed guide, and other supporting documentation, the DSS review team can acquaint itself with the system and its functionality. States should complete and submit the review guide in hard copy and electronically as a Word document.

States may want to provide a demo version of the application (if available) or give access to the training database that does not contain confidential case data. Making screen shots of application screens or web pages does not demonstrate any functionality.

TIP:

The best source for this is the introduction to the SARGe, which discusses the preparatory steps for review. The SARGe is found on the Children’s Bureau website at: http://www.acf.hhs.gov/programs/cb/systems/sacwis/process.htm

3. Writing Action Plans

The SACWIS review process can be finalized only when the State has either modified the system and implemented the required changes or created acceptable corrective action plans for the identified issues.

ADDITIONAL RESOURCES:

· Administration for Children and Families, Technical Bulletin #1: Action Plan Guidance and Examples at: http://www.acf.hhs.gov/programs/cb/systems/sacwis/bulletin1.htm

· National Resource Center for Child Welfare Data and Technology, “Guidelines for Action Plans Submitted with the SACWIS Assessment Review Report (SARR),” (8/12/05 Conference Call) at: http://www.nrccwdt.org/resources/sacwis_archive/sacwis_archive.html

· National Resource Center for Child Welfare Data and Technology, “Five Action Plan Examples,” (8/12/05 Conference Call) at: http://www.nrccwdt.org/resources/sacwis_archive/sacwis_archive.html

C. Child and Family Services Review

The Child and Family Services Review (CFSR) is the process through which ACF assesses each State’s conformity with Federal requirements for child protection, foster care, adoption, family preservation and family support, and independent living services. The CFSR focuses on program outcomes for child safety, permanency, and child and family well being. The review evaluates the State’s capacity to deliver child welfare services and achieve performance-based outcomes.

The CFSR enables the Children’s Bureau to ensure that State child welfare agency practice conforms to Federal child welfare requirements, to determine what is actually happening to children and families as they are engaged in State child welfare services, and to assist States to enhance their capacity to help children and families achieve positive outcomes.

The SACWIS project manager will most likely be an active participant in the CFSR and may be asked to write or develop input into the agency’s child welfare Program Improvement Plan (PIP) related to the automated system.

D. AFCARS Review

The purpose of the AFCARS review is assessing a State’s information systems capability to accurately collect, extract, and transmit the AFCARS data to ACF. The AFCARS review also assesses the State child welfare staff’s ability to accurately collect and document the AFCARS information related to the foster care and/or adoption case of a child. The review process goes beyond the edit checks that must be met to pass the AFCARS compliance error standards. The review focuses on ascertaining a State’s degree of compliance with all the AFCARS requirements and the quality of its data. Therefore, AFCARS reviews have a separate and distinct purpose from SACWIS reviews and may be conducted before, during, or after a SAR.

E. NCANDS Data

The National Child Abuse and Neglect Data System (NCANDS) is a Federally sponsored effort that collects and analyzes annual data on child abuse and neglect. The 1988 amendments to the Child Abuse and Prevention and Treatment Act (CAPTA) directed HSS to establish a national data collection and analysis program. The Children’s Bureau in the Administration on Children, Youth and Families, ACF, HHS collects and analyzes the data.

The report is submitted by the States and the District of Columbia and is used for an annual Child Maltreatment report, which is published each spring. In addition, this data is used in several efforts by the Children’s Bureau to measure the impact and effectiveness of CPS. Data from NCANDS is used in the CFSRs of the States (resulting in potential PIPs) and the Annual Child Welfare Outcomes Report.

ACF collects case-level NCANDS data on all children who received an investigation or assessment by a CPS agency. States that are unable to provide case-level data submit aggregated counts of key indicators. Case-level data include information on the characteristics of referrals of abuse or neglect that are made to CPS agencies, the children referred, the types of maltreatment that are alleged, the dispositions (or findings) of the investigations, the risk factors of the child and the caregivers, the services that are provided, and the perpetrators.

F. NYTD Reporting

Public Law 106-169 established the John H. Chafee Foster Care Independence Program (CFCIP), which provides funding to States to implement programs that assist youth in making the transition from foster care to self-sufficiency. The law requires ACF to develop a data collection system for tracking independent living services youth received while in care and to develop outcome measures that will be used to assess the effectiveness of the States’ independent living programs. To meet the law’s mandate, ACF published a proposed rule in the Federal Register on July 14, 2006, and final rule on February 26, 2008. The rule establishes the National Youth in Transition Database (NYTD) and related reporting requirements. States have until October 1, 2010, to implement the rule at which time they must begin to collect data. The first submission of data to ACF will be due no later than May 15, 2111.

States must use the same encrypted identification number the youth has in AFCARS for the NYTD report. This allows ACF to track youth from the beginning of their initial foster care episode until the youth leaves the care of the State and beyond. Data reported to ACF must comply with specific data standards. If data is out of compliance, States will be able to transmit corrected data files by the subsequent reporting period or face potential financial penalties.

VI.Top Tips for Project Success

Garnered from the experience of project managers from successful child welfare information system projects from around the country, these top tips are worth embracing.

Involve end users throughout the SDLC, especially when identifying stakeholder needs and defining user requirements.
When seeking the services of outside vendors, document the business problem, not the solution. This approach will open your project to creative solutions and new products of which you may not be aware.
Establish a governance structure for your project that involves local office directors and private provider agencies. This strategy improves local buy-in and is particularly helpful in State-supervised, county-administered States.
Cultivate champions and mentors from the executive office to each local office.
Provide tools (e.g., reports and monitoring tools) useful to local managers so that they see the need for and utility of accurate and timely data; they will then encourage workers to pay close attention to the quality of the data being entered into the system.
Design the system to be friendly and usable, giving workers the tools that they need without burdening them with entering too much data.
Communicate, communicate, and communicate in as many ways and as often as possible.
Develop functionality that maps to the workers mental model of how a process should work, that allows workers to find the data they want to work with, and provides easy navigation to the functionality the worker needs.
Create positive and partnering relationships with the vendor, private providers, and your ACF Federal analyst.

LIST OF ACRONYMS

ACF	Administration for Children and Families
AFCARS	Adoption and Foster Care Analysis and Reporting System
APD	Advance Planning Document
APDU	APD Update
AT	Action Transmittals
BAFO	Best and Final Offer
CAPTA	Child Abuse and Prevention and Treatment Act
CBA	Cost Benefit Analysis
CB	Children’s Bureau
CEO	Chief Executive Officer
CFCIP	Foster Care Independence Program
CFO	Chief Financial Officer
CFSR	Child and Family Services Review
CIO	Chief Information Officer
COTS	Commercial off-the-shelf
DSS	Division of State Systems
FFP	Federal financial participation
HHS	U. S. Department of Health and Human Services
HTML	HyperText Mark-up Language
I	impact
IAPD	Implementation Advance Planning Document
IAPDU	IAPD Update
IT	information technology
LOE	Level of Effort
NCANDS	National Child Abuse and Neglect Data System
NYTD	National Youth in Transition Database
P	probability
PAPD	Planning APD
PI	Program Instructions
PIP	Program Improvement Plan
QA	Quality Assurance
RE	risk exposure
RFI	Request for Information
RFP	Request for Proposal
SACWIS	Statewide Automated Child Welfare Information System
SAR	SACWIS Assessment Review
SARGe	SACWIS Assessment Review Guide
SARR	SACWIS Assessment Review Report
SDLC	System Development Life Cycle
SOA	Service-Oriented Architecture
SOW	Statement of Work

REFERENCES AND BIBLIOGRAPHY

Please note that links to websites included in this document are subject to change and will be updated periodically.

2004 CHAOS Report, “CHAOS Chronicles,” The Standish Group.

Action Transmittal ACF-AT-93-03, Automatic Data Processing Equipment and Services – Conditions for Federal Financial Participation, January 3, 1993.

http://www.acf.hhs.gov/programs/cb/systems/sacwis/at9303.htm

Action Transmittal ACF-OISM-001, Automation of Child Welfare Programs, February 24, 1995.

http://www.acf.hhs.gov/programs/cb/systems/sacwis/federal.htm

Action Transmittal ACF-OSS-05, SACWIS Policy Guidance – Interfaces; Personal Responsibility and Work Opportunity Reconciliation Act (PRWORA); Cost Allocation & Other Issues, August 21, 1998. http://www.acf.hhs.gov/programs/cb/systems/sacwis/federal.htm

Advance Planning Document – 101, video recording of presentation by Tom Wetterhan, ACF Division of State Systems, at the 2006 Child Welfare National Training Conference.

http://www.state-itc.org/Webinars_Webcasts/Navigation/Resource_Events/NTC/NTC2006_accessible_index_webcasts.html

Advance Planning Document (APD) 101 and Cost-Benefit Analysis 101, online courses. http://www.state-itc.org/cw_resources.html

Advance Planning Document and Cost Benefit Analysis Presentations, 2006 National Child Welfare Training Conference. http://www.state‑itc.org/ntc2006/accessible/ntc2006_accessible_index.html

ASMB C-10, Implementation Guide for OMB Circular A-87, September 30, 1998.

http://www.hhs.gov/grantsnet/ogamat5.htm

Change Management in Child Welfare, Child Welfare National Training Conference 2004:

http://www.state-itc.org/ntc2004/accessible/NTC2004_accessible_index.html.

Cost-Benefit Analysis – 101, presentation by Tom Wetterhan, ACF Division of State Systems, at the 2006 Child Welfare National Training Conference. http://www.state-itc.org/Webinars_Webcasts/Navigation/Resource_Events/NTC/NTC2006_accessible_index_webcasts.html

Foster Care Independence Act of 1999, Public Law 106-169, 106^th Congress.

http://www.acf.hhs.gov/programs/cb/laws_policies/cblaws/public_law/pl106_169/pl106_169.htm

Getting IT Right and Keeping it Right: Processes, Strategies, and Tools to Develop and Manage Requirements, Online Course: http://www.state-itc.org/cw_resources.html

Getting IT Right and Keeping It Right: Processes, Strategies, and Tools to Develop and Manage User Requirements, Regional training materials, sponsored by the U.S. Department of Health and Human Services, Administration for Children and Families, Children’s Bureau. http://www.state-itc.org/cw_resources.html

Getting IT Right the First Time and Keeping It Right: Processes, Strategies, and Tools to Develop and Manage Requirements. Webcast of the Regional Child Welfare Training Conference 2006: http://www.state-itc.org/Webinars_Webcasts/Navigation/Resource_Events/RTC/RTC2006_accessible_Presentation_webcasts.html

Health and Human Services, Administration for Children and Families – Companion Guide 2: Cost/Benefit Analysis Illustrated for Child Welfare Systems. May 1996.

http://www.acf.hhs.gov/programs/cb/systems/sacwis/federal.htm

Health and Human Services, Administration for Children and Families – Feasibility, Alternatives, and Cost/Benefit Companion Guide. October 6, 1993.

http://www.acf.hhs.gov/programs/cb/systems/sacwis/federal.htm

Health and Human Services, Administration for Children and Families and Health Care Finance Administration – State Systems APD Guide. September 1996.

http://www.acf.hhs.gov/programs/cb/systems/sacwis/apdguide/index.htm

Information Memorandum OISM-IM-93-1, ADP System Security Requirements and Review Process – Federal Guideline,. October 1, 1992.

http://www.acf.hhs.gov/programs/cb/systems/sacwis/federal.htm

Information Memorandum ACYF-CB-IM-04-07, Technical Assistance for Developing Cost Allocation Methodologies for Multi program Information System Projects, Cost Allocation Toolkit, July 2, 2004. http://www.acf.hhs.gov/programs/cb/laws_policies/policy/im/2004/im0407.htm

Information Memorandum ACYF-CB-IM-05-02, Federal/State Information Technology Policy: Optional checklist for states and territories to use in requesting an exemption of prior approval for Information Technology acquisition documents, May 3, 2005.

http://www.acf.hhs.gov/programs/cb/laws_policies/policy/im/technologyindex.ht m

Information Memorandum 05-04, FEDERAL/STATE INFORMATION TECHNOLOGY POLICY – Use of Enterprise Level Commercial-Off-the-Shelf (COTS) Software in Automated Human Services Information Systems. http://www.acf.hhs.gov/programs/cse/stsys/ref/im_05_04.htm

Knapp, Karl M., Ed.D. Winning E-Learning Proposals: The Art of Development and Delivery, Fort Lauderdale, Florida: J. Ross Publishing and the Institute for Interactive Technologies, May 2003.

Managing the Alignment of Technology with Practice, Child Welfare National Training Conference 2005: http://www.state-itc.org/Webinars_Webcasts/Navigation/Resource_Events/NTC/NTC2005_accessible_index_webcasts.html

Managing Change Inside the Agency: Making Our Own Processes Better.

http://www.state-itc.org/Webinars_Webcasts/RTC_Events.html

National Resource Center for Child Welfare Data and Technology, Guidelines for Action Plans Submitted with the SACWIS Assessment Review Report (SARR), 8/12/05 Conference Call.

http://www.nrccwdt.org/resources/sacwis_archive/sacwis_archive.html

National Resource Center for Child Welfare Data and Technology, Five Action Plan Examples, 8/12/05 Conference Call. http://www.nrccwdt.org/resources/sacwis_archive/sacwis_archive.html

OMB Circular A-87, Cost Principles for State, Local, and Indian Tribal Governments. May 10, 2004: http://www.whitehouse.gov/omb/circulars/a087/a87_2004.html

Overview of Project Start-Up, Project Management Methodology, State of Kansas:

www.da.ks.gov/kito/Rel23/4startup.doc.

Program Instruction ACYF-CB-PI-06-01, Requirements and Level of Federal Financial Participation (FFP) Based on Status of a State’s Statewide Automated Child Welfare Information System (SACWIS), February 16, 2006.

http://www.acf.hhs.gov/programs/cb/laws_policies/policy/pi/2006/pi0601.htm

Program Instruction ACYF-CB-PI-01-05, Cost Allocation Policies for Operational SACWIS Systems, April 16, 2001.

http://www.acf.hhs.gov/programs/cb/laws_policies/policy/pi/2001/pi0105.htm

Project Management for the Project Life Cycle, Regional Child Welfare Training Conference 2004 at: http://www.state-itc.org/rtc2004/accessible/rtc2004_accessible_index.html

Public Sector IT Procurement online course. http://www.state-itc.org/cw_resources.html

Resource Estimation and Allocation Presentation, 6^th Annual National Child Welfare IT Managers’ Meeting. http://www.state-itc.org/Webinars_Webcasts/Navigation/Resource_Events/NTC/NTC2007_accessible_index_webcasts.html

Resource Estimation and Allocation and Driving Quality presentations, 6^th Annual National Child Welfare IT Managers’ Meeting: http://www.state-itc.org/Webinars_Webcasts/Navigation/Resource_Events/NTC/NTC2007_accessible_index_webcasts.html

Risk Management for Child Welfare Information Systems online course.

http://www.state-itc.org/cw_resources.html

SACWIS Assessment Review Guide (SARGe). http://www.acf.hhs.gov/programs/cb/systems/sacwis/process.htm

Title 45 Public Welfare and Human Services Code of Federal Regulations (CFR), Part 95 – General Administration – Grant Programs (Public Assistance, Medical Assistance, and State Children’s Health Insurance Programs). October 1, 2005.

http://www.access.gpo.gov/nara/cfr/waisidx_05/45cfr95_05.html

Title 45 Public Welfare and Human Services Code of Federal Regulations (CFR), Part 1355 – General. October 1, 2005.

http://www.access.gpo.gov/nara/cfr/waisidx_05/45cfr1355_05.html

Title 45 Public Welfare and Human Services Code of Federal Regulations (CFR), Part 1356 – Requirements Applicable to Title IV-E. October 1, 2005.

http://www.access.gpo.gov/nara/cfr/waisidx_05/45cfr1356_05.html