About Evaluation Task Force

The Evaluation Policy Task Force is appointed to assist AEA in developing an ongoing capability to influence evaluation policies that are critically important to the practice of evaluation. The following outlines the intended scope of this work.

• Focus on Evaluation Policies. The Evaluation Policy Task Force should focus on evaluation policy, rather than policy in general. While evaluation can help inform substantive policies in a wide range of areas, and this is a recognized central purpose for much evaluation, influencing substantive policy is not the emphasis of this Task Force effort. Instead, the Task Force should concentrate on evaluation policies. Examples of general areas of evaluation policy might include (although may not be limited to) policies regarding:

  • Evaluation definition. How, if at all, is evaluation defined in an agency or in legislation? In such contexts, how is evaluation formally distinguished from or related to other functions such as program planning, monitoring, performance measurement or implementation?
  • Requirements of evaluation. When are evaluations required? What programs or entities are required to have evaluations? How often are evaluations scheduled?  What procedures are used to determine when or whether evaluation takes place?
  • Evaluation methods. What approaches or methods of evaluation are recommended or required by legislation or regulation, for what types of programs or initiatives?
  • Human resources regarding evaluation. What requirements exist for people who conduct evaluations? What types of training, experience or background are required?
  • Evaluation budgets. What are the standards for budgeting for evaluation work?
  • Evaluation implementation. What types of evaluation implementation issues are guided by policies? For instance, when are internal versus external evaluations required and how are these defined?

  • Evaluation ethics. What are the policies for addressing ethical issues in evaluation?

• Focus on U.S. Federal Evaluation Policy. At least initially, it would be desirable to focus this Task Force on the Federal level because evaluation policy decisions at that level have broad implications for many AEA members and for the field generally. It may be possible in the future to expand this effort to other arenas (nonprofits, state-governments, businesses, international, etc.), but we expect that initially at least the effort will be focused on Federal policy.

• Target Only a Few Specific Legislative and Executive Policies. It will be necessary to limit the effort to one or two potentially important evaluation policy formulation opportunities on both the congressional and administrative side of the Federal government. For instance, we might identify one or two major pieces of legislation and one or two areas where administrative regulations are likely over the next few years.

• Focus on Selected Substantive Areas. The workgroup anticipates that it will be necessary to focus the Task Force’s work on evaluation policy within a limited set of substantive areas.  For instance, the fields of education and health evaluation are areas of primary interest to a broad range of AEA members where AEA could build on its current strengths. The Task Force should focus its evaluation policy shaping activities in substantive areas such as these that are related to AEA’s history and current emphases.

• Be Appropriately Opportunistic.  Given the preceding thoughts about limiting the Task Force’s work, we recognize the value of remaining flexible and of responding (thoughtfully and selectively) as unexpected opportunities arise that might fall outside the stated initial scope. The Task Force should look to take advantage of such opportunities and should consult with the AEA Board as such arise.

Tasks and Responsibilities

The Task Force will advise and assist the AEA Board, Executive Committee and AEA staff in addressing two primary parallel tasks. The first task will consist of a focused, targeted consultative campaign designed to identify and provide useful consultation to U.S. federal legislators and executive branch staff on legislation and regulation in a select number of areas. The second and parallel task will be to develop the public presence that AEA projects in direct support of the consultative campaign, in particular through the development of collateral materials that can be utilized in that campaign, and perhaps through other forms of outreach as well (e.g., website presence).

The primary role of the Task Force will be to provide overall guidance on these two tasks. The Task Force is not responsible for implementation of the tasks, although some Task Force members may be directly involved in the implementation. To accomplish the work envisioned here, AEA intends to engage the services of a consultant who will play a major role in implementation of the consultative campaign. The Task Force is intended to advise and support this consultant, and to make policy recommendations to the AEA Executive Committee and Board on both of the tasks as appropriate.

Consultative Campaign Task. The primary purpose of this task is to influence federal evaluation policy in a manner consistent with the Mission of AEA in select areas identified by this Task Force. This task is intended to be a relatively “quiet” effort designed to develop experience and expertise on how best to influence federal evaluation policy directly.

Specific sub-tasks on which the Task Force will provide input and guidance include:

• Identifying and summarizing attempts to influence federal policy by other professional organizations that are similar in nature and scope to this effort.
• Identify and coordinate with evaluators within the Federal Government who are connected to the specific evaluation policy context.
• Advising the AEA Board on legal and ethical issues and concerns associated with efforts to influence evaluation policy.
• Developing processes for engaging and communicating with the AEA membership on this evaluation policy effort, including communicating with membership and inviting their input at sessions at the annual conference of the American Evaluation Association.
• Formulating and making recommendations to the AEA Board as appropriate.
• Consulting on evaluation policy planning processes, including methods for: scanning and identifying prospective policy opportunities; managing networks of contacts; entering into and managing specific policy influence efforts; responding to requests for input on policy formulation; monitoring policy efforts; and managing relationships with any consultants engaged by AEA to accomplish this task.
• Providing guidance and advice on efforts to scan federal legislative and regulatory opportunities for relevant policies that could be targets for influence.
• Advising on the selection of specific policy opportunities.
• Advising on how best to manage the emerging network of contacts in both the legislative and executive branch that can assist AEA in influencing evaluation policy.
• Monitoring and providing input on the evaluation policy efforts and progress made.
• Reporting to the Executive Committee and Board on progress of the consultative campaign.
• Preparing and making recommendations to AEA Board on how the evaluation policy influence effort should be implemented subsequent to the two-year appointment period of the Task Force.

Because of the potentially broad nature of this work, the consultative campaign will initially be limited to only a few policy targets. The consultant, AEA Staff, and any Task Force members directly involved in implementation will be primarily responsible for identifying an initial set of potential policy targets and will bring these to the Task Force for consideration. The Task Force will be expected to play a major role in discussing and advising on what policies should be targeted in this effort. On the congressional/legislative side, this might involve: identifying one or two major areas of upcoming legislation (e.g., one each in education and health); identifying congressional staff members that are critical to the legislation; developing contacts and building a network of personal relationships with staff members; conducting background research on evaluation policies that currently exist in the area or other relevant aligned areas; identifying potential policies that would enhance evaluation; making contact with identified staff; and, responding as needed to any requests for input on policy formulation (e.g., testifying at congressional hearings; drafting potential language for legislation). On the executive side, this effort might involve: identifying one or two major areas where evaluation policy is likely to be formulated within the next few years; determining who the major agencies and people are who will formulate the policies; determining how direct personal contact will be made with them; advising on the development of a network of personal relationships; conducting background research on existing relevant evaluation policies; and, consulting on potential policy changes or regulatory language. The Task Force will provide general guidance and advice on these activities; implementation will be handled by the consultant, AEA staff, and select Task Force members appointed to engage in these activities.

Public Presence Task. The primary purpose of this task is to support the consultative campaign through efforts that help position AEA as the leading U.S. association in evaluation and through the development of collateral materials that represent AEA and can be utilized in discussions with policymakers. Again, while the Task Force will be expected to advise and provide guidance on this effort, and may elect to draft materials that support it, the primary implementation will be the responsibility of the consultant, AEA staff, and Task Force members who are directly involved in implementation. The Task Force will be expected to provide advice and guidance on the following:

• Development of general “talking points” on evaluation and its role that would be likely to have broad consensus from AEA members and could be used in consultative work on evaluation policies (possible examples of talking points might be, for example, that: evaluation is a critically important endeavor; legislation and regulation should have explicit written requirements for evaluation of federally-funded programs; a broad range of evaluation methods are likely to be applicable and legislation and regulation should require that multiple methods and approaches appropriate to the program being assessed should be considered; etc.)
• Development and maintenance of a roster of leading evaluators who can be called upon as needed as spokespersons or contact persons for the media, for meetings with legislators or executives in government, for testifying in formal committees or meetings, and/or as consultants on evaluation policy issues.
• Development and enhancement of AEA collateral materials (e.g., brochures, Guiding Principles, public statements) that will support the consultative campaign
• Enhancement of the AEA website so that it is more useful for influencing evaluation policy formulation and positioning AEA as a leader in advising on evaluation policy. This should include working with the consultant to develop a “Policy Watch” function for notifying and engaging AEA members regarding important impending evaluation policy changes and initiatives.

The Task Force is not expected to address the AEA’s public presence generally. Their efforts should be concentrated on guidance that will specifically support the consultative campaign. In this work, the Task Force will communicate and collaborate with other AEA Committees such as the Public Affairs or Professional Development committees on areas of common relevance.

Appointment, Time Commitment, Structure and Support

The Evaluation Policy Task Force is appointed for a two-year period beginning in July, 2007. The Evaluation Policy Task Force is a Task Force of the AEA Executive Committee and, as such, will report through the EC and may use the EC as a sounding Board as desired.

The Task Force will consist of 5 members, the Executive Director, and a liaison from the Public Affairs Committee (either the Chair or a PAC member). The Chair of the Task Force will be a member of the Executive Committee. The Evaluation Policy Consultant will be integrally involved in the Task Force and will participate in meetings and discussions as appropriate. The Task Force Chair and AEA Executive Director will identify potential members, subject to approval of the Executive Committee, with the following criteria to guide selection of members:

• Commitment to supporting the mission and goals of the American Evaluation Association.
• Knowledge of and a history of prior involvement with the American Evaluation Association.
• Familiarity with the field of evaluation and capacity to understand and represent the field to others.
• A broad perspective on evaluation and willingness to advocate for the many and diverse views of evaluation.
• Knowledge of the federal policymaking process.
• Experience with policy development initiatives in the Federal Government.
• Experience with public presentations of evaluation to a variety of audiences.
• Diversity of the Task Force and representativeness of the breadth of members and interests of AEA.

The Task Force will provide a written annual report to the AEA Board for review at its Winter Board meeting and updates as needed through the Task Force Chair at other Board meetings. At the completion of the first year of their work, the Task Force will be expected to report to the Board with their recommendations about the feasibility and desirability of continuation of this effort and the best mechanisms for doing so (e.g., continuation of the Task Force; establishment of a standing committee or assignment of this effort as a subcommittee to an existing committee). The AEA Board will provide ongoing feedback and guidance based on its reviews of these reports.

The Task Force will be expected to convene at least once every two months over the two year appointment period. Most of these meetings will be by teleconference, although the Task Force should attempt to meet face-to-face at least annually at the AEA annual conference. The Task Force will set its own agenda and specific activities in pursuit of the tasks outlined above. The Task Force is encouraged to develop work groups that invite people who are not on the Task Force to assist in work to address specific tasks as needed.

The Task Force will most likely need to develop a mechanism that enables rapid response to situations that arise under the consultative campaign. For instance, they might wish to develop a roster of senior evaluators who are willing to be called to help respond in the event that a meeting is needed to be held immediately, legislative or statutory language needs to be drafted quickly or public testimony is requested.

The AEA office will provide administrative support for the Task Force, including assistance in setting and managing meetings and Task Force materials.

AEA will provide contracted consultative support for technical assistance in identifying, contacting and managing specific policy shaping opportunities.

Over the course of the two year timeframe, the success of this initiative will be judged on: the level and appropriateness of the activities; the extent of its influence on specific policies and policy language; and, the quality and potential value of the network developed that connects AEA, the consultant, and policymakers.

MODIFICATIONS AS OF JULY 2009

The Evaluation Policy Task Force is appointed for an additional two-year period beginning in July, 2009.

The Task Force will consist of up to 10 members, including the Executive Director, a member of the Presidential rotation, and a liaison from the Public Affairs Committee (either the Chair or a PAC member). The Chair of the Task Force will be a member of the Executive Committee.