评估及策划资源
As higher education professionals, we understand that assessment is essential as it helps us determine whether programs and services are meeting the needs of our students. 评估必须从一开始就有一个计划。 We cannot leave assessment to the last minute, otherwise we will not effectively collect the best information, which means we don’t have the most optimal information to guide our decisions. 评估 is intentional and takes times. There are many components involved in developing assessment, which is why it is necessary to develop an assessment plan; it provides us with the necessary steps in conducting assessment. An assessment plan can be developed at the institutional, divisional, departmental or program/service level and ensures that we look at all possible steps needed to produce reliable and valid assessment.
评估计划的步骤如下：
明确任务和目标
任务 是确定我们努力的路径和目标的主要目的。 It describes who our program serves, the main functions or activities, and the primary intention of the program. It sets the foundation that leads to the design of the program’s goals and outcomes. 为了进行评估，一份良好的书面陈述是必不可少的。 The mission statement helps us understand the master plan for a program and assessment helps us determine if our mission is actually being accomplished. For an academic program, it should convey the scope and purpose of faculty members’ expectations for student learning.
目标 是非常宽泛的陈述，描述了程序应该完成什么。 目标 are important because they detail how the mission will be achieved. For that reason, the goals need to align with the mission. For academic programs, these are most often presented as characteristics of the student.
创建可衡量的项目和/或学习成果
结果 are specifically what you want the end result of your efforts to be, the changes you want to occur. 结果就是你如何衡量目标。 There are two different types of outcomes:
- 学习成果 are statements of what students will be able to think, know, do, or feel because
of a given educational experience (including co-curricular). A learning outcome consists
of four components: Audience, Behavior, Condition, and Degree (which will typically
be identified in the targets).
- 情感性-感觉或情绪的成长（态度）
- 认知-心理技能（知识）
- 精神运动-体力或体力技能（技能）
- 程序的结果 are statements focused on what the program should accomplish each year with students in terms of program quality, efficiency, and productivity (e.g. retention and graduation rates), but not in terms of actual student learning. A program outcome consists of five components: Specific, Measurable, Achievable, Relevant, and Time-Sensitivity.
确定如何衡量结果
Once the outcomes have been identified, the next step is to determine how the outcomes will be measured. Depending on the wording of your outcomes, you may elect to use qualitative, quantitative, and/or mixed methods data collection.
- Quantitative: Methods that assess objectives by collecting numeric data and analyzing the data using statistical techniques; useful for comparing and measuring across individual students or student population. Examples: rubrics, checklists, pre/post-tests, survey questions using scales, data from student information system.
- 定性：依赖和评估描述而不是数字数据的方法； 有助于理解经验的深度和丰富性。 Examples: written reflections, focus groups, interviews, open response survey questions.
- Mixed Methods: Sometimes referred to as mixed methodology, multiple methodology or multi-methodology research, mixed methods research offers the best of both worlds; the in-depth, contextualized, and natural but more time-consuming insights of qualitative research coupled with the more-efficient but less rich or compelling predictive power of quantitative research.
决定是否收集直接/间接证据和形成性/总结性证据
It is also important to select whether you will be conducting direct or indirect assessment and formative or summative assessment. Many times you will have a combination to meet your needs.
- Direct Measure includes student products or performances that demonstrate that specific learning has taken place. （例如，考试，使用评分的报告）。
- Indirect Measure asks students to reflect on their learning or experiences rather than to demonstrate it. （例如学生对学习的认知）。
- Formative 评估 is conducted during the program to provide feedback and used to shape, modify, or improve the program; A means of gathering information about student learning that is built into and a natural part of the teaching-learning process.
- Summative 评估 is conducted after the program, which provides the opportunity to make judgment on quality, worth, or compares to standards and can be incorporated into future plans.
确定要进行何种类型的评估
使用数字：跟踪参与程序或服务
学生需求：让你了解学生群体或特定人群。
Program Effectiveness: level of satisfaction, involvement, effectiveness, helpfulness, etc.
成本效益：提供的项目/服务与成本相比如何？
校园气候或环境：评估校园的行为/态度。
比较（基准）：将程序/服务与比较组进行比较。
学习成果: assess how a participant will think, feel, or act differently as a result of your program/course/service.
如果使用间接评估，发现哪些数据已经存在
Sometimes the information may already be available through institutional-wide surveys or data from the Student Information System. Make sure that you are looking into what already exists before jumping into developing a new assessment tool. Here are some examples:
- Skyfactor （EBI）学生事务评估套件
- 全国学生参与调查（NSSE）
- Mapworks
- 建议调查
- 退出调查
- 课程评价
- 毕业生调查
- 保持率
- 教师/学生比率
- 入学趋势
- 毕业率
- 学生群体的多样性（人口统计）
- Other surveys administered in Student Affairs (Welcome Week, Orientation, Resident Satisfaction)
- 校友的调查
确定评估对象
Obviously if you are assessing a program or service, you will want to study those that utilize the program or service. You do still need to determine if you want to include all those students or a sample.
确定管理的时间表和细节
接下来，你需要确定何时进行评估。 Who is responsible for administration of the assessment? If conducting qualitative assessment, is the facilitator unbiased? For example, the Director of Residence Life shouldn’t facilitate focus groups with housing students to determine their satisfaction. When do you think the best time will be and does it coincide with another assessment that will affect your response rate (survey fatigue), if applicable? Are you planning to conduct this just once or over the course of time (longitudinal study)? How will you be administering the survey, will it be online or on paper, will it be in the classroom, will you use tablets in the Student Center? Also, how will you communicate the assessment to students and will you incentivize it? 你的评估预算是多少？
设定绩效目标/最低绩效标准
Once you have selected how you are going to assess your outcomes, you need to set your targets. The targets provide a specific level which will define success towards achieving the outcome. For example, we want to determine if the following learning outcome is being achieved in a Biochemistry program: “Students will demonstrate the ability to dissect a problem into its key features and to test hypotheses through interpretation of experiments.” It is determined that a final exam will be used to assess this learning outcome. The targets are that 80% of students will correctly answer 80% of exam questions designed to assess application and analysis utilizing key principles and 80% of students will correctly answer 75% of exam questions designed to assess synthesis and evaluation of new ideas based on their knowledge of key principles. So if only 50% of the students on the first target are answering 80% of questions correctly, than we need to look at how to improve that number.
设定目标应该基于证据（如果有的话）。 It involves knowing where you are now (looking at past levels of student performance) and at what level you are trying to achieve. If you do not have historical data, you might consider using information from outside data sources. Targets are about setting a desired level of performance, not about the level you are currently achieving. Remember that it should be challenging, but realistic.
决定如何以及由谁来分析数据
Now that you have collected and have access to the data, you will need to start analyzing the data. How you analyze the information depends on what type of assessment methods you selected (quantitative vs. qualitative).
Quantitative Analysis – If you have selected quantitative, there are many ways that you can analyze your information. You will want to apply statistical methods to your information including descriptive statistics, correlational statistics, and/or inferential statistics, depending on what you are trying to discover.
描述性统计 （描述或总结回应的措施）
- 集中趋势（平均反应/得分是多少？）：
- 平均值-数值平均值
- 中位数：按从低到高的顺序显示的中位数
- 模式-最频繁出现的分数
- 可变性/分散性（分数/反应的一致性或分散性如何？）
- 范围-分数从最高到最低的范围
- 标准偏差-衡量一组分数与平均值的关系
- 分布（每个类别中有多少个回复？）
- 频率-一个类别中案例/分数的数量或比例（%）
相关与推论统计（推断变量之间的关系）
- 相关性统计（事物相关吗？）
- Correlation coefficient – numerical representation of the strength and directions of the relationship between two variables – correlation ≠ causation
- 推论统计学（一组与另一组不同吗？）
- T-test – used to examine the relationship between measures for two groups (e.g. mean scores)
- ANOVA (analysis of variance) – similar to t-test, but used when there are more than two groups to compare
- Chi-square test (X^2) – measure of relationship of nominal/categorical variables (e.g. retention data)
Also think about how you will disaggregate the information and what groups you will use for comparisons.
Qualitative analysis – This is very different compared to quantitative analysis because it is focused on descriptive vs. numeric and the administrator of the assessment is considered the instrument for analysis. This type of analysis is inductive and comparative, and systematic yet flexible. Below are a couple methods that can be employed for qualitative analysis:
通用数据分析示例：
- 整理数据
- 熟悉数据
- 生成类别、主题和模式
- 编码数据
- 寻找其他的解释
- 写报告
Constant-Comparative方法
- 反射
- 开放的编码
- 轴向编码
- 查看其他数据
- 主题编码
- 排序
有效地沟通结果
不仅进行评估很重要，而且分享结果也很重要。 Think of who would benefit from the information gained from the assessment and determine how you will share it. 会在网站上公布吗？ Will you present to various groups? 你会通过社交媒体分享吗？ Will you develop posters to hang around campus to share with students?
决定如何处理结果
Once the results have been shared with the various groups on campus, the most important step is to determine how you will utilize those results. Will you use them to defend offering a new program or service to students as it was determined as a need? Will it help you determine how you can change aspects of a program or service? Remember that once you have implemented initiatives based on assessment results, communicate what has been changed to students. It will show them that we are actually interested in what they have to say.
评估方案的评价
The last and final step is to evaluate your assessment plan, especially if you plan to conduct the assessment more than once.