Fall 2011 will mark the launch of a new course evaluation system at Utah State University. The IDEA Center Student Ratings of Instruction assess effectiveness by focusing on learning and curricular objectives. The IDEA system will replace USU's current course evaluation process. The hallmark of the IDEA system is the opportunity for students to provide feedback on their progress in achieving specific learning objectives that are identified by the instructor, while adjusting for extraneous circumstances like class size. The IDEA system has a documented history of reliability and validity, and the IDEA "Diagnostic" also provides specific feedback on teaching methods and practice.
IDEA's focus on Learning Objectives is a change from USU's current course evaluation system, and faculty's selection of objectives and communication of those objectives to students is critical component of the new process. Below, you will find some guidance on selecting IDEA Objectives. At the bottom of the page please find links to documents that can assist faculty in communicating their learning objectives to students (through the syllabus, course announcements, etc.). All faculty teaching courses will receive an email 6-8 weeks prior to the end-date for their classes. This email will include a link to a web page where faculty must enter their selected learning objectives. These objectives are then used for student responses. For more information about the IDEA system, please contact the AAA Office at 435.797.0001
Selecting IDEA Objectives
How many Objectives should I select?: IDEA recommends that a total of 3-5 objectives is a good rule of thumb. You can select them as either "important," or "essential," with essential objectives double weighed (counting twice) in the calculation of your averages. Both important and essential objectives should be counted towards your overall total (e.g. pick 3-5, not 6-10).
The best approach to selecting objectives is to ask three questions:
1. Is this a significant part of the course?
2. Do I do something specific to help students accomplish this objective?
3. Does the student's progress on this objective affect his or her grade?
If you can answer "yes" to all three questions for a particular objective, it should be selected as either "important" or "essential."
Faculty should be "true to their course" when deciding how many objectives to select. In other words, if you are teaching a lab course where only one objective seems appropriate, then just select one. If you are teaching a senior capstone class for the major, and you feel you should select more than five, then feel free to do so. Be true to your course.
The selection of objectives on the Faculty Information Form is a crucial activity for two reasons. First, the IDEA System evaluates teaching by assessing student progress on these unique, instructor-chosen objectives. Second, objectives provide guidance for selecting teaching methods; those that promote progress on one type of objective may differ from those that promote progress on other types. Differential objectives make each course a unique learning experience.
Although objectives can be stated in a variety of ways, they should always focus on expected effects on students, not on the instructor's actions or procedures designed to promote learning. It is desirable for each instructor to develop statements or objectives as precisely and comprehensively as possible. In order to participate effectively in the IDEA program, it will be necessary to interpret these statements within the framework provided by the 12 objectives listed on the Faculty Information Form. The following discussion is intended to help users differentiate meaningfully and accurately among the 12 objectives of the IDEA System.
HOW MANY OBJECTIVES SHOULD BE SELECTED?
Although each of the 12 IDEA objectives is desirable in the abstract, it is unrealistic to think that, in a single course, students can make significant progress on all, or even most, of them. Most instructors will be unable to address seriously more than three to five objectives. Those choosing more than five objectives commonly spread their efforts too thinly to impact student learning significantly in all areas.
In selecting "Essential" or "Important" objectives for a particular course, ask three questions:
1. Is this a significant part of the course?
2. Do I do something specific to help the students accomplish this objective?
3. Does the student's progress on this objective affect his or her grade?
If the answer to each of these questions is "Yes," then that objective should be identified as E or I on the Faculty Information Form (FIF). The phrase, "Of no more than minor importance," does not mean that such objectives are unimportant. It simply recognizes that such objectives are of considerably less importance than those chosen as E or I; even if some attention is given to them, an M should be selected on the FIF.
WHAT IS MEANT BY EACH IDEA OBJECTIVE?
The 12 IDEA objectives have been developed over a period of 25 years, both through literature reviews and by consulting faculty who have used the IDEA system. The intent is to provide a useful, practical way to describe the objectives of most college courses.
A brief description of each of these objectives, together with some comments about how they may be compared or contrasted, follows. The objectives are organized into six groups on the basis of statistical and conceptual similarities. The number used to identify each objective (1-12) corresponds with that used on the Faculty Information Form.
The first two IDEA objectives focus on the development of a basic background in the subject.
1. Gaining factual knowledge (terminology, classifications, methods, trends)
2. Learning fundamental principles, generalizations, or theories
This objective stresses learning at its most basic level. Frequently what is required is that students memorize and remember the information taught; because the emphasis is on acquiring information, comprehension is not an issue.
Two other objectives focus on applying what has been learned to solve problems, make decisions, or perform specialized functions. These are:
3. Learning to apply course materials (to improve rational thinking, problem solving and decisions)
4. Developing specific skills, competencies and points of view needed by professionals in the field most closely related to this course
Only in courses where a primary goal is to develop a generalized skill in applying course materials to concerns or problems should Objective 3 be selected as "Essential" or "Important." Such skills and attitudes should be reflected in the performance of a professional assignment, not simply by knowing a "correct" answer.
Two objectives focus on learning to express one's individuality:
6. Developing creative capacities (writing, inventing, designing, performing in art, music drama, etc.)
8. Developing skills in expressing oneself orally or in writing
Creative capacities are important not only in the humanities, but also in science, technology, social sciences, and many professional courses (business, education, law, etc.). The development of creative capacities is expected more often in upper division courses than in lower division courses. For this (or any other) objective to be classified as "Essential" or "Important", it should be a significant emphasis in the course and there should be specific instruction or assignments designed to promote its development.
Effective communication is an appropriate objective for any course designed to promote clear, grammatically correct writing and/or listening and speaking skills. However, if the objective is chosen as "Essential" or "Important", it is not enough to simply provide opportunities to exercise communication skills; there must also be deliberate attempts to improve these skills.
Three objectives emphasize higher-level intellectual skills:
7. Gaining a broader understanding and appreciation of intellectual-cultural activity (music, science, literature, etc.)
10. Developing a clearer understanding of, and commitment to, personal values.
11. Learning to analyze and critically evaluate ideas, arguments, and points of view.
This objective should be selected as important or essential if the focus of the course is on broadening intellectual background and increasing the breadth of students' interests and appreciations. Such courses are usually directed to non-majors, although they may serve as introductory courses for majors as well.
Some college courses offer students the opportunity to explore alternative value systems as a means of facilitating this process. Although such courses usually are oriented around disciplinary content, their true subject matter is the student. If the emphasis is on stimulating a consideration of alternative goals and life styles, on differentiating the basic from the superficial, and on assuming personal responsibility for life choices, then this objective should be considered as important or essential.
The ability to critically evaluate and reason is a valuable trait in nearly every line of employment; therefore, it is expected that many upper division.
courses intended for majors will give some emphasis to its development. But because it is also a distinguishing characteristic of the educated person, those teaching courses intended to support an institution's general education program will also find this objective relevant to their purposes.
Two objectives are concerned with the development of skills helpful in the continual pursuit of learning after the formal educational experiences are over.
9. Learning how to find and use resources for answering questions or solving problems.
12. Acquiring an interest in learning more by asking questions and seeking answers.
Objective 9 should be chosen as "Important" or "Essential" for courses that emphasize the development of independent learning skills.
This objective is especially appropriate for courses in which students have already acquired sufficient background information and skill to permit them to identify the trends or unresolved problems that are likely to form the basis for future advances in the field.
Objective 12 stresses developing capacities for identifying salient questions.
While this objective is frequently cited as one of the purposes of general education, all faculty members who want the success of their teaching to be judged by the degree to which students acquire an "inquiring mind" should select it as "important" or "essential."
Objective 5 overlaps from the objectives just reviewed. For this reason, it is classified separately.
5. Acquiring skills in working with others as a member of a team.
The development of team skills involves a complex set of attributes, including capacities for accepting and appreciating human diversity, for listening, for communicating, for compromising, and for sharing responsibility in developing creative proposals and recommendations. Instructors who emphasize the development of team skills will note the overlap with several other objectives included on the IDEA list, including Objective 8 Communication Skills, Objective 3 Applications for Problem Solving, and Objective 6 Creative Capacities. If the expectation is that all of these skills will be developed, together with skills involving effective interpersonal relationships, then this objective should be identified as "Important" or "Essential." Faculty members who establish student teams or subgroups for more limited purposes, such as learning communication skills or developing interpersonal sensitivities, should select less complex objectives when completing the FIF.
CONCLUSION
Hopefully, this discussion will help you select IDEA objectives for your courses. To summarize, there are three criteria that are useful in selecting objectives:
(1) The objective is a significant part of the course;
(2) Specific and substantive techniques and assignments are employed to help the student achieve the objective; and
(3) Relevant assessments are made of student achievement of the objective.
Objectives should not be regarded as important unless a substantial and explicit effort on the part of the instructor is directed to the achievement of that objective and unless achievement on the objective is meaningfully reflected in the appraisal of student progress.
Frequently, there are differences between the instructor's and the students' perception of the relevance of a given objective. It is recommended that the course objectives be discussed with the students, preferably early in the term.
Ask them to reflect on their understanding of the course's purposes and the way in which they believe the various parts of the course fit into each of the 12 objectives. Student learning will be enhanced if they are committed to clearly formulated objectives.
Based on interviews with a small number of students, we do not believe that holding such a discussion before the students fill out the IDEA Survey Form will bias results. Students claim that their report of progress on objectives is uninfluenced by the knowledge that the instructor selected the objective as relevant. However, progress is more likely if students and faculty are agreed on the major purposes of the course. Hence, a discussion of such purposes will not only improve the quality of responses to the IDEA form but also be beneficial to the learning process.