Outcomes Data
Chemistry Outcomes Data
Assessment of Learning Objective 1
Fundamental knowledge in the diverse fields of Chemistry
Criteria 1
CHEM 4990 capstone course exit exam.
Each of the sixty exit exam questions has been mapped to specific learning objectives associated with the core chemistry degree program courses. This allows the department to track overall performance in the six major areas of chemistry associated with the degree program (General, Organic, Inorganic, Analytical, Physical and Biochemistry). It also allows feedback to faculty teaching the specific courses as to areas requiring additional emphasis to enhance student performance.
The average of the % correct for each question in a section
STD
The standard deviation of the % correct for each question in a section
Results From Chemistry Exit Examination
Summary Chem 4990 - Assessment Exam Results for Fall 2020
| Area | Mean | STD |
|---|---|---|
| Gen Chem Questions | 60.0% | 23.6% |
| Inorganic | 37.0% | 23.1% |
| Organic | 59.0% | 17.3% |
| Analytical | 53.0% | 28.3% |
| Physical | 42.0% | 29.4% |
| Biochemistry | 49.0% | 19.1% |
| Exam | 50.0% | 23.5% |
Chem 4990 - Exam Results for 2019
| Area | Mean | STD |
|---|---|---|
| Gen Chem Questions | 55.33% | 29.81% |
| Inorganic | 38.69% | 19.90% |
| Organic | 66.52% | 19.89% |
| Analytical | 55.21% | 29.48% |
| Physical | 35.65% | 27.71% |
| Biochemistry | 47.83% | 25.39% |
| Exam | 49.87% | 25.36% |
Exam Results (5 Year Average: 2013-2017)
| Area | Ave % Correct | ACS Percentile |
|---|---|---|
| Gen Chem Questions | 64 ± 8 | 71st |
| Inorganic | 49 ± 7 | 50th |
| Organic | 56 ± 5 | 61st |
| Analytical | 58 ± 3 | 52nd |
| Physical | 44 ± 3 | 50th |
| Biochemistry | 51 ± 6 | 35th |
Chem 4990 - Assessment Exam Results for Spring 2018
| Area | Mean | STD |
|---|---|---|
| Gen Chem Questions | 56.3% | 29.0% |
| Inorganic | 55.0 % | 25.1% |
| Organic | 62.5% | 20.4% |
| Analytical | 62.5% | 25.7% |
| Physical | 31.3% | 29.6% |
| Biochemistry | 47.5% | 21.1% |
Chem 4990 - Exam Results for 2017
| Area | Mean | STD |
|---|---|---|
| Gen Chem Questions | 55.3% | 28.6% |
| Inorganic | 40.0% | 20.1% |
| Organic | 64.0% | 19.9% |
| Analytical | 54.0% | 27.6% |
| Physical | 44.0% | 28.8% |
| Biochemistry | 42.7% | 21.6% |
| Exam | 50.0% | 25.2% |
Criteria 2
ACS examinations covering a specific chemistry area in depth.
Several core chemistry courses required for a chemistry degree have implemented the appropriate ACS standard examination as a final exam. Data below is an example of the spring 2020 results for students taking the most current version of the ACS Instrumental Analysis examination as part of CHEM 5640 (a core chemistry class required for the chemistry degree).
Historic Data
- 2020: Average score of 35/50 = 90th percentile nationally
- 2019: Exam was not administered
- 2018: Exam was not administered
- 2017: Average score of 31/50 = 85th percentile nationally
- 2016: Average score of 30/50 = 82nd percentile nationally
- 2015: Average score of 28/50 = 73rd percentile nationally
- 2014: Average score of 32/50 = 88th percentile nationally
- 2013: Average score of 30/50 = 82nd percentile nationally
- 2012: Average score of 28/50 = 73rd percentile nationally
The nationally normalized ACS Instrumental Analysis examination is used as the final exam for CHEM 5640 (a core senior-level chemistry class required for the chemistry degree). Reviewing the results for specific exam questions showed students generally performed better on questions related to concepts and applications of spectroscopy and chromatography. As has been seen in the past, this is typical as this material is stressed in both the class and the associated laboratory. When students take both the class and laboratory concurrently (which is common and recommended), this tends to reinforce the class concepts and likely leads to better learning outcomes.
Assessment of Learning Objective 2
Proficiency in basic laboratory techniques and laboratory safety; experience with modern lab instrumentation and experience working as a member of a team.
These objectives are assessed in low enrollment laboratory courses run under the direct supervision of a faculty member. This ensures students have developed proficiency in basic laboratory techniques and laboratory safety prior to graduation.
One measure is the performance of students in less structured laboratory environments such as faculty-directed Undergraduate Research Problems (CHEM 4800). This is an elective course and not all undergraduates take this course, although those that do are required to submit a detailed written project report at the end of the semester.
Another assessment occurs in the less structured laboratory experience that is part of the Chemistry degree core curriculum, the CHEM 5650 laboratory. In CHEM 5650, students perform eight structured labs and write detailed reports for each. During the final month of the semester, students collaborate on a multi-week, team centric project. Student groups (typically 3 students per group) choose and design their own project, the results of which are presented in a detailed written final report.
For 2020, due to Covid-19 and the move to on-line laboratories, no final projects were possible.
For 2019, all of the submitted final projects and their reports were judged good to excellent. The average score for the final projects in the class was 87%.
For 2018, two of the three final projects and their associated reports were judged good or excellent. One of the final projects and its associated report was judged satisfactory. The overall average score for the final projects in the class in 2017 was 80%, which is below what is normally observed in a typical year. This was primarily due to the small number of groups (three) in the 2017 course and the lower performance on the final project by one of the groups.
Assessment of Learning Objective 3
Communicate in written and oral formats scientific information to chemists and non-chemists.
Writing skills are developed and assessed via the university designated communication intensive (CI) courses that the department requires to earn a chemistry degree.
Required CI courses for chemistry degrees include: CHEM 3080/3090 (Physical Chemistry Laboratory I & II) and CHEM 4990 (Undergraduate Seminar Capstone class). As part of the two semester CHEM 3080/3090 CI courses, students are required to write a detailed laboratory report for each of a semester's experiments. This is typically eight laboratory reports per semester of 8-10 pages each. For the CHEM 4990 seminar course, students attend all departmental seminars from visiting scholars are required to write a minimum of five, 1-2 page critiques of each seminar. The critiques require that the student follow the material being presented in the seminar and to be able to coherently discuss what was learned from the presentation in a written format. The seminars also provide examples of various presentation styles that students can try to emulate (or not) in their own presentations. CHEM 4990 student posters and seminar presentations were affected by the move to on-line courses due to Covid-19. Student seminar presentations were presented via Zoom video recordings and a Zoom Question and Answer session on the seminar presentation (8-10 minutes per student). Presentations were equal or better than the live seminars students normally give. We will examine the possible advantages recorded presentations might offer to students who are not as comfortable or use to live presentations. Student posters were prepared as normal and printed out for display within the Chemistry building and were displayed for 4 days before being taken down. Students were assessed on the quality of the posters, both in terms of content and professional quality of the poster layout.
Oral communication skills of seniors are further developed, and assessed, in the Chemistry degree capstone course (CHEM 4990). Students are instructed on how to properly research a scientific topic of their choosing using a variety of library and Internet database tools. Students are also instructed on how to present research results to a general scientific audience. Students do an in-class practice seminar presentation and receive feedback after the presentation to improve the quality of their presentations. Finally, students are required to present both an oral presentation and a poster-based presentation to their undergraduate peers, selected faculty members and graduate students who provide constructive feedback on their presentations. The poster presentations (typically 90 minutes) are given during a departmental poster session and allow a less formal presentation that is typical of many science conferences and allows students to interact one-on-one with those attending the poster session.
For the Fall 2019 and 2020 oral and poster seminar presentations, all students were deemed to have done very well to excellent on their final presentations. All students received combined scores of between 90% and 95% on their 2 presentations. The 2020 poster and seminars were impacted, somewhat, by Covid-19 but despite the changes outlined above, students still did very well to excellent (similar to previous years).
Biochemistry Outcomes Data
Assessment of Learning Objective 1
Acquiring Fundamental Knowledge
- Students taking the capstone Chem 4890 course (a requirement of the major, taken during the senior year) are given the ACS Biochemistry exam.
- Each of the ACS exam questions have been mapped to one of the six objectives supported by the American Society of Biochemistry and Molecular Biology (ASBMB) visit https://goo.gl/LGH6qC. Transitioning to these objectives will facilitate future efforts to gain accreditation from the ASBMB. In future years we will adopt the ASBMB exit exam in place of the ACS Biochemistry exam.
- Each year the student scores will be compared to the national average for the same questions reported for the ACS exam.
- The 60 questions in the most recent ACS exam map as shown in Table 3.
- The mean raw score achieved by the 2017-18 class on the ACS Biochemistry exam was 55% compared to the national mean percentile of 50%. For comparison, the 2016-17 class scored in the 51st percentile.
- Figure 1 shows how the ACS exam results mapped onto the ASBMB objectives.
Assessment of Learning Objective 2
Read, Present, and Analyze Scientific Literature and Data
- These skills are honed and assessed in Chem 5700/5710 (Biochemistry), Chem 5720 (Biochemistry lab), and Chem 4890 (Seminar Capstone).
- In Chem 5700/5710, new short writing assignments have been included in the syllabi. Grading rubrics include content, grammar, and scientific correctness. Average student scores increased in 5700 and 5710 by 25% and 13% over the semester, respectively as shown in Figures 2 and 3.
- In Chem 5720, students are required to orally defend a research proposal and write extensive lab reports. In 2015 a new rubric for the lab reports was devised in conjunction with the STEM center to assess learning based on scientifically validated criteria. For the 2017-2018 class, rubric scores increased by 25% over the course of the semester (from 68% for the first lab report to 93% for the last lab report) as shown in Figure 4.
- In Chem 4890, students are required to read the literature and to do several presentations that include both poster and verbal formats. Rubrics for assessment are the same as those used in the graduate seminar class. Poster and verbal presentations by USU students in the spring of 2017-18 were scored at 85% or greater (see Figure 5).
Assessment of Learning Objective 3
Design Experiments; Choose and Apply Appropriate Techniques to Achieve a Research Goal; and Demonstrate Good Quantitative Skills
- This objective is assessed using Chem 5720, the Biochemistry Laboratory course. The course is designed to give students a genuine research experience, and includes a two-month research project that requires students to develop their own protein purification protocol for a “mystery protein.” They are required to develop preliminary data on the protein expression and write a proposal for purifying the protein to at least 90% purity. The students are then required to give an oral defense of their proposal to a group that includes the instructor and several graduate students. Once the proposal is approved, the students are given two weeks in the lab to purify their protein. This is a difficult project for many students in part because it requires them to (1) clearly understand topics presented in lecture and described in the primary literature, and (2) apply the material in a real-life setting.
- As a means of numerically assessing student skills, the following rubric is used for the projects:
- Project proposal
- (150 points) This is a detailed description of your project justification, strategy, and timeline. The protein expression experiment must be fully described as part of the project justification/preliminary data. An itemized list of required chemicals and reagents is also required. The TA and/or the instructor may request revisions before students are allowed to begin the experiment.
- Proposal Q&A.
- (20 points) As a group, you will be required to meet with the TA and/or instructor to answer specific questions about your proposal. Each member of the group will receive a separate grade.
- Lab notebook
- (25 points) As with the other labs, you will be expected to keep a lab notebook each throughout the project. Each member of your group will turn in their own notebook at the end of the project.
- Protein purification
- (50 points) Full credit will be given to those who obtain 90% purity or better.
- Final report
- (135 points) The report will be similar to other lab reports, but only one report will be submitted for each group. Consequently, it is expected that the report will be more detailed than those submitted for the other labs.
- Team evaluation
- (20 points) Each member of the group will evaluate the contributions of every other member of the group. If it becomes apparent that some individuals contributed more to the project than others (as judged by the evaluations and observations by the TA and instructor), the final grades for the project will be adjusted accordingly.
- Project proposal
- 3. The performance of students in the 2018 calendar year is shown in Figure 6. The average grade amongst 34 students was a 94%.
Assessment of Learning Objective 4
Overall Preparedness to Successfully Gain Entrance into High Quality Graduate Schools, Admission to Professional Schools, and Securing Quality Careers in the Chemical Sciences.
Tracking of our graduates (where they are hired, and graduate/professional school acceptances) represents a major deficiency in our program. While we maintain contact with some students (acceptance into high quality graduate and professional schools such as UCLA, Penn State, Duke, New York School of Medicine, University of Utah School) we are lacking in a comprehensive understanding of how the whole of our student body meets their post baccalaureate challenges.
Chemistry Teaching Major Outcomes Data
Prior to student teaching all teaching majors must meet a State of Utah mandated passing score in the ETS PRAXIS exam in their content area. Knowledge of the content area for chemistry teaching majors as assessed by the PRAXIS exam for the past five years is below.
PRAXIS Physical Sciences Content Exam Scores 2016-2019
Chemistry, Physics and General Science
- Attempted 11
- Passed 7
Physical Science - Content Knowledge
- Attempted 1
- Passed 1
PRAXIS Physical Sciences Content Exam Scores 2011-2016
2016 Chemistry, Physics and General Science
- Attempted 1
- Passed 1
2016 Physical Science - Content Knowledge
- Attempted 2
- Passed 2
2015 Chemistry, Physics and General Science
- Attempted 0
- Passed 0
2015 Physical Science - Content Knowledge
- Attempted 0
- Passed 0
2014 Chemistry, Physics and General Science
- Attempted 0
- Passed 0
2014 Physical Science - Content Knowledge
- Attempted 6
- Passed 5
2013 Chemistry, Physics and General Science
- Attempted 1
- Passed 1
2013 Physical Science - Content Knowledge
- Attempted 0
- Passed 0
2012 Chemistry, Physics and General Science
- Attempted 11
- Passed 1
2012 Physical Science - Content Knowledge
- Attempted 1
- Passed 1
2011 Chemistry, Physics and General Science
- Attempted 0
- Passed 0
2011 Physical Science - Content Knowledge
- Attempted 1
- Passed 1
Chemistry teaching majors take the chemistry major core classes. Their outcomes data for the chemistry core are included in the data for the chemistry major.