Executive Summary
The School of Journalism & Graphic Communication (SJGC) follows the University’s STARS process to assess student learning annually, collecting data each semester. The goals for student learning include ACEJMC’s Professional Values and Competencies. An annual Program Evaluation Plan includes five objectives, each linked to student learning outcomes and each having two assessment measures (direct and indirect). The school’s Assessment Committee reviews the data, and the faculty discusses assessment at their division meetings. The assessment process aligns with the University’s strategic priorities and goals. Each September, a report is sent to the University assessment office, and the faculty reflects on outcomes to suggest improvements. Using the STARS framework strengthens the assessment continuity for SJGC programs. In 2020, the faculty approved a revised curriculum that reflects both changes in industry and curriculum in competing universities. Assessment measures have been added for the new courses as well.
Assessment of Student Learning Outcomes – Written Assessment Plan
The unit utilizes a holistic approach to assessment and continuous improvements to ensure that it offers quality degree programs. SJGC maintains a focus on consistent identification and evaluation of salient student learning outcomes (SLOs) through FAMU’s annual assessment process utilizing the STARS framework for programmatic assessment. Within this structure, faculty and staff within SJGC formulate SLOs aligned with the course goals and ACEJMC competencies, which are tracked using a variety of direct and indirect assessment measures.
During the year, SJGC discusses student performance on the assessment measures and what the results mean in relation to curriculum and instruction for the programs. Assessment reports include reflection on how to drive improvement forward within SJGC; faculty and staff return to the assessment process in the following year to reflect on the impact of implemented strategies and to close the loop on continuous improvement efforts. This process facilitates compliance with ACEJMC’s Standard 3: Assessment of Student Learning Outcomes.
The annual assessment of student learning is guided by the STARS assessment process. The assessment plan is entered into FAMU’s assessment platform, Nuventive, an online system that helps educators manage assessment data.
Data is collected from faculty each semester; this information along with analysis of the data, improvement narratives, attachments, and an overall reflection are entered into Nuventive. This information is also recorded in a Program Evaluation Plan and distributed to faculty for their review. The plan lists the standard, the program goal/outcome for the standard, the assessment tool, the data results, a reflection of continuous improvement and faculty who are responsible for assessing the measure. Within this framework, the unit is responsible for submitting an annual plan. In compliance with FAMU’s requirements, Assessment Plans must have at least five (5) objectives with at least two (2) assessment measures per objective. At least one assessment measure must be a direct measure. Currently, Journalism has 7 active objectives, each of which is a student learning objective, and 22 assessment measures. Of those assessment measures, 15 are direct and 7 are indirect. These plans are updated on an annual basis and posted on Nuventive.
Each September, FAMU’s Office of University Assessment reviews assessment reports for the prior year and plans for the current year to ensure that all instructional programs are routinely pursuing continuous improvement and closing the loop on enhancements from year to year. The objectives reflect the course objectives that align with the program goals, school strategic plan, and University strategic plan. The objectives are aligned with contemporary concerns and competencies necessary to provide students with the necessary foundation to excel as communication professionals.
Contemporary concerns such as the impact of diversity on journalism and mass communication in a domestic and global society, use of digital media in journalism, and the First Amendment and freedom of the press are represented in the student learning objectives to evince their integration into the curriculum.
Following the STARS framework and submitting an annual plan and report to Florida A&M University’s Office of University Assessment has strengthened the unit’s leveraging of the continuous improvement process. Faculty collectively, through SJGC’s assessment committee, define expected outcomes and measures to demonstrate success as outlined in each step of the STARS process.
Through this process, the unit has consistently identified the core competencies and reflected those in the objectives set forth for assessment, set assessment measures and benchmarks for success that lead to actionable data that can be used to craft strategies to enhance student performance, and circle back to consider the impact of those strategies. Thus, the systematic process of assessment facilitates a method to address curriculum and instruction, and to make the journalism and public relations programs responsive to the rapidly changing digital world.
Defined SLOs and Integration of Values & Competencies (Learning Outcome Statements)
The approach towards quality assessment for the journalism and public relations academic programs centers on annual assessment of SLOs that are aligned to ACEJMC’s core values and programmatic goals, using course-level assessment which includes formative and summative feedback. Upon graduation, it is expected that graduates of SGJC will demonstrate pertinent skills applicable to their profession and the ability to increase the nation’s workforce, through competency in the following objectives:
- Communication
- Critical Thinking
- Content/Discipline Specific Knowledge
- Cultural Diversity
- Research
- Technology Literacy
- Ethical Values
These objectives were used to frame SLOs that convey the skills that graduates must gain to enter the profession. The objectives were built to represent the intersection between course learning and the Council’s professional Values and Competencies.
The objectives adopted for assessment are
presented in the table below.
| Major | Direct | Indirect |
|---|---|---|
| Journalism |
|
|
| Public Relations |
|
|
Indirect & Direct Measures
SJGC employs a mixture of direct and indirect measures to evaluate student learning in a summative and formative capacity. To develop the current measures, faculty reviewed the course-level outcomes to determine where a specific SLO should be assessed. Additionally, the faculty team reviewed the SLOs to discuss alignment with courses to confirm where assessment of the SLO would occur. The team then identified what assessment was occurring within the course to determine what assignment or task would be used as the assessment measure. Assessment criteria were established based on previous performance by students to determine an aspirational benchmark for success.
Direct measures vary depending on the type of course, but may include pre- and post-tests, rubrics, assessments within comprehensive examinations and written assignments, and evaluations of internships and documentary production (see below for specific examples). The unit’s indirect measures are exit surveys. SJGC uses the University’s Graduate Exit Survey that is administered every semester to every graduating student. Additionally, the unit is developing an exit survey to administer to students as they conclude a course, in addition to items to add to the FAMU Graduate Exit Survey. These surveys are intended to pose questions specific to the program coursework.
Direct Measures
The following direct measures have been used in the assessment process to provide a variety of methods to gauge students’ performance on the SLOs:
- Pretest/Posttest: Administered every fall and spring semester for students in the following courses: JOU1040 Language Skills for Media Professionals, JOU1112 Use of Information Resources, JOU2100C Reporting & Writing I and MMC2000 Intro to Mass Media. The goal is to test students’ knowledge of topics related to introductory core classes.
- Internship Evaluations: Every student in journalism and public relations is required to complete an internship with an approved company before graduation. These internships include an evaluation of performance using a 5-point Likert scale, performance is evaluated to determine areas such as critical thinking, writing, and professionalism. The internship evaluation form completed by the journalism and mass communications professionals who supervise students on the internship. Professionals answer whether the student met deadlines, if they carried out assignments in a timely fashion and their overall professionalism in the workplace. The internship supervisor is encouraged to give comments on the student’s overall performance and recommend a grade for the intern.
- Project-driven assignments: Various courses assess students’ knowledge through projects assigned during the course. For example, MMC4481, Communication Law & Media Ethics has a First Amendment Project that gives students the opportunity to develop a creative project that addresses ACEJMC’s competencies related to knowing and applying the First Amendment. In PUR3622 Social Media/Data Analytics (typically offered in the fall), public relations and journalism students create a social media campaign proposal. Based on the market trends and analytics for a target audience, audience engagement and increased sales, as well as lessons learned from a social media simulation program, students create a comprehensive social media plan to target travelers and shoppers during the holiday season.
- Research Paper: The PUR4500 course culminates in a 20-page original data research paper covering topics as diverse as HBCU students’ understanding of financial aid and how colorism plays a role in all racial groups and cultures. All public relations students are required to take this course. Students in the course are evaluated based on their mastery in the development of each chapter included in a social science research paper. The course also encourages the students to learn about multicultural history as it relates to research paper topics.
- Capstone Presentation/Professional Review: The journalism and public relations curriculum ends with a final Capstone course and presentation that takes place in students’ senior year. This course encapsulates all that students have learned and gives them the opportunity to present a summary of what they achieved during their time at FAMU. Included in the presentation are a website portfolio, review of student’s resume, visual/digital and creative work, and 10 articles/press releases written over the course of their tenure at FAMU. In addition, students are given questions related to communication law and ethics. The evaluation of the project and presentation is done by a combination of faculty members and program alumni/media professionals.
Indirect Measures
The following indirect measures are leveraged to provide a comprehensive picture of student learning, by gathering data related to the intangible aspect of learning – how students feel about their competency on the SLOs:
- FAMU Exit Survey: The Office of University Assessment conducts a survey of graduating seniors in every School. The survey asks students about their experiences at FAMU, rating them via a 5-point Likert scale. For example, questions range from students’ satisfaction level with the registration process, financial aid, and academic advising. For the academic experience, students assess their abilities to express themselves verbally and in writing, their ability to adhere to high standards of integrity, their ability to problem solve and show consideration of others. These questions are then applied to courses that are related. Students’ responses to these questions are generally very positive about our programs.
- FAMU Student Satisfaction Survey: The Office of University Assessment conducts a survey measuring student satisfaction in many areas. This allows the unit to get to the students’ perception of the school and their critical thinking skills, communication skills, and general feedback about their courses.
The Annual Assessment Plan and Report for 2023-2024 academic year is shown below as evidence of the program’s commitment to quality assessment of student learning outcomes. It includes an assessment measure for every course taught in both the journalism and public relations curriculum. The faculty participates in continuous improvement of their measures to ensure they effectively provide knowledge and reflect the ACEJMC values and principles as core components of the SLOs for each program (see Table AY2023-24 Assessment Plan and Results below).
Table AY2023-24 Assessment Plan and Results (Journalism & Public Relations)
| Student Learning Outcome | Measurement Criteria | Actual Results | Use of Results for Continuous Improvement |
|---|---|---|---|
| Students will demonstrate knowledge of the mechanics of the Associated Press style and the ability to apply those mechanics to their written communication. | JOU1040-A 20% increase in overall class average from pretest to post-test on the comprehensive test in JOU 1040 (Language Skills for Media Professionals). | Fall 2023: the average pre-test was 71%, the post-test was 68%, a decrease of 3%. Spring 2024: the average pretest was 63%, the post-test was 72%, an increase of 9%. Summer 2024: the average pretest was 64%, the posttest was 85%, an increase of 21%. | Faculty need to discuss the difficulty that students have in improving their grammar exam grades. |
| JOU2100- A 20% increase in overall class average from pretest to post-test on the comprehensive test in JOU 2100 (Newswriting and Reporting I). | Pretest Section 01 had a 48.2% average. Section 04 had a 56.0% average. Post-test Section 01 had a 55.3% average. Section 04 had a 66.0% average. Students had an average score of 51.86% on the pretest. The highest score was 75 and the lowest was 30. Students had an average score of 60.31% on the post-test. The highest score was an 85, with a 35 being the lowest score. | Looking closer into the numbers, among the 32 students that took both exams, 20 of the students saw an increase from the pretest to the posttest. Among the remaining 12, 7 received the same score, with 5 students recording scores on the second test that were lower than their first test score. | |
| JOU4181- 70% of stories will produce video stories for production on The FAMUAN. (Capital Bureau) | Fewer than half of the 20 students in the classes produced video stories, but those who did often produced as many as three or four during the course of the semester. | One of the challenges for Capital Bureau course is developing relevant stories in the fall, when the Legislature is not in session. During the fall term the students are reporting on the implications for laws that have recently gone into effect. They are also expanding the government landscape for stories by including the local school district as well as the city and county commissions. | |
| JOU4342- 70% of students will publish 10 or more stories in the course. (Newswriting & Reporting III) | 80%, 16 of the 20 students, completed the goal. | (JOU4342) It is a work in progress. Some students enter NW&R 3 with little if any confidence in their writing skills, despite having passed NW&R 1 and 2. In some cases the professor will work with the student one-on-one in their office to help them better understand how to execute a story. That often goes a long way toward overcoming a student’s lack of confidence, and the quality of their work improves quickly — and sometimes dramatically. | |
| Exit Survey- At least 75% of students will agree/strongly agree that the program enhances their communication skills. | 97% of the students strongly or somewhat agreed that their skills in communication increased. | Students believe their communication skills have increased. Should the unit increase the requirement for meeting criteria? | |
| Students will apply tools and technologies for the developing journalism world, including digital media. | RTV3322- 70% of students in RTV 3322 (Advanced TV News) will score a 3 or better on a rubric assessing their ability to use current technical/electronic journalism tools. | 53% of students scored a 3 or better on the rubric assessing use of electronic journalism tools. | In the two classes, 8 out of the 15 total students passed the assignment, but 4 students did not turn in the assignment and 3 failed the assignment. |
| JOU1112- 80% of students will earn 75% or higher on the post-test. (Use of Information Resources) | Final scores on the posttest ranged from 71-97. 100% of the class earned a C or higher on the post test. | Measure was met. In future measures, the unit will raise the expectations of the students’ final grade. Also, focus will be given to the writing component. | |
| RTV2230- 70% of students will score 80% or higher on this project. (Multimedia Oral Engagement) | Fall 2023 Results: 90% of students passed. Spring 2024 Results: 73.6% of students passed. | In the future, have students do solo podcasts for this assessment rather than co-host. | |
| RTV2520- 60% of students in the course will earn at least an 80% on the assignment. (Visual Storytelling) | 37 students submitted a final project. Among them, 24 students received an 80 or higher (65%). Seven students received a 90 or higher (19%). In the two classes, 8 out of the 15 total students passed the assignment, but 4 students did not turn in the assignment and 3 failed the assignment. | Looking closer into the numbers, among the 32 students that took both exams, 20 of the students saw an increase from the pretest to the post-test. Among the remaining 12, 7 received the same score, with 5 students recording scores on the second test that were lower than their first test score. | |
| Students will be able to analyze professional ethical principles and relate said principles to working ethically in pursuit of truth, accuracy, fairness, and diversity. | MMC4920- 80% of students will earn a 4 or higher (out of 5) on an ethical scenario. (Senior Capstone Colloquium) | 52.6% of the students made a 4 or higher on the Ethics question in Capstone. | This measure was recently adjusted when the Communication Law & Ethics courses were combined. To address the poor results, the unit will have more dedicated class time in the Capstone course reviewing the Ethics topics. |
| Exit Survey – 80% of students will strongly agree that they are satisfied with their ability to adhere to a high standard of academic integrity and personal conduct, based on senior exit survey. | 93.02% of students agreed they adhere to standards with high integrity and personal conduct. | Students strongly believe they have high ethical standards. | |
| Upon graduation students will be able to identify the principles and laws of the First Amendment, including freedom of speech and freedom of the press. | MMC4200/MMC4181- 75% of the students will earn 8 out of 10 points on a rubric assessing the quality of the students’ creative artifacts in its explanation and application of the First Amendment to real life issues. (Communication Law & Ethics) | In Communication Law & Ethics (MMC4181) Spring 2024, students earned a 7.33 average score on the assignment, but when students who did not complete the assignment were removed, the average increased to 8.76. In total, 81% of the students earned an 8 or higher on the assignment. | This is a good method for the students to learn about the First Amendment in practice. |
| MMC4920- 80% of students will earn a 4 or higher (out of 5) on a Law-related question. (Senior Capstone Colloquium) | 48.7% of students earned a 4 or higher. | This measure was recently added when the Communication Law & Media Ethics courses were combined. To address the poor results, the unit will have more dedicated class time in the Capstone course reviewing the Law topics. | |
| Students will agree that they can distinguish the range of systems of freedom of expression around the world. | MMC4200/MMC4181- 75% of students will score an 18 out of 20 on an essay comparing freedom of expression rights in the US with other countries. (Communication Law & Ethics) | In the Fall, one class scored an average of 15.26, but when students who made a zero were removed, the average was 18.2. The second class had an average of 15.39. When students who made a zero were removed, the average was 18.14. The Spring course was not given the assignment. | The assignment was initially a one-day assignment, but the professor extended it to two days, with positive results. Students seem to attain a lot out of the assignment. They generally learn about the limitations other countries may have as it relates to freedom of expression. |
| Exit Survey: Students will agree that they can distinguish the range of systems of freedom of expression around the world. | No measures were taken. | The creation of a student survey that questions students’ course competencies (self-analysis). | |
| Students will demonstrate knowledge of the history of mass media and the professionals who have shaped communications to this point. | MMC2000- 20% increase in the class average from pretest to post-test. (Intro to Mass Media) | Pretest average is 41 and the post-test average is 84. This is an increase of 43%. | Since adapting Achieve (MacMillan’s online course) and moving online, the students’ grades on the pre and post-test have increased steadily. |
| Exit Survey- 80% of students will agree/strongly agree that the program enhances their knowledge of the professional principles and history of journalism | 88.02% of students agree that they have a level of competence in their field of study. | Students believe they have competence in their area, which is connected to their knowledge of principles and history. | |
| Students will effectively apply learned theories of journalism to presentations of images and information in the field. | MMC2920- 70% of MMC2920 students will get a job after graduation in the career field they listed in their college career plan. (Professional Development Colloquium) | Students’ college career plans were collected, but they are still in school, so we are not yet able to determine the success of their choices. | The unit will begin following up to determine how many college career plans were aligned with their immediate post-graduation careers. The faculty will discuss if this is an effective measure. |
| MMC4945- 85% of students will be rated a 70% or better by their internship supervisor as it applies to their critical thinking skills. (Internship) | Fall 2023, 92.85% of MMC 4945 students earned a score of 70% or better on their internship supervisor evaluation, as it applies to their critical thinking skills. Spring 2024, 80.76% of MMC 4945 students earned a score of 70% or better on their internship supervisor evaluation, as it applies to their critical thinking skills. | This evaluation form gives a rich amount of data. In the future, we will use more of this in assessment. Also, review the data as faculty to see where improvements should be made. | |
| RTV3332- 40% of students’ projects will be chosen for final screening. (Documentary & Film) | Ten (10) documentaries were produced in the class. Fall 2023 – five (5) documentary teams/JSJ = 10 students/15 students in the class: Front Line, Pretty Perfect, Bearing The Burden, Florida Book Bans, Counted Out: FAMU Football. Spring 2024 – five (5) documentary teams/JSJ = 11 students/only 13 students in class: Roll N’ Soul, One Nation Under God, A Piece of Home, Sound It Out, When We See Ourselves. | Working to increase attendance at the public screening and greater audience awareness throughout the region will continue. This includes seeking out more assistance from our PR Agency course and utilizing the student learning outcomes of both journalism and public relations students more. | |
| Exit Survey- 80% of students will strongly agree that they have the ability to apply theoretical knowledge to practical situations. | Exit Survey – 97.67% of students agree that they have the ability to apply theoretical knowledge to practical situations. | Students overwhelmingly feel like they can apply theoretical knowledge in the field. | |
| Students will demonstrate a knowledge of the history of public relations and the professionals who have shaped communications to this point. | PUR3000-Introduction to Public Relations – The history of pre/post- test in this class has been ineffective in understanding and predicting student academic performance. Therefore, the professor chose to have discussions with students early in the class to better understand their base knowledge of public relations and to devise an effective course plan to ensure maximum student mastery of the topic. | Instead of pre/post-tests, students and the professor held discussions at the start of the class about their knowledge and understanding of the public relations profession. Based on previous assessments using the Pre/Post tests, student class performance could not be sufficiently assessed, thereby using a qualitative discussion method in its place. However, final class grades at a C level or higher for 80% of the class were a better assessment of students’ class topic mastery/performance. In the Spring, it was found that 67% of students mastered the course content at the 80th percentile. This means that the assessment measure goal was not achieved. For the pretest 36% of students who took the pretest earned a score of 75% or higher on the test. | The assessment measure allows for maximum student mastery of the course topic. Faculty assessment will be based on students’ final grade. |
| Exit Survey- 80% of students will agree/strongly agree that the program enhances their knowledge of the professional principles and history of the public relations industry. | |||
| Students will be able to apply tools and technologies for the developing world of public relations, including digital media. | PUR3622- 80% of students will score a B or higher on the project. (Social Media/Data Analytics) | 80% of the class (4/5 groups) earned a B and above, and one group scored a C. | Will create an assignment or element within the project that will give an individual grade for students instead of group grade. |
| PUR4800- 80% of students will earn a 75% or higher on the posttest. (PR Agency) | For the pretest 36% of students who took the pretest earned a score of 75% or higher on the test. For the post-test 63% of students who took the post-test earned a score of 75% or higher on the test. | Creating an assessment that can be a long-term assignment for the class. | |
| MMC3603- 80% of students should earn a 75% or higher on the final PR Plan. (Strategic Communication) | For the final presentation 12 students earned 90%+, 5 students earned 80%+, and 2 Students earned 75%+ while 5 students earned below 75%. Therefore, 79.17% of the students were successful in the class. | This assignment achieves its goals of making sure students can create a public relations plan. | |
| Exit Survey- 80% of students will indicate they agree that they are satisfied with their ability to use technology to support classroom learning. | 95.35% of students agree they are satisfied with their ability to use technology to support classroom learning. | A majority of students feel strongly about their ability to use technology to help their classroom learning. | |
| Students will demonstrate ability to utilize research skills, methods, and concepts appropriate to the public relations profession | PUR4500- 80% of students will earn a 75% or higher on the research paper. (PR Research & Strategies) | 77% of the submitted class papers mastered the course subject matter at the 80th percentile. This indicates that the assessment measure was not met. The assessment measure sought to demonstrate the overall class passage rate at the 80th percentile. | Future assessments will be based on a rubric. |
| Exit Survey– 85% of students will agree/strongly agree that the program has enhanced their research skills. | 95.35% of students believe that the program has increased their ability to do research. | A majority of students agree that their ability to do research has been enhanced. |
Application of Data to Improve Curriculum & Instruction
The School of Journalism & Graphic Communication has a standing Assessment Committee, appointed by the dean and composed of faculty representing all areas of study. Members of the Assessment Committee review data collected at the course-level to determine if defined student learning outcomes are met on an annual basis. During monthly division meetings, the Assessment Committee discusses these findings with faculty. The committee and faculty collaborate on analysis of the results and construct improvements based on their analyses. These improvements comprise various actions including modifications to courses or assessment measures, regular review of processes and methods of evaluation, and revisions when needed. The Assessment Committee works closely with the curriculum committee to ensure that the measures align with student learning outcomes (SLOs) and what is addressed in the curriculum.
The Assessment Committee formulates formal recommendations to the faculty regarding proposed changes for improvement. Faculty who teach courses work with the SJGC Assessment Committee to determine if changes should be adopted and implemented. Courses with multiple faculty teaching appoint a captain responsible for overseeing assessment and submitting data. If elected to be adopted, a plan of action and timeline for implementation follows. Some of the questions considered in this phase are: (1) What worked well?; (2) Did previous improvements result in positive outcomes?; and (3) What future improvements are needed to continue to strengthen the academic programs?
Utilizing the steps outlined above, program faculty can approach assessment of student learning outcomes in a more meaningful way and consistently over a period of time. Proposed changes to assessment plans are evaluated to ensure that the SLO’s determined by faculty were written in measurable terms, that the appropriate level of Bloom’s Taxonomy was achieved based on the faculty’s expectation of student learning, and that appropriate targets were identified using direct and indirect measures of evaluation. The Assessment Committee and faculty capitalize on the actionable data collected through the assessment process to not only make modifications to curriculum, instruction, courses, and assessment measures, but to track the impact of these changes from year to year to identify what strategies and changes have enhanced student learning and where there is still room for improvement.
By “closing the loop” on the assessment cycle on the subsequent year’s assessment report, journalism and public relations faculty have an opportunity to gauge the impact of continuous improvement efforts. Based on the assessment, the faculty approved a revision of the curriculum in journalism and public relations. The goal of these revisions was to improve curriculum and instruction, better align the SJGC courses with those of equivalent and competing journalism programs and to update the program to align with skill sets necessary in current media industries.
The update included:
- Communication Law and Media Ethics courses are combined to create Communication Law & Ethics. This was done to bring the school in line with the University required credit hour limits. Additionally, research found that other universities that have a Journalism program offered this combined course.
- For Journalism students, JOU4181 SJGC Capital Bureau and for public relations students, PUR4800 PR Agency were created to give students practical experience in professional settings and better prepare them for the expectations awaiting them as media professionals. For the Capital Bureau course, students cover the Florida Legislative Session held between January and March and any other newsworthy events in the Capitol. For PR Agency, students are hired by an outside company to create a PR campaign.
- MMC2605 Social Justice, Thought & the Media has been added to address the growing importance of social justice and advocacy as an area of journalism.
We are still incorporating these courses into the program and are beginning to assess them. The Journalism Division will continue to track these measures to determine the lasting impact of the curriculum modifications.
Link for the ACEJMC Assessment Matrix
