This case study provides an example of the type of placement process that can occur within a structured state policy that allows institutions the flexibility to identify and guide students that are near a college-ready cut score. In this instance, Brazosport College in Texas set a “bubble” range for students beneath the state-determined cut score and guided these students through a holistic course placement process. This case study provides a description of the state policy, the placement process implemented on campus, and lessons that can be applied to similar institutions and systems.
Methodology and Data Sources
This is one of eight institutional case studies developed by Research for Action as part of a larger research project on the use of multiple measures for college placement. Each case study presents a snapshot of implementation based on a comprehensive document review and campus site visits conducted in the spring of 2016. Sites were selected based on state or system recommendations on leading institutions in multiple measures reform. Field work included interviews with nine administrators and seven faculty members. Institutional documents and online resources were reviewed prior to field work. Once drafted, this case study was provided to the primary institutional contact for review and verification.
Texas College Placement Policy: State Requirements with Some Institutional Flexibility
The Texas Success Initiative, implemented in August 2013, was designed to assess college readiness and improve developmental supports. Under this legislative mandate, all non-exempt degree-seeking students enrolling in a two-year or four-year institution are required to take the new Texas Success Initiative (TSI) Assessment, which consists of a standardized exam and computer-graded essay.1 The TSI Assessment determines placement for reading, writing, and math.
Students are exempt from taking the TSI if they meet cut scores on the ACT, SAT, Texas Assessment of Academic Skills (TAAS), Texas Assessment of Knowledge and Skills (TAKS), or State of Texas Assessments of Academic Readiness (STAAR) end-of-course assessments. Students are also exempt if they are a veteran, transfer in college-level coursework, or pursue a technical certificate rather than an associate degree.2 Students who are not eligible for exemption from the TSI Assessment are required to take the test. The test was developed by the College Board and is based on the Accuplacer.
The mandate also provides institutional flexibility to determine a “bubble score” just below the college-ready cut score. Students scoring within the bubble score engage in another set of multiple measures to determine if they should enroll in a college-level course, a college-level course with co-requisite support, or a developmental course. To place these bubble score students, institutions also consider: (1) high school grade point average (institutionally determined) and class ranking; (2) prior academic coursework and/or workplace experiences; (3) non-cognitive factors (e.g., motivation, self-efficacy); and (4) family-life issues (e.g., job, childcare, transportation, finances).3
The state gave institutions early notice that the policy would take effect and provided meetings and webinars to help inform institutions about the movement to the TSI. At the same time, the state mandated that developmental reading and writing merge into one course. To assist in assessment administration, the state requires students to engage in mandatory TSI Pre-Assessment Activities (PAA) to prepare for the exam. The state plans to raise the TSI cut scores in the fall of 2017 and the fall of 2019.
Background: A Small Regional College
Brazosport College (BC) was founded in 1968 to serve students of southern Brazoria County, a rural county 40 miles from Houston, TX.4 BC offers two four-year degrees but primarily awards two-year degrees and certificates. The characteristics of students at Brazosport College as of fall 2015 are outlined in Table 1.
Table 1. Student Characteristics at Brazosport College5
Primed for a Shift to Multiple Measures
Under the previous state policy, colleges and universities used Compass, then assigned students to specific courses depending on their score. The state mandated that institutions use cut scores, but each institution could determine the score. However, Brazosport College had been considering the use of multiple measures prior to the TSI mandate. Staff had been researching and were beginning to implement new placement methods based on recommendations from Achieving the Dream and had been using holistic advising practices that took into account past experiences and motivation level to help place students. When the TSI plan was unveiled in 2012, BC abandoned the system they were developing, but administration and staff were primed for the use of multiple measures.
Placement Process: Exemptions, State Assessment, and Holistic Advising
Multiple measures placement practice at BC consisted of the TSI, a range of exemptions, and, for some students, use of additional measures and holistic advising. First, BC determined if a student was exempt from taking the TSI. When students enrolled at BC, they were given information on the TSI policy. Students were encouraged to provide the institution with evidence of an exception if they had one. Criteria for exemptions from the TSI are outlined in Table 2, below. All other students were required to take the test.
Table 2. Exemptions from taking the TSI Assessment6
The BC placement process is depicted in Figure 1 below.
Figure 1. Brazosport College Placement Process
The Placement Process for Non-Exempt Students
Figure 1 maps out BC’s process for determining college placement for students who do not automatically place into college-level courses. Each step is described below.
Step 1. Preparing for the TSI: Before taking the TSI, students completed a mandatory online Pre-Assessment Activity (PAA) to help them prepare for the test. Students were led through a series of videos explaining the importance of the TSI, what to expect of the TSI, and how to prepare for the TSI. The preparation portion included tips for overcoming test anxiety, tips for taking multiple choice exams, and a link to the TSI Accuplacer App, where students could access sample questions and a practice test. Students were also given information on the math concepts covered by each of the math courses, with instructional videos and sample problems so students could gauge their familiarity with the concepts. Finally, students were encouraged to visit the Student Success Center for individualized help and tutoring. Students entered their student ID number after clicking through the PAA to indicate to the institution that they had completed the activities. However, it was possible for students to click through the pages without viewing videos or engaging in activities. After completing the PAA, the student was eligible to take the TSI.
Step 2. Taking the TSI: Non-exempt students were mandated to take the TSI at their campus testing center before registering for classes. Students could retake the assessment as many times as they liked.
Step 3. Assessing the TSI Results: The TSI provided a raw score that was used as the college-readiness cut score, along with sub-scores (known as diagnostic scores) on topics such as beginning and intermediate algebra and statistics. Decision points were as follows:
- If a student scored above a cut score they were eligible for college-level coursework.
- Texas provided colleges with the flexibility to place students in college-level courses, with or without co-requisite support for students who scored within a bubble range. BC chose a five-point range as a bubble.
- Students below the cut score, but within the five-point bubble score range, were referred to an advisor for a holistic advising session.
- Students below the bubble score received a recommendation to enroll in developmental coursework or, if they were far below the bubble score, ABE (Adult Basic Education). It is important to note, however, that students had the final say over their course selections.
Step 4. Holistic Advising: All students, regardless of their score, had the option of going through a holistic advising session that helped them plan their schedule, but the session was highly recommended for students in or below the bubble. For students in the bubble, placement was based on their TSI scores and other indicators listed in the TSI policy, such as their previous academic record, employment experience, planned degree and major, and work and family commitments. Advisors made a strong recommendation to each advisee. Advisors at BC documented each advising session to maintain a record of the recommended courses and reasons for the recommendation.
Implementation and Impact: State Support for Campuses, Campus Support for Students
Our examination of Brazosport College’s placement policies uncovered the following successes and challenges:
- Campus support for multiple measures was high, but respondents noted some areas for improvement. Both faculty and administrators believed the multiple measures process was superior to BC’s previous placement process of primarily utilizing the COMPASS exam, which was discontinued due to a lack of predictive value, and faculty noted they identified very few misplaced students each year. However, respondents also noted ways that the process could be improved. Respondents suggested following up with students placed through holistic advising mid-semester to determine if they felt they were properly placed. Some faculty respondents felt the placement process could also be improved by moving further away from standardized tests in favor of hand-graded essays and non-cognitive measures.
- The state provided helpful implementation supports through meetings, webinars, and PAA. The state communicated the TSI policy change to institutions about a year in advance, allowing institutions time to plan appropriately. Participants noted that the state also hosted informative regional meetings and online webinars to guide administrators through the change. In order to assist students statewide in preparing for the exam, the state also provided pre-assessment activities that institutional websites linked to in order to provide students with as much information about the exam as possible.
- While the state determined the cut score, the policy allowed for institutional flexibility to determine the right bubble score. State policy mandated the college-ready cut score, but the policy also provided institutions with the flexibility to determine the bubble score that was right for their population, allowing more students to partake in holistic advising to assist in their placement. This approach was considered a success of the policy, and respondents praised the use of a bubble score and holistic advising. However, multiple respondents stated they felt the bubble score should be larger, allowing more students to be placed through holistic advising and non-cognitive measures.
- Holistic advising allowed BC the opportunity to use a wider array of information when placing students but also increased the need for well-trained advisors and a more sophisticated student information system. The TSI policy allowed institutions to consider student characteristics beyond the placement test score, such as high school GPA, prior coursework or workplace experiences, non-cognitive factors, and family-life issues. This gave BC more data points to use when determining placement, but it also required more effort from advisors and added an element of subjectivity to placement. BC responded with the following supports:
- Placement training sessions for advisors,
- Creation of a flowchart to assist in utilizing the full range of data to determine placement,
- Adding a function to the student information system that allowed advisors to enter in notes on each advising session and to view notes on past advising sessions, and
- Training sessions for advisors on using the improved system.
In spite of these efforts, respondents stated there was still some variability and subjectivity in placement decisions. While some advisors exhibited a preference for placing students in “reach” classes, others preferred to place students in “safe” classes.
Students were given information about the placement process in advance and required to prepare for the TSI.7 BC counselors provided incoming students with a four-page brochure explaining the TSI policy and process. It explains what the test is, what the exemptions are, and goes over question types and some preparation strategies. It also refers students to the mandatory PAA. The amount of information and preparation students are given was considered a success of the policy. While students were given information on holistic advising, interview respondents noted the challenge that students still understood the process to be mainly test-driven. Respondents referenced students re-testing multiple times or visiting the Success Center for tutoring to improve their score by a point or two. Respondents indicated that requiring students to participate in the PAA was a step in the right direction; however it was possible that some students are clicking through the exercises without engaging in them.
- The placement process may benefit from increased data collection on the efficacy of multiple measures. While respondents felt BC’s placement process was an effective way to place students, they also recognized that improving data collection could strengthen it. Multiple respondents suggested asking bubble-score students mid-semester if they felt they were placed correctly or tracking their success in college-level courses for a year. Both of these actions would allow the institution to make a data-based decision on whether to expand or contract its bubble score. More data collection on the use of the PAA and the success of students who are placed solely on the TSI or other assessments are also mentioned as areas for data collection improvement.
Lessons for the Field: Multiple Supports for Students in the Bubble
Through interviews with administrators and faculty, our case study of Brazosport College examined how the institution responded to a statewide multiple measures policy that allows for institutional flexibility in identifying and supporting students near assessment cut scores. From this research, the following lessons may be applied in other states and institutions:
- Use of a bubble score can help identify students whose capabilities are not accurately reflected in their test scores. Many placement processes rely on a hard cut score to determine if a student is ready for college-level coursework, but students close to the cut score may benefit from a closer look at their capabilities. Bubble scores can help institutions identify students who would benefit from an additional level of placement measures, giving institutions the opportunity to more thoroughly evaluate if a student is college ready.
- The use of non-cognitive measures and holistic advising provides more data points to assist in placement. Rather than relying on one or two standardized tests to place students, examination of a range of information—including previous experiences, motivation level, and current life responsibilities—may give a more complete picture of a student’s capabilities and help advisors more accurately determine appropriate course levels.
- Consider making placement test preparation mandatory and applicable to a variety of learning styles. TSI policy mandates that students complete pre-assessment preparation, and BC adds additional supports by providing in-person test preparation at the tutoring centers. These supports accommodate a variety of learning styles, giving students the option to prepare online, through videos and practice exams, or in person with knowledgeable instructors. These supports provide content knowledge and also let students know what to expect from the test.
- Build in processes for evaluating the efficacy of multiple measures. Institutionalizing data collection on the success of students placed through different placement measures can improve data-based decision making. This could also help institutions determine which measures are more or less successful in placing students and when cut or bubble scores could be adjusted.