Creating a Healthy Analytics Culture on Campus
Differing definitions in higher education
In higher education, new ideas have the potential to take off quickly. With a plethora of publications, Listservs, and professional networks, good concepts can spread from institution to institution in a matter of hours. Ideas like microlearning, competency-based assessment, student-centered learning, and immersive experiences began, at some point in time, on one campus or in the mind of one individual. Today, they have become commonplace at institutions across the globe. Yet, as each concept has been passed along and adopted on different campuses, it has been adjusted and branded for the unique needs of that institution. Analytics — including big data analytics — is one such concept, with various campuses using the term to potentially reflect different activities.
From descriptive and explanative to predictive and prescriptive
The true power of analytical tools lies in the understanding that in higher education, the data points represent students. Unlike when large corporations use algorithms to predict consumer behavior, colleges and universities are chosen by students and parents to help them grow and develop. In short, there is a higher level of expectation for how student data will be used by a campus. Gone are the days of merely describing or even explaining institutional statistics. For individuals conducting research about students on a campus, the types of questions being posed have noticeably changed over the past decade. We have moved — rather quickly — to asking predictive questions, and in many cases determining how to seemingly shape the future. In other words, we’ve advanced from asking what is happening to why is it happening to what will happen to how can we make something happen.
From outputs to insights
With each progression, the contributions of data scientists and researchers have become increasingly important. And this increasing importance has not necessarily required conducting more research — just learning the right questions to ask and the right ways to bring data together from across campus silos to answer them. The future of institutional research means far more than simply creating outputs for others to consume. What’s needed is a clear transition from data as simple output to data insights that help directly motivate action and change for a better student experience. In a competitive higher education environment — with internal and external pressures mounting daily — it is the job of data scientists and institutional researchers to ensure campuses are fully prepared to make optimal use of their data.
So what principles can guide the creation of a healthy data culture on a campus?
1. Data points equate to students
Every piece of data on a college campus relates — directly or indirectly — to students. These students are not databases and spreadsheets of random data or mechanical entities; they’re human beings, and typically young, developing ones at that. Institutional researchers are interacting with information that both illuminates and impacts the lives of students on campus. As one example, a predictive model based on noncognitive factors may suggest an entering freshman is only 20 percent likely to be academically successful on campus. While knowing this information can be useful, it is not enough to simply run a model. Institutions should then leverage this insight to improve the odds for student success. It could be used to alert members of a student’s success network that the student may need some extra help or encouragement to reach their maximum level of success. Ultimately, stakeholders should remember that data points represent actual human beings who want to succeed in multiple facets of life.
2. Practical significance trumps statistical significance
The campus data culture should include a sound data ecosystem that helps institutional researchers and effectiveness professionals to avoid “analysis paralysis,” along with the temptation to cherry-pick statistics or fall back on isolated anecdotes. Rather than worrying about showing we are correct through the intentional selection of outlying cases or becoming too engrossed in designing novel research or analytical methodologies, we need to focus on identifying actionable data points and relationships that can be used to improve a campus. And being too idealistic or too clinical can lead us to overlook meaningful results. For example, maybe there isn’t a statistically significant difference in retention for students who live on campus their first year compared to those who do not. But even if the gap is four or five points, enrollment managers might still want to hear about the finding.
3. Don’t be afraid to data mine
For a trained social scientist with years of experience, it is not easy to approach data without a research question or hypothesis in mind. But, in the interests of students, sometimes allowing our data to lead us to questions or trends to explore can be highly beneficial. After all, the goal of institutional research is to benefit the campus and its students. This does not, however, suggest that researchers should simply fire darts at the dartboard. Instead, they should be willing to follow the data and conduct after-the-fact analyses in order to inform future predictive and prescriptive decisions. Thus, while data mining may not be something a veteran researcher would look fondly at, a willingness to follow the data trail can help a campus address questions they may not have even envisioned yet.
4. Decisions should be thoughtfully data-driven
At the end of the day, many — if not most — higher educational professionals claim they are in favor of data-driven decision-making, but remain largely unwilling to set aside personal beliefs when the data suggests they may not be entirely correct. A healthy analytics culture uses available data points from across all areas of campus to augment decisions that still include a degree of the human touch. Consider predictive analytics. Perhaps a campus has established a model for applicants that produces the odds of the applicant retaining for their first year and then for graduating on-time. While the spreadsheet should unquestionably be considered as part of the admissions process, once validated and found to be reliable, should a model automatically generate acceptance and denial letters? Likely not. Admissions officers should use data to drive decisions but also allow for human capital to weigh in as necessary.
Rather than being mere stargazers or technical experts, institutional researchers will benefit from remembering that every piece of data that comes across their desk directly represents or impacts students in some way. A healthy analytics culture on campus ensures data is collected and used to show improvement and effectiveness — for the benefit of the entire community.
