August 14, 2018
Every year since the establishment of Computer Science in the 1960s, 30-60% of CS college majors have failed their Introduction to Computer Science course because they simply could not learn to program. Despite hours of studying and tutoring, most of these underperforming students struggle with, and many ultimately give up on, programming as a career. So here is the ultimate question- Can a simple algebra test predict programming aptitude?
Education researchers have been looking for the answer to that question for well over five decades. In that time, they’ve looked at dozens of possible predictors of programming aptitude, including:
Most of these studies have found little to no predictive power in the variables they looked at. Or worse, when they do find something that’s predictive of programming aptitude, the techniques appeared to be so complicated that other researchers are unable to replicate their results at another university. Our impression is that research in this field has slowed. At this point, there seems to be no silver bullet for predicting one’s aptitude for early programming success.
It’s fairly common for incoming Computer Science majors to ask the question, “Why do I have to learn all this math if I just want to learn to program?” The correlation above suggests a possible answer: The ability to understand basic mathematics is likely correlated with the ability to “think algorithmically,” which is well-known to be a foundational skill for expert programmers. Computer Science was founded in mathematics, after all.On a more practical level, we’ve possibly found a simple method for sorting out the ninja programmers from the less capable programmers. So, if you’re a college student considering a major in Computer Science, take a look back at your algebra scores from high school to get a sense of your algebraic aptitude. And if you’re a recruiter for a growing tech company, try adding a basic algebra test to the interview process when hiring new programmers. In both cases, a 1-hour math test could save you thousands of dollars.
At the beginning of the Codeup Bootcamp program, students are given between 35 and 55 minutes (depending on the test version) to complete a basic algebra test. The test contains 25-40 questions, such as:Adriana's age is 1/3rd of her dad's age. If her dad is 36 years old, how old is Adriana?andThere are 3 consecutive integers with a sum of 69. What are they?No partial credit is given.To determine a student’s overall stack ranking, we have our instructors periodically rank each student according to how well they have been performing in the current programming module. Students are assigned a 4.0 if they’re at the top of their class, and a 1.0 if they’re struggling to understand and apply coding concepts. The majority of the students receive rankings of 2.0 and 3.0 such that the stack rankings for the entire class form a normal distribution. A student’s overall stack ranking is simply the average of their stack rankings over the entire Bootcamp.The chart above shows the linear regression over the performance data of all 62 students who have graduated from Codeup Bootcamp. These 62 students come from 3 separate cohorts who took 3 distinct algebra tests. Scroll to the graphs at the bottom of this post to see that, when applied to the data from individual cohorts, all linear regressions show the same positive correlation between test score and programming aptitude with p < 0.01. This finding has therefore been replicated twice with three distinct cohorts of students.In summary, these math tests – along with stack rankings – have been a handy diagnostic tool for us, and we’d like to share our findings with the world in an effort to both shed light on our own experience and hear plenty of feedback from others. In this new and growing industry, there is so much to learn, and we hope that this study provides food for thought and space for conversation.Feel free to send lots of comments, questions, and suggestions our way!(See discussion about this post on Hacker News: https://news.ycombinator.com/item?id=8741868).Interested in learning more about our program and admissions process? Contact us here and we would be happy to chat!