Probability Theory, the LSAT, and You (Part 3)
Today, let’s look at a specific use case of Bayesian probability theory on the LSAT!
(If that sounds like gibberish nonsense, check out this post to learn about Bayesian probability theory and this post to relate it to our favorite law school application test.)
In Part 2, I promised to write more about this potential benefit of applying Bayesian inferences to the LSAT:
“Comparing the answer choices we haven’t ruled out once we’ve read them all”
It comes down to a fundamental truth about all tests, in fact: there’s a difference between figuring out which answer is right and which answer is better. And as it turns out, this difference can help us save time, reduce stress, and maybe even score some points.
Let’s discuss.
Right vs. Better on the LSAT
When we think about answer choices on the LSAT, we usually think about right versus wrong. And that’s totally fine—all answer choices are one or the other. Our LSAT scores depend on how many questions we find the right answers for, naturally, so we’re pretty invested in doing just that.
So isn’t it true that the right answer is always the better answer?
In my opinion, no. In fact, it’s possible that the wrong answer is the better one.
Here’s why:
When we think of right/wrong, we think in terms of the answer key. Which choice will directly impact our score?
But when we think of better/worse, we can involve other factors at play in the LSAT. By weighing our options as better or worse, we’re asking ourselves to account for factors like question difficulty, timing, and stress, in addition to getting the raw point from that single question we’re grappling with.
An Example
Consider this situation:
You’re stuck between (A) and (D) on question 22 of a Logical Reasoning section. It was a behemoth of a question, and you don’t feel confident about either of those choices — although you’re pretty sure the correct answer is either (A) or (D).
The right answer would be the choice that correctly answers the question, of course. But what if it took you another two minutes to figure out which option was right? What if aiming only for correct answers led you to run out of time, and you left questions 25 and 26 blank?
If this seems familiar, then you’re in good company! This is a problem many people face in the bigger “danger zones” on the LSAT—especially questions 18 to 22 on Logical Reasoning, for example.
Generally speaking, the questions at the end of an LR section tend to be a bit easier than the questions almost at the end. By leaving questions blank, you’re giving up points that you may very well have gotten quickly.
What’s the Takeaway?
Given the above, then it seems fair to say that the better answer might be the choice that lets you move ahead to question 23 faster, giving you enough time to hit every question in the section. You may not get this question right, but you’re better off on the whole, since you’ve seen every question.
And if we’re optimizing for overall test score, rather than getting every question right, then we want to make sure we manage our timing in order to see every question and sweep up the points that we deserve! (Remember—you can get a 180 on the LSAT without getting every question right.)
One common aphorism that fits this case is “The perfect is the enemy of the good.” (Or to quote Shakespeare, because why not, “Were it not sinful then, striving to mend, / To mar the subject that before was well?”)
Of course, we’d love to be perfect…
But it’s really a matter of scope. Doing perfectly on the test means doing as well as you can—it doesn’t mean getting every single question right. Perfectionism can be a great boon while studying, but in my experience, it’s a dangerous habit to fall into on the LSAT.
Talk Bayesian To Me
So we’ve discussed the difference between right answers and better answers—the first refers to an answer key, and the second refers to thinking holistically about the entire section or test.
But why is this Part 3 of a series about Bayesian probability theory?
You might recall that in Part 1, we talked about how Bayes’ Theorem can be used to think of beliefs in terms of probabilities rather than binaries (true or false). Then, in Part 2, we linked that up to the LSAT by introducing the idea of right/wrong vs. likely/unlikely. We agreed on this:
Just because one answer is 100% correct doesn’t mean that we need to be 100% confident that it’s 100% correct. We just need to be confident enough.
Now let’s factor in the LSAT’s timing constraint and the pattern of variability in question difficulty—a.k.a. the fact that you have a time limit and that we can somewhat predict which questions in a section will be easier or harder than others.
We want to select answers that we’re confident enough about. And we figure out what “confident enough” means by considering how tough the question is and how much time left we have for the rest of the questions. We might call this “confident enough” level our confidence threshold, if we wanted to be fancy-pants, which we always do.
In my experience, it can be pretty useful to assign probabilities to your answer choices, and pick the answer that hits your confidence threshold. It’s “good enough”—good enough for you to feel comfortable with it and good enough for you to at least make an educated guess on every question in the section. The answer that satisfies these conditions is the better answer… Even if it ends up being the wrong answer.
So by thinking in terms of “Which answer is more likely?” rather than “Which answer is right?” we can avoid the pitfalls of timing constraints, as long as we adjust our confidence threshold.
A Mathematical Example
Let’s take Bob.
Bob loves the LSAT, but he spends too much time on tough questions, and often doesn’t get to the end of each section.
In a set of 6 tough questions, Bob the perfectionist might get the first 3 questions correct and then guess on the last 3. That gives perfectionist Bob an average score of 3.6:
100% + 100% + 100% + 20% + 20% + 20%
The actual formula would be (Question points) * (Probability of getting the question correct), but each question is only worth one point, so we can get rid of that. Bob gets 20% for the 3 questions he didn’t answer, rather than 0%, because he still has a 1-in-5 chance of blindly guessing correctly.
Now let’s take Jimmy.
Jimmy also loves the LSAT, but he knows that spending too much time on a question might mean that he won’t finish the section. Here’s how non-perfectionist Jimmy can match perfectionist Bob’s average score of 3.6:
60% + 60% + 60% + 60% + 60% + 60%
This means that Jimmy eliminated all but two answer choices and then picked the answer that seemed slightly better than the other. And he broke even with perfectionist Bob!
But we can go further. What if Jimmy spends a bit more time on the first 3 questions, like Bob did…
80% + 80% + 80% + 50% + 50% + 50%
This gives Jimmy an average score of 3.9!
(If you want to quibble with the difference in total probabilities, then I would say it’s generally easier to eliminate 2-3 answer choices than it is to select between the remaining 2 on a difficult question. Most of Bob’s wasted time is spent deliberating between two remaining choices, not eliminating.)
Finally, what happens when Jimmy adjusts his confidence threshold higher or lower, depending on how difficult the question seems and how much time he has left? He might get a distribution like this:
70% + 50% + 60% + 70% + 80% + 80%
This would give Jimmy an average score of 4.1.
These differences seem small, but they can add up! More importantly, Jimmy is taking the LSAT on his own terms and tackling it in a way that fits flexibly with his strengths and weaknesses.
Whether you dive deep into the Bayesian probability theory rabbit hole or just think this is a useful tool for you to use on the LSAT, I’d love to hear your thoughts! Email me or comment below! ?
Don’t forget that you can attend the first session of any of our online or in-person LSAT courses absolutely free. We’re not kidding! Check out our upcoming courses here.
Ben Rashkovich
is a Manhattan Prep LSAT instructor based in New York, NY. He’s a graduate of Columbia University, and he scored a 172 on the LSAT. He enjoys the mental challenge and logical acrobatics of the LSAT—and he feels that studying for the test can teach everyone to approach problems more rationally. You can check out Ben’s upcoming LSAT courses here!