Test-Prep Advice Post

When charged with a crime you didn't commit, would you hire AI representation, instead of an actual lawyer?  

When filing a complicated tax return, would you allow AI to fill out the forms, instead of a certified public accountant?  

Sure – I might choose to trust a Waymo automated rideshare driver more than a random Uber one, and doctors are known for using technology and computers to aid with surgery.  However, one thing you will notice is that when the task is complex and/or highly consequential, a real human expert is almost always involved.

I recently took a hard LSAT question, and a hard SAT question, for example, and entered each verbatim into 3 of the most popular AI engines (ChatGPT, Claude, and Gemini).

Guess what?  A highly confident 0 for 6.   And when I corrected these "agents," they each capitulated, quickly admitting their mistakes.

Conclusion: AI isn’t qualified to replace professional test-prep tutors like me anytime soon.  And good luck having an actual spoken conversation with these "chat" bots, instead of just relying on their usual text explanations: the voice chat features are even worse.

Another consideration: free written explanations to popular standardized test questions have already been around for decades, hosted on websites such as GMAT Club and the PowerScore LSAT discussion forums.  Yet even though detailed text explanations to official SAT, ACT, GRE, GMAT, and LSAT questions are often freely available, students will find that many of these questions still require a real chat with a human expert to fully unlock.

In other words, there is a difference between having access to written (or even video) explanations, and being able to truly learn from them.  So why are we expecting these AI-created explanations, which are mostly derived from the already existing human ones, to be any better?  In reality, AI question solutions exhibit the same flaws as their human-created counterparts: they are are too complicated, reverse-engineered from the correct answer, over-simplified, incomplete, contradictory, hard to understand and/or categorize, redundant, difficult to execute under the time pressure of the actual exam, et cetera.  And YouTube videos, though easy to watch, often make for even worse learning tools due to the drawbacks of passive learning.  

Until AI can answer every question correctly, and until it can also hold a real, natural, customized conversation about the nuances of the test and best preparation methods, you’re certainly better off choosing an experienced private tutor for the task -- if you can afford it.  After all, though mastery of the exam is a prerequisite, mastery alone is not a guarantee of a skilled instructor.  In the same way that elite athletes don't always make the best coaches, even those who are able to ace the exam themselves are often surprisingly lousy at teaching it.

In addition, tests such as the SAT, ACT, LSAT, GRE, GMAT, ISEE, and SSAT are assigned outsize importance during the admissions process, and should be prioritized accordingly.

Am I biased?  Of course: test-prep tutoring is how I make a living.  But after 25 years and counting as a pro tutor, I also know that private tutoring gets results -- and that in the vast majority of cases, artificial intelligence does not.  You can’t AI your way into a great college or grad school in 2026.  Having access to an (often incorrect) written explanation is not the same as actually understanding your mistakes by discussing it with a tutor with decades of instruction.  In fact, it's not even close.

Of course, we all use AI for grunt, repetitive tasks sometimes, but standardized test scores are simply too important to trust to a generic computer algorithm: some law-school admissions experts estimate, for example, that for every 1-point improvement in one's LSAT score, students are eligible for an additional $10,000 in scholarships and grants.  Though deciding to use AI for your prep is certainly the cheaper option, that choice may well end up costing you in the long run.

Enjoying genuine, human, question-and-answer conversations with master educators is a key aspect of learning -- one that computers will never be able to fully replicate.  Soon the AI bubble will pop, and we will go back to giving experts in education their proper due.  Until then, I’ll still be chugging along, adding to my 25,000 hours of experience ... and maybe writing a blog post to complain every once in while.