Test-Prep Advice Post

When accused of a crime, would you hire AI representation, instead of a human lawyer?  

When filing a complicated tax return, would you allow AI to fill out the forms, instead of a certified public accountant?  

Sure – I might sometimes trust a Waymo automated driver more than a random Uber driver, and doctors are known for using technology and computers to aid with their jobs.  However, one thing you will notice is that when the task is complex and/or highly consequential, a human expert is almost involved.

I recently took a hard LSAT question, and a hard SAT question, for example, and entered each verbatim into 3 of the most popular AI engines (ChatGPT, Claude, and Gemini).

Guess what?  A highly confident 0 for 6.   And when I corrected these AI bots, they each capitulated, quickly admitting their mistakes.

Conclusion: AI isn’t taking the job of private test-prep tutors anytime soon.  And good luck having an actual spoken conversation with these "chat" bots, instead of just relying on their usual text explanations.  The actual voice chat feature is even worse and less accurate than are the text responses.

Another consideration: free written explanations to exam questions have been around for decades already, hosted on websites such as GMAT Club and the Powerscore LSAT discussion forums.  Yet even though many detailed text explanations to official questions are freely available, many difficult questions still require a real conversation with a human expert to fully unlock.  So why are we expecting the AI-written explanations, which are mostly derived from the human ones, to be any better?

Until AI can answer EVERY question correctly, and until it can also hold a real, natural conversation with nuance to explain them, you’re certainly better off choosing an expert human for the task -- if you can afford it.  After all, though mastery of the exam is a prerequisite, it is not a guarantee of a skilled instructor.  Even humans who are able to ace the test are sometimes lousy at teaching it!   In addition, exams such as the SAT, ACT, LSAT, GRE, GMAT, ISEE, and SSAT are assigned outsize importance during the admissions process, and should be prioritized accordingly.

Am I biased?  Of course: test-prep tutoring is how I make a living.  But after 25 years of professional experience, I also know that private tutoring gets real results -- and that generally speaking at least, artificial intelligence does not.  You can’t AI your way into a great college or grad school in 2026, because it’s a tool that everyone has access to, and is already aware of.  Having access to an (often wrong!) written explanation is not the same as actually understanding your mistakes by discussing it with a human teacher with decades of instruction.  In fact, it's not even close.

Of course, we all use AI for grunt, repetitive tasks sometimes, but standardized test scores are simply too important to trust to a generic computer algorithm: some law-school admissions experts estimate, for example, that for every 1 point improvement in one's LSAT score, students are eligible for an additional $10,000 in scholarships and grants.  Though deciding to use AI for your prep is certainly the cheaper option, that decision may well end up costing you money in the long run.

Enjoying real, human conversations with others is a key aspect of learning -- one that computers will never be able to fully replicate.  Soon the AI bubble will pop, and we will go back to giving real people in education their proper due.  Until then, I’ll be here, chugging along, adding to my 25,000 hours of experience ... and maybe writing a blog post to complain every once in while.