We wanted to improve our candidate sourcing.

The problem: Our recruiters were spending hours manually searching LinkedIn for qualified candidates.

The solution (allegedly): An AI-powered candidate matching tool.

The promise: "Our AI scans millions of profiles and automatically identifies the best matches for your roles. It learns from your company's hiring patterns and surfaces candidates you'd never find on your own."

What actually happened: The AI only recommended people from our CEO's LinkedIn network—including his dentist, his fantasy football league, and someone he met once at an airport Chili's in 2017.

How The AI Was Supposed To Work

The vendor's pitch:

"Our AI analyzes your job descriptions, company culture, and successful hires. Then it scans millions of candidate profiles across LinkedIn, Github, and other platforms. It ranks candidates by fit and automatically sends you the top 20 matches for each role."

Sounds amazing, right?

We signed a 12-month contract.

Week One: The Results Seemed Too Good To Be True

We posted a Software Engineer role.

The AI sent us 20 candidate recommendations within 24 hours.

Me: "Wow, this is fast."

I opened the list.

Candidate #1: David Chen

• Senior Software Engineer at Google
• 10 years experience
• Perfect skill match

Candidate #2: Maria Garcia

• Lead Developer at Microsoft
• 8 years experience
• Exactly what we needed

Candidate #3: James Wilson

• Principal Engineer at Amazon
• 12 years experience
• Dream candidate

Me: "This AI is incredible."

Then I noticed something odd.

Next to each name: "2nd-degree connection to [CEO's name]"

Me: "Huh. Interesting coincidence."

I checked the remaining 17 candidates.

ALL of them were 2nd-degree connections to our CEO.

Me: "Okay, that's weird. But maybe the AI just happened to find great candidates who are loosely connected to him. LinkedIn is a small world."

Week Two: The Fantasy Football League

We posted a Marketing Manager role.

The AI sent 20 recommendations.

Candidate #7: Brad Thompson

• Title: Sales Director at a pharmaceutical company
• Location: Austin, TX
• Experience: 6 years in sales, zero marketing experience

Me: "Why is the AI recommending a sales director for a marketing role?"

I clicked on Brad's profile.

In his "About" section: "Passionate about sales, college football, and crushing my fantasy football league."

I checked our CEO's LinkedIn.

CEO's activity: Tagged in a photo with Brad Thompson at a fantasy football draft party.

Caption: "Ready to destroy @BradThompson in fantasy this year! #FantasyFootball #Rivalry"

THE AI RECOMMENDED THE CEO'S FANTASY FOOTBALL RIVAL FOR A MARKETING JOB HE WASN'T QUALIFIED FOR.

I checked the other 19 recommendations.

6 more were in the CEO's fantasy football league.

Including:

• An accountant (for a software engineer role)
• A dentist (for a project manager role)
• A real estate agent (for a data analyst role)

None of them were qualified for the roles.

But all of them were connected to our CEO.

Week Three: The "How Do You Know Him?" Investigation

I pulled every candidate recommendation the AI had sent for the past three weeks.

Total recommendations: 147 candidates across 8 roles

Candidates who were direct LinkedIn connections to our CEO: 38

Candidates who were 2nd-degree connections to our CEO: 94

Candidates with no LinkedIn connection to our CEO: 15

That's 90% of recommendations coming from our CEO's network.

I started investigating the connections.

Sample candidates the AI recommended:

Candidate: Dr. Robert Martinez

• Recommended for: Software Engineer
• Actual job: Dentist
• Connection: Our CEO's dentist (they're LinkedIn connections)

Candidate: Susan Park

• Recommended for: Data Analyst
• Actual job: Yoga instructor
• Connection: Went to college with our CEO (LinkedIn friends since 2004)

Candidate: Kevin O'Brien

• Recommended for: Product Manager
• Actual job: Bartender
• Connection: Our CEO once tagged him in a post about "best cocktails in SF"

Candidate: Amanda Foster

• Recommended for: Marketing Manager
• Actual job: High school math teacher
• Connection: Daughter of our CEO's college roommate

NONE OF THESE PEOPLE WERE REMOTELY QUALIFIED.

But all of them were in our CEO's LinkedIn network.

I Called The Vendor

Me: "Your AI is only recommending people from our CEO's LinkedIn connections."

Vendor: "That's not possible. Our AI scans millions of profiles."

Me: "90% of your recommendations are people connected to our CEO. Including his dentist."

Vendor: "His dentist?"

Me: "Yes. For a software engineer role."

Vendor: "Let me check the algorithm."

Hold music

Vendor: "I found the issue."

Me: "What is it?"

Vendor: "When you set up the account, whose LinkedIn did you connect?"

Me: "Our CEO's. He's the account owner."

Vendor: "Right. The AI uses the connected account as a 'trust signal.'"

Me: "What does that mean?"

Vendor: "The AI prioritizes candidates who are connected to or have interacted with the account holder, because those represent 'warm leads' that are more likely to respond."

Me: "So the AI is basically just scraping our CEO's LinkedIn network?"

Vendor: "It's not scraping. It's prioritizing trusted connections."

Me: "Our CEO's dentist is not a software engineer."

Vendor: "Right, but he's a trusted connection, so the AI surfaced him in case there was a fit."

Me: "There is no fit. He's a dentist."

Vendor: "The AI doesn't always get it right."

Me: "90% of your recommendations are unqualified people from our CEO's network. That's not 'doesn't always get it right.' That's completely broken."

The "Airport Chili's" Incident

The most absurd recommendation:

Candidate: Mark Sullivan

• Recommended for: Senior Product Manager
• Actual job: Insurance adjuster in Tampa
• Connection: Our CEO posted about him once in 2017

I found the post.

CEO's LinkedIn post (May 2017):

"Sitting in an airport Chili's next to a guy who's explaining insurance fraud investigation techniques. Fascinating stuff. Small world! #Travel #Networking"

Tagged: Mark Sullivan

They met ONCE. In an airport Chili's. 8 years ago.

And the AI recommended him for a Senior Product Manager role.

Mark had zero product management experience.

But he was "connected" to our CEO.

So the AI thought he was a great fit.

Week Four: The "Your Nephew" Email

Our CEO got an email from the AI matching tool.

Subject: "Top Candidate Match for Marketing Manager Role"

Body:

"We found an excellent match for your Marketing Manager opening!

Candidate: Jason Williams Current role: Junior Account Executive Experience: 2 years Match score: 94%

Jason is a 1st-degree connection of yours. Would you like to reach out?"

Our CEO forwarded it to me.

CEO: "Why is the AI recommending my college roommate's nephew for a senior marketing role? He's 23 and works in sales."

Me: "Because he's connected to you on LinkedIn."

CEO: "This tool is useless."

Me: "Correct."

We Ran An Experiment

I wanted to test the AI's actual capabilities.

Experiment:

1. I created a fake LinkedIn profile (using a stock photo and fake name)
2. I made the profile a "Senior Engineering Manager" with 15 years of experience and perfect qualifications for our open roles
3. I did NOT connect the profile to our CEO

Then I waited.

The AI never recommended this fake profile.

Not once. Across 4 engineering roles over 2 weeks.

Then I connected the fake profile to our CEO.

24 hours later: The AI recommended the fake profile for 3 different roles.

Match scores: 91%, 88%, 94%

The AI didn't care about qualifications.

It only cared about LinkedIn connections.

We Tried To Fix It (Spoiler: We Couldn't)

Me to vendor: "Can you disable the 'trust signal' prioritization?"

Vendor: "Not without significantly reducing match quality."

Me: "Your current match quality is terrible. 90% of recommendations are unqualified."

Vendor: "That's because your CEO has a very diverse network."

Me: "His network includes his dentist and his fantasy football league. That's not 'diverse' in a useful way."

Vendor: "We can lower the connection weight in the algorithm."

Me: "Do that."

They adjusted the settings.

New results:

Week One after adjustment:

• 63% of recommendations were still from CEO's network (down from 90%)
• Match quality slightly better but still poor
• Still recommended the dentist (now with an 82% match score instead of 94%)

Week Two after adjustment:

• The AI started recommending completely random people
• A high school student for a VP role
• Someone whose LinkedIn profile was entirely in Portuguese (we only hire English speakers)
• A candidate who listed "Chief Snack Officer" as their current job title

The AI was worse with the "fix" than it was before.

We Canceled The Contract

Me: "We're canceling. This tool doesn't work."

Vendor: "We offer a satisfaction guarantee. We can work with you to improve results."

Me: "You've been 'working with us' for 6 weeks. Your AI still recommends our CEO's dentist for software engineer roles."

Vendor: "AI matching is a complex problem."

Me: "Apparently."

Vendor: "Would you consider staying on for another month while we retrain the model?"

Me: "No."

We got a partial refund and canceled.

What We Learned

AI candidate matching sounds amazing:

• Automatically source qualified candidates
• Save recruiters time
• Find people you wouldn't discover manually

In reality, it:

• Prioritized LinkedIn connections over qualifications
• Recommended unqualified candidates based on social proximity
• Wasted weeks of our time reviewing terrible matches
• Couldn't distinguish between "my CEO's dentist" and "qualified software engineer"

We're back to manual sourcing.

It's slower.

But at least we're not reaching out to dentists for engineering roles.

The Bottom Line

If you're buying AI sourcing tools:

1. Ask how the AI prioritizes candidates (ours prioritized LinkedIn connections)
2. Test it with roles you've already filled (we didn't)
3. Check if recommended candidates are actually qualified (we eventually did)
4. Don't assume "AI" means "better" (it often doesn't)

Or just have your recruiters do it manually.

They might not scan millions of profiles.

But they also won't recommend your CEO's fantasy football league for senior leadership roles.

(Unless your league is really impressive. Which, to be fair, it might be.)