TAMPA — Companies often send out simulated (or “fake”) phishing emails to its employees to see who take the bait and click. Those who fall for the simulated scam usually get an on-the-spot lesson meant to help them catch suspicious messages next time.
These phishing simulations — called embedded training because once users fail, they are sent into training mode — are widely considered a “best practice” in the cybersecurity anti-phishing industry.
But new research co-led by University of South ֻƬ’s Muma College of Business faculty finds that approach might not be the best way to help employees learn from their mistakes.
The study published today in MIS Quarterly, was co-authored by Dezhi Yin and Matthew Mullarkey from ֻƬ’s Muma College of Business, Gert-Jan de Vreede from Stevens Institute of Technology, and Moez Limayem, president and professor at the University of North ֻƬ, who was selected this month to become the president-elect of ֻƬ.
“Giving feedback only to the people who clicked the ‘fake’ phishing email misses a big opportunity. We found that employees learn better when everyone — even those who didn’t fall for it — gets a follow-up message explaining the phishing test.”
Dezhi Yin
This new research identifies two shortcomings of embedded training:
- The instant feedback can be limited in reach. Only those who got duped are given training, while those who passed may end up falling for a real phishing attack later.
- Catching employees at the exact moment of failure, called “just-in-time” training, can be counterproductive. This on-the-spot training can cause negative reactions in employees who feel exposed and may get defensive.
Instead, the researchers recommend taking a non-embedded approach. By providing feedback to everyone after the entire simulation is over, the exercise turns into a broader and more positive learning opportunity.
large-scale experiments
The researchers conducted three large-scale experiments using a real phishing simulation platform. Thousands of students received realistic (but simulated) phishing emails, some with immediate feedback after clicking, and others with follow-up messages sent days later. The team then tracked how likely participants were to fall for future simulated scams over the next several weeks and months.

Dezhi Yin
“Giving feedback only to the people who clicked the ‘fake’ phishing email misses a big opportunity,” Yin said. “We found that employees learn better when everyone — even those who didn’t fall for it — gets a follow-up message explaining the phishing test.”
Among the study’s key insights, researchers discovered:
- Sharing lessons with the entire group, not just those who got duped, helped people recognize scams more effectively and stay alert for months afterward.
- Training does not need to be delivered at the point of failure to be effective. A time-delayed but all-encompassing approach ultimately builds a better defense against real attacks.
“Phishing training companies can directly make use of our key insights in designing more effective software tools, and we heard that KnowBe4 is already doing that,” Mullarkey said.
The project began with support from KnowBe4, a Clearwater-based cybersecurity company that donated software licenses for more than 12,000 users and provided technical expertise and research funding.

Matt Mullarkey
“This is an example of research that literally would not exist without the industry partnership,” Mullarkey said. “KnowBe4 gave us access to the platform, helped us understand how to launch phishing simulations and track user behavior, and even funded research stipends.”
The study’s findings could help companies strengthen their cybersecurity defenses as phishing scams grow more sophisticated using artificial intelligence.
“Employees are widely considered the last line of defense in the anti-phishing training industry,” Yin said. “Non-embedded training provides a more effective alternative to fortify this last defense than the status quo.”
