Skip to main content
  • Cybersecurity

Should you run a blind baseline phishing test?

By Jon Lewis
Product Marketing

When an organization starts phishing simulations for the first time, we often get asked if they should do a “blind phishing test” before announcing their program to their staff.

Cybersecurity awareness training and phishing simulations are already a controversial, hotly debated topic in cybersecurity circles for a variety of reasons, and whether blind tests are effective or not is one of the many popular topics in those conversations.

The thought process behind doing a blind test makes sense, but ultimately I don’t think they’re as important as most might believe them to be. I actually think that they can cause more harm than good, and the benefits are short-term at best, if benefits are even realized.

So, let’s break down what a blind phishing test is, why IT teams think they want to do them, and why we think it’s better that you don’t do them at all.

What is a blind phishing test?

A blind phishing test is where you send out a phishing campaign to all (or most) of your staff without telling them first.

The theory behind a blind test is that you want to get a controlled, unbiased set of data around the phishing susceptibility of your organization. By telling staff that you will be testing them, you may make them more alert in the short-term, causing your data to be skewed positively.

Why do cybersecurity and IT teams want that “accurate” phishing susceptibility rate? Because they think it makes it easier for them to paint a picture of the current, pre-training state of the organization to themselves and senior management. This allows them to prove that their investment in awareness training has had a positive impact after several weeks or months of courses and phishing tests.

Blind phishing tests are not statistically relevant

The first major problem I see with blind tests is a data problem.

A single phishing test, using a single template at a single point in time, is not an accurate representation of your phishing susceptibility.

Your phishing test could be sent at a time where people are more or less skeptical of emails. An organization doing a blind test in February will have dramatically different results from that same organization choosing to do their blind test in May, just a couple months after the COVID-19 pandemic really started to be at the forefront of everyone’s minds.

The phishing template you chose might not contain relevant content or have an enticing subject line that otherwise would have made your staff want to engage with it. Your employees might be good at ignoring social media phishing scams, but could be less savvy at detecting CEO fraud templates. Depending on the template you chose, you could derive that your employees are better (or worse) at detecting phishing emails than they actually are.

Blind phishing tests can erode trust

The second problem I see with blind phishing tests is a bit fuzzier, and involves your culture, communications, and trust with your staff.

Cybersecurity is a sensitive topic for some organizations, especially if they’ve been hit with ransomware or other incidents in the past. We believe that a healthy cybersecurity culture is one built on open communication and trust between IT and the groups that they support.

Blind phishing tests can actively go against that philosophy. I’ve heard countless stories of organizations where phishing tests made their users feel like they were being tricked or pranked by their IT team, and that resulted in negative attitudes about not only their awareness training program, but also IT and cybersecurity in general.

That’s the opposite outcome we’re trying to achieve with cybersecurity awareness training. When attitudes like this set in, your staff are less likely to follow policies, adopt new cybersecurity tools, and raise their hand when they detect something suspicious.

Ultimately, an organization with a poor cybersecurity culture is one that will see more cybersecurity incidents.

What should you do instead of blind phishing tests?

First and foremost, we believe that you should approach your cybersecurity awareness training program, including your phishing tests, with full transparency and open communications for all your staff.

When organizations take this approach, they engage their users to be part of the solution to external threats, instead of making them feel like they are risks to be managed.

Tell them what you’re doing, when, and why. Give them the support they need in order to identify phishing emails (through your training courses) and how to report them safely (through a tool like a Report-a-Phish button or a forwarding address.)

For your very first phishing simulation, you should tell everyone that you’re planning on doing one to everyone in a wide time window.

We’re not saying you should tell people the exact style and time of your test, because that will influence your results. What you should do is tell everyone that you will be testing within a one or two week time window, and then send a few templates out randomly to all staff.

It’s important that you include all employees in your phishing test so it doesn’t seem like you are picking on any specific department or individual. This means testing your leadership teams.

From there, you should be transparent about what your future phishing test plan is. We recommend monthly, randomized phishing tests to all staff, but you should set expectations about cadence and what you will share after each test regardless of how frequently you’re doing your tests.

Afterward, your first phishing test, and periodically after that, you should share the results with your employees. You’ll be surprised how interested they are in your findings since you included them from the beginning—and this is the purpose of being transparent. You want your staff to feel like they are part of your cybersecurity solution.

Ready to start phishing tests in your organization?

CIRA’s Cyber Security Awareness Training makes it easy to automate a monthly phishing program and create custom spear phishing templates.

Learn more and book a demo

About the author
Jon Lewis

Jon champions the people-side of cybersecurity as the marketing lead for CIRA’s cybersecurity awareness training platform. His background in enterprise data marketing and teaching organizational behaviour at the university level allows him to develop resources for Canadian businesses to help them engage their employees and empower them to reduce their cyber risk.