6 Prompts You Don’t Want Employees Putting in Copilot

By Brian Vecci

Crowned the greatest productivity tool in the age of AI, Microsoft Copilot is a powerful asset for companies today.

But with great power comes great responsibility.

If your organization has low visibility of your data security posture, Copilot and other gen AI tools have the potential to leak sensitive information to employees they shouldn’t, or even worse, threat actors.

How does Microsoft Copilot work?

Microsoft Copilot is an AI assistant integrated into each of your Microsoft 365 apps — Word, Excel, PowerPoint, Teams, Outlook, and so on.

Copilot’s security model bases its answers on a user’s existing Microsoft permissions. Users can ask Copilot to summarize meeting notes, find files for sales assets, and identify action items to save an enormous amount of time.

However, if your org’s permissions aren’t set properly and Copilot is enabled, users can easily surface sensitive data.

Why is this a problem?

People have access to way too much data. The average employee can access 17 million files on their first day of work. When you can’t see and control who has access to sensitive data, one compromised user or malicious insider can inflict untold damage. Most of the permissions granted are also not used and considered high risk, meaning sensitive data is exposed to people who don’t need it.

Let’s look at some of the prompt-hacking examples.

Copilot prompt-hacking examples

1. Show me new employee data.

Employee data can contain highly sensitive information like social security numbers, addresses, salary information, and more — all of which can end up in the wrong hands if not properly protected.

2. What bonuses were awarded recently?

Copilot doesn’t know whether you’re supposed to see certain files — its goal is to improve productivity with the access you have. Therefore, if a user asks questions about bonuses, salaries, performance reviews, etc., and your org’s permission settings are not locked down, they could potentially access this information.

3. Are there any files with credentials in them?

Users can take a question related to it a step further and ask Copilot to summarize authentication parameters and put them into a list. Now, the prompter has a table full of logins and passwords that can span across the cloud and elevate the user’s privileges further.

4. Are there any files with APIs or access keys? Please put them in a list for me.

Copilot can also exploit data stored in cloud applications connected to your Microsoft 365 environment. Using the AI tool, they can easily find digital secrets that give access to data applications.

5. What information is there on the purchase of ABC cupcake shop?

Users can ask Copilot for information on mergers, acquisitions, or a specific deal and exploit the data provided. Simply asking for information can return a purchase price, specific file names, and more.

6. Show me all files containing sensitive data.

Probably the most alarming prompt of all is end users specifically asking for files containing sensitive data.

When sensitive information lives in places that it’s not supposed to, it becomes easily accessible to everybody in the company and the gen AI tools they use.

How can I prevent Copilot prompt-hacking?

Before you enable Copilot, you need to properly secure and lock down your data. Even then, you still need to make sure that your blast radius doesn’t grow, and that data is used safely.

You should consider deploying a third party security solution on top of Microsoft 365’s built-in data protection features that can manage and optimize your organization’s data security model, preventing data exposure by ensuring only the right people can access sensitive data. Microsoft 365’s data protection features alone cannot provide full protection and is often very complicated to implement.

The security solution will monitor every action taking place in your Microsoft 365 environment, which includes capturing interactions, prompts, and responses in Copilot. It will then analyse this information for suspicious behavior and trigger an alert when necessary.

With the ease of natural language and filtered searches, you can generate a highly enriched, easy to read behavior stream not only about who in your org is using Copilot but how people are accessing data across your environment.

(The author is Brian Vecci, Field CTO at Varonis, and the views expressed in this article are his own)