A while back, my wife was telling me about a book she’d just read called “Animals in Translation” by animal behaviourist Professor Temple Grandin. In the book, Professor Grandin explains her work to improve conditions for animals at meat-packing facilities (and if you’re wondering what on earth this has to do with privacy, keep reading - I’ll get there.)
One of the ways originally used to assess conditions at these facilities was by means of lengthy, detailed audit inspections, with audit questionnaires often running well in excess of a hundred pages. These questionnaires were by their very design cumbersome to complete and consequently attracted voluminous but poor quality responses that gave little real insight into animal welfare or how to improve it.
Professor Grandin’s insight was that audits could be considerably improved by reducing the questionnaires to just a few, select, critical questions. For example, asking what percentage of animals displayed signs of lameness replaced the need to ask several questions about the detail of the animal-handling process. Why? Because the detail was largely irrelevant - if more than a certain percentage of animals displayed lameness, then it stood to reason that something was going wrong and needed addressing.
As privacy professionals, we can all learn something from this drive towards audit simplicity. Having worked with many businesses on privacy assessments ranging from full-blown audits to Privacy Impact Assessments for new product launches, and vendor due diligence projects to staff awareness questionnaires, I’ve often been struck by how unnecessarily long and detailed many audit questionnaires are.
This length is more often than not attributable to a concern that, if questioners don’t ask every last question they can think of, they might miss something. That’s a fair concern, but what it overlooks is that a questionnaire is the beginning and not the end of a well-run audit - it’s simply the means by which you start gathering information in order to identify potential issues needing follow-up. Not only that, but it’s far better to get a high response rate to a few well-selected questions than a low-to-zero rate on a more detailed set.
So the next time you have to circulate an audit questionnaire to an internal development team about some new product launch, or to an external vendor about its compliance practices, or maybe to another business unit about its data-handling procedures, then keep the following practical tips in mind:
1. Keep it short and sweet. Nobody likes answering lengthy questionnaires and so, if you send one out, expect that it either won’t get answered or won’t get answered well. A one to two page questionnaire of a few select questions will encourage a relatively high response rate; at three to four pages, you’ll watch your response rate fall off a cliff; and anything longer that you’re almost guaranteeing next to no responses - no matter how much chasing, cajoling, or threatening you do!
2. Ask critical questions only. Think carefully about what you really need to know for initial issue-spotting purposes. Focus, for example, on what data are you collecting, why, who it will be shared with, where it will be processed, and what security is in place. Don’t worry too much about the detail for now - you can (and should!) follow-up that up later if the initial responses ring any alarm bells (“You’re sending data to Uzbekistan? Why? Who will have access to it? What protections are in place?").
3. Use open questions. It’s tempting to design a checklist questionnaire. In theory, this makes the respondent’s job easier and encourages consistency of responses. In practice, however, checklists aren't always well-suited for privacy assessments. It's difficult to capture the wide array of potential data types, uses, recipients etc. through checklists alone unless you use very lengthy checklists indeed - resulting in the response-rate issues already mentioned above. Open questions, by contrast, are generally simpler and shorter to ask, and lead to more, and often more informative, responses (e.g. Consider “Who you will share data with and why?" vs. “Who will you share data with? Tick all of the following that apply: Customers [ ] Customers’ end users [ ] Prospective customers [ ] Employees [ ] Contractors [ ] Vendors [ ] etc.”).
4. Don’t lose sight of the wood for the trees. In many of the questionnaires I’ve seen, questioners often delve straight into the detail, asking very specific questions about the nature of the data processed, the locations of privacy notices, specific security standards applied and so on. What’s often missing is a simple, upfront question asking what the particular product, service or processing operation actually is or does. Asking this provides important context that will necessarily influence the depth of review you need to undertake. Better still, if you can actually get to see the “thing" in practice, then seize the opportunity - this will crystallize your understanding far better than any questionnaire or verbal description ever could.
5. We never talk anymore. As already noted, a questionnaire is just the beginning of an assessment. Use it to gather the initial data you need, but then make sure to follow up with a meeting to discuss any potential issues you identify. You will learn far more from person-to-person meetings than you ever can through a questionnaire - interviewees will often volunteer information in-person you never thought to ask on your questionnaire, and some interview responses may prompt you to think of questions “in the moment” that you’d never otherwise have thought of back in the isolation of your office.
6. Not all audits are created equal. The above are just some practical suggestions borne out of experience but, obviously, not all audits are created equal. As a questioner, you need to have the flexibility to adapt your questionnaire to the specific context in hand - a "lessons learned" audit following a data breach will by necessity have to be more detailed and formal than an internal staff awareness assessment. Exercise good judgement to tailor your approach as the situation requires!
A while back, my wife was telling me about a book she’d just read called “Animals in Translation” by animal behaviourist Professor Temple Grandin. In the book, Professor Grandin explains her work to