“Black Cap is the head-dress worn by the judge in pronouncing the sentence of death. It is part of the judicial full dress, and is worn by the judges on occasions of special state.”
— The Law Dictionary (Featuring Black’s Law Dictionary Free Online Legal Dictionary 2nd Ed.)
Art + Science
A client recently expressed an entirely unexpected level of exasperation.
How could two Auditors, he asked, with similar compliance experience, review the same files and come to very definite and contradictory positions?
Without having reviewed the files myself, there was little reassurance I could offer him.
While facts might be incontrovertible, the interpretation of those facts – and the inferences that are drawn from those facts, documents and accounts – are subject to the preferences, assumptions and (unconscious) biases of the reviewers.
In the absence of an Australian Standard for compliance reviews, there will be variability (and sometimes incredible variability) between the expert opinions of Reviewers. Differences are exacerbated if the Reviewers methodologies place different emphasis on formal or substantive compliance. Although Commissioner Hayne denounced the value of ‘formal’ compliance, it’s easier to industrialise than methodologies that focus on substance, context and principles.
Fundamentally, each reviewer assesses the facts through the lenses of their own experience, expertise and understanding. Unfortunately, few reviewers recognise that, notwithstanding their intentions
“all perceptions, judgments and beliefs are inferences and not direct readouts of reality”
Richard Nisbett’s recognition of the limits of ‘reason’, his focus on ‘framing’ and his observation that “situational and contextual factors” are more important than ‘dispositions such as traits, abilities and preferences”, should alert you to the biases and weaknesses of many compliance reviews.
Any review that discounts construal elements and applies a binary overlay to the application of nuanced legal principles should be considered critically. Because interpreting nuances and determining context require inferences, the best a Licensee can hope for is a Reviewer that approaches matters objectively, follows a consistent methodology and is conscious of the applicable laws, community expectations and their own biases.
All reviewers have biases. Some are black caps, convinced that all advisers are recalcitrant, dishonest, incompetent and self-interested. Some are cheerleaders, convinced that advisers are well-intentioned, ethical, professional and community focused.
Regardless of their philosophy, Auditor’s results tend to reflect, and validate, their presumptions.
As a Licensee (or as an adviser), you have little capacity to uncover Reviewers’ biases, judgments and beliefs – particularly since these are often unconscious – but you can be aware of the main (unconscious) challenges to their objectivity and impassivity. These include:
- Consistency – the reviewer’s conclusions will tend to accord with their previous conclusions or the previous judgments of their peers. It’s often neither a deliberate or conscious choice but a reflection of the difficulty in critically evaluating a previous judgment.
- Cheerleading – the Reviewer will adopt and promote a client’s perspective (or their employer’s perspective) to a degree that compromises their objectivity.
- Comfort ─ sometimes, a close, long or sympathetic relationship with an adviser or Licensee can bias their conclusions and their ability to identify, and escalate, issues.
- Self-interest – As REP515 identified, sometimes financial or other interests compromise, or inappropriately influence, a reviewer’s professional judgement. In other cases, actual or perceived pressures, can intimidate or influence reviewers.
- Framing – often linked to self-interest, this simply reflects the reality that Reviewers are subject to confirmatory biases too and, if told about particular issues, will tend to identify, and escalate, those issues (often at the expense of both objectivity and good risk management). This can often be seen as a blinders effect where reviewers miss (or overlook) significant compliance issues because they’re focusing on flagged issues.
“If the individual has the relevant expertise, she will recognise the situation, and the intuitive solution that comes to her mind is likely to be correct. .. When the question is difficult and a skilled solution is not available, intuition still has a shot: an answer may come to mind quickly – but it is not an answer to the original question.”
— Daniel Kahneman, “Thinking fast and slow”
The ‘audit’ is, and should be, a key component of a Licensee’s compliance framework.
Licensees have obligations to monitor and supervise their representatives, to ensure they are trained and competent and ensure, as far as practicable, that they comply (and continue to comply) with the financial services laws.
I’m not for one moment suggesting that an adviser review is either irrelevant or unreliable, but simply identifying that the Reviewer’s conclusions should not be interpreted as unassailable truths but instead recognised as inferences and observations subject to the Reviewer’s own perspective, biases and perceptions.
Given the impact that a review may have, and the consequences and implications of adverse findings, it’s incumbent on you to at least understand the reviewer’s processes, perceptions and pressures.
At Assured Support, we try to mitigate the effect of biases by:
- implementing a defined and pre-determined review methodology;
- defining clear parameters;
- automating reporting and escalation;
- Structuring the stages of the review to reconcile intuitive and deliberate responses;
- imposing a defined and structure taxonomy;
- regularly reviewing (and validating) the observations, trends, impact and assessment framework;
- acknowledging contextual and situational factors;
- issuing draft reports with formal appeal mechanics;
- encouraging advisers and Licensees to challenge (or question) our findings; and
- acknowledging, and accepting, that our Reviewers may sometimes make mistakes.
Using the OpenAFSL compliance platform we follow a methodology that is explicitly designed to:
- establish a transparent, objective, consistent methodology for determining “advice quality”;
- embed and reflect values of creativity, excellence and integrity;
- focus on quality, process and the management and avoidance of conflicts of interest;
- prioritise clients’ interests and consumer expectations;
- provide for the effective identification, classification and management of risks;
- promote professionalism and continuous improvement;
- provide consistent grading, assessment and consequent management measures; and
- focus on training, remediation and effective behavioural change.
We understand better than most that financial services is both complex and complicated, and we’ve done what we can to ensure that we deliver consistency, confidence and certainty to our clients.
After reviewing several thousand advice files in the past few years, we think we have a proven methodology and a consistent practice, but we also appreciate that most biases and assumptions are unconscious.
What additional controls, concessions or amendments would you propose?
Let us know what you’d suggest.
We look forward to hearing from you.