Banking Exchange Magazine Logo

Adopting a Culture that Will Prevent Algorithm Bias

A timely commentary in light of this week’s news about Apple and Goldman Sachs

  • |
  • Written by  Stuart Dobbie, VP of Product at Callsign 
  • |
  • Comments:   DISQUS_COMMENTS
Adopting a Culture that Will Prevent Algorithm Bias

A flood of businesses are looking to AI to solve business challenges, automate processes and differentiate their solutions in highly competitive markets. But with the pressure to bring these solutions to market quickly, there needs to be early adoption of good processes from the outset.

The key first key process is understanding your dataset. Previous algorithm failures in other industries – Insurance and e-Commerce, for example – have occurred due to the predominant bias found in the gender and socioeconomic status of their training datasets. It is important to understand how bias may be found naturally in your own customer base, and how this may affect your inferences, since these naturally need checks and balances to avoid sampling bias.

Second, for real-time automated decision systems, it would be wise to create circuit breakers. These will raise alerts when a propensity of the decisions or distribution of outcomes becomes heavily skewed beyond some fail safe values. This is a good proactive measure to monitor performance and expectations of algorithm decision making. 

AI responsibility is two-fold. We’re looking at the strict implementation of governance processes fortified by a culture of open communication and scrutiny. Data governance best practices are critical, but players implementing AI from across the business ecosystem need to also be vigilant about engaging a diverse range of internal contributors and stakeholders. This could be sharing private and public proofs of concept, working with the academic and research communities to participate in peer reviews, or even involving consenting members of the public to help validate their AI models.

In Europe, GDPR actually encourages this culture of algorithm scrutiny. Failing to have the appropriate checks in place can result in repercussions with data processing compliance and lines of recourse for automated decision making; in particular, data subjects being allowed to contest automated decisions and request human intervention, review and maintenance of fairness. An inability to explain a black box score or reach a similar conclusion when manually reviewed will raise further scrutiny.

Trust is inherently tied to AI, so I urge any business in this space to constantly push for transparency – both with stakeholders and how there are communicating data collection. It is this environment of open discussion and scrutiny in addition to holistically-constructed governance practices – that will help deter bias in AI.

Stuart Dobbie is VP of Product at Callsign

back to top


About Us

Connect With Us


Webinar: From KYC to IDV

How three leading banks are utilizing cutting-edge
digital tools to onboard, win, and wow customers

Time/Date: June 23, 2021 11:00 a.m. ET

Digital adoption, already moving at warp speed, accelerated seven years into the future during the COVID-19 pandemic. As the number of bank branches continues to fall, with at least one study predicting all branches will disappear by 2034 (Fox Business) and foot traffic declining (Vox), today’s most innovative banks are charting a new, digital-first path to win over customers while increasing security, meeting KYC compliance requirements, and winning customers to drive revenue.

In this webinar, you’ll hear from John Baird, Founder & CEO of Vouched, Tyler Crawford, COO of Bankers Healthcare Group, Anand Sathiyamurthy, CPO of Flagstar Bank and Daniel Sheehan, Chairman & CEO of Professional Bank as they describe their vision for digital transformation and how customer expectations are changing to digital first. They’ll also explore how fostering an innovation mindset creates new ways to tackle complex KYC problems and allows them to quickly compete in new markets and win customers.


This webinar is brought to you by:
Vouched Logo