Apple Card’s algorithm is sexist, gives women a much lower credit limit than their spouse

Text size:
A-
A +

HHailed as the biggest credit card innovation in 50 years, the Apple card is starting to look more like something from the 1960s and 1970s: Women would be given a fraction of their spouse’s borrowing limits. This is another troubling example of the shortcomings of machine learning.

Just months after its launch, New York regulators say they are investigating Goldman Sachs Group Inc., the bank behind the card, and the algorithm it uses to determine creditworthiness. Goldman denies any discrimination, but that hasn’t stopped Apple Inc. co-founder Steve Wozniak from calling on the U.S. government to get involved. “We don’t have transparency on how these companies set these things up and operate,” he told Bloomberg News.

Wozniak’s inquiry and comments came in response to a Twitter post by tech entrepreneur David Heinemeier Hansson, in which he said the Apple card gave him a credit limit 20 times that of his wife. . This despite his superior credit rating and their joint tax returns. Wozniak says he received 10 times the limit given to his wife.

The bone of contention here is what Apple’s customer service representatives called, according to Hansson, “the algorithm.” When asked why his wife was being treated differently, he was told the algorithm was responsible.

Yet blaming the algorithm – while saying an exception would be made for Hansson’s wife and her adjusted credit score, as Apple did – seems a tacit admission that said algorithm is flawed. At the very least, this raises questions about the degree of “accountability” of these systems. Customers don’t know the details of how Apple-Goldman credit calculations work, how much they depend on artificial intelligence (or, more specifically, machine learning), the inputs they use, or even the share of proprietary technology. to both companies.

If the system is indeed making such egregious decisions, should it really be used? At least when there is human error or bias, there is an easier way to correct it. While a business can ask a person how they arrived at an individual decision, this is usually not possible with machine learning. Instead, you need to look at the “big data” entries that informed the algorithm and see if that caused any set of biases.

Of course, the artificial intelligence bias is not unique to the Apple card. It has made its way into the criminal justice system, the job market, healthcare, facial recognition, app recommendations and beyond. In each case, understanding what gave rise to the prejudices is key to addressing it. And in each case, it’s easier said than done. John Giannandrea, head of AI at Apple, said in 2017 that data bias was the biggest danger in machine learning.

This is not Goldman’s first consumer credit misstep. The Wall Street firm may be at the forefront of finance, but its foray into consumer credit has stuck in rookie mistakes. Its consumer lending arm, Marcus, is said to have started without a team of debt collectors, resulting in early losses for defaulting borrowers.

Apple CEO Tim Cook, meanwhile, hinted that he was looking for a partner to bring the Apple card to Europe. The Hansson and Wozniak episodes show that it would be quite a gamble. The European Union’s General Data Protection Regulation, introduced last year, includes the “right to an explanation” for consumers – exactly what Hansson demands. Failure to provide a satisfactory reason may result in financial penalties. As we can see, with AI algorithms, such explanations are not easy to extract. Bloomberg

---

Read also: Apple plans to launch standalone AR and VR gaming headset by 2022

---

Subscribe to our channels on YouTube and Telegram