An Ethics Case Study
Between 2013 and 2015, according to a Detroit Free Press article, “at least 20,000 Michigan residents—and possibly as many as 40,000—were wrongly accused of fraud” based on determinations made by a computer system that the state implemented “without human supervision and with an error rate as high as 93%.”
The residents impacted faced high penalties; some also had their wages garnished or their tax refunds seized by the state.
Referencing the ancient legend about King Midas, who despaired after being granted the wish to have everything that he touched turn into gold, a related article published in the IEEE Spectrum’s “Computing” section argued that in Michigan’s case, “algorithm alchemy created lead, not gold.” As of May 2020, some of the residents impacted had received $20 million in refunds; however, ongoing lawsuits contend that the amount should be much higher.
MiDAS (the “Michigan Integrated Data Automated System”) is one example of a growing number of automated tools that various states have deployed in an effort to curtail fraud or expenditures, or to streamline the administration of various governmental programs. A Time magazine article titled “States’ Automated Systems Are Trapping Citizens in Bureaucratic Nightmares with Their Lives on the Line” details several such tools that have come under criticism (and at times led to lawsuits) in Arkansas, Idaho, and Illinois. In many cases, people whose lives were impacted by the systems found it difficult to even understand what led to the decisions made about their benefits—let alone to appeal those decisions.
As the Time article notes, “when problems with bureaucratic software arise,… officials have tended to blame the unknowable nature of the algorithms themselves, rather than take responsibility for their output.” In this “accountability gap,” “neither the designer nor the state takes responsibility for an algorithm’s decisions.”
Lawsuits may yet assign responsibility, but such resolution can take a very long time. It was not until March of 2021, for example, that a federal judge held that Michigan residents who had been wrongly accused of fraud could move forward with their lawsuit against state officials and the companies involved in the design and implementation of MiDAS. In the meantime, some of the people impacted had filed for bankruptcy and had been contending with the consequences for years.
Discussion questions
Before answering these questions, please review the Markkula Center for Applied Ethics’ Framework for Ethical Decision-Making, which details the ethical lenses referenced below.
- Who are the stakeholders involved in this case?
- What are some of the causes of the “accountability gap”?
- Consider the case through the lenses of rights, justice, utilitarianism, the common good, virtue, and care ethics; what ethical aspects does each of them highlight?
