Reading notes from Monday 10/14.
A Convex Framework for Fair Regression
What is the problem the paper identifies?
Regression problems must draw a balance between accuracy and fairness, the latter often compromising efficiency.
What is the contribution the paper presents?
The paper measures this accuracy-fairness balance with a value they call the Price of Fairness (PoF), which is found by testing different weights of the fairness regularizer.
Give more detail about the specific problem the paper is addressing.
The use of machine learning to predict the "value" of a person (in situations such as credit, employment, education, and criminal sentencing) is often discriminatory on the basis of individual characteristics. Attempts to fix this issue have resulted in an element of "fairness" that is used to re-evaluate both groups and individuals.
However, it has previously been shown that increased fairness always comes at a cost of decreased accuracy. This fairness-accuracy tradeoff is the primary concern of this paper, as it uses varying weights on its fairness regularizer and the use of linear and logistic regression settings to maintain a new metric of accuracy degradation ("Price of Fairness"). Ultimately, the authors conclude that the trade-off between fairness and accuracy must be made on a domain-specific basis, as there is no standard definition of fairness.
What terms or concepts are you confused about, and how important do you think each will be to your understanding of the paper?
How does the author show that their approach is successful? What evidence does the author provide? Is it compelling?
The paper uses an empirical approach based off of 6 datasets:
Each dataset corresponds to an accuracy v.s. fairness model created from the varying fairness regularizers.
While in theory, the paper notes that group fairness is seen as less costly than individual fairness, the slim majority of their datasets displayed the opposite. Since the scale of fairness loss differed so substantially among the datasets, the authors required a normalization technique that would allow them to compare attributes. This became the Price of Fairness (PoF).
Yet even after utilizing the PoF to normalize the datasets, there remained a large degree of variation. The PoF would at times lead to greater fairness at the cost of greater accuracy, and vice versa. The authors conclude that practitioners must take care to define the type of fairness they prefer when measuring accuracy-fairness tradeoffs.
How does the work in this paper build on what has been done before?
This paper separates itself from the preceding work on the subject in a three key ways:
The authors also mention that similar work has been done using statistical parity in regression settings; however, they differentiate themselves from this work by introducing the concept of "similarity of pairs," which displays a different group notion.
Lastly, the paper continually emphasizes the importance of deciding on a fairness notion based on the task at hand. For their own work, they decide to allow for "disparate treatment" based on the idea that "similar instances should be treated similarly" (despite the possibility of discriminatory consequences).
To the best of your understanding, what is proposed in this paper? The answer to this question should contain as much detail as possible, and should be a few paragraphs long, most likely.
While describing the related work that has already been completed in similar subjects, the authors describe several techniques — statistical parity, equality of odds, equal means — and concluded that none of them work for all cases. The use of machine learning to predict classifications is too broad of a subject to encompass in a single technique. Rather, it is the responsibility of the practitioners who decide to proceed algorithmically to determine a method that will best serve their interests.
These interests must consider several attributes, but the primary differentiation to make is whether to proceed in the interest of fairness or accuracy. The paper states early on that their mutually exclusive has already been established, and therefore the problem at hand for anyone using machine learning to predict peoples' "value" is to find an appropriate balance between the two.
To demonstrate their findings, the authors use a new method of combining regression in their fairness regularizer, as well as taking an empirical approach. They then measure the disparity between fairness and accuracy using a number they call the Price of Fairness (PoF) to normalize their results.
What are the key concepts/terms/ideas that are blocking your more complete understanding of this paper? These should be the things you highlighted as confusions that you feel are most important for you to understand in order to understand this paper.
Much of the statistical reasoning was lost on me, as I haven't yet completed a college-level statistics class. Therefore, important concepts — most notably that of regression, fairness regularizers, and normalization — went over my head.
I also understand that this paper encompasses both machine learning and statistics, but my knowledge of both subjects is not yet deep enough to see where the two overlap.