Group items tagged

"Creating an algorithm that discriminates or shows bias isn't as hard as it might seem, however. As a first-year graduate student, my advisor asked me to create a machine-learning algorithm to analyze a survey sent to United States physics instructors about teaching computer programming in their courses."

"The founders of Predictim want to be clear with me: Their product-an algorithm that scans the online footprint of a prospective babysitter to determine their "risk" levels for parents-is not racist. It is not biased. "We take ethics and bias extremely seriously," Sal Parsa, Predictim's CEO, tells me warily over the phone. "In fact, in the last 18 months we trained our product, our machine, our algorithm to make sure it was ethical and not biased. We took sensitive attributes, protected classes, sex, gender, race, away from our training set. We continuously audit our model. And on top of that we added a human review process.""

"Admission algorithms can make or break your academic plans. A Washington Post investigation found 44 schools use prediction software to give applicants a score out of 100 for its admissions process. The score considers different aspects of a student's application from test scores, home address, transcripts, and even what websites they've visited. That's all calculated to rate how strong of a match a student is for a school."

"Apart from biases in the training databases, it's hard to know how well face-recognition systems actually perform in the real world, in spite of recent gains. Anil Jain, a professor of computer science at Michigan State University who has worked on face recognition for more than thirty years, told me, "Most of the testing on the private venders' products is done in a laboratory environment under controlled settings. In real practice, you're walking around in the streets of New York. It's a cold winter day, you have a scarf around your face, a cap, maybe your coat is pulled up so your chin is partially hidden, the illumination may not be the most favorable, and the camera isn't capturing a frontal view.""