Group items tagged

some examples of how the 3 types of machine learning - supervised, unsupervised, and reinforcement - have already been used for journalistic purposes, and using those to explain what those are along the way. Examples include: supervised learning to investigate doctors and sex abuse; unsurprivsed learning to identify motifs in Wes Anderson films; reinforcement learning to create a rock-paper-scissors that can beat you...

Roughly put, building a machine-learning model involves training it on a large number of examples and then testing it on a bunch of similar examples that it has not yet seen. When the model passes the test, you're done. What the Google researchers point out is that this bar is too low. The training process can produce many different models that all pass the test but-and this is the crucial part-these models will differ in small, arbitrary ways, depending on things like the random values given to the nodes in a neural network before training starts, the way training data is selected or represented, the number of training runs, and so on. These small, often random, differences are typically overlooked if they don't affect how a model does on the test. But it turns out they can lead to huge variation in performance in the real world. In other words, the process used to build most machine-learning models today cannot tell which models will work in the real world and which ones won't.

At The Economist, we take data visualisation seriously. Every week we publish around 40 charts across print, the website and our apps. With every single one, we try our best to visualise the numbers accurately and in a way that best supports the story. But sometimes we get it wrong. We can do better in future if we learn from our mistakes - and other people may be able to learn from them, too. After a deep dive into our archive, I found several instructive examples. I grouped our crimes against data visualisation into three categories: charts that are (1) misleading, (2) confusing and (3) failing to make a point. For each, I suggest an improved version that requires a similar amount of space - an important consideration when drawing charts to be published in print.

Every year in Australia, the National Assessment Program - Literacy and Numeracy (NAPLAN) results show Indigenous school students are well behind their non-Indigenous peers. Reducing this disparity is a vital part of Australia's national Closing the Gap policy. ... Using an updated version of our equivalent year levels metric, introduced in Grattan Institute's 2016 report Widening Gaps, we estimate year nine Indigenous students in very remote areas are: five years behind in numeracy six years behind in reading, and seven to eight years behind in writing. In other words, the average year nine Indigenous student in a very remote area scores about the same in NAPLAN reading as the average year three non-Indigenous city student, and significantly lower in writing. But it would be a big mistake to see this only as a problem for isolated outback communities. Most Indigenous students live in cities or regional areas. So, even though learning outcomes are worse in remote and very remote areas, city and regional students account for more than two-thirds of the lost years of learning.

But because a machine learning model learns the boundaries of its world from its input data, just three people informed how any model using that dataset would interpret if 'childbirth' was emotional. This led to a perspective that has informed all of my work since: a dataset is a worldview. It encompasses the worldview of the people who scrape and collect the data, whether they're researchers, artists, or companies. It encompasses the worldview of the labelers, whether they labeled the data manually, unknowingly, or through a third party service like Mechanical Turk, which comes with its own demographic biases. It encompasses the worldview of the inherent taxonomies created by the organizers, which in many cases are corporations whose motives are directly incompatible with a high quality of life.

The "What's going on in this graph' series is from the NYT Learning Network, and is about interpreting graphs that represent real data to tell a story. It's aimed at high school students but that just means the examples and explanations are a really great introduction to visualising and interpreting data!

When creating a machine-learning algorithm, designers have to make many choices: what data to train it on, what specific questions to ask, how to use predictions that the algorithm produces. These choices leave room for discrimination, particularly against people who have been discriminated against in the past. For example, training an algorithm to select potential medical students on a data set that reflects longtime biases against women and people of color may make these groups less likely to be admitted. In computing, the phrase "garbage in, garbage out" describes how poor-quality input leads to poor-quality output. In this case we might say, "White male doctors in, white male doctors out."

hen the AP United States history students at Aragon High School in San Mateo California, scanned the professionally designed pages of www.minimumwage.com, most concluded that it was a solid, unbiased source of facts and analysis. They noted the menu of research reports, graphics and videos, and the "About" page describing the site as a project of a "nonprofit research organization" called the Employment Policies Institute. But then their teacher, Will Colglazier, demonstrated how a couple more exploratory clicks-critically, beyond the site itself-revealed that the Employment Policies Institute is considered by the Center for Media and Democracy to be a front group created by lobbyists for the restaurant and hotel industries. "I have some bright students, and a lot of them felt chagrined that they weren't able to deduce this," said Colglazier, who videotaped the episode last January. "They got duped."

Over recent years, data has become an indispensable source for journalists and news organisations, providing excellent material for investigative work as well as storytelling. This has led to the emergence of data journalism, which, broadly speaking, uses information science and analytical techniques in conjunction with journalistic workflows to produce compelling stories rooted in data. Despite the relative maturity of data journalism and the growing application of data in editorial workflows, there is still a lot to learn about the systematic, seamless and effective integration of data and computational tools in newsrooms. It is time time for a holistic assessment of this emerging field by looking deeply into the ways newsrooms across the world have adopted data in their day-to-day workflows, the formation of their data teams, their best practices for producing high quality data driven investigative work, their success and failure stories, and emerging training requirements.

In August 2016, the Australian government released an "anonymised" data set comprising the medical billing records, including every prescription and surgery, of 2.9 million people. Names and other identifying features were removed from the records in an effort to protect individuals' privacy, but a research team from the University of Melbourne soon discovered that it was simple to re-identify people, and learn about their entire medical history without their consent, by comparing the dataset to other publicly available information, such as reports of celebrities having babies or athletes having surgeries.

Women continue to earn less than men, for a variety of reasons. Discrimination is one, research shows. Women are also likelier than men to work in lower-paying jobs like those in public service, caregiving and the nonprofit sector - and to take time off for children. Employers often base a starting salary on someone's previous one, so at each job, the gender pay gap continues, and it becomes seemingly impossible for women to catch up. Salary history bans are too new for researchers to have studied their effects extensively. But other research has found that people are overly influenced by an opening bid, something social scientists call anchoring bias. This means that if employers learn an applicant's previous salary and it's lower or higher than they were planning to offer, it's likely to influence their offer.

The reasons for the disparity in pay vary from company to company but the broader source of the gap can be seen in the distribution of high earners by gender. Of all the companies that have reported to date the top pay quartile, the highest paid 25% of employees, is male-dominated. Almost two-thirds of the top quartile is made up of men, while conversely 57% of the lowest-paid employees are women.

just knowing facts doesn't necessarily guarantee that one's opinions and behaviors will be consistent with them. For example, many people "know" that recycling is beneficial but still throw plastic bottles in the trash. Or they read an online article by a scientist about the necessity of vaccines, but leave comments expressing outrage that doctors are trying to further a pro-vaccine agenda. Convincing people that scientific evidence has merit and should guide behavior may be the greatest science communication challenge, particularly in our "post-truth" era. Luckily, we know a lot about human psychology - how people perceive, reason and learn about the world - and many lessons from psychology can be applied to science communication endeavors.

Sometimes it's hard to know what statistics are worthy of trust. But we shouldn't count out stats altogether ... instead, we should learn to look behind them. In this delightful, hilarious talk, data journalist Mona Chalabi shares handy tips to help question, interpret and truly understand what the numbers are saying.

Statistics have played an important role both in our understanding of the coronavirus pandemic, and our attempts to fight it. The RSS sets out ten lessons the government can learn, and a series of recommendations for what they should do now, to ensure that the country's data infrastructure is prepared for the next crisis - whatever form it takes.

By pulling this chart apart and then helping you put it back together, this story aims to help you understand: how quickly coronavirus is spreading in different countries; where Australia fits into the global picture; what we can learn from countries that appear to have curbed the rise of COVID-19; and what you can do to help keep Australians safe. But first, one concept that's vitally important to understanding a pandemic is exponential growth. This is a pattern viruses tend to initially follow, due to the way they're spread. The result is that what might seem like a small difference in the rate of growth can actually have enormous impacts on how many people are infected overall.

We learned this week that the Trump campaign may have tried to dissuade millions of Black voters from voting in 2016 through highly targeted online ads. The investigation, by Channel 4, highlighted a still little-understood online advertising technique, microtargeting. This targets ads at people based on the huge amount of data available about them online. Experts say Big Tech needs to be much more transparent about how microtargeting works, to avoid overblown claims but also counter a potential threat to democracy.

An excellent online course from the Open Data Institute on working with data to find stories

Most people pass through some type of public space in their daily routine - sidewalks, roads, train stations. Thousands walk through Bryant Park every day. But we generally think that a detailed log of our location, and a list of the people we're with, is private. Facial recognition, applied to the web of cameras that already exists in most cities, is a threat to that privacy. To demonstrate how easy it is to track people without their knowledge, we collected public images of people who worked near Bryant Park (available on their employers' websites, for the most part) and ran one day of footage through Amazon's commercial facial recognition service.

A series of short articles, with videos and browser addons "investigating algorithmic injustice and the formulas that influence our lives."