Group items matching

in title, tags, annotations or url

in a discipline that regards ingenuity as the ultimate virtue, those who engage in the grunt work of data cleaning and replication receive few rewards. Nobel prizes are not awarded for constructing new historical estimates of GDP that allow policy analysis to be extended back in time.

How could a flawed study have appeared first in the prestigious working-paper series of the National Bureau of Economic Research (NBER) and then in a journal of the American Economic Association? And, if this was possible, why should policymakers and a discerning public vest any credibility in economic research?CommentsView/Create comment on this paragraphIt was possible because economists are not obliged to make their data and programs publicly available when publishing scientific research. It is said that NBER working papers are even more prestigious than publication in refereed journals. Yet the Bureau does not require scholars to post their data and programs to its Web site as a condition for working-paper publication.

Statistics are helpful. But in economics, as in other lines of social inquiry, they are no substitute for proper historical analysis.

Measuring and assessing academic performance is now a fact of scientific life.

Yet current systems of measurement are inadequate. Widely used metrics, from the newly-fashionable Hirsch index to the 50-year-old citation index, are of limited use1

Existing metrics do not capture the full range of activities that support and transmit scientific ideas, which can be as varied as mentoring, blogging or creating industrial prototypes.

While usage-based billing and data caps are often talked about in terms of their ability to curb congestion, it's rarely suggested that making Internet access more expensive is a positive move for the content industries. But Castro has a whole host of such suggestions, drawn largely verbatim from his 2009 report (PDF) on the subject.

Should the US government actually fund antipiracy research? Sure. Should the US government “enlist” Internet providers to block entire websites? Sure. Should copyright holders suggest to the government which sites should go on the blocklist? Sure. Should ad networks and payment processors be forced to cut ties to such sites, even if those sites are legal in the countries where they operate? Sure.

Castro's original 2009 paper goes further, suggesting that deep packet inspection (DPI) be routinely deployed by ISPs in order to scan subscriber traffic for potential copyright infringements. Sound like wiretapping? Yes, though Castro has a solution if courts do crack down on the practice: "the law should be changed." After all, "piracy mitigation with DPI deals with a set of issues virtually identical to the largely noncontroversial question of virus detection and mitigation."

On the benefit side, people become more engaged when they can filter information that is presented visually and make discoveries on their own. On the risk side, Professor Shneiderman says, tools as powerful as visualizations have the potential to mislead or confuse consumers. And privacy implications arise, he says, as increasing amounts of personal, housing, medical and financial data become widely accessible, searchable and viewable.

In the 1990s, Professor Shneiderman developed tree mapping, which uses interlocking rectangles to represent complicated data sets. The rectangles are sized and colored to convey different kinds of information, like revenue or geographic region, says Jim Bartoo, the chief executive of the Hive Group, a software company that uses tree mapping to help companies and government agencies monitor operational data. When executives or plant managers see the nested rectangles grouped together, he adds, they should be able to immediately spot anomalies or trends. In one tree-map visualization of a sales department on the Hive Group site, red tiles represent underperforming sales representatives while green tiles represent people who exceeded their sales quotas. So it’s easy to identify the best sales rep in the company: the biggest green tile. But viewers can also reorganize the display — by region, say, or by sales manager — to see whether patterns exist that explain why some employees are falling behind. “It’s the ability of the human brain to pick out size and color” that makes tree mapping so intuitive, Mr. Bartoo says. Information visualization, he adds, “suddenly starts answering questions that you didn’t know you had.”

data visualization is no longer just a useful tool for researchers and corporations. It’s also an entertainment and marketing vehicle.

1) Any errors, however inconsequential, will be taken Very Seriously and accusations of fraud will be made.

2) If you adjust the raw data we will accuse you of fraudulently fiddling the figures whilst cooking the books.3) If you don't adjust the raw data we will accuse you of fraudulently failing to account for station biases and UHI.

7) By all means publish all your source code, but we will still accuse you of hiding the methodology for your adjustments.

Egon Willighagen commenting on Peter Murray-Rusk response to my blogpost  writes: Open Data is *not* about how to present (governmental) Data in a human readable way to the general public to take advantage of (though I understand why he got that idea), but Open Data is about making this technically and legally *possible*. He did not get that point, unfortunately.

“Open Data” as articulated above by Willighagen has the form of a private club—open “technically” (and “legally”) to all to join but whose membership requires a degree of education, ressources, technical skill such as to put it out of the reach of any but a very select group.

Parminder Jeet Singh in his own comments contrasts Open Data with Public Data—a terminology and conceptual shift with which I am coming to agree—where Public Data is Data which is not only “open” but also is designed and structured so as to be usable by the broad “public” (“the people”).

The Swiss economists Bruno Frey and Alois Stutzer argue that people do not appreciate the real cost of a long commute. And especially when that commute is unpredictable, it takes a toll on our daily well-being.

imagine if we shared our commuting information so that we could calculate the average commute from various locations around a city. When the growing family of four pulls up to a house for sale for in New Jersey, the listing would indicate not only the price and the number of bathrooms but also the rush-hour commute time to Midtown Manhattan. That would be valuable information to have, since buyers could realistically factor the tradeoffs of remaining in a smaller space closer to work against moving to a larger space and taking on a longer commute.

In a cover story for the New York Times Magazine, the writer Gary Wolf documented the followers of “The Data-Driven Life,” programmers, students, and self-described geeks who track various aspects of their lives. Seth Roberts does a daily math exercise to measure small changes in his mental acuity. Kiel Gilleade is a "Body Blogger" who shares his heart rate via Twitter. On the more extreme end, Mark Carranza has a searchable Database of every idea he's had since 1984. They're not alone. This community continues to thrive, and its efforts are chronicled at a blog called the Quantified Self, co-founded by Wolf and Kevin Kelly.

Despite the revolutionary technological advance of the printing press in the 15th century, there is precious little economic evidence of its benefits. Using data on 200 European cities between 1450 and 1600, this column finds that economic growth was higher by as much as 60 percentage points in cities that adopted the technology.

Historians argue that the printing press was among the most revolutionary inventions in human history, responsible for a diffusion of knowledge and ideas, “dwarfing in scale anything which had occurred since the invention of writing” (Roberts 1996, p. 220). Yet economists have struggled to find any evidence of this information technology revolution in measures of aggregate productivity or per capita income (Clark 2001, Mokyr 2005). The historical data thus present us with a puzzle analogous to the famous Solow productivity paradox – that, until the mid-1990s, the data on macroeconomic productivity showed no effect of innovations in computer-based information technology.

In recent work (Dittmar 2010a), I examine the revolution in Renaissance information technology from a new perspective by assembling city-level data on the diffusion of the printing press in 15th-century Europe. The data record each city in which a printing press was established 1450-1500 – some 200 out of over 1,000 historic cities (see also an interview on this site, Dittmar 2010b). The research emphasises cities for three principal reasons. First, the printing press was an urban technology, producing for urban consumers. Second, cities were seedbeds for economic ideas and social groups that drove the emergence of modern growth. Third, city sizes were historically important indicators of economic prosperity, and broad-based city growth was associated with macroeconomic growth (Bairoch 1988, Acemoglu et al. 2005).

better management of flood risks in Australia will depend up better data on flood risk.  However, collecting such data has proven problematic

As many Queenslanders affected by January’s floods are realising, riverine flood damage is commonly excluded from household insurance policies. And this is unlikely to change until councils – especially in Queensland – stop dragging their feet and actively assist in developing comprehensive data insurance companies can use.

? Because there is often little available information that would allow an insurer to adequately price this flood risk. Without this, there is little economic incentive for insurers to accept this risk. It would be irresponsible for insurers to cover riverine flood without quantifying and pricing the risk accordingly.

Humans make errors. We make errors of fact and errors of judgment. We have blind spots in our field of vision and gaps in our stream of attention.

These weaknesses put us at a disadvantage. We make decisions with partial information. We are forced to steer by guesswork. We go with our gut.

Others use data.

Whether individual Likert items can be considered as interval-level data, or whether they should be considered merely ordered-categorical data is the subject of disagreement. Many regard such items only as ordinal data, because, especially when using only five levels, one cannot assume that respondents perceive all pairs of adjacent levels as equidistant. On the other hand, often (as in the example above) the wording of response levels clearly implies a symmetry of response levels about a middle category; at the very least, such an item would fall between ordinal- and interval-level measurement; to treat it as merely ordinal would lose information. Further, if the item is accompanied by a visual analog scale, where equal spacing of response levels is clearly indicated, the argument for treating it as interval-level data is even stronger.

When treated as ordinal data, Likert responses can be collated into bar charts, central tendency summarised by the median or the mode (but some would say not the mean), dispersion summarised by the range across quartiles (but some would say not the standard deviation), or analyzed using non-parametric tests, e.g. chi-square test, Mann–Whitney test, Wilcoxon signed-rank test, or Kruskal–Wallis test.[4] Parametric analysis of ordinary averages of Likert scale data is also justifiable by the Central Limit Theorem, although some would disagree that ordinary averages should be used for Likert scale data.

Skeptics may by now be very familiar with recent attempts in Canada to ban wifi from public schools and libraries.  In short: there is no valid scientific reason to be worried about wifi.  It has also been revealed that the chief scientists pushing the wifi bans have been relying on poor data and even poorer studies.  By far the vast majority of scientific data that currently exists supports the conclusion that wifi and cell phone signals are perfectly safe.

So I wrote about that particular topic in the summer.  It got some decent coverage, but the fear mongering continued. I wrote another piece after I did a little digging into one of the main players behind this, one Rodney Palmer, and I discovered some decidedly pseudo-scientific tendencies in his past, as well as some undisclosed collusion.

One night I came home after a long day at work, a long commute, and a phone call that a beloved family pet was dying, and will soon be in significant pain.  That is the state I was in when I read the news about Palmer and Parliamentary committee.

The inquiry, by the Commerce Department’s inspector general, focused on e-mail messages between climate scientists that were stolen and circulated on the Internet in late 2009 (NOAA is part of the Commerce Department). Some of the e-mails involved scientists from NOAA.

Climate change skeptics contended that the correspondence showed that scientists were manipulating or withholding information to advance the theory that the earth is warming as a result of human activity.

In a report dated Feb. 18 and circulated by the Obama administration on Thursday, the inspector general said, “We did not find any evidence that NOAA inappropriately manipulated data.”

Regarding the “hide the decline” email, Jones has explained that when he used the word “trick”, he simply meant “a mathematical approach brought to bear to solve a problem”. The inquiry made the following criticism of the resulting graph (its emphasis): [T]he figure supplied for the WMO Report was misleading. We do not find that it is misleading to curtail reconstructions at some point per se, or to splice data, but we believe that both of these procedures should have been made plain — ideally in the figure but certainly clearly described in either the caption or the text. [1.3.2] But this was one isolated instance that occurred more than a decade ago. The Review did not find anything wrong with the overall picture painted about divergence (or uncertainties generally) in the literature and in IPCC reports. The Review notes that the WMO report in question “does not have the status or importance of the IPCC reports”, and concludes that divergence “is not hidden” and “the subject is openly and extensively discussed in the literature, including CRU papers.” [1.3.2]

As for the treatment of uncertainty in the AR4’s paleoclimate chapter, the Review concludes that the central Figure 6.10 is not misleading, that “[t]he variation within and between lines, as well as the depiction of uncertainty is quite apparent to any reader”, that “there has been no exclusion of other published temperature reconstructions which would show a very different picture”, and that “[t]he general discussion of sources of uncertainty in the text is extensive, including reference to divergence”. [7.3.1]

Regarding CRU’s selections of tree ring series, the Review does not presume to say whether one series is better than another, though it does point out that CRU have responded to the accusation that Briffa misused the Yamal data on their website. The Review found no evidence that CRU scientists knowingly promoted non-representative series or that their input cast doubt on the IPCC’s conclusions. The much-maligned Yamal series was included in only 4 of the 12 temperature reconstructions in the AR4 (and not at all in the TAR).

One of the scientists was Richard Muller from University of California, Berkeley. Muller has been working on an independent project to better estimate the planet's surface temperatures over time. Because he is willing to say publicly that he has some doubts about the accuracy of the temperature stations that most climate models are based on, he has been embraced by the science denying crowd.

A Koch brothers charity, for example, has donated nearly 25 percent of the financial support provided to Muller's project.

Skeptics of climate science have been licking their lips waiting for his latest research, which they hoped would undermine the data behind basic theories of anthropogenic climate change. At the hearing today, however, Muller threw them for a loop with this graph: