Recently I wrote a blog post that mentioned “Superiority” as a type of A/B test decision. In this post I want to talk about all five types of A/B test decision that I think are relevant. This is an adaptation and extension of a talk I gave last year at the Quant UX conference (it’s a great event, you should check it …

Recently, tech-journalism site The Markup ran a long, detailed, critical investigation of a predictive machine learning model used by the State of Wisconsin to identify public school students at risk of not graduating. I mostly agree with the conclusions of the piece -- the system appears not to be fit for purpose and …

The "best practice", when evaluating the results of an online controlled experiment (A/B test), is to use classical statistical tests, proceeding with a change if (and only if) the result of the test includes a p value of less than 0.05. But, the American Statistical Association (ASA) said in a prominent 2016 …

A/B testing is a tool for supporting decision-making in business, and so in addition to getting the statistics right, it’s really important to communicate well with the non-statisticians who will have the final say on the go/no-go decision. Most A/B tests in practice are testing ratios, conversion rates of various …

I recently read Will Larson's excellent book Staff Engineer: Leadership beyond the management track. Larson covers the individual contributor (IC, not management) roles that software engineers fill after they are promoted past Senior Software Engineer, with titles like Staff and Principal ("Staff-plus"). In …

Suppose you’re a data scientist at an e-commerce web site that sells shoes, responsible for supporting A/B tests. Many A/B tests are easy, and there are a number of companies that sell tools that make the easy cases as simple as clicking a few buttons and looking at pretty graphs. But A/B tests can get statistically …

Just a quick post here to note a few professional accomplishments:

1. I just added a new publication to my vita -- a peer-reviewed conference proceeedings article about abstractions for building repeated, related versions of similar predictive models. Check out some longer thoughts on Medium, or read the full article. …

This is my first new post on harlan.harris.name for a while. The occasion is a change of scenery. For about 10 years, my primary blog has been on WordPress, more recently supplemented by Medium. But WordPress and Medium are limited for technical writing, and the trend among data people recently has been to publish …

This post was originally published on Medium

There’s recently been some interesting opinionated writing in the R statistical programming community about how and when to teach the abstracted, easy-to-use approaches to solving problems, versus the underlying nitty-gritty. David Robinson, Data Scientist at Stack Overflow, …

This post was originally published on Medium

I recently attended two small conferences — the ISBIS (International Society for Business and Industrial Statistics) 2017 conference, held at IBM Research in Westchester County, and the Domino Data Lab Popup, held in West SoHo. I was invited to speak at ISBIS (slides here, …