Machine-Learning on Harlan D. Harris

A Big Win from a Small Search UI Change and a Predictive Model

harlan.harris@gmail.com (Harlan Harris) — Thu, 09 Jul 2026 00:00:00 +0000

One of my biggest commercial wins was a seemingly small UI tweak that led to a roughly 1.5% marketplace revenue bump. Teachers Pay Teachers (TPT, think "Etsy for Teachers" -- educational resources created by and sold to other educators) was a mature marketplace, where revenue wins from search system changes were typically under 0.5%. How we got this win is, I think, instructive, reinforcing the value of a search scientist, and of having a holistic view of search UI/UX and algorithms.

Are LLMs a simple solution? And if so, for what problems?

harlan.harris@gmail.com (Harlan Harris) — Thu, 14 Sep 2023 00:00:00 +0000

The other day, I was reading a post by Venkatash Rao (thousands of words of under-edited brilliance, as usual), and was struck by this note about the complexity of climate solutions:

I tend to take as an article of faith the systems science rule of thumb that the complexity of solutions generally matches the complexity of the problems. If it doesn’t, then you either got lucky, or there are negative externalities you’re ignoring.

A Critique of The Markup’s Investigation into Predictive Models of Student Success

harlan.harris@gmail.com (Harlan Harris) — Mon, 29 May 2023 00:00:00 +0000

Recently, tech-journalism site The Markup ran a long, detailed, critical investigation of a predictive machine learning model used by the State of Wisconsin to identify public school students at risk of not graduating. I mostly agree with the conclusions of the piece -- the system appears not to be fit for purpose and needs to be substantially improved -- but I want to comment on several aspects of the model and the Markup’s reporting. Although I know nothing about the Wisconsin model beyond what is reported, I know a lot about predictive student success risk models, having led a team of data scientists who built related models used by colleges and universities when I worked at EAB from 2014 through 2016.

New Publications and Upcoming Talks

harlan.harris@gmail.com (Harlan Harris) — Tue, 04 Sep 2018 00:00:00 +0000

Just a quick post here to note a few professional accomplishments:

I just added a new publication to my vita -- a peer-reviewed conference proceeedings article about abstractions for building repeated, related versions of similar predictive models. Check out some longer thoughts on Medium, or read the full article.
Earlier this year, I added an incredibly old project! A paper that I had contributed a bit to in... 2005! finally got published! It has something to do with the way information flows during speech perception... I think...
I'll be talking at two Meetups this Fall -- the RecSys NYC Meetup on Sept. 18th, and the Dataiku Data Science Meetup on Sept. 26th. In both Meetups, I'll be talking about (different) aspects of recommendations systems I'm building at WayUp.
And I'll be talking at two conferences this Fall too -- the Predictive APIs conference in Boston in October, and DataEngConf in NYC in November. At both of those conferences, I'll be talking about the software architecture aspects of building job recommendation systems that need to provide compelling recommendations just seconds after a user creates a rich profile.

A few things I learned from two small Data Science conferences

harlan.harris@gmail.com (Harlan Harris) — Sun, 16 Jul 2017 00:00:00 +0000

This post was originally published on Medium

I recently attended two small conferences — the ISBIS (International Society for Business and Industrial Statistics) 2017 conference, held at IBM Research in Westchester County, and the Domino Data Lab Popup, held in West SoHo. I was invited to speak at ISBIS (slides here, if you’re curious), but for this post, I want to summarize some insights from other people’s talks.

In chronological (to me) order… First a few talks from ISBIS that I particularly liked (note that I only saw a fraction of all the talks):

What do Data Scientists mean by “Scaling”?

harlan.harris@gmail.com (Harlan Harris) — Fri, 09 Jun 2017 00:00:00 +0000

This post was originally published on Medium

Occasionally when chatting with other data scientists, especially with others who are interested in integrating predictive models into production software system, the word “scaling” comes up.

Not this. Although some West Coast data scientists are into this kind of scaling too.

I think this is a great question, but it’s a little underspecified. There seem to be at least three qualitatively different notions of “scaling” in data science, and it’s worth the effort to clarify each of them, and address how people tackle them.

Transparency, Trust, and Proprietary Predictive Analytics

harlan.harris@gmail.com (Harlan Harris) — Mon, 27 Feb 2017 00:00:00 +0000

This post was originally published on Medium

A particularly good talk at Strata NY last year was by Brett Goldstein, former CIO of Chicago, who talked about accountability and transparency in predictive models that affect people’s lives. This struck a strong chord with me, so I wanted to take some time to write down some thoughts. (And a rather longer time to publish those thoughts…) I’m sure others’ have thought about this more and have better takes on this — please comment and provide links!

Insights from a Predictive Model Pipeline Abstraction

harlan.harris@gmail.com (Harlan Harris) — Mon, 07 Nov 2016 00:00:00 +0000

This post was originally published on Medium

When building a complex system, it’s often helpful to think about the design of that system using patterns and abstractions. Architects and software engineers do so frequently, and the experience of implementing predictive modeling pipelines has recently led to a variety of patterns and best practices. For instance, when dealing with large amounts of streaming data, some organizations use the Lambda Architecture to handle both real-time and computationally-intensive use-cases.

Patterns for Connecting Predictive Models to Software Products

harlan.harris@gmail.com (Harlan Harris) — Tue, 12 Jul 2016 00:00:00 +0000

This post was originally published on Medium

You’re a data scientist, and you’ve got a predictive model — great work! Now what? In many cases, you need to hook it up to some sort of large, complex software product so that users can get access to the predictions. Think of LinkedIn’s People You May Know, which mines your professional graph for unconnected connections, or Hopper’s flight price predictions. Those started out as prototypes on someone’s laptop, and are now running at scale, with many millions of users.