Software Architecture on Harlan D. Harris

New Publications and Upcoming Talks

harlan@harris.name (Harlan Harris) — Tue, 04 Sep 2018 00:00:00 +0000

Just a quick post here to note a few professional accomplishments:

I just added a new publication to my vita -- a peer-reviewed conference proceeedings article about abstractions for building repeated, related versions of similar predictive models. Check out some longer thoughts on Medium, or read the full article.
Earlier this year, I added an incredibly old project! A paper that I had contributed a bit to in... 2005! finally got published! It has something to do with the way information flows during speech perception... I think...
I'll be talking at two Meetups this Fall -- the RecSys NYC Meetup on Sept. 18th, and the Dataiku Data Science Meetup on Sept. 26th. In both Meetups, I'll be talking about (different) aspects of recommendations systems I'm building at WayUp.
And I'll be talking at two conferences this Fall too -- the Predictive APIs conference in Boston in October, and DataEngConf in NYC in November. At both of those conferences, I'll be talking about the software architecture aspects of building job recommendation systems that need to provide compelling recommendations just seconds after a user creates a rich profile.

What do Data Scientists mean by “Scaling”?

harlan@harris.name (Harlan Harris) — Fri, 09 Jun 2017 00:00:00 +0000

This post was originally published on Medium

Occasionally when chatting with other data scientists, especially with others who are interested in integrating predictive models into production software system, the word “scaling” comes up.

Not this. Although some West Coast data scientists are into this kind of scaling too.

I think this is a great question, but it’s a little underspecified. There seem to be at least three qualitatively different notions of “scaling” in data science, and it’s worth the effort to clarify each of them, and address how people tackle them.

Insights from a Predictive Model Pipeline Abstraction

harlan@harris.name (Harlan Harris) — Mon, 07 Nov 2016 00:00:00 +0000

This post was originally published on Medium

When building a complex system, it’s often helpful to think about the design of that system using patterns and abstractions. Architects and software engineers do so frequently, and the experience of implementing predictive modeling pipelines has recently led to a variety of patterns and best practices. For instance, when dealing with large amounts of streaming data, some organizations use the Lambda Architecture to handle both real-time and computationally-intensive use-cases.