App developers need new paradigms for AI programming

We need a middle ground between trusting black box APIs and becoming AI researchers

goldilocks

The year is 2023, but we’re in a 2010ish moment when it comes to AI and app development.

2010 is when Backbone.js, the groundbreaking web framework, was first released. From that moment forward, we saw a rapid evolution of web frameworks but more importantly programming paradigms. Functional reactive programming (e.g., React, Redux) eventually won the day, but we had to go through several alternatives—for example, Angular’s two-way data binding—to get there.

We’re at the very beginning of going through those paradigms today when it comes to building AI-powered apps. It’s clear that AI will change everything about the software engineering and product process, from the interfaces we create to our infrastructure to the way we write code.

However, there’s a yawning chasm right now between AI IaaS providers like OpenAI and Replicate, which offer opaque APIs for inferences and fine-tuning, and fine tuning your own Llama 2 instance with Hugging Face libraries. The former is too expensive at the moment and too much of a black box for many applications. The latter is too close to AI research, requiring devs to learn lots of machine learning fundamentals just to use the library.

AI programming paradigms are the missing middle. What’s going to be the reactive programming equivalent that enables the React of LLMs? We don’t know yet.

The first step to getting there is to give developers a set of new “mental primitives.” We need to acknowledge that we’re transitioning from “classical programming” to “AI programming” and that the two are very different. In other words, we need to develop idioms, shorthands, and a way of reasoning around AI app development that’s distinct from how researchers talk about AI/ML.

This conviction has grown out of my frustration with the standard recommendations for devs who want to learn AI, such as Andrej Karpathy’s YouTube videos or the fast.ai course. The barrier to entry is still too high.

Instead, we need stuff closer to Brev.dev’s straightforward guide on how to fine tune Llama 2. The problem there is that when something goes wrong—like when after fine tuning your model still doesn’t provide the desired output—you have to get your hands dirty with Hugging Face TRL’s APIs, and those APIs are still too low-level for the average developer.

Another analogy is to look at SQL databases. The information theory behind SQL and all the things that make Postgres work is way above my head. Yet as a web dev, I intuitively understand concepts like atomic commits and ACID. More importantly, concepts like ACID enable and inspire higher-level paradigms like CRUD, the bread and butter of many a app/web developer’s career.

I’d like to explore potential paradigms in this blog as I learn more about AI and machine learning. Hopefully others will find them useful. Consider this post my clarion call 📯.