API Bakery: Building APIs that build APIs

API Bakery is a project generator SaaS. More specifically, it generates API services that can be used for web or mobile app backends or exposed to the public. In a way, it’s an API for generating APIs – with a twist. Based on a recipe that describes the project and data on a high level, API Bakery generates the complete ready-to-use project in a way that makes it easy to modify and build upon.

Code generation pros and cons

Generative AI helping programmers be more productive is all the rage now, but using programs to generate new programs is nothing new. As an industry, we’ve been doing that for decades. This is especially true for APIs. In the broadest strokes, an API can be understood as a specification of how to call (interface with) some program or service. And we programmers are lazy: if we have the spec, why not generate code for it?

The problem is two-fold: we can rarely automagically create all the functionality that we need; and auto-generated code is usually an ugly, unmaintainable mess.

The many code-generator tools that exist can’t handle all imaginable cases and situations. Because of this, they focus on covering the most common functionality, or what’s the easiest to automate. For APIs, that’s usually the client SDKs that wrap the HTTP calls and parse the responses. This works, but the resulting code isn’t as nice to use as something crafted by hand. On the server side, the generated code usually consists of empty wrapper methods that need to be filled in by the user to actually provide the working code.

This brings us to the second problem: having to update and maintain that code manually. Often, the generated code doesn’t conform to the coding style, best practices, or patterns of the framework or platform being used. Because it doesn’t have a wider context of the entire application, it can also be sub-optimal or have potential security issues. Nobody likes having to go in and deswamp that.

Trying existing code and API generators

On the other hand, code generators can save you a lot of time. In our web development agency work, we’ve worked on a lot of projects where the API was mostly standard CRUD (Create, Read, Update, Delete) methods, plus a few things like payments and project-specific flows. Writing this almost-boilerplate API seemed like a waste of time and clients’ money, so we started using existing project generators and building an internal tool to automate as much of it as possible.

Most modern web frameworks have one (or a few) project generators. We work with Django (a web framework for Python) a lot, and there are at least a dozen Django project generators, with the most popular being cookiecutter. However, we found that those generators can only provide a barebones skeleton for a new empty project. All those CRUD APIs we still had to write ourselves.

The more we tried to customize the generators to help us generate more code, the more we had to maintain those customizations, in effect creating our own framework on top of Django. This was definitely not the way we wanted to go.

We also tried using Swagger (an open API specification that can be used to generate documentation or client or server code) and found out that writing a complete OpenAPI spec was almost the same amount of work as writing the Django code for it!

How we built API Bakery

What we ended up doing was writing our own code generator from scratch, focusing on two things: making it take a description of a project only to generate what’s needed and making sure the generated code looks and works great, like it was written by an actual person on the team.

This means, for example, that the code passes code-style and quality checks (linters), has unit tests (we aim for 90%+ coverage out of the box), and that when you open any file in the editor, the layout, variable naming, etc. all feels “normal”. On the flip side, we wanted to make sure we didn’t generate any generic code that would do more than asked: we aimed for the simplest, straightforward solution like a human programmer would.

We sorely underestimated how hard it would be! It took us a few rewrites of the generator itself to make it to a point where we’re satisfied with the output and with the complexity of the generator.

Realizing how useful it could be to other developers beyond our agency, we decided to spin it off into a separate standalone software as a service – API Bakery.

How API Bakery works

API Bakery combines project scaffolding (creating a structure for a new empty project), database model generation, CRUD API generation, and tests (and other features, but these are the main areas). Of those, the project scaffolding part is straightforward and doesn’t need to be customized much – it’s the most like other generator tools like cookiecutter. For a new project, we just create a couple dozen files with (mostly) boilerplate code.

The database model generation needs to know the structure of your data. This structure is an input to API Bakery in a form of a JSON recipe. The recipe describes the database models and their relationships. For example, a webshop might have Products that are grouped into Categories. Each Product might have a few Variants (SKUs), and so on. API Bakery users configure this through a web-based UI, and it is passed as a JSON recipe to the project generator. Based on this, the actual code to create and manage the data is created.

The API generation builds on this. API Bakery generates the standard CRUD methods for accessing and managing the data through a REST API. The code for this is generated in such a way that it’s easy to add custom API (like payments or more involved data flows) later. We realize that no generator can generate all the functionality, so we take care to make it easy to expand it with custom code.

Finally, we generate automated tests. This has proven the most tricky part because tests must be tailored to the API for the custom data models. Things like default or required fields, allowed values, and permissions all must be handled here; there are tons of edge cases that we have to handle.

Finally, we have a Docker-based isolated environment in which we run the linters and tests to make sure the generated code does what it’s supposed to do. Automated tests with high coverage make this easier – we just run the tests we generated in the previous steps.

For the user, this all takes a few seconds, and the project is baked and ready!

The AI future?

Will AI replace us all? The jury is still out, but in the meantime, we’ve started exploring how it can make our projects better. For API Bakery, we’re very happy with the code generator and don’t think GPT or other coder AIs are there yet when it comes to creating a full non-trivial project. Where we see the value is in combining AI’s ability to understand imprecise, implicit, and context-dependent human instructions.

Our current integration takes advantage of this – users can describe their project in as much detail as they like in normal human language. We then use GPT4 to turn these instructions into a recipe, which then feeds into our existing code generator. The results have been striking. You can literally type one sentence and get a fully working and tested project for what you describe.

Where AI can be of great use is also in side-coding or co-piloting, i.e., helping the programmer write code faster and better. This is an area that API Bakery doesn’t currently cover – once you have the initial project generated, you’re on your own. Using AI to add custom code to an existing project and to go beyond CRUD API is an exciting proposition and within the realm of feasibility already.

With continuing AI progress, the life of API developers will just get easier!