Service Framework: The Journey Behind Service to Service Communication at Conversion

Background

Conversion is building the marketing automation platform of the future. As we have continued to scale our product features, we have also scaled the amount of endpoints reaching over 350 managed endpoints across 10 services. This post will detail the journey we took with service to service communication used to handle >30 million requests a day.
‍
With the introduction of generics, we’ve seen new frameworks like huma and fuego which handle endpoint routing and generating documentation. This blog post details our Service Framework which goes one step further and allows services to call each other without an intermediate representation (and thus build step).

OpenAPI Generated Handlers

Within the software industry debate there is a heavy debate between monolithic architectures and microservices. At Conversion, we believe that services should be “macroservices” which means for basic functions we call a function instead of making a network request but get the benefit of separation for large concepts to keep our codebase manageable. Although we try to reduce “service sprawl”, we’ve seen the business need to separate into a few macroservices due to scaling limitations, domain specification, and centralizing common resources.

Our first approach to service to service communication was to generate handlers based on the OpenAPI specification we had for each service. At this point in our journey, all of our functions were commented in Golang with the documentation that could generate OpenAPI specifications using swag. The natural step was to use a framework to generate client handlers based on this specification.

We accomplished this with go-swagger which allows us to specify templates to generate files with Golang code based on the OpenAPI specification. While this served us well for over a year, our service complexity continued to increase and we saw shortcomings in the lack of strong typing, slow generation times (>2 mins), and poor developer experience requiring developers to regenerate handlers manually. Additionally, engineers needed to context switch between the terminal, repositories, and their IDE — disrupting the coding “flow” and creating a poor developer experience. We had clearly reached the limits of this setup and needed the next evolution.

Generic Schema and Handlers

For a while, there seemed to be no great answer to our problem without adopting heavy handed tools like protobuf / gRPC / bazel. However, we decided that for our scale (~7 engineers) these tools would create a large maintenance overhead and preferred simpler to maintain solutions.

Fortunately, in Go 1.18, Golang introduced generics which are a strong language primitive allowing logic and code to be re-used for different types. This, in combination with the fact that our backend is completely written in Golang and uses Fiber, allowed us to come up with a framework that defines schemas directly in Golang and use them in handlers without reflection or code generation.

At a high level, developers write a schema which defines the types that this endpoint uses. This schema is then used to instantiate a route which requires the handler function to match and accept the specified types. This guarantees via typing that the function is valid to represent this endpoint / schema. Finally, other services have all the metadata required to call this endpoint via the schema and can require a type safe request to be constructed all in Golang.

An example schema would be the following which defines a single endpoint that has a path with the templateId parameters:
‍

package emailschema

type TemplatePathParams struct {
    TemplateId uuid.UUID `json:"templateId" validate:"required"`
}

var (
    TakeTemplateScreenshotV1 = schema.EndpointWithPath[TemplatePathParams, string, EmailApiHandler]{
        EndpointSchema: schema.NewEndpointSchema[EmailApiHandler](
            "TakeTemplateScreenshotV1",
            "/v1/templates/:templateId/screenshot",
            http.MethodPost,
        ),
    }
)

We can then define a type safe endpoint using the following pattern:
‍

func (a *App) TakeTemplateScreenshotV1(ctx context.Context, params emailschema.TemplatePathParams) string {
  return "success"
}

routing.RouteEndpointWithPath(
  emailschema.TakeTemplateScreenshotV1, 
  endpointBuilder, 
  a.TakeTemplateScreenshotV1
)

Finally, we can call this endpoint from another service using a generic handler invocation:

resp, err := emailschema.TakeTemplateScreenshotV1.
  New(app.EmailHandler).
  WithPathParams(emailschema.TemplatePathParams{
    TemplateId: uuid.MustParse('UUID')
  }).
  Do(ctx)

All of this is made possible by our Service Framework! We now have a type safe service endpoint and type safe way to call other services. This is all done natively with generics greatly simplifying our build process (just go build!), improving developer experience with realtime feedback, and using an interface definition language (IDL) that must be correct.
The above example is a slightly abbreviated example of what it looks like to use our service framework. While we cannot open source our entire implementation due to its deep integration with our tech stack, we have created a simple version of this framework in an open source repository at https://github.com/tapp-ai/service-framework-example! This repository includes a simpler version of the machinery we use and a full example of an implementation.

The new Service Framework has been in production for over a month and we continue to migrate services to use the new framework. Beyond the many benefits including no build steps and greatly speeding up the developer experience, we’ve also been able to leverage this new framework to build infrastructure level tools like detailed observability, rate limiting, and request validation.

It’s an understatement to say that this has led to a big improvement in the intangible of “how fun it is to write code at Conversion” while delivering numerous business benefits.

Acknowledgements

The Service Framework is a collaboration and achievement shared by all of the backend engineering team at Conversion. This wouldn’t have been possible without the inspiration and leadership of Tayler who has been the internal champion of the new framework and led many of the early proof of concepts.

This achievement would also not have been possible without the help of Charlie, James, Naasir, and Swamik with their help in migrating our existing and critical endpoints to use the new framework.

If you love tackling interesting technical problems like the ones in this article, Conversion is hiring! We’re constantly working to build new features, improve reliability, and scalability. Please reach out if you’d like to join us on the mission of building the future of marketing.

‍