In the first part of our series on Building, Deploying, and Observing SDKs as a Service, we delved into the world of APIs and successfully deployed our own REST APIs by wrapping the existing pet store APIs. Now, it’s time to take our journey further and unlock the true potential of SDKs. In this second part, we’ll explore how to build an SDK for the pet store API using the OpenAPI spec and the OpenAPI Generator project.
What is an SDK
SDKs, or Software Development Kits, play a pivotal role in empowering developers to leverage existing APIs and services seamlessly. By encapsulating the complexities of API integration, SDKs eliminate the need for developers to manually handle low-level API interactions and protocol details. Instead, they can focus on utilizing the intuitive methods and functions provided by the SDKs to accelerate their development process. These SDKs serve as an abstraction layer that hides the intricacies of the underlying APIs, enabling developers to work more efficiently and effectively.
Moreover, SDKs facilitate seamless integration across a variety of platforms and programming languages. They act as bridges between different systems, enabling developers to build applications that can effortlessly communicate and interact with diverse services. This interoperability opens up a world of possibilities for developers and integrators, as they can quickly and easily leverage SDKs to connect their applications with various APIs and services, regardless of the underlying technologies.
However, managing SDKs can be a challenging task due to the multitude of platforms and programming languages they support. Ensuring that SDKs are up to date with the latest API versions, maintaining compatibility across different language versions, and addressing platform-specific nuances are just a few of the complexities involved. Therefore, generating and maintaining SDKs can be a time-consuming and resource-intensive process.
SDKs should have the following properties:
- Be easy to use
- Support for popular programming languages
- Secure, with regular security updates
- Play well with other SDKs, although this one can be a little tricky
- Up to date with the latest versions of the API
What is SDK as a Service
Building and maintaining SDKs that support multiple programming languages are hard and take a lot of time. SDK as a service provides a self hosted way to support the automatic generation of SDKs for multiple programming languages and platforms at scale.
There is a few ways to approach the task of SDK generation
Using API Gateway
AWS APIGateway directly gives you the option to generate SDKs for several programming languages. However, it is important to note that this approach has certain limitations in terms of control and customization over the generated SDKs. It is particularly suitable for generating SDKs intended for internal usage, such as applications developed in-house. Based on previous experience, leveraging API Gateway for generating these internally utilized SDKs can prove to be a valuable method.
Using OpenAPI Generator
When it comes to the world of SDK generation, one tool stands out as a true gem: the OpenAPI Generator. This open-source project, which I have extensively worked with in the past, never fails to amaze me with its incredible capabilities, solidifying its place as one of my all-time favorite projects. It takes the rich specifications of OpenAPI, which elegantly describe the nuances of RESTful APIs, including endpoints and the intricacies of request and response structures. With a wave of its metaphorical wand, the OpenAPI Generator transforms these specifications into fully functional SDKs for a multitude of programming languages.
Whether you’re working with Java, Python, JavaScript, TypeScript, PHP, or one of the many other supported languages, the OpenAPI Generator has got you covered. By embracing the power of the OpenAPI specification format, you can define your APIs. The OpenAPI Generator then springs into action, automatically crafting SDKs that align with your chosen language. It’s an incredible asset that streamlines the process of SDK creation at scale, empowering you to focus on building innovative applications without getting bogged down in the intricacies of coding SDKs from scratch.
The OpenAPI Generator offers three methods for generating SDKs:
- Command-line Interface (CLI): Developers can utilize the CLI to generate SDKs by specifying the input specification file and the desired output language. The CLI offers customization options, enabling developers to tailor the generated SDKs according to their specific requirements.
- IDE Integrations: The OpenAPI Generator integrates seamlessly with popular Integrated Development Environments (IDEs) such as Visual Studio Code, IntelliJ IDEA, and Eclipse. These integrations provide developers with a graphical interface to configure and generate SDKs directly within their preferred development environment.
- Containers: The OpenAPI Generator offers containerized solutions, allowing developers to generate SDKs within lightweight, isolated environments. By leveraging containers, developers can easily incorporate SDK generation into their continuous integration and deployment pipelines, ensuring consistent and reproducible SDK generation.
These methods enable developers to generate SDKs in a variety of scenarios, whether through the CLI for command-line convenience, IDE integrations for a visual and integrated experience, or containerized solutions for automated and scalable SDK generation.
In this post we are going to deploy the containerized version to ECS, Then use that to generate a SDK.
Deploying, Testing and Monitoring the SDK Service
For this we are going to use the OpenAPI Online Generator as it is the quickest way to get up and running in a containerised deployment. Additionally with Lumigo connected to my aws, there’s also some fun metrics that can be gathered around resource utilization. It’s important to note that for the most part this will likely be an internally used tool, because it is used with API deployment events, we still want to be able to see the performance and potential capacity of our SDK generator service.
To Deploy the online service we have 2 options, Docker pull into a local container environment then tag and send up to AWS ECR (Elastic Container Registry) to deploy as an ECS task. Alternatively you can also deploy directly from the OpenAPI Generator docker hub as a task. The OpenAPI generator community has some great getting started guides to help get you up and running super quick, so head on over to the Getting Started guide for more on that.
With this demo deployment, Ive created a task definition with minimal resources directly from the docker hub, then deployed that as a service to a cluster.
All going well with the deployment you should end up with the default OpenAPI Online landing page available. This page will also give you further information on how to generate and query SDKs relative to your newly deployed service.
To test that we can then run the following command from a CLI with Curl available, this request is going to use the OpenAPI generator petstore test yaml to generate a Ruby SDK
curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' \
-d '{"openAPIUrl": "https://raw.githubusercontent.com/openapitools/openapi-generator/master/modules/openapi-generator/src/test/resources/3_0/petstore.yaml"}' \
'http://{ECS task IP}:8080/api/gen/clients/ruby'
This should then return the location of the generated SDK package, which can then be downloaded. You should get back a json response looking something like this
{"code":"3af73bf1-697e-410e-a4aa-bc8ff651fc39","link":"http://{ECS task IP}:8080/api/gen/download/3af73bf1-697e-410e-a4aa-bc8ff651fc39"}
Once generated, you can easily obtain the newly created SDK by running a wget command, but please note that the download link provided will only remain valid for a single download before being removed by the OpenAPI Generator app.
The following will download the SDK zip from the generator instance and name the zip output to `ruby.zip`
wget -O ruby.zip http://{ECS task IP}:8080/api/gen/download/3af73bf1-697e-410e-a4aa-bc8ff651fc39
After decompressing the zip file, you will have a fully functional SDK at your disposal, complete with documentation.
It is crucial to remember that the quality of the generated SDK relies heavily on the specification used for its generation. The goal here is to establish a reliable and iterative process capable of generating SDKs for multiple programming languages and platforms from a single API specification.
When operating at scale, it is crucial to monitor resource utilization and detect potential issues that may arise during the generation of one or multiple SDKs from the deployed container service. By connecting my AWS account to Lumigo, I gain valuable insights into runtime information about the deployed SDK service, allowing me to closely monitor resource consumption and promptly address any issues that may impact performance.
This becomes super important when you’re dealing with multiple API deployments and generating the necessary SDKs as part of your CI/CD pipeline. The openapi generator deployment I’m using right now has a slight delay of around 5 to 10 seconds for a single SDK (depending on language), mainly because I kept things minimal in the ECS deployment specs.
When it comes to automated generation, this delay could cause some resource consumption issues. So, it might be a good idea to beef up the cluster’s resources when the demand is high. Keep things running smoothly. Keeping a close watch on those metrics can work wonders in mitigating such issues by ensuring that everything stays in order.
Furthermore, since this is a Java-based application, you can enhance trace insights more by also building the container with our Open Telemetry Java distribution. This will show more granular level trace data to see where particular issues may be arising with OpenAPI spec files being used to generate SDKs or within the deployment itself.
Taking SDK Development Further
In this article, we delved into the significance of SDKs and explored the essential properties of a good SDK. Through the utilization of the OpenAPI Generator, we successfully generated the SDK for our pet store API and experienced its capabilities firsthand. However, it is important to remember that the OpenAPI Generator is an open-source project that thrives on community contributions.
This article marks the completion of part two in our series on building, deploying, and observing SDKs as a service. In the final installment, part three, we will delve into the critical aspects of testing and monitoring SDK. We will explore how to effectively test the generated SDKs and discuss the importance of continuous monitoring to ensure optimal performance and reliability. Join us in the next part as we take our SDK journey to the next level, while also giving back to the open-source ecosystem.
Lastly, if you find the OpenAPI Generator to be a valuable asset in your SDK generation process, consider giving back to the project. You can contribute by providing feedback, reporting any issues you encounter, or even submitting improvements to the codebase. A heartfelt shout-out goes to the amazing OpenAPI Generator community, who not only engaged in a productive discussion in their community Slack but also went the extra mile by submitting a pull request that resolved a Java image issue I had discovered. Their active involvement truly exemplifies the collaborative spirit of the open-source community 🙌.