API Advice

How To Wrap Your Terraform Provider for Pulumi

Thomas Rooney

Thomas Rooney

September 15, 2023

Featured blog post image

How To Wrap Your Terraform Provider for Pulumi

In this post, we'll show you how to make a Terraform provider available on Pulumi. The article assumes some prior knowledge of Terraform and Pulumi, as well as fluency in Go. If you are new to Terraform providers, please check out our Terraform documentation.

This post will provide instructions for building a Pulumi provider, however the resultant provider is not intended for production use. If you want to maintain a Pulumi provider as part of your product offering, we recommend you get in touch with us. Without a partner, providers can become a significant ongoing cost.

Why Users Are Switching From Terraform to Pulumi

Following the recent HashiCorp license change (opens in a new tab), many users are exploring Pulumi as an alternative to Terraform.

The license change came as a surprise. In response, many companies are considering alternatives to manage their infra as code set up. Given the enormous scale of the Terraform ecosystem, it's unlikely that the license change will lead to Terraform dissapearing. However, we can expect to see some fragmentation in the market.

If you're used to Terraform, you might notice that Pulumi has fewer providers available. Currently, Pulumi offers 125 providers in their registry, while Terraform boasts an incredible 3,511.

We know the comparison isn't completely fair, though – some users take the "everything-as-code" approach to the extreme, with providers for ordering pizza (opens in a new tab), building Factorio factories (opens in a new tab), and placing blocks in Minecraft (opens in a new tab). But even accounting for the hobbyist providers, it's clear that Terraform has significantly more 3rd party support.

So, let's look at how we can shrink that provider gap...

How Pulumi Differs From Terraform

Pulumi and Terraform are both infrastructure-as-code tools, but they differ in many ways (opens in a new tab), most importantly in the languages they support.

Terraform programs are defined in the declarative HashiCorp Configuration Language (HCL), while Pulumi allows users to create imperative programs using familiar programming languages like Python, Go, JavaScript, TypeScript, C#, and Java.

This difference has some benefits and drawbacks. With Pulumi's imperative approach, users have more control over how their infrastructure is defined and can write complex logic that isn't easily expressed in Terraform's declarative language. However, this also means that Pulumi code can be less readable than Terraform code.

Bridging Terraform and Pulumi

Pulumi provides two tools to help maintainers build bridge providers. The first is Pulumi Terraform Bridge (opens in a new tab), which creates Pulumi SDKs based on a Terraform provider schema. The second repository, Terraform Bridge Provider Boilerplate (opens in a new tab), is a template for building a new Pulumi provider based on a Terraform provider.

While creating a new provider, we'll use the Terraform Bridge Provider Boilerplate, but we'll often call functions from the Pulumi Terraform Bridge.

Pulumi Terraform Bridge is actively maintained, so bear in mind that the requirements and steps below may change with time.

How the Pulumi Terraform Bridge Works

Pulumi Terraform Bridge plays an important role during two distinct phases: design time and runtime.

During design time, Pulumi Terraform Bridge inspects a Terraform provider schema, then generates Pulumi SDKs in multiple languages.

At runtime, the bridge connects Pulumi to the underlying resource via the Terraform provider schema. This way, the Terraform provider schema continues to perform validation and calculates differences between the state in Pulumi and the resource state.

Pulumi Terraform Bridge does not use the Terraform provider binaries. Instead, it creates a Pulumi provider based only on a Terraform provider's Go modules and provider schema.

Step by Step: Creating a Terraform Bridge Provider in Pulumi

The process of creating a Pulumi provider differs slightly depending on how the Terraform provider was created. In the past, most Terraform providers were based on Terraform Plugin SDK (opens in a new tab). More recent providers are usually based on Terraform Plugin Framework (opens in a new tab).

The steps you'll follow depend on your Terraform provider. Inspect the go.mod file in your provider to see whether it depends on github.com/hashicorp/terraform-plugin-sdk or github.com/hashicorp/terraform-plugin-framework.

Prerequisites

We manually installed the required tools before noticing that the Terraform Bridge Provider Boilerplate contains a Dockerfile to set up a development image. The manual process wasn't too painful, and either method should work.

The following must be available in your $PATH:

Terraform Plugin SDK to Pulumi

As an example of a Terraform provider based on Terraform Plugin SDK, we'll use this Spotify Terraform provider (opens in a new tab), a small provider for managing Spotify playlists with Terraform.

To create a new Pulumi provider, we'll start with the Terraform Bridge Provider Boilerplate (opens in a new tab).

Clone the Terraform Bridge Provider Boilerplate

Go to the Terraform Bridge Provider Boilerplate (opens in a new tab) repository in GitHub and click on the green Use this template button. Select Create a new repository from the dropdown.

Select your organization as the owner of the new repository (we'll use speakeasy-api) and create a name for your Pulumi provider. In Pulumi, it is conventional to use pulumi- followed by the resource name in lowercase as the provider name. We'll use pulumi-spotify.

Click Create repository.

Use your Git client of choice to clone your new Pulumi provider repository on your local machine. Our examples below will use the command line on macOS.

In the terminal, replace speakeasy-api with your GitHub organization name in the code below and run it:

git clone git@github.com:speakeasy-api/pulumi-spotify.git
cd pulumi-spotify

Rename Pulumi-Specific Strings in the Boilerplate

The Pulumi Terraform Bridge Provider Boilerplate in its current state is primarily an internal tool used by the Pulumi team to bring Terraform providers into Pulumi. We need to replace a few instances where the boilerplate assumes Pulumi will publish the provider in their GitHub organization.

The Makefile has a prepare command that handles some of the string replacement. In the terminal, replace speakeasy-api with your GitHub organization name in the command below and run it:

make prepare NAME=spotify REPOSITORY=github.com/speakeasy-api/pulumi-spotify

The make command will print the sed commands it ran to replace the boilerplate strings in two files in the repo.

Next, we'll use sed to replace strings in the rest of the repo:

In the terminal, replace speakeasy-api with your GitHub organization name, then run:

find . -not \( -name '.git' -prune \) -not -name 'Makefile' -type f -exec sed -i '' 's|github.com/pulumi/pulumi-spotify|github.com/speakeasy-api/pulumi-spotify|g' {} \;

In the Makefile, replace the ORG variable with the name of your GitHub organization.

Finally, in the provider/resources.go file, manually replace the values in the tfbridge.ProviderInfo struct. Many of these values define names and other fields in the resulting Pulumi SDK packages.

Import Your Terraform Provider

Back in the provider/resources.go file, replace the github.com/terraform-providers/terraform-provider-spotify/spotify import with github.com/conradludgate/terraform-provider-spotify/spotify.

Fix a Dependency Version

This temporary step is a workaround related to the version of terraform-plugin-sdk imported in the boilerplate.

During our testing, we encountered a bug in terraform-plugin-sdk that was fixed in v2.0.0-20230710100801-03a71d0fca3d.

In provider/go.mod, replace replace github.com/hashicorp/terraform-plugin-sdk/v2 => github.com/pulumi/terraform-plugin-sdk/v2 v2.0.0-20220824175045-450992f2f5b9 with replace github.com/hashicorp/terraform-plugin-sdk/v2 => github.com/pulumi/terraform-plugin-sdk/v2 v2.0.0-20230710100801-03a71d0fca3d. Note the difference in the version strings.

Remove Outdated make Step

This temporary step removes a single line from the Makefile that copies a non-existent scripts directory while building the Node.js SDK. In earlier versions of Pulumi, the Node.js SDK included a scripts folder containing install-pulumi-plugin.js, but Pulumi no longer generates these files (opens in a new tab).

In the Makefile, remove the cp -R scripts/ bin && \ line from build_nodejs:

build_nodejs:: VERSION := $(shell pulumictl get version --language javascript)
build_nodejs:: install_plugins tfgen # build the node sdk
	$(WORKING_DIR)/bin/$(TFGEN) nodejs --overlays provider/overlays/nodejs --out sdk/nodejs/
	cd sdk/nodejs/ && \
        yarn install && \
        yarn run tsc && \
		cp -R scripts/ bin && \ # remove this line
        cp ../../README.md ../../LICENSE package.json yarn.lock ./bin/ && \
		sed -i.bak -e "s/\$${VERSION}/$(VERSION)/g" ./bin/package.json

Remove the Non-Existent ‘prov.MustApplyAutoAliasing()’ Function

In provider/resources.go, remove the line prov.MustApplyAutoAliasing() from the Provider() function.

Build the Generator

In the terminal, run:

make tfgen

Go will build the pulumi-tfgen-spotify binary. You can safely ignore any warnings about missing documentation. This can be resolved by mapping documentation from Terraform to Pulumi, but we won't cover that in this guide, because the Pulumi boilerplate code does not include this step yet.

Build the Provider

In the terminal, run:

make provider

Go now builds the pulumi-resource-spotify binary and outputs the same warnings as before.

Build the SDKs

In the final step, Pulumi generates SDK packages for .NET, Go, Node.js, and Python.

We hope this comparison of Terraform and Pulumi and our step-by-step guide to bridging a Terraform provider into Pulumi has been useful to you. You should now be able to create a Pulumi provider based on your Terraform provider.

Speakeasy can help you generate a Terraform provider based on your OpenAPI specifications. Follow our documentation (opens in a new tab) to enter this exciting ecosystem.

Speakeasy also has a Pulumi target in Beta. Join our Slack community (opens in a new tab) to discuss our upcoming Pulumi release or for expert advice on Terraform providers.