How we used Tilt to accelerate our product feedback loop

The last 3 months the Archera team has been going through rapid technology, design, and marketing changes. Our company, formerly known as Reserved.ai, began by providing reservation and commitment management for AWS, but it is ever evolving based on customer needs. We are now Archera and while the .ai has stuck with us a lot has changed.

Our mission has been to improve Engineering and Finance teams lives, initially this was through commitment management but as we showed customers more of what our tools could do, their requests expanded into advanced scenario planning. This meant rebuilding our architecture around an API first extensible system so customers could ala cart features they wanted. We also wanted to improve and maintain consistency across our new UI/UX, and set up engineering and product to be able to scale in parallel. Specifically we wanted to accelerate customer feedback on features and design in usability studies and product feedback sessions. Essentially, we wanted to create our own little sandbox for each team member to accelerate their workflow. No more engineers having to wait for an Argo update or the product manager having to wait for a staging deploy to get customer feedback. We took a look at a number of tools and, with our dogfood approach and internal use of Kubernetes, we were happy to discover Tilt.

In this post our team members who led this process, software engineer Bikramdeep Singh and product manager Jason Burt, explain how they did it.

Here's the breakdown of various solutions ranging from simple tools to complex platforms which we explored as part of our investigation.

Tools for local development environments:

These were probably the simplest to integrate for getting something up and running quickly. However, both failed to check many boxes from our original set of requirements. These tools lack many features from our set of requirements e.g: developer collaboration and reproducible environments among other things.

Collaborative development platforms:

Although impressive, both platforms were well suited for browser-based coding which was not our priority. Our goals at the time were to perform, test, and share changes in the app and were not centered around collaborative coding. These platforms also provide limited support for custom deployment infrastructure. Though Gitpods was more mature, it's paid for cloud version and self-hosted version had many shortcomings .

Kubernetes specific solutions:

Since we were already using kubernetes (k8s) + helm for our production and staging environment it made sense for us to explore solutions offering dev capabilities on k8s environments.

Skaffold is a really powerful tool for local k8s app development. It is built upon a pluggable architecture, however requires significant work to build out of the box capabilities offered by other solutions. Skaffold is highly recommended if you are creating a tool/platform from scratch for k8s app development.

We finally honed in on Garden and Tilt for proofs of concept to compare their capabilities and relative pros and cons.

Garden is probably the most comprehensive and mature tool out there for app development environments. Their live update in-cluster image building capabilities provide a lot of value. We managed to deploy Garden on our cluster and test a few microservices. During that process we noticed that the process of defining a workflow in Garden is a bit complex and time consuming due to their stack graph style dependency structure. Though the platform is pluggable and extensible, it is decoupled from native k8s requiring some additional work while defining providers. This meant more work initially to translate our services, ingress endpoints although it might yield more value in the future as we diversify the underlying requirements.

Last but obviously not the least we tried Tilt. Since we were already using k8s + helm, the initial setup was just writing a Tiltfile which uses a python dialect describing deployment steps. Tilt can either be pointed to a local or remote k8s cluster and uses your local Kubeconfig for authentication. It offers a live update feature similar to Garden and tracks and pushes changes after initial image build/push. It also comes with a UI to track progress of various steps. Tilt still requires local docker for image building and falls short for use cases requiring more complex setup, but these were not concerns for our particular use case.

All the tools we researched are great.Though some might have overlapping capabilities, each of them is meant to target a particular problem domain.The most impressive part of Tilt was the initial setup time - we were able to get up and running with just a 20 or so line file and firing tilt up. Also, Tilt’s performance with regards to monitoring the changes and building/deploying was impressive. After due research we shortlisted Tilt for our case because of following key capabilities:

Also, we have a never-ending ❤️ for open-source.

This was amazing! 15 minutes and Bikramdeep was up and running! Immediately here are the seamless live updates!

Instead of waiting half an hour for staging Tilt was up in seconds. This was our goal for the feedback loop with Tilt.

The above diagram shows an overview of our current Tilt setup. An important thing to notice here is the development feedback loop became shorter and more agile after we introduced Tilt in our pipeline since developers no longer need to wait for their changes to be deployed on staging environment in order to get feedback. This is especially helpful for scenarios like minor frontend changes, bug fixes where quick turnaround time is required.

For the product and design team, Tilt allowed us to do rapid prototyping and feedback. We could pull up specific branches right before a customer call and get immediate feedback from our design partners on the line.

For usability experiments we would set up standard 30 minute calls with customers and kick up a tilt environment for the window of the call. It was so fast sometimes we would do it at the tail end of other customer account calls where there was extra time.

This combination of the rapid ability to test internally and get customer feedback in parallel collapsed conversations that would take months to schedule and set up down to weeks. Shortening the feedback loop allowed us to not only hit our launch date ahead of schedule but be prepared to adjust during launch.

Now that we have the basic Tilt setup for testing, we would like to explore more capabilities to streamline our local dev environments. This includes:

We're also excited to test out some of Tilt's recently released features, including their new API and UI featuresfor creating things like custom buttons.