How we used Tilt to accelerate our product feedback loop
The last 3 months the Archera team has been going through rapid technology, design, and marketing changes. Our company, formerly known as Reserved.ai, began by providing reservation and commitment management for AWS, but it is ever evolving based on customer needs. We are now Archera and while the .ai has stuck with us a lot has changed.
Our mission has been to improve Engineering and Finance teams lives, initially this was through commitment management but as we showed customers more of what our tools could do, their requests expanded into advanced scenario planning. This meant rebuilding our architecture around an API first extensible system so customers could ala cart features they wanted. We also wanted to improve and maintain consistency across our new UI/UX, and set up engineering and product to be able to scale in parallel. Specifically we wanted to accelerate customer feedback on features and design in usability studies and product feedback sessions. Essentially, we wanted to create our own little sandbox for each team member to accelerate their workflow. No more engineers having to wait for an Argo update or the product manager having to wait for a staging deploy to get customer feedback. We took a look at a number of tools and, with our dogfood approach and internal use of Kubernetes, we were happy to discover Tilt.
Tools and Research
Here's the breakdown of various solutions ranging from simple tools to complex platforms which we explored as part of our investigation.
Tools for local development environments:
These were probably the simplest to integrate for getting something up and running quickly. However, both failed to check many boxes from our original set of requirements. These tools lack many features from our set of requirements e.g: developer collaboration and reproducible environments among other things.
Collaborative development platforms:
Although impressive, both platforms were well suited for browser-based coding which was not our priority. Our goals at the time were to perform, test, and share changes in the app and were not centered around collaborative coding. These platforms also provide limited support for custom deployment infrastructure. Though Gitpods was more mature, it's paid for cloud version and self-hosted version had many shortcomings .
Kubernetes specific solutions:
Skaffold is a really powerful tool for local k8s app development. It is built upon a pluggable architecture, however requires significant work to build out of the box capabilities offered by other solutions. Skaffold is highly recommended if you are creating a tool/platform from scratch for k8s app development.
We finally honed in on Garden and Tilt for proofs of concept to compare their capabilities and relative pros and cons.
Garden is probably the most comprehensive and mature tool out there for app development environments. Their live update in-cluster image building capabilities provide a lot of value. We managed to deploy Garden on our cluster and test a few microservices. During that process we noticed that the process of defining a workflow in Garden is a bit complex and time consuming due to their stack graph style dependency structure. Though the platform is pluggable and extensible, it is decoupled from native k8s requiring some additional work while defining providers. This meant more work initially to translate our services, ingress endpoints although it might yield more value in the future as we diversify the underlying requirements.
Last but obviously not the least we tried Tilt. Since we were already using k8s + helm, the initial setup was just writing a Tiltfile which uses a python dialect describing deployment steps. Tilt can either be pointed to a local or remote k8s cluster and uses your local Kubeconfig for authentication. It offers a live update feature similar to Garden and tracks and pushes changes after initial image build/push. It also comes with a UI to track progress of various steps. Tilt still requires local docker for image building and falls short for use cases requiring more complex setup, but these were not concerns for our particular use case.
Key Features of Tilt
All the tools we researched are great.Though some might have overlapping capabilities, each of them is meant to target a particular problem domain.The most impressive part of Tilt was the initial setup time - we were able to get up and running with just a 20 or so line file and firing
tilt up. Also, Tilt’s performance with regards to monitoring the changes and building/deploying was impressive. After due research we shortlisted Tilt for our case because of following key capabilities:
- Quick and simple setup
- Clear and concise documentation
- Native support for k8s + helm
- Live update capabilities
- UI for tracking build progress
- Reproducible Snapshots
Also, we have a never-ending ❤️ for open-source.
Trying Tilt for the first time
This was amazing! 15 minutes and Bikramdeep was up and running! Immediately here are the seamless live updates!
Instead of waiting half an hour for staging Tilt was up in seconds. This was our goal for the feedback loop with Tilt.
The above diagram shows an overview of our current Tilt setup. An important thing to notice here is the development feedback loop became shorter and more agile after we introduced Tilt in our pipeline since developers no longer need to wait for their changes to be deployed on staging environment in order to get feedback. This is especially helpful for scenarios like minor frontend changes, bug fixes where quick turnaround time is required.
What we experienced in our product design flow
For the product and design team, Tilt allowed us to do rapid prototyping and feedback. We could pull up specific branches right before a customer call and get immediate feedback from our design partners on the line.
For usability experiments we would set up standard 30 minute calls with customers and kick up a tilt environment for the window of the call. It was so fast sometimes we would do it at the tail end of other customer account calls where there was extra time.
This combination of the rapid ability to test internally and get customer feedback in parallel collapsed conversations that would take months to schedule and set up down to weeks. Shortening the feedback loop allowed us to not only hit our launch date ahead of schedule but be prepared to adjust during launch.
We loved it and are integrating it more into our process!
”I’ve worked with a lot of DevOps tools and overall as a product manager I would bring Tilt into any organization working with Kubernetes to help product, engineering, and UI/UX rapidly engage with customers. We look forward to the next set of upgrades that we will be doing to provide the Sales team with environments that they can spin up for trial customers.”
Jason Burt - Product Manager
Now that we have the basic Tilt setup for testing, we would like to explore more capabilities to streamline our local dev environments. This includes:
- Exploring how we can use snapshots and python debuggers to reproduce and debug environments like staging and production.
- Exploring custom extensions for running integration tests.
- Importing custom metrics from Tilt into Datadog for performance analysis.