In today's fast-paced technological landscape, optimizing developer productivity is paramount for businesses to remain competitive. At ThoughtSpot, we've recognized the potential of remote development environments to revolutionize how our engineering teams work. Inspired by the success of remote development environments at other tech giants like Uber, we embarked on a journey to create a solution that would transform the way our developers work. DevPod, our innovative solution, provides a seamless and efficient way for developers to collaborate and build applications from anywhere in the world.
Before DevPod, our development workflow faced significant challenges. Local development environments were often inconsistent, with developers spending countless hours setting up tools, dependencies, and configurations. These discrepancies led to the infamous "it works on my machine" problem, causing delays in debugging and collaboration. Resource-intensive tasks like builds and tests strained local machines, slowing down productivity. Onboarding new team members was time-consuming, as they had to navigate complex setup processes. These pain points not only hampered individual productivity but also created bottlenecks in team collaboration and project delivery.
It was clear that we needed a better approach, one that would eliminate these inefficiencies and provide a consistent, scalable, and secure development environment. This search for a solution led us to DevPod, a platform designed to address these challenges head-on.
In this article, we’ll explore the intricacies of DevPod, highlight its benefits and implementation, and explain how it enhanced developer productivity at ThoughtSpot.
Table of contents:
DevPod is a containerised cloud-based remote development environment, designed to provide developers with a self-contained workspace that is pre-configured with all the necessary tools and dependencies. It eliminates the need for lengthy setup processes, allowing developers to focus on coding and innovation from the very beginning. It runs as a container on Kubernetes, with the necessary tools and compute resources to enable seamless development on ThoughtSpot’s large-scale monorepos.
DevPod offers the following benefits:
Enhances developer productivity: By providing a consistent and efficient development environment, DevPod reduces context switching and accelerates development cycles.
Improves collaboration: DevPod facilitates seamless collaboration among distributed teams, enabling knowledge sharing and code reviews in real-time.
Reduces infrastructure overhead: By centralizing development environments in the cloud or on-prem infrastructure, DevPod eliminates the need for managing local development machines, reducing IT overhead.
Ensures security and compliance: DevPod provides a secure and compliant development environment, safeguarding sensitive data and adhering to industry standards.
Deploying DevPods in an on-prem Kubernetes environment offers three core advantages:
enhanced performance
seamless setup
heightened security
These benefits translate into a significant increase in productivity, especially with DevPods’ ability to persist data across sessions. The persistence feature is a game-changer for developer workflows, which ensures that their custom setups, files, and work are preserved even after their environments are restarted or redeployed.
With developer environments on the latest stable software version, administrators can patch vulnerabilities or update applications outside of working hours. The centralized nature of Kubernetes architecture also makes it easier to monitor and detect potential threats or malicious activity. Unlike laptops, Kubernetes pods aren’t constrained by battery life, allowing us to perform thorough scans and security audits during off-hours.
Enhanced performance
DevPods leverage the full capabilities of on-prem Kubernetes clusters, delivering a significant boost in performance for tasks such as Git operations, builds, and the overall integrated development environment (IDE) experience. Developers can tap into high-performance resources, including up to 48 CPU cores and 128 GB of RAM per environment and eliminate slow responses and workflow lag.
Here’s how DevPods improve developer productivity:
IDE Experience
DevPods allow seamless integration with various IDEs, enabling developers to write, test, and debug code in a standardized environment.
Developers can use their preferred IDE locally, for example, VSCode, Intellij, or Cursor, while running the actual development environment in DevPod to avoid conflicts and ensure a clean workspace.
Git Performance
Optimized Git settings improve the speed of operations.
Kubernetes’s Linux-based file system is more efficient than traditional laptop file systems, further enhancing performance.
Builds
Dev artifacts are preloaded, which reduces wait time during build processes.
Access to robust compute resources significantly accelerates the build process.
Use of nearby remote build caches minimizes re-build times.
As a result, laptops no longer freeze or overheat during intensive tasks. Developers can work across multiple environments simultaneously with DevPod’s activities isolated from other processes on their machines. This ensures smooth workflows and minimal interruptions.
Seamless setup
Setting up a development environment is tedious—downloading tools, configuring settings, and troubleshooting issues even before you write your first line of code. However, with DevPods, this hassle is eliminated. DevPods come pre-configured with all the tools, libraries, and environments your project needs, so you can skip the setup and start coding right away. Whether you're working on a monorepo or a smaller project, DevPods ensure everything is ready to go from the moment you start.
Here’s how DevPods simplify developers’ work:
Quick and easy provisioning
With just a single command or a few clicks, DevPods spin up a fully functional development environment on demand. It’s like having a personal assistant who sets up your workspace in seconds. You’ll have access to a complete, production-like environment. All the tools, libraries, and dependencies you need are pre-installed, and your repository is already cloned and ready to go. It’s like walking into a fully-stocked kitchen where all the ingredients are prepped and waiting for you to start cooking
The lifecycle of DevPods are designed to be flexible, catering to different workflows and project needs. Here’s how their lifecycle typically works:
On-Demand Provisioning: Users can spin up a DevPod whenever they need it, whether for a new feature, bug fix, or experiment. This ensures resources are used efficiently and only when required.
Long-Lived or Short-Lived: DevPods can be long-lived for ongoing projects or short-lived for specific tasks like feature development or unit testing. Once the task is complete, the DevPod can be shut down to free up resources.
Persistent Data Storage: Even if a DevPod is deleted (for example: due to lease expiry), the data is stored in a persistent volume. This means users can recreate the DevPod later and pick up right where they left off, without losing any work.
Lease-Based Management: DevPods are provisioned with a specific lease period, ensuring resources are not wasted. Users can extend the lease if needed or let it expire to automatically clean up unused environments.
Perfect for teams and solo developers
Whether you’re part of a large team or working on your own, DevPods make collaboration and productivity effortless. New team members can get started in minutes, and everyone can work in the same environment, ensuring consistency and reducing onboarding time.
In short, DevPods are like having a magic wand for your development workflow. They take the pain out of setup, ensure consistency, and let you focus on what really matters—building amazing software. Whether you’re a seasoned developer or just starting out, DevPods are here to make your life easier and your coding experience smoother.
Improved security by design
Security is baked into DevPods. With automated overnight updates, developers no longer have to worry about manually keeping their environments up-to-date with the latest security patches and tools. This seamless updating process ensures that the DevPod environment is always running with the latest and most secure versions of software.
DevPod offers the following security advantages:
Controlled toolchain: A pre-curated, hardened toolchain ensures a secure development environment with minimal risk of vulnerabilities.
Seamless updates: Tooling and configurations are updated automatically, ensuring developers are always using the latest versions.
Pre-release security scans: Each DevPod image undergoes a security scan before being promoted to stable, which reduces the risk of vulnerabilities in the system.
Persistence: Always-on workflows and seamless collaboration
One of the most valuable features of DevPods is their ability to persist and preserve settings. DevPods retain all data, configurations, and states across sessions, ensuring that developers don’t lose their work when their environments are stopped or restarted. This persistence is achieved through durable storage volumes in Kubernetes, which are mounted to each pod.
Here are the key benefits of DevPod persistence:
Persistent storage: Kubernetes-backed persistent volumes ensure that all files, configurations, and setups are retained, even when the pod or container is shut down or redeployed.
Stateful workflows: Developers can resume their work exactly where they left off, without the need to reconfigure their environments or restore files.
Collaborative development: The persistent nature of DevPods also enables collaboration between multiple engineers within a single environment and makes it easy to pick up right where you left off—on any device, making it easier to work together on complex projects.
By leveraging persistent volumes in Kubernetes, the DevPod environment ensures durability and reliability, which are essential for developers working on long-term or complex projects that span multiple sessions. This framework makes their environments faster, easier to set up, and more secure, while ensuring consistency across teams and devices. It’s a game-changer for productivity and security alike.
While DevPods bring a host of advantages to the table, like any tool, they come with their own set of challenges and considerations. It’s important to be aware of these potential hurdles to make informed decisions and ensure a smooth experience. Here are some key challenges and considerations to keep in mind:
Learning curve for new users
Developers new to DevPods or containerized development environments may find it challenging at first. Understanding how to set up, configure, and troubleshoot DevPods can take time, especially for teams transitioning from traditional local setups.
Dependency on infrastructure
DevPods often rely on cloud or remote infrastructure to function effectively. If your internet connection is unstable or your cloud provider or remote infrastructure experiences downtime, it could disrupt your workflow.
Customization limitations
While DevPods come pre-configured, they may not always align perfectly with every developer’s preferences or project requirements. Customizing the environment might require additional effort.
Resource costs
Running DevPods on cloud infrastructure can incur costs, especially for larger teams or resource-intensive projects. If not managed properly, these costs can add up quickly. At ThoughtSpot, we closely monitor and carefully manage these costs to ensure efficiency and control.
Performance overhead
Running development environments in containers or on remote servers can introduce performance overhead, especially for tasks that require heavy computational resources or low-latency interactions.
For example, Bazel builds can be quite demanding on system resources, particularly when working with large monorepos. If Bazel is configured to automatically use parallel threads for builds, and the DevPod environment has limited resources, such as CPU or memory, the builds might slow down significantly or even fail due to resource constraints.
Debugging and troubleshooting
Debugging issues in a containerized or remote environment can be more complex than debugging on a local machine. Network issues, container misconfigurations, or differences between local and remote setups can further complicate the process.
Integration with existing tools
Integrating DevPods with existing tools, workflows, or CI/CD pipelines can sometimes be tricky, especially if those tools aren’t designed to work with containerized environments.
Cultural shift
Adopting DevPods often requires a cultural shift within teams, especially if developers are accustomed to local environments. Resistance to change can slow down adoption.
DevPods are a powerful tool for modern development workflows, but they’re not without their challenges. By understanding these considerations and planning ahead, we can mitigate potential issues and make the most of what DevPods have to offer. With the right approach, the benefits of using DevPod can far outweigh the challenges, leading to a smoother and more efficient development experience.
Standardize environments: Use version-controlled configuration files, for example, Dockerfiles and scripts, to ensure consistent setups across all DevPods.
Optimize resource allocation: Allocate sufficient CPU, memory, and storage to handle resource-intensive tasks like builds or testing.
Idle sensing for better resource utilization : Implement idle detection mechanisms to identify when a DevPod is not in use. Automatically pause or scale down resources during idle periods to better utilize the resources.
Leverage caching: Use remote caches for build tools (e.g., Bazel, Gradle) to speed up builds and avoid redundant computations.
Monitor and optimize performance: Use monitoring tools to track resource usage and optimize DevPod configurations for better performance.
DevPod is a valuable asset for ThoughtSpot, empowering our developers to work more efficiently and collaboratively. By adopting remote development environments, businesses can unlock new levels of productivity and innovation. DevPods streamline workflows, eliminate setup friction, and ensure consistency across teams, allowing developers to focus on what they do best, building great software.
Whether you're tackling complex monorepos, scaling your team, or simply looking to modernize your development process, DevPods offer a powerful solution. With their ability to enhance security, improve collaboration, and provide a reproducible environment, DevPods are more than just a tool, they're a game-changer for modern development.
As the technology landscape continues to evolve, embracing tools like DevPods can help your team stay ahead of the curve. By investing in a seamless, scalable, and secure development experience, you're not just improving workflows, you're paving the way for faster innovation and long-term success.
