Why turborepo is fast? | Turborepo

Ever stare at your terminal, yearning for that build progress bar to budge a little faster? We've all been trapped in the molasses of slow builds, our development rhythm reduced to a frustrating crawl. Enter Turborepo, the revolutionary build system redefining how we develop. Forget plodding through sequential tasks and redundant work. Turborepo embraces a smarter, faster approach, leaving traditional build tools in the dust.

How is turborepo fast?

• Caching: Meticulously tracks tasks and outputs, skipping unchanged code for near-instant incremental builds.

• Parallelism: Analyzes dependencies for conflict-free parallel execution, maximizing CPU power for dramatically reduced build times.

• Smart Scheduling: Prioritizes critical tasks for faster feedback loops and uninterrupted development flow.

• Remote Caching: Shares build-cache across devices and teams, enabling instant builds for those who've already benefited from previous runs.

• Pruned Subsets: Generates deployment packages containing only necessary code, streamlining the process and reducing resource usage.

Caching: How does it work?

When we run a task with Turborepo, such as turbo run build, the following process occurs:

1. Calculating the Task Hash: Turborepo evaluates the inputs of your task and generates a unique hash based on these inputs.

2. Checking for Cached Results: Turborepo checks its local filesystem cache or remote build caches like at the vercel server, if configured. for a matching cache artifact based on the calculated hash.

3. Executing the Task or Using the Cache: If Turborepo doesn't find a matching artifact in the cache, it executes the task as usual. After successful completion, Turborepo saves all specified outputs, including files and logs, into a new cache artifact addressed by the hash in ./node_modules/.cache/turbo by default.

4. Reusing Cached Results: If you rerun the task without changing its inputs, Turborepo recognizes the unchanged hash and retrieves the cached artifact. Instead of rerunning the task, Turborepo replays the output, restoring files and logs from the cache almost instantly.

   

Decide what to cache

Turborepo defaults to caching tasks unless explicitly specified otherwise. For example, the following configuration, and Turborepo defaults to caching the "lint" task are the same:

// turbo.json
{
  "$schema": "https://turbo.build/schema.json",
  "pipeline": {
    "lint": {
      "cache": true
    }
  }
}

It is preferred to include, linting, building, and testing as cached tasks. The following can be some of the reasons why you may opt not to cache the tasks:

• Tasks with their caching mechanisms, like Docker's layer Cache, could lead to conflicts or complexity when combined with Turborepo's caching.

• Tasks that finish in less time than a network round trip might not gain significant benefits from caching, especially when using remote caching.

• Tasks that generate large output assets, such as Docker containers, could be better off without caching due to the overhead of caching and transferring these assets.

• Simple file operations that execute faster than caching operations may not see noticeable improvements from caching.

To exclude any task from being included in the cache just add cache: false to that task in turbo.json for example:

{
  "$schema": "https://turbo.build/schema.json",
  "pipeline": {
    "test": {
      "cache": false
    }
  }
}

Parallelism and Smart Scheduling

It analyzes dependencies between tasks and identifies those that can run simultaneously. Think of it as having multiple build crews working on different parts of the project at the same time. This parallel execution leverages the power of your multi-core CPU, dramatically reducing build times.

Not all tasks are created equal. Some, like compiling critical code, are more important than others. Turborepo understands this and prioritizes tasks accordingly. It puts the urgent jobs at the front of the queue, ensuring you get the feedback you need quickly, and keeping you in the development flow. Imagine it as a VIP lane on the expressway, where critical tasks zoom past less important ones. Below is the illustration of how turbo repo handles non-dependent tasks in parallel:

The combination of parallelism and smart scheduling is a game-changer. It's like having a highly efficient pit crew working on your car – tasks are executed concurrently, and the most important ones are tackled first. This translates to significantly faster builds, smoother development workflows, and happier developers.

Remote Caching

Turborepo's remote caching shatters build silos and unlock a new level of teamwork. Here's how it transforms your development experience:

1. Shared Cache, Shared Speed:

   Store build artifacts in a central location, for example, vercel servers, accessible to everyone on the team and eliminate redundant builds – if someone's already built it, you can grab it from the cache.

1. CI/CD Integration, Deployment Delight:

   Integrate remote caching with your CI/CD pipelines for blazing-fast deployments. Optimize build times on servers, leading to smoother releases and happier users.

We will delve into this sole topic alone in upcoming blogs in this series.

Next Steps

We've peeled back the hood on Turborepo's speed secrets, leaving you with one burning question: how do I get my monorepo humming at hyper-speed? As the next step, we will see how to set up CI/CD workflows in turborepo efficiently.

Dive into the detailed documentation and start experimenting with parallelism, smart scheduling, and the game-changing remote cache. Prioritize critical tasks, unleash parallel builds, and share your pre-built components with the team. Then, spread the turbo love – evangelize efficiency and watch your team's workflows transform. Turborepo isn't just a tool; it's a philosophy of speed, collaboration, and developer joy. Open your terminal, embrace the future of monorepo development, and start turbocharging your workflow today!