Many companies start with a monorepo and then move to a polyrepo setup as they grow. However, as these companies become larger and more complex, many of them transition back to a monorepo to standardize their internal processes and improve collaboration and code quality. As companies grow and evolve, so do their code management practices.

This is a story of this evolution.

Episode 1 – monorepo – the force awakens

A long time ago, in a galaxy far, far away…

A small Rebel Alliance who started a company. As they were a small team, they began with a monorepo, which made the collaboration and knowledge sharing easier. The master Jedi Martin Fowler had shared that starting with monolith architecture is the right approach for most companies. A monolith architecture also allowed them to continue collaborating into a monorepo.

Episode 2 – polyrepos – attack of the clones

As the empire grew in power and the codebase expanded, the monorepo became too large and unwieldy, making it difficult to work with and slowed down development.

The primary reason for slow development?  Growing build times. Running the entire build on every change caused build times to become the bottleneck. Furthermore, as the complexity grew, these builds became unstable, leading developers to encounter flaky tests. And the darkness grew.

Also, as the Rebel Alliance tried to maintain control, concerns about code ownership grew. With all code stored in a single repository, managing who had access to which parts of the codebase became harder, and ensuring code changes were properly reviewed and approved by the correct code owners became a challenge.

Eventually the clone wars began, emboldening the Stormtroopers to continue cloning and creating new repositories across the galaxy.

Episode 3 –  A new hope

With the First Order, developers became completely disconnected and had no understanding of what was happening in other parts of the galaxy. Coordinating their efforts and keeping track of changes proved challenging.

From a galaxy far, far away, a new Resistance was born. They knew that if they could harness the power of the Monorepo, they could achieve greater efficiency and collaboration.

The Separatists rejected the concept of a monorepo, as they had seen scaling issues with it in the past. This allowed them to operate more independently but made it difficult to coordinate their efforts, leading to inconsistencies in their codebase.

However, the Resistance knew that, despite the challenges of working with monorepos, the benefits of a monorepo setup outweighed the drawbacks. They also understood that winning the war would not be easy, as managing monorepos at scale comes with a significant overhead.

Episode 4 – Return of the Jedi

Transitioning from a polyrepo setup to a monorepo at scale was a complex and challenging process. A Jedi task force assembled to handle this mission. The Jedis came from some of the largest galaxies, like Google and Facebook, who had seen monorepos work at scale. To transition successfully, they:

• Carefully planned the process and created a migration plan that outlined the steps involved.
• Ensured that all teams were on board with the change and understood the benefits of a monorepo setup.
• Identified the most critical parts of the codebase and migrated them first.
• Addressed concerns about code ownership and access control by implementing appropriate access control mechanisms and policies.
• Used automated tools to help with the migration process, such as those for dependency management, code reviews, and testing.
• Tested the new monorepo setup thoroughly before rolling it out to the entire organization.
• Provided training and support to developers to help them adjust to the new workflow and processes.
• Monitored the performance of the new monorepo setup and made adjustments as needed.

Episode 5 – A new order

By using modern tools, the small team of Jedi was able to bring the developers back to the light side of the Force. They learnt that it’s not that monorepos don’t work at scale, it’s rather the lack of tooling that makes this hard.

Here are a few things that if done well, can bring the benefits of poly repos in a monorepo setup while providing the standardization and collaboration benefits that monorepos offer:

• Code ownership – define granular code ownership and use CODEOWNERS file properly to provide enough independence to each team while managing consistency across the org.
• Dynamic builds – At scale, you cannot afford to run full builds on every commit. Define build rules that help run a subset of changes. Most of the CI systems these days offer opportunities to create dynamic pipelines, you can also leverage Bazel like build systems to cache partial builds.

Conclusion – May the force be with you

While moving from a poly repo to a monorepo setup can be challenging and complex, it can bring significant benefits to companies looking to standardize their internal processes and improve collaboration and code quality. Companies like Google, Facebook, and Microsoft have successfully transitioned back to a monorepo setup and have reported improved development speed and better collaboration as a result.

However, transitioning back to a monorepo requires careful planning and execution. Companies need to address the challenges of dealing with legacy code, ensuring team buy-in, and managing the migration process without disrupting ongoing development work. Gradually migrating code and using automated tools can help to make the process smoother.

In summary, while monorepos and poly repos both have their advantages and disadvantages, companies should carefully evaluate their needs and consider the benefits of a monorepo setup when looking to improve collaboration, code quality, and internal processes. It’s not that monorepos are unfit at scale, it’s more likely that the tooling isn’t in place to support it.

May the 4th be with you!

Aviator: Automate your cumbersome merge processes

Aviator automates tedious developer workflows by managing git Pull Requests (PRs) and continuous integration test (CI) runs to help your team avoid broken builds, streamline cumbersome merge processes, manage cross-PR dependencies, and handle flaky tests while maintaining their security compliance.

There are 4 key components to Aviator:

1. MergeQueue – an automated queue that manages the merging workflow for your GitHub repository to help protect important branches from broken builds. The Aviator bot uses GitHub Labels to identify Pull Requests (PRs) that are ready to be merged, validates CI checks, processes semantic conflicts, and merges the PRs automatically.
2. ChangeSets – workflows to synchronize validating and merging multiple PRs within the same repository or multiple repositories. Useful when your team often sees groups of related PRs that need to be merged together, or otherwise treated as a single broader unit of change.
3. FlakyBot – a tool to automatically detect, take action on, and process results from flaky tests in your CI infrastructure.
4. Stacked PRs CLI – a command line tool that helps developers manage cross-PR dependencies. This tool also automates syncing and merging of stacked PRs. Useful when your team wants to promote a culture of smaller, incremental PRs instead of large changes, or when your workflows involve keeping multiple, dependent PRs in sync.

Try it for free.