Code reviews and code ownership are team based concepts. Building an efficient code review culture requires having a strong partnership between the authors and the reviewers. Additionally, to truly improve the code review cycle times, it’s even more important for the individual responses to come quickly than it is for the whole process to happen rapidly.

FlexReview Teams is built with those principles in mind. It provides a customizable framework for managing code review rules, assignments, and notifications, allowing teams to establish their own policies and objectives while facilitating inter-team best practices.

Teams

At the core of the system is a Team representation. Each FlexReview Team is 1:1 mapped with a unique GitHub Team, and it inherits the configuration from its parent team as specified in GitHub. That means, in a multi-level hierarchy of various engineering teams set up in GitHub, the configurations can be defined at any level, and be trickled down to all children teams.

Code reviews SLO

Service level objective (SLO) for code review defines the suggested time it should take the reviewer to respond to a code change. This is NOT the time it takes for a PR to be approved, but rather just getting a response. This helps the team focus on individual response times instead of focusing on the complete cycle times.

Think of the code review SLO as an agreement between the author and the reviewer for a pull request. With this agreement, the author is incentivized to create smaller PRs that fall within the bounds of the SLO agreement, and reviewers are incentivized to stay on top of the PRs.

Configuration

There are 3 core configurations for each FlexReview Team:

1. Reviewer assignment

FlexReview Teams support 3 different configurations for reviewer assignment:

• Expert reviewer: Aviator uses heuristics from the past code reviews to calculate the expert reviewer for each file in each change. Based on the expertise score and the current review load, it chooses the “best fit” reviewer from the team.
• Oncall reviewer: A team can manage oncall schedule using an oncall software, e.g. Pagerduty to automatically assign reviewers based on the rotation priority.
• Load balanced: Reviews can also be evenly distributed in the team purely based on the current review load.

2. Review SLO

Managing the review SLO configuration is critical for a healthy code review experience. The configuration is based on:

• Internal response time: An internal team review is the code review where both the author is part of the same team that is reviewing the code
• External response time: The ones where the author is not part of the
• Max number of modified lines: These are the total modified lines in a pull request that qualifies it to be within the expected SLO range. If a pull request is larger than the specified lines, it won’t be tracked on the team’s SLO dashboard.

The SLO is calculated for every author-reviewer combination. That means the same pull request may have both an internal and an external code review SLO if there are two separate reviewers assigned.

3. Automated rules

Set up automated rules for stale PRs that are pending action from the reviewer(s). Notify the reviewer on Slack or automatically reassign the review to someone else on the team.

Review reassignment

Based on the automated rules, a review may be automatically reassigned to another team member. This ensures that PRs do not get stalled due to unaddressed reviews, keeping the review process efficient and timely.

Review reassignment follows the same strategy as specified in the reviewer assignment configuration. This automation tries to identify the next best person who can review the provided code change. If no other valid reviewer exists, Aviator would not reassign the reviewer, and report the error as a GitHub comment.

• Expert reviewer – the review would be reassigned to another expert likely with a lower score or lower review load.
• Oncall reviewer – the review would be reassigned to the next person defined in the escalation policy.
• Load balanced: the review would be reassigned to another person with the least amount of load

Team’s SLO dashboard

Once the Review SLO config is specified, each team can monitor the response times for their incoming code reviews. This dashboard provides real-time insights into code review response times, highlighting areas for improvement and ensuring compliance with set objectives. It splits the review responses for internal and external team reviews and also excludes any reviews that fall outside of the SLO max lines requirement.

Demo

To learn more and try out FlexReview Teams, visit: aviator.co/flexreview