In large organizations that follow microservices architecture, onboarding new software engineers is cumbersome. New developers often soon realize that the codebase has multiple components and uses many third-party services.

• They struggle to find documentation for these components and services since they were integrated by different developers who had their own opinions about documentation.
• They also struggle to find credentials for some services as they were never documented.
• They might spend days developing fixes for bugs assigned to them and deploying them to production—only to realize that their changes broke some third-party integration they didn’t know about because its usage wasn’t documented.

With new features and improvements, products become more complicated, and the effort required to maintain information about all the components grows exponentially. Not enforcing any standardized development practice can also cause different teams to settle on different practices, leading to inefficiency. It’s quite easy for situations to grow out of control, which can slow down development.

Internal developer platforms (IDPs) like Backstage are designed to solve this problem. IDPs let you consolidate information about all the organization’s components and services in one place—dependencies, credentials, documentation, developer contact, version control history, and more. IDPs also allow administrators to enforce standards, such as code styles, documentation, and best practices, for all projects to keep project quality in check and reduce the cognitive load of developers and administrators.

In this article, you will learn more about IDPs and their benefits. You’ll also get an overview of what Backstage, one of the most popular IDPs, offers.

What Is an Internal Developer Platform and Why Use It

An IDP is like a typical web portal, but instead of being customer-facing, it’s targeted toward developers and internal to an organization. IDPs provide developers with tools, services, and resources to streamline the software development lifecycle and improve developer productivity, collaboration, and the overall efficiency of development operations:

IDPs are mainly used to enforce the golden path and enable developer self-service. How an IDP can be used depends on the exact features provided by the IDP, but most IDPs let you realize the following benefits:

Standardizing Tooling

IDPs can provide a standardized set of tools, frameworks, libraries, and development environments across teams and projects to ensure consistency in development practices and reduce the time developers spend setting up their development environments.

For example, in Backstage, if you want all your projects to include documentation in the docs directory, you can set up Software Templates so that any time developers use the template to create their project, it automatically includes the docs directory.

Consolidating Developer Resources

If developers do not have access to a centralized repository of the different components and services used by the organization, it hinders the development process. Without the big picture of all the components of a project, developers must spend time looking for documentation and information, and the risk of accidentally breaking code also increases. Onboarding new hires also becomes challenging as they struggle to get familiar with the tech stack.

The cataloging capabilities of an IDP give developers a big-picture view of the different parts of their codebase—all the apps, libraries, third-party services, APIs, credentials, and more. IDPs can also catalog critical metadata about components, such as an API’s version number or a third-party service’s website URL.

This catalog of components also helps developers collaborate better because they can see who owns the different components, and it helps new hires be onboarded faster. At Spotify, Backstage reduced the time it takes for a newcomer to merge their tenth pull request from sixty days to less than twenty.

By integrating issue trackers and pull requests into the catalog, the IDP can also give developers a better idea of the health of the project.

Incorporating Developer Productivity Tools

Many IDPs offer productivity-enhancing tools, such as code generators, templates, code snippets, and integrations with third-party services. By enforcing best practices and automating repetitive tasks, these tools reduce the cognitive load on developers and allow developers to focus on writing high-quality code. According to the Forrester Opportunity Snapshot, 74 percent of DevOps leaders agree that an IDP boosts developer productivity.

Enforcing Security and Compliance

You can use IDPs to enforce security best practices and compliance requirements by integrating centralized security scanning tools, access controls, and compliance checks into the development workflow.

Without an IDP, each project may have its own set of security rules, and it’s easy to miss a critical security issue. With an IDP, the security measures are consolidated in a central place, providing security checks for all the projects in the organization. This helps mitigate security risks and ensures that software meets regulatory standards.

Reducing Costs

Companies can reduce overhead expenses by unifying all their various tools and resources into a single IDP. An IDP allows you to optimize resource allocation and identify and reduce cost-bearing inefficiencies, which cuts down on expenses. Because IDPs streamline and automate tasks, they also help reduce manual labor costs.

Challenges of an IDP

Although IDPs are versatile tools that can increase developer productivity and code quality, they also come with some challenges.

Integrating with Existing Systems

Integrating an IDP with existing systems, tools, and workflows can be challenging, particularly in large organizations with diverse technology stacks. Backstage aims to simplify the process of integrating with existing software by including a catalog-info.yaml file in the project’s codebase. However, it’s still a considerable effort for companies with a large codebase as you must add the catalog-info.yaml file to every component in use.

Security and Compliance

Securely implementing the IDP can also be a challenge. Given the access an IDP has to sensitive data, code, and infrastructure resources, you need to ensure you’re maintaining robust security controls, enforcing access policies, and monitoring for security threats.

Adoption

Although IDPs make the lives of developers easier by reducing the effort required to maintain the codebase, developers can still resist adoption because of resistance to change, reliance on existing tools and processes, and skepticism about the benefits of the platform. Developers may also find it challenging to navigate the platform or understand how to use the available tools and services.

This means your IDP should be designed with a balance between standardization and flexibility by keeping the developer experience in mind. Organizations also need to invest in communicating the value proposition of the platform effectively and training developers to use it.

Time and Money investment

An IDP comes with all the costs of another piece of software. Implementing an IDP, customizing it according to the organization’s need, and keeping it running requires a significant investment in infrastructure resources and ongoing maintenance and support.

The IDP itself may also come with a price tag. And even self-hosted IDPs like Backstage incur server and maintenance costs, especially if they’re not optimized correctly.

Keeping the IDP Updated

Just like any other piece of software, the IDP needs to be periodically updated to fix any security issues and introduce new features. For a managed IDP, the updates are usually performed automatically without minimal intervention. However, the vendor decides when you get the updates. On the other hand, a self-hosted IDP gives you the ability to update it whenever you want, but you’ll need to invest time and money and will require experts to minimize any potential issues.

It’s also a hassle for engineering teams to keep the IDP data up-to-date. As teams introduce new components or modify existing ones, they must remember to also update the data in the IDP accordingly. Otherwise, there’s a risk of mismatched data, thus impacting the advantages of an IDP.

When Should You Get an IDP

Even though an IDP offers many benefits and solves important pain points, the challenges associated with successfully implementing one means you must carefully consider whether you need one.

If your organization is small, with only a few components, it likely doesn’t make sense to invest in an IDP. You can avoid the need for the resource consolidation capabilities of an IDP by maintaining well-structured documentation about applications and code practices.

However, if you’re part of a growing organization with a rising number of components being used, investing in an IDP now can save you from the costs of a disconnected development team in the long run. If your organization is already large enough that teams are feeling this disconnect, you should implement an IDP to prevent further decline.

Another consideration is developer productivity. If a lack of self-service abilities is causing your developers to spend precious time waiting for ops to set up their environments, you should consider implementing an IDP, regardless of the size of your organization.

Getting Started with Backstage

To give you a better idea of what it’s like getting started with an IDP, let’s take the example of Backstage, an open source IDP developed by Spotify.

Spotify developed Backstage to solve the issue of managing Spotify’s growing architecture. Spotify donated Backstage to the Cloud Native Computing Foundation (CNCF) in 2020, where Backstage developed into a community-driven effort aimed at simplifying development.

Backstage comes with many different features that make it one of the most highly sought-after IDPs:

• Its powerful Software Catalog offers a centralized repository to consolidate and track ownership, interdependencies, and metadata for all your components (apps, services, websites, libraries, etc).
• TechDocs allows you to create a central documentation hub inside the IDP. Backstage can generate the central documentation hub automatically from the docs included with the code.
• Templates lets you create components from Backstage using a predefined template. It ensures best practices and style guides are followed for maximum code quality. It also saves time since developers don’t have to create projects from scratch.

Backstage also provides integrations with many third-party services. You can customize it using existing plugins, or you can create your own, and it’s free and open source. When you create a Backstage app, you get complete access to the Backstage codebase, which you can modify however you want, and you can host it on your own server.

Exploring Backstage’s vast amount of features is out of scope for this article, so this article focuses only on the Software Catalog in more detail.

Conclusion

An IDP like Backstage helps boost developer productivity by consolidating development resources, such as components and documentation; enforcing consistent standards; and reducing cognitive load and manual effort.

IDPs can be challenging to integrate and adopt, but their benefits make them a worthwhile investment, especially for growing organizations and large enterprises.