Want to have an npm proxy between you and the public registry? Want to make sure all teams are using on the same package versions? Want to delegate the responsibility of making releases?
Using a Bytesafe private registry with Upstreams make this possible!
This post aims to guide you through the concept of upstreams. Read on for insight into:
- Push & pull of package versions when using Bytesafe with upstreams
- Advantages of using and configuring upstreams in your organization
- Example upstream configurations to fit your organization’s workflow
What are upstreams?
Upstreams are linked registries that are used both as a destination and source for packages in your private registries.
Upstreams can be:
- Public npm registry (registry.npmjs.org)
- Other Bytesafe registry
- URL to any npm compatible registry
Package versions can be downloaded (“pulled”) from an upstream to a registry (either manually or when running
Packages can also be published (“pushed”) from a Bytesafe registry up to an upstream.
Configuring upstreams with private registries
In the simplest use-case, Bytesafe offers private registries to manage packages for internal use within an organization.
If a registry is configured without any upstreams:
- Developers can publish package versions to the private registry
- Package versions can NOT be published from the private registry to public npm or any other registry
- Installing packages from the registry requires the package version to be available, else the install fails
This is the default scenario when a new empty Bytesafe registry is created by the user.
For some teams this case may be the preferred option, as they use the registry only for internal packages. But most teams also depend on package versions available in the public registry.
Let’s look at what adding the public npm registry as a upstream looks like.
Add npmjs as upstream to the registry
Adding the public npm registry as an upstream adds new options and features for a team’s Bytesafe registry. The Bytesafe registry is still private and only the team members can access its contents.
Package versions pulled by a developer will also be downloaded and stored in the Bytesafe registry, in essence creating a npm proxy (with the security benefits that entails).
If the public npm upstream is configured with a valid npm token, package versions can also be pushed from the bytesafe registry to the public registry.
- Mix of private and public package versions
- Registry works as npm proxy, adding a firewall to public registry
- Centralized hub to manage public npm tokens and curating the package versions available for your organization
Pushing and pulling packages
Bytesafe supports two different ways of interacting with a registry, using the regular npm cli or using the bytesafe cli / web console.
The main difference between using npm and bytesafe commands lies in the scenario where you use them, where
npm is typically used by developers interacting with the registry from a project workspace and
bytesafe is used for moving versions between registries.
npm publish &
Both commands push package versions to a target (upstream):
- npm publish - publishes from a project workspace to a registry.
- bytesafe push pushes between registries and upstreams (public npm or other registry).
npm install &
Both commands retrieve packages from a source (upstream):
- npm install - pulls a package (and dependencies) from a upstream (and installs it in your project workspace).
- bytesafe pull - pulls a package (without dependencies) from a upstream into a registry.
npmcommands for pushing or pulling directly from a developer’s project workspace.
bytesafecommands when managing package versions in registries and upstreams
Bytesafe is designed to let developers preserve the regular npm workflow.
Installing packages from a registry with upstreams
Installing a package with
npm install is a three-step process. First a list of all available versions for the package is downloaded,
then a specific version is chosen from the list, and last the actual contents are downloaded.
When a bytesafe registry have one or more upstreams,
npm will receive all available versions of the package, including the versions available in the upstream registries.
When the contents of a specific version is then downloaded, bytesafe will pull the contents from an upstream if needed.
The upstreams are searched in order, which can be changed in the registry settings in the console.
Different upstream configurations for different workflows
Bytesafe allows a lot flexibility to organize registries and upstreams to fit your specific workflows.
Use the following example configurations as an inspiration for whats possible.
Use Case: Chaining registries
For certain organizations it may not be ideal to have a setup where a developer-facing Bytesafe registry is directly connected to the public registry.
Teams might use a dedicated registry but still want to share private package versions, or maybe a new registry is setup for each sprint or release cycle, or a central registry is needed to control the flow of package versions from the outside.
In this case adding one or more layers of registries between developer teams and the public registry may be beneficial.
Each private registry is only configured with the layer above it as an upstream.
This way of structuring the flow of packages between teams and the outside is especially powerful in combination with policies and plugins.
- Added security, inserting a central point to control the flow of package versions (and enforce policies)
- Ensure the whole organization is working with the same package versions (private and public)
- Centralize releases to the public registry
Remember, pulling packages is recursive
So what happens when a developer want to pull new package versions?
As pulls using Bytesafe are recursive, the pull request action will be passed from registry to registry until the package is found (or it fails).
So although the developers registry is only configured with another private registry as a upstream, they are able to continue working using third party packages.
If the same developer want to push a new package version it only has to push it to the layer above, where it can be distributed to other organization members. No need to keep track of credentials for the public registry or who has the right to publish a new package version. That responsibility is handled by the layer above.
Use Case: Registry with multiple upstreams
What if your team or organization want to use packages, both third party and internal, from different sources without setting up a chain of registries?
Bytesafe supports registries configured with multiple upstreams where the priority of the upstreams decides the order in which they are queried for package versions.
If a team uses a Bytesafe registry that has two upstreams configured with the following priority:
- Priority 1: partner registry (e.g. other team, outsourced dev team etc.)
- Priority 2: public npm registry
When a developer installs a package from the private team registry:
- If the private registry contains the package it will be returned immediately
- If not the package will be pulled from the upstream with the highest priority (i.e the partner registry)
This is can be very useful if the partner registry contains a patched version of a public package for example!
Continued reading - Plugins and policies
The concept of upstreams introduce potential for controlling the flow of packages between registries that your organization control and the public npm registry. By properly understanding upstreams you can already setup a flow where only a whitelisted packages are available for downstream registries or control who is allowed to publish to public npm.
Continue reading our series on Using Bytesafe, to see how Plugins & Policies can secure your code supply chain!
- simplify workflow for users, freeing up time for other tasks
- ensure better control over the flow of packages and your organizations dependencies