Exhaustive by default
Every version of every artifact you add, and recursively every version of everything it depends on. Filters narrow down what you ask for, never what it needs.
Open source · MIT licensed
Backpack collects every version of the packages you choose, and everything they depend on, into S3-compatible storage. Then it tells you exactly what's new, so each transfer carries only what changed.
Features
Backpack runs on the connected side of your network. It doesn't serve packages; it makes sure that everything your builds will ask for is already collected.
Every version of every artifact you add, and recursively every version of everything it depends on. Filters narrow down what you ask for, never what it needs.
Every newly collected file is recorded in a daily delta folder, so an incremental transfer copies exactly what was added since the last one.
Files are stored under the host and path they came from, grouped by ecosystem:
npm/.
Processors understand ecosystems, collectors speak protocols, and RabbitMQ connects them. Deploy only what you use, scale what's busy, add your own in any language.
Sign in with your own OIDC provider. Require an administrator's approval per ecosystem before anything is collected, and give automation API keys.
Kubernetes with the Helm chart or Docker Compose, MinIO, SeaweedFS, Ceph or AWS S3, and OpenTelemetry traces and metrics.
How it works
Request express, Newtonsoft.Json or
docker.io/library/nginx in the web interface or through the REST API.
The ecosystem's processor lists every version, its files and its direct dependencies. The gateway follows each dependency in turn until the tree is complete.
Collectors download what isn't stored yet, write it to your bucket and record it in the day's delta. Scheduled re-tracking picks up new releases.
Copy the delta folders since your last transfer to the isolated network, and load them into the registries and proxies you already run there.
npm/registry.npmjs.org/express/express-4.21.2.tgz.Ecosystems
Each ecosystem is a small service you deploy when you need it.
Deploy
Backpack needs RabbitMQ, MongoDB, Redis, S3-compatible storage and an OIDC provider.
Install with the Helm chart and point it at your RabbitMQ, MongoDB, Redis, storage and OIDC provider. Add the processors and collectors you need to its values.
Every value is documented in the chart README.
helm repo add backpack https://linus-berg.github.io/helm-charts/
helm show values backpack/backpack > my-values.yaml
# set oidc, rabbitmq, mongo, redis and s3
helm install backpack backpack/backpack \
-n backpack --create-namespace -f my-values.yaml
Try a complete single-machine stack, infrastructure and sign-in included, with npm ready to go.
See the example README for signing in and adding ecosystems.
git clone https://github.com/linus-berg/backpack.git
cd backpack/example
# first add "127.0.0.1 keycloak" to /etc/hosts
docker compose up -d
# then open http://localhost:3000
Documentation
Components, queues and the life of an artifact.
Infrastructure, images, sign-in and security notes.
Every environment variable, processors and schedules.
How files are laid out in the bucket, and delta links.
Add an ecosystem with the .NET template or any language.
Follow an artifact through the system and fix common problems.