abx-pkg

`abx-pkg` 📦 `apt` `brew` `pip` `npm` `cargo` `gem` `go_get` `nix` `docker` ₊₊₊
_{Simple Python interfaces for package managers + installed binaries.}

It’s an ORM for your package managers, providing a nice python types for packages + installers.

This is a Python library for installing & managing packages locally with a variety of package managers.
It’s designed for when requirements.txt isn’t enough, and you have to detect or install dependencies at runtime. It’s great for installing and managing MCP servers and their dependencies at runtime.

pip install abx-pkg

python
>>> from abx_pkg import Binary, apt, pip, npm

>>> curl = Binary('curl').load()
>>> print(curl.abspath, curl.version, curl.exec(cmd=['--version']))
/usr/bin/curl 7.81.0 curl 7.81.0 (x86_64-apple-darwin23.0) libcurl/7.81.0 ...

>>> npm.install('puppeteer')

📦 Provides consistent interfaces for runtime dependency resolution & installation across multiple package managers & OSs ✨ Built with pydantic v2 for strong static typing guarantees and easy conversion to/from json 🌈 Usable with django >= 4.0, django-ninja, and OpenAPI + django-jsonform to build UIs & APIs 🦄 Driver layer can be pyinfra / ansible / or built-in abx-pkg engine

_{Built by ArchiveBox to install & auto-update our extractor dependencies at runtime (chrome, wget, curl, etc.) on macOS/Linux/Docker.}

Source Code: https://github.com/ArchiveBox/abx-pkg/

Documentation: https://github.com/ArchiveBox/abx-pkg/blob/main/README.md

from abx_pkg import Binary, apt, brew, pip, npm, env

# Provider singletons are available as simple imports — no manual instantiation needed
dependencies = [
    Binary(name='curl',       binproviders=[env, apt, brew]),
    Binary(name='wget',       binproviders=[env, apt, brew]),
    Binary(name='yt-dlp',     binproviders=[env, pip, apt, brew]),
    Binary(name='playwright', binproviders=[env, pip, npm]),
    Binary(name='puppeteer',  binproviders=[env, npm]),
]
for binary in dependencies:
    binary = binary.load_or_install()

    print(binary.abspath, binary.version, binary.binprovider, binary.is_valid, binary.sha256)
    # Path('/usr/bin/curl') SemVer('7.81.0') AptProvider() True abc134...

    binary.exec(cmd=['--version'])   # curl 7.81.0 (x86_64-apple-darwin23.0) libcurl/7.81.0 ...

Binary.min_version is optional. Leave it as None when any discovered version is acceptable, or set it to a SemVer/string to enforce a minimum version after load/install.

from abx_pkg import Binary, apt, pip

# Use providers directly for package manager operations
apt.install('wget')
print(apt.PATH, apt.get_abspaths('wget'), apt.get_version('wget'))

# our Binary API provides a nice type-checkable, validated, serializable handle
ffmpeg = Binary(name='ffmpeg').load()
print(ffmpeg)                       # name=ffmpeg abspath=/usr/bin/ffmpeg version=3.3.0 is_valid=True binprovider=apt ...
print(ffmpeg.abspaths)              # show all the ffmpeg binaries found in $PATH (in case there's more than one available)
print(ffmpeg.model_dump())          # ... everything can also be dumped/loaded as json-ready dict
print(ffmpeg.model_json_schema())   # ... OpenAPI-ready JSON schema showing all available fields

from pydantic import InstanceOf
from abx_pkg import Binary, BinProvider, BrewProvider, EnvProvider

# You can also instantiate provider classes manually for custom configuration,
# or define binaries as classes for type checking
class CurlBinary(Binary):
    name: str = 'curl'
    binproviders: list[InstanceOf[BinProvider]] = [BrewProvider(), EnvProvider()]

curl = CurlBinary().install()
assert isinstance(curl, CurlBinary)                                 # CurlBinary is a unique type you can use in annotations now
print(curl.abspath, curl.version, curl.binprovider, curl.is_valid)  # Path('/opt/homebrew/bin/curl') SemVer('8.4.0') BrewProvider() True
curl.exec(cmd=['--version'])                                        # curl 8.4.0 (x86_64-apple-darwin23.0) libcurl/8.4.0 ...

Supported Package Managers

So far it supports installing/finding installed/updating/removing packages on Linux/macOS with:

apt (Ubuntu/Debian/etc.)
brew (macOS/Linux)
pip (Linux/macOS/Windows)
npm (Linux/macOS/Windows)
cargo (Linux/macOS)
gem (Linux/macOS)
go get / go install (GoGetProvider) (Linux/macOS)
nix (Linux/macOS)
docker (Linux/macOS, using local wrapper scripts that run docker run)
env (looks for existing version of binary in user’s $PATH at runtime)
vendor (you can bundle vendored copies of packages you depend on within your source)

Planned: apk, pkg, and more using ansible/pyinfra…

DockerProvider expects image refs as install args, typically via overrides on a Binary. It writes a local wrapper script for the binary and executes it via docker run ...; the binary version is parsed from the image tag, so semver-like tags work best.

NpmProvider prefers a real npm executable when both npm and pnpm are installed. If npm is unavailable, it can still drive installs and metadata lookups through pnpm using the same provider API.

Usage

pip install abx-pkg

Lazy Provider Singletons

All built-in providers are available as lazy singletons — just import them by name:

from abx_pkg import apt, brew, pip, npm, env

apt.install('curl')
env.load('wget')

These are instantiated on first access and cached for reuse. If you need custom configuration, you can still instantiate provider classes directly:

from abx_pkg import PipProvider

custom_pip = PipProvider(abspath_handler={...})

Version Floors

Binary.min_version is enforced after a provider resolves or installs a binary. Provider discovery can still succeed, but the final Binary will be rejected if the loaded version is below your required floor.

from abx_pkg import Binary, SemVer, env, brew

curl = Binary(
    name="curl",
    min_version=SemVer("8.0.0"),
    binproviders=[env, brew],
).load_or_install()

Use min_version=None to explicitly disable version floor checks.

`BinProvider`

Implementations: EnvProvider, AptProvider, BrewProvider, PipProvider, NpmProvider

This type represents a “provider of binaries”, e.g. a package manager like apt/pip/npm, or env (which finds binaries in your $PATH).

BinProviders implement the following interface:

.INSTALLER_BIN -> /opt/homebrew/bin/brew provider’s pkg manager location
.PATH -> PATHStr('/opt/homebrew/bin:/usr/local/bin:...') where provider stores bins
get_install_args(bin_name: str) -> InstallArgs(['curl', 'libcurl4', '...]) find installer args for a bin
install(bin_name: str) install a bin using binprovider to install needed packages
update(bin_name: str) update a bin using the binprovider’s package manager
uninstall(bin_name: str) remove a bin using the binprovider’s package manager
load(bin_name: str) find an existing installed binary
load_or_install(bin_name: str) -> Binary find existing / install if needed
get_version(bin_name: str) -> SemVer('1.0.0') get currently installed version
get_abspath(bin_name: str) -> Path('/absolute/path/to/bin') get installed bin abspath
get_abspaths(bin_name: str) -> [Path('/opt/homebrew/bin/curl'), Path('/other/paths/to/curl'), ...] get all matching bins found
get_sha256(bin_name: str) -> str get sha256 hash hexdigest of the binary

install() and update() return a loaded binary with fresh abspath / version / sha256 metadata. uninstall() removes the package and returns True when the provider action succeeded.

from abx_pkg import env, apt, pip

### Example: Finding an existing install of bash using the system $PATH environment
bash = env.load(bin_name='bash')      # Binary('bash', provider=env)
print(bash.abspath)                   # Path('/opt/homebrew/bin/bash')
print(bash.version)                   # SemVer('5.2.26')
bash.exec(['-c', 'echo hi'])          # hi

### Example: Installing curl using the apt package manager
curl = apt.install(bin_name='curl')   # Binary('curl', provider=apt)
print(curl.abspath)                   # Path('/usr/bin/curl')
print(curl.version)                   # SemVer('8.4.0')
print(curl.sha256)                    # 9fd780521c97365f94c90724d80a889097ae1eeb2ffce67b87869cb7e79688ec
curl.exec(['--version'])              # curl 7.81.0 (x86_64-pc-linux-gnu) libcurl/7.81.0 ...

# Example: Updating and removing curl with the same provider
curl = apt.update(bin_name='curl')
assert curl.is_valid
assert apt.uninstall(bin_name='curl') is True

### Example: Finding/Installing django with pip (w/ customized binpath resolution behavior)
from abx_pkg import PipProvider
custom_pip = PipProvider(
    abspath_handler={'*': lambda self, bin_name, **context: inspect.getfile(bin_name)},  # use python inspect to get path instead of os.which
)
django_bin = custom_pip.load_or_install('django') # Binary('django', provider=custom_pip)
print(django_bin.abspath)             # Path('/usr/lib/python3.10/site-packages/django/__init__.py')
print(django_bin.version)             # SemVer('5.0.2')

`Binary`

This type represents a single binary dependency aka a package (e.g. wget, curl, ffmpeg, etc.).
It can define one or more BinProviders that it supports, along with overrides to customize the behavior for each.

Binarys implement the following interface:

load(), install(), update(), uninstall(), load_or_install() -> Binary
binprovider: InstanceOf[BinProvider]
abspath: Path
abspaths: list[Path]
version: SemVer
sha256: str

Binary.install() and Binary.update() return a fresh loaded Binary. Binary.uninstall() returns a Binary with binprovider, abspath, version, and sha256 cleared after removal. Binary.load(), Binary.install(), Binary.load_or_install(), and Binary.update() all enforce min_version consistently.

from abx_pkg import BinProvider, Binary, BinProviderName, BinName, ProviderLookupDict, SemVer, BrewProvider
from abx_pkg import env, pip, apt

class CustomBrewProvider(BrewProvider):
    name: str = 'custom_brew'

    def get_macos_packages(self, bin_name: str, **context) -> list[str]:
        extra_packages_lookup_table = json.load(Path('macos_packages.json'))
        return extra_packages_lookup_table.get(platform.machine(), [bin_name])


### Example: Create a reusable class defining a binary and its providers
class YtdlpBinary(Binary):
    name: BinName = 'ytdlp'
    description: str = 'YT-DLP (Replacement for YouTube-DL) Media Downloader'

    binproviders_supported: list[BinProvider] = [env, pip, apt, CustomBrewProvider()]
    
    # customize installed package names for specific package managers
    overrides: dict[BinProviderName, ProviderLookupDict] = {
        'pip': {'install_args': ['yt-dlp[default,curl-cffi]']}, # can use literal values (install_args -> list[str], version -> SemVer, abspath -> Path, install -> str log)
        'apt': {'install_args': lambda: ['yt-dlp', 'ffmpeg']},  # also accepts any pure Callable that returns a list of packages
        'brew': {'install_args': 'self.get_macos_packages'},    # also accepts string reference to function on self (where self is the BinProvider)
    }


ytdlp = YtdlpBinary().load_or_install()
print(ytdlp.binprovider)                  # BrewProvider(...)
print(ytdlp.abspath)                      # Path('/opt/homebrew/bin/yt-dlp')
print(ytdlp.abspaths)                     # [Path('/opt/homebrew/bin/yt-dlp'), Path('/usr/local/bin/yt-dlp')]
print(ytdlp.version)                      # SemVer('2024.4.9')
print(ytdlp.sha256)                       # 46c3518cfa788090c42e379971485f56d007a6ce366dafb0556134ca724d6a36
print(ytdlp.is_valid)                     # True

# Lifecycle actions preserve the Binary type and refresh/clear loaded metadata as needed
ytdlp = ytdlp.update()
assert ytdlp.is_valid
ytdlp = ytdlp.uninstall()
assert ytdlp.abspath is None and ytdlp.version is None

from abx_pkg import BinProvider, Binary, BinProviderName, BinName, ProviderLookupDict, SemVer
from abx_pkg import env, apt

#### Example: Create a binary that uses Podman if available, or Docker otherwise
class DockerBinary(Binary):
    name: BinName = 'docker'

    binproviders_supported: list[BinProvider] = [env, apt]
    
    overrides: dict[BinProviderName, ProviderLookupDict] = {
        'env': {
            # example: prefer podman if installed (falling back to docker)
            'abspath': lambda: os.which('podman') or os.which('docker') or os.which('docker-ce'),
        },
        'apt': {
            # example: vary installed package name based on your CPU architecture
            'install_args': {
                'amd64': ['docker'],
                'armv7l': ['docker-ce'],
                'arm64': ['docker-ce'],
            }.get(platform.machine(), 'docker'),
        },
    }

docker = DockerBinary().load_or_install()
print(docker.binprovider)                 # EnvProvider()
print(docker.abspath)                     # Path('/usr/local/bin/podman')
print(docker.abspaths)                    # [Path('/usr/local/bin/podman'), Path('/opt/homebrew/bin/podman')]
print(docker.version)                     # SemVer('6.0.2')
print(docker.is_valid)                    # True

# You can also pass **kwargs to override properties at runtime,
# e.g. if you want to force the abspath to be at a specific path:
custom_docker = DockerBinary(abspath='~/custom/bin/podman').load()
print(custom_docker.name)                 # 'docker'
print(custom_docker.binprovider)          # EnvProvider()
print(custom_docker.abspath)              # Path('/Users/example/custom/bin/podman')
print(custom_docker.version)              # SemVer('5.0.2')
print(custom_docker.is_valid)             # True

`SemVer`

from abx_pkg import SemVer

### Example: Use the SemVer type directly for parsing & verifying version strings
SemVer.parse('Google Chrome 124.0.6367.208+beta_234. 234.234.123')  # SemVer(124, 0, 6367')
SemVer.parse('2024.04.05)                                           # SemVer(2024, 4, 5)
SemVer.parse('1.9+beta')                                            # SemVer(1, 9, 0)
str(SemVer(1, 9, 0))                                                # '1.9.0'

These types are all meant to be used library-style to make writing your own apps easier.
e.g. you can use it to build things like: playwright install --with-deps)

Django Usage

With a few more packages, you get type-checked Django fields & forms that support BinProvider and Binary.

[!TIP] For the full Django experience, we recommend installing these 3 excellent packages:

django-admin-data-views

django-pydantic-field

django-jsonform
pip install abx-pkg django-admin-data-views django-pydantic-field django-jsonform

Django Model Usage: Store `BinProvider` and `Binary` entries in your model fields

pip install django-pydantic-field

Fore more info see the django-pydantic-field docs…

Example Django models.py showing how to store Binary and BinProvider instances in DB fields:

from typing import List
from django.db import models
from pydantic import InstanceOf
from abx_pkg import BinProvider, Binary, SemVer
from django_pydantic_field import SchemaField

class Binary(models.Model):
    name = models.CharField(max_length=63)
    binary: Binary = SchemaField()
    binproviders: list[InstanceOf[BinProvider]] = SchemaField(default=[])
    version: SemVer = SchemaField(default=(0,0,1))

And here’s how to save a Binary using the example model:

from abx_pkg import Binary, SemVer, env

# find existing curl Binary in $PATH
curl = Binary(name='curl').load()

# save it to the DB using our new model
obj = Binary(
    name='curl',
    binary=curl,                                  # store Binary/BinProvider/SemVer values directly in fields
    binproviders=[env],                           # no need for manual JSON serialization / schema checking
    min_version=SemVer('6.5.0'),
)
obj.save()

When fetching it back from the DB, the Binary field is auto-deserialized / immediately usable:

obj = Binary.objects.get(name='curl')    # everything is transparently serialized to/from the DB,
                                                  # and is ready to go immediately after querying:
assert obj.binary.abspath == curl.abspath
print(obj.binary.abspath)                         #   Path('/usr/local/bin/curl')
obj.binary.exec(['--version'])                    #   curl 7.81.0 (x86_64-apple-darwin23.0) libcurl/7.81.0 ...

For a full example see our provided django_example_project/…

Django Admin Usage: Display `Binary` objects nicely in the Admin UI

Django Admin binaries list view Django Admin binaries detail view

pip install abx-pkg django-admin-data-views

For more info see the django-admin-data-views docs…

Then add this to your settings.py:

INSTALLED_APPS = [
    # ...
    'admin_data_views'
    'abx_pkg'
    # ...
]

# point these to a function that gets the list of all binaries / a single binary
ABX_PKG_GET_ALL_BINARIES = 'abx_pkg.views.get_all_binaries'
ABX_PKG_GET_BINARY = 'abx_pkg.views.get_binary'

ADMIN_DATA_VIEWS = {
    "NAME": "Environment",
    "URLS": [
        {
            "route": "binaries/",
            "view": "abx_pkg.views.binaries_list_view",
            "name": "binaries",
            "items": {
                "route": "<str:key>/",
                "view": "abx_pkg.views.binary_detail_view",
                "name": "binary",
            },
        },
        # Coming soon: binprovider_list_view + binprovider_detail_view ...
    ],
}

For a full example see our provided django_example_project/…

Note: If you override the default site admin, you must register the views manually...

admin.py:


class YourSiteAdmin(admin.AdminSite):
    """Your customized version of admin.AdminSite"""
    ...


custom_admin = YourSiteAdmin()
custom_admin.register(get_user_model())
...
from abx_pkg.admin import register_admin_views
register_admin_views(custom_admin)

Django Admin Usage: JSONFormWidget for editing `BinProvider` and `Binary` data

Expand to see more...

> [!IMPORTANT] > This feature is coming soon but is blocked on a few issues being fixed first: > - https://github.com/surenkov/django-pydantic-field/issues/64 > - https://github.com/surenkov/django-pydantic-field/issues/65 > - https://github.com/surenkov/django-pydantic-field/issues/66 ~~Install `django-jsonform` to get auto-generated Forms for editing BinProvider, Binary, etc. data~~ ```bash pip install django-pydantic-field django-jsonform ``` *For more info see the [`django-jsonform`](https://django-jsonform.readthedocs.io/) docs...* `admin.py`: ```python from django.contrib import admin from django_jsonform.widgets import JSONFormWidget from django_pydantic_field.v2.fields import PydanticSchemaField class MyModelAdmin(admin.ModelAdmin): formfield_overrides = {PydanticSchemaField: {"widget": JSONFormWidget}} admin.site.register(MyModel, MyModelAdmin) ```

For a full example see our provided django_example_project/…

Logging

abx-pkg uses the standard Python logging module. By default it stays quiet unless your application configures logging explicitly.

import logging
from abx_pkg import Binary, env, configure_logging

configure_logging(logging.INFO)

python = Binary(name='python', binproviders=[env]).load()

To enable Rich logging:

pip install "abx-pkg[rich]"

import logging
from abx_pkg import Binary, EnvProvider, configure_rich_logging

configure_rich_logging(logging.DEBUG)

python = Binary(name='python', binproviders=[EnvProvider()]).load()

Debug logging is hardened so logging itself does not become the failure. If a provider/model object has a broken or overly-expensive repr(), abx-pkg falls back to a short ClassName(...) summary instead of raising while formatting log output.

configure_rich_logging(...) uses rich.logging.RichHandler under the hood, so log levels, paths, arguments, and command lines render with terminal colors when supported.

You can also manage it with standard logging primitives:

import logging

logging.basicConfig(level=logging.INFO)
logging.getLogger("abx_pkg").setLevel(logging.DEBUG)

Examples

Advanced: Implement your own package manager behavior by subclassing BinProvider

from subprocess import run, PIPE

from abx_pkg import BinProvider, BinProviderName, BinName, SemVer

class CargoProvider(BinProvider):
    name: BinProviderName = 'cargo'
    
    def on_setup_paths(self):
        if '~/.cargo/bin' not in sys.path:
            sys.path.append('~/.cargo/bin')

    def on_install(self, bin_name: BinName, **context):
        install_args = self.on_get_install_args(bin_name)
        installer_process = run(['cargo', 'install', *install_args.split(' ')], capture_output = True, text=True)
        assert installer_process.returncode == 0

    def on_get_install_args(self, bin_name: BinName, **context) -> InstallArgs:
        # optionally remap bin_names to strings passed to installer 
        # e.g. 'yt-dlp' -> ['yt-dlp, 'ffmpeg', 'libcffi', 'libaac']
        return [bin_name]

    def on_get_abspath(self, bin_name: BinName, **context) -> Path | None:
        self.on_setup_paths()
        return Path(os.which(bin_name))

    def on_get_version(self, bin_name: BinName, **context) -> SemVer | None:
        self.on_setup_paths()
        return SemVer(run([bin_name, '--version'], stdout=PIPE).stdout.decode())

cargo = CargoProvider()
rg = cargo.install(bin_name='ripgrep')
print(rg.binprovider)                   # CargoProvider()
print(rg.version)                       # SemVer(14, 1, 0)

Note: this package used to be called pydantic-pkgr, it was renamed to abx-pkg on 2024-11-12.

TODO

Implement initial basic support for apt, brew, and pip
Provide editability and actions via Django Admin UI using django-pydantic-field and django-jsonform
Add preinstall and postinstall hooks for things like adding apt sources and running cleanup scripts
Implement more package managers (apk, ppm, pkg, etc.)

Other Packages We Like

https://github.com/MrThearMan/django-signal-webhooks
https://github.com/MrThearMan/django-admin-data-views
https://github.com/lazybird/django-solo
https://github.com/joshourisman/django-pydantic-settings
https://github.com/surenkov/django-pydantic-field
https://github.com/jordaneremieff/djantic

This site is open source. Improve this page.

abx-pkg

abx-pkg 📦 apt brew pip npm cargo gem go_get nix docker ₊₊₊Simple Python interfaces for package managers + installed binaries.

Supported Package Managers

Usage

Lazy Provider Singletons

Version Floors

BinProvider

Binary

SemVer

Django Usage

Django Model Usage: Store BinProvider and Binary entries in your model fields

Django Admin Usage: Display Binary objects nicely in the Admin UI

Django Admin Usage: JSONFormWidget for editing BinProvider and Binary data

Logging

Examples

Advanced: Implement your own package manager behavior by subclassing BinProvider

TODO

Other Packages We Like

`abx-pkg` 📦 `apt` `brew` `pip` `npm` `cargo` `gem` `go_get` `nix` `docker` ₊₊₊
_{Simple Python interfaces for package managers + installed binaries.}

`BinProvider`

`Binary`

`SemVer`

Django Model Usage: Store `BinProvider` and `Binary` entries in your model fields

Django Admin Usage: Display `Binary` objects nicely in the Admin UI

Django Admin Usage: JSONFormWidget for editing `BinProvider` and `Binary` data