This is derived from rust-forge project.
The following pixi tasks are provided:
The build takes 3 parameters with defaults.
The following commands do the same thing. The first example sets all the arguments, the second uses defaults.
pixi run build-the digitalpy linux-64
pixi run build-the digitalpyBuild all of the packages in the default pkgs folder.
pixi run buildRetract packages from prefix.dev channels. This permanently removes packages from the repository.
pixi run retract-pd foo --channel mychannel --versions "1.0.0"
pixi run retract-pd foo --channel mychannel --versions "1.0.0,1.1.0,1.2.0"
pixi run retract-pd foo --channel mychannel --versions "1.0.0" --forceRetract packages from S3-based conda channels.
pixi run retract-s3 foo --channel s3://my-bucket/channel --versions "1.0.0"
pixi run retract-s3-local foo --channel s3://local-bucket/channel --versions "1.0.0"The traditional approach to building software not provided by a host package manager is to build it from source on the same system it will be run on. For unprivileged users running in HPC environments, all software may need to be built from source.
The conda package format is a universal binary package format that works in any installation prefix in privileged and unprivileged environments. While originally developed for distributing Python packages, The conda format now provides relocatable, precompiled packages for any language. There is a rich set of tooling which may be used to build, publish, and install conda packages. This project uses the following tools:
The workflows in this repository provide an easy way to upload Conda recipes and have the packages built and uploaded to a personal channel using GitHub Actions. This means you don’t have to build the Conda packages in an HPC environment or in a virtual machine on your personal computer. Instructions for doing this are provided in a separate document.
Building and uploading Conda packages to a personal channel should not be a substitute for submitting and maintaining these packages on:
This setup should be used when developing and testing packages with the intention of submitting them to upstream channels when they are ready. Please see the wiki for more information on how you should prepare your software for public use.
Make a fork of the 'main' branch of this repository.
You will need to set up a pers 8000 onal Anaconda channel and add an API key for the channel to the repository secrets. See the wiki for a step-by-step walkthrough.
You will also want to modify the
default channels configuration
to use whatever channels you want.
The order of the channels in this config file determines their precedence for dependency resolution
- pixi will search for the dependency in the channels in the order they are given in the file.
You should usually leave the last 3 channels as conda-forge, bioconda and nodefaults
in that order so that any custom channels you add before them are checked first,
that bioconda is checked after conda-forge, and the legacy defaults channels are ignored.
To add a new package, clone the repository locally and create a new branch.
Each package recipe should be in its own folder with a recipe.yaml,
and when needed, a build.sh script.
Some complex recipes may use multiple shell scripts.
Push the branch to the repository and open a pull request from it.
Once the pull request has been opened,
the rattler-build_linux-64.yml workflow will attempt to build the recipes
using the ratter-build_container.df linux container image.
If the recipes build successfully,
you can merge the pull request to trigger the rattler-build_upload.yml workflow,
which will upload the conda packages to your personal channel.
This is the step that requires the ANACONDA_API_TOKEN secret and ANACONDA_OWNER variable.
The ANACONDA_API_TOKEN is used in the following jobs: .github/workflows/dispatch_build_linux64.yml
link:.github/workflows/build-packages.yml[role=include]Previously, it was necessary to use a token but http://prefix.dev mesolab allows prefered publishers.
-
Enable "Allow all operations"
-
Choose an informative name
-
Select an expiration date that is sufficiently far in the future
Copy the secret and then go to your fork of this repository and go to "Settings"
Here is an example of how to run a build for testresources.
It pulls from two channels, mesomorph and conda-forge.
PKG_NAME=node-red
rattler-build build --recipe "./packages/$PKG_NAME" --target-platform linux-64 --channel mesomorph --channel conda-forgeUploading the constructed package.
PACKAGE_FILE=$(find output -type f \( -name "*.conda" -o -name "*.tar.bz2" \) )
ANACONDA_OWNER: mesomorph
ANACONDA_API_KEY: ***
RATTLER_BUILD_LOG_STYLE: fancy
rattler-build upload anaconda -v $PACKAGE_FILEPresume that the ANACONDA_API_KEY has been stored locally as a secret.
echo -n <the anaconda api key token> | secret-tool store --label="ANACONDA_API_KEY" ANACONDA API_KEY