How to do dynamic "uses" for reusable workflow #45342
-
Select Topic AreaQuestion BodyMy use case:
I have researched previous discussions here and all I could find was a centralized, single-branch solution, and using different workflows (ie. customer-deploy-development.yaml, customer-deploy-staging.yaml, etc). While this would solve the pipeline call issue, it would then significantly complicate the pipeline for infrastructure rollout (each branch can be different), bordering unmaintainable. Another approach would be separating each environment into its own file, tracking a single branch per workflow file. This means we'd have to go over thousands of customer repositories creating the same file, with only the Anyway, to solve our use case scenario, we are attempting to do the following: name: customer-deploy
on:
push:
branches:
- development
- staging
- production
jobs:
deploy:
uses: "octocat/pipelines/.github/workflows/customer-deploy.yaml@${{ github.ref_name }}"
with:
environment_name: ${{ github.ref_name }}
repository_name: ${{ github.repository }}
repository_ref_name: ${{ github.ref_name }}
repository_commit_hash: ${{ github.sha }}
secrets: inheritHowever, we experience this error: We have attempted multiple variations of this, without success. Even build another job that only computes the string and use it, like this: ...
jobs:
metadata-pipeline:
runs-on: self-hosted
env:
PIPELINE: "octocat/pipelines/.github/workflows/customer-deploy.yaml"
RELEASE: ${{ github.ref_name }}
outputs:
pipeline: ${{ steps.release.outputs.pipeline }}
steps:
- name: Extract release pipeline
id: pipeline
run: |
echo "pipeline=${PIPELINE}@${RELEASE##*/} >> $GITHUB_OUTPUT
deploy:
uses: ${{ needs.metadata-pipeline.outputs.pipeline }}
needs:
- metadata-pipeline
with:
environment_name: ${{ github.ref_name }}
repository_name: ${{ github.repository }}
repository_ref_name: ${{ github.ref_name }}
repository_commit_hash: ${{ github.sha }}
secrets: inheritWhich also gives me: How do we achieve this? |
Beta Was this translation helpful? Give feedback.
Replies: 9 comments 5 replies
-
Unfortunately the answer is "you don't". The value for |
Beta Was this translation helpful? Give feedback.
-
|
Do you have a finite amount of possible environments? If no, you can skip reading this. I would classify this as a workaround, like already mentioned expressions are forbidden in uses. on: workflow_call
# TODO add some inputs if you really need them, you example inputs could be removed without loss of information.
jobs:
development:
if: github.ref_name == 'development'
uses: "octocat/pipelines/.github/workflows/customer-deploy.yaml@development"
with:
environment_name: ${{ github.ref_name }}
repository_name: ${{ github.repository }}
repository_ref_name: ${{ github.ref_name }}
repository_commit_hash: ${{ github.sha }}
secrets: inherit
staging:
if: github.ref_name == 'staging'
uses: "octocat/pipelines/.github/workflows/customer-deploy.yaml@staging"
with:
environment_name: ${{ github.ref_name }}
repository_name: ${{ github.repository }}
repository_ref_name: ${{ github.ref_name }}
repository_commit_hash: ${{ github.sha }}
secrets: inherit
production:
if: github.ref_name == 'production'
uses: "octocat/pipelines/.github/workflows/customer-deploy.yaml@production"
with:
environment_name: ${{ github.ref_name }}
repository_name: ${{ github.repository }}
repository_ref_name: ${{ github.ref_name }}
repository_commit_hash: ${{ github.sha }}
secrets: inheritAll consumers just have this file name: customer-deploy
on:
push:
branches:
- development
- staging
- production
jobs:
deploy:
uses: "octocat/pipelines/.github/workflows/customer-deploy.yaml@main"
secrets: inheritYou may take special care about required status checks, the different environments use different job names and skipped jobs are counting as success. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks @airtower-luna and @ChristopherHX for the answers, it is greatly appreciated. |
Beta Was this translation helpful? Give feedback.
-
|
I have a different requirement that requires the same requested solution. We are in the processing of migrating workflows between orgs. The workflow in the new org will continue to be developed but the old workflow is frozen. We planned on storing the org name in a variable that the caller workflow would interrogate to call the right reusable workflow. In this case, the setup job gets the value org value and expects the prepare job to use it to call the right workflow. Subsequent jobs require the output of the prepare step. e.g. So using the solution suggested above doesn't really work since each job must have a unique name (e.g. development, staging, production). Is the only solution to essentially duplicate the entire caller workflow with unique job names that are conditional based off of the value of the org name? |
Beta Was this translation helpful? Give feedback.
-
|
Sad that this not a possibility, I understand the potential for injecting a foreign workflow, but we could at least be allowed to call workflows that are from the same repo or org. |
Beta Was this translation helpful? Give feedback.
-
|
any new? |
Beta Was this translation helpful? Give feedback.
-
|
I have a different use case, where I have a separate repository containing reusable workflows and reusable actions. The repository is semanticly versioned. Let's call that the "reusable repository" Some of the reusable workflows use the actions in that same repository : And in a different repository that is calling the ** reusable_repository/.github/workflows/some-workflow.yaml@v3.5.2** I would like the reusable action "some-action" to be called from the same ref_name |
Beta Was this translation helpful? Give feedback.
-
|
The problem is even worse: |
Beta Was this translation helpful? Give feedback.
-
|
We have the same use case, and AFAIK (in contrast to referencing actions/workflows in the same repository where they're executed from) in this case you have to specify the full path, including the organization and repo name. I.e. in "reusable repository" you have to specify the full In our own such "reusable repository" we've created a workflow that warns you if you forget to adjust the references, see here: https://github.com/Lombiq/GitHub-Actions/blob/dev/.github/workflows/validate-this-gha-refs.yml |
Beta Was this translation helpful? Give feedback.
This comment was marked as spam.
This comment was marked as spam.
-
|
While dynamic uses is not directly supported, you can achieve what you want by nesting the remote reusable workflow inside another reusable workflow that is path based. Just before running your path based workflow job is run, you can create the prepare:
runs-on: ubuntu-latest
env:
DYNAMIC_WORKFLOW: .github/workflows/dynamic.yml
steps:
- name: Setup dynamic workflow
uses: actions/github-script@v6
with:
script: |
const path = require('path');
const fs = require('fs/promises');
const workflow = {
name: "Dynamic workflow",
on: {
workflow_call: {}
},
jobs: {
actual-deploy: {
uses: "octocat/pipelines/.github/workflows/customer-deploy.yaml@${{ github.ref_name }}",
with: {
environment_name: "${{ github.ref_name }}",
repository_name: "${{ github.repository }}",
repository_ref_name: "${{ github.ref_name }}",
repository_commit_hash: "${{ github.sha }}"
},
secrets: "inherit"
}
}
};
await fs.writeFile(${{ env.DYNAMIC_WORKFLOW }}, JSON.stringify(workflow));and then invoke the file in the next job, note that this job must start after deploy:
needs: prepare
uses: ".github/workflows/dynamic.yml"
secrets: inheritNote that there is limitation on level of nesting as described here, if you are not exceeding that you can use this approach. |
Beta Was this translation helpful? Give feedback.
-
|
This seems like a great workaround but I can't get it working. I get an error that the |
Beta Was this translation helpful? Give feedback.
-
|
@soumyamahunt you may be misunderstanding how GitHub actions work
Therefore what you have described cannot work because
https://docs.github.com/en/actions/using-workflows/reusing-workflows |
Beta Was this translation helpful? Give feedback.
-
|
If triggering using |
Beta Was this translation helpful? Give feedback.
Do you have a finite amount of possible environments? If no, you can skip reading this.
I would classify this as a workaround, like already mentioned expressions are forbidden in uses.
Create a proxy workflow
octocat/pipelines/.github/workflows/customer-deploy.yaml@main, which references each of the possible environments. You only need to update it if you modify inputs and outputs.