Maybe we're doing this wrong but we find ourselves writing playbooks that talk to multiple platforms requiring different credentials to log in. For example, I have a backup playbook that logs into the BIG-IPs in production and creates a backup. Then either that playbook needs to scp from BIG-IP to the backup server, or in a separate play or role, the backup server needs to log into the BIG-IP to store the backup. Either way, I need both the BIG-IP and backup server credentials in the same playbook. Unfortunately, both platforms require a different user and password.

To work around Tower's current behavior, we will typically write the playbook using the BIG-IP inventory and BIG-IP Tower credentials, and then within the playbook at run-time, we perform a separate lookup to get the backup server account credentials and store it in a variable. This is brittle and requires side-band resources to execute the playbook successfully. Further, it disallows us from using ansible modules like 'copy', and we instead have to run 'shell' and scp. Our previous approach was to store the credentials in a vault file, but that complicated SCM of our playbook source code any time the password or the vault password changed because this vault file had to be in many many playbooks.

This is a common patter in our environment and we see no way around it due to limitations with network appliances like BIG-IP, Cisco, Arbor, etc. Many devices in our network don't auth like a Linux server and having the same system account on every device doesn't work well for us at this time.

Regarding this RFE, what about a method similar to a 'survey' or vars_prompt. What if Tower's credential subsystem simply allowed for arbitrary key/value pairs, where the "key/value" pair is stored within Tower's Credential subsystem encrypting the "value" like everything else and exposing the key/value at run-time if the credential is associated with the Job Template.

Tower's Credential screen would have something like the following in addition to the default screen values:

Name: My Compound Credentials
Description: Collection of credentials
Organization: Default
Type: Machine
Type Details

• Username: Dave
• Password: SomethingSecret
• Private Key Passphrase: SomethingSuperSecret
• Vault Password: SomethingSecret
  Private key:

Additional Parameters:
Key:Value
machine1_pass: "somethingsecret"
machine2_key: "my private key string"
mysshkeypass: "somekeypasswd"

While it won't solve all of the challenges, it would allow us to keep credentials in one place. Eventually, core modules could be extended to include user/pass fields. This would allow modules like 'copy' to use a different credential set as needed.

... how is this different than the existing custom credential type support? See https://www.ansible.com/blog/ansible-tower-feature-spotlight-custom-credentials and http://docs.ansible.com/ansible-tower/latest/html/userguide/credential_types.html

We're in the process of setting up Ansible for our entire infrastructure in AWS.

We have a number of legacy Ubuntu servers where the default username is ubuntu.

We also have a good chunk of newer Debian based servers, where the default username is admin.

There's a small number of other legacy servers with various other user setups.

Finally the servers are launched with a plethora of differing base private keys.

Currently with the "only one SSH key per template" restriction, it's troublesome to get AWX/Ansible access to all of the servers. I'm currently settling on the option of making a simple "re-key" playbook, that will create an ansible user, add a shared private key, and enable sudo privileges. Then I'm going to duplicate the template for every credential combination in our infrastructure, and make sure that each template only run on the correct subset of instances.

Overall this ansible account is useful to streamline the configuration of the servers, and also useful for auditing purposes, such that I can see that some command was run by ansible.

On the other hand, the current option is cumbersome, as long as we're running more than a single linux distribution, as each distribution have differing conventions for default usernames.

I would like to have the ability to assign machine credential by inventory group as well

Link ansible/ansible-runner#51

Another use case to support is using WinRm and SSH connection in the same playbook / template. Currently both SSH and Winrm credentials represent with the same "Machine" type which makes it impossible to use both in the same run.

Is this feature planned to be released soon? A number of enterprises with rigid governance policies do not allow same credentials across multiple hosts.

It is not currently being worked, when it is, more information will be placed here.

This is really useful feature in case clients are in multi domain environment so multiple creds can be used in job templates.

very interesting on this as well .. we are a large enterprise customer planning to host +100K endpoints thru AT.
This feature is crucial for our project.

I'd sell my soul to see this feature implemented!!!!

@wenottingham can you please check if what I ask in ansible/ansible-runner#51 makes sense? It should be easy to setup, but need some kind of confirmation before proceeding.

Can you create this feature by giving option to include a credential with a host in inventory itself. I think that will be much simplified and highly distributive.

we also would sell our soul for this feature. this makes using AWX at a large scale difficult, as even if you can use the same credential for more than 1 host you more than likely can't use that same credential to target 3000+ servers at a single time, thus making AWX unusable.

One partial work around to this is to create custom credentials and then shim the path into ansible_ssh_private_key. It's extremely unpleasant, and it does not scale, but this is what it looks like:
In AWX, you create a custom credential that will accept your key value and set an env with the path:

Input cofiguration

fields:
  - id: switch_ssh_private_key
    type: string
    label: Switch SSH Private Key
    format: ssh_private_key
    secret: true
    multiline: true
required:
  - switch_ssh_private_key

Injector configuration:

env:
  AWX_SWITCH_SSH_KEY_FILE: '{{ tower.filename.switch_ssh_key_file }}'
file:
  template.switch_ssh_key_file: '{{ switch_ssh_private_key }}'

Then you create a credential of that type and input your private key. After that, you can attach it to your job template. You can do this for as many keys as you want/need, but you'll need to change the var names. E.g., Changing the word switch above to something else that's unique to each key you create.

After that, you can do gymnastics to use them in tasks/plays:

    - name: Try to grab a custom ssh key file for the switch
      run_once: True
      set_fact:
        pm_custom_switch_ssh_key_file: "{{ lookup('env','AWX_SWITCH_SSH_KEY_FILE') }}"

    - name: Chmod the tmp key
      run_once: True
      delegate_to: localhost
      file:
        path: "{{ pm_custom_switch_ssh_key_file }}"
        mode: 0600

and

      
    - name: Add switch to inventory
      add_host:
          name: "{{ switch_name }}"
          ansible_ssh_host: "{{ switch_name }}"
          ansible_ssh_private_key_file: "{{ pm_custom_switch_ssh_key_file }}"

or

    - name: Add switch to inventory
      set_fact:
          ansible_ssh_private_key_file: "{{ pm_custom_switch_ssh_key_file }}"

With this, it becomes possible to create a dictionary to reference the right env value for the host you want based on some variable in group_vars etc.

I agree with @mkkeffeler, this feature would be extremely useful for Windows environments as well - especially since Kerberos/AD auth is far less flexible than SSH.

I would've sold my soul in July, and i'd still sell it now.

So, if I have a 1000 hosts that I want to manage, I need to create a 1000 different custom credential types? That's quite unreasonable. Is there any other work-around for this issue?

It really defeats the purpose of using Ansible Tower/AWX at scale.

When I want to execute job template toward windows and linux inventory, I have to use credential for linux and group_vars credential with plain text for windows host.
That is not good for me.

I guess splitting job template and executing via workflow is the workaround, but it's really bad.

So, if I have a 1000 hosts that I want to manage, I need to create a 1000 different custom credential types? That's quite unreasonable. Is there any other work-around for this issue?

It really defeats the purpose of using Ansible Tower/AWX at scale.

@saurabh02 but if it can be managed using the API, then it becomes easier. I think this issue is intended to enable the functionality, and offer the ability to manage it via the API so that the issue you described doesn't become unmanageable.

Worth looking into #11121 at the same time.

If you look at our current SSH credential, these are our inputs:

But if you look at the implementation of the credential, many of these fields are incompatible with multiple SSH keys. I'll look at become_username as an easy example:

This sets the become username for the entire playbook. If using multiple SSH credentials, then they would clearly need to all either have the same value or not have a value. Or, we might be able to do the configuration differently.

Coming from the ansible-playbook CLI perspective, could anyone help outline specific use cases that we should look into supporting? I mean, how do you do this from your own machine, and what inputs would those map to in the credential type code I linked here.

@AlanCoding

Here is how I am able to conduct OS patching across many hosts while pulling secrets dynamically per host using CyberArk Conjur Secrets Manager. Each server has a unique SSH key that is needed for connection: https://github.com/infamousjoeg/instruqt/blob/22654a6ae7681846ae516489c5594935456fea33/tracks/conjur.org/secure-ansible-automation/09-secure-playbook/solve-host01#L13

My issue and suggestion is #11121.

In our AWS org we create EC2 keys that differ by region and by account via CloudFormation, letting CFN create the keypair dynamically. We could also make them differ between tiers within the same account within the same region if needed. However, a template and workflow template can only have a single machine credential so unfortunately for us, in a single account, machines that boot up in any region need to pull a dynamically created public key to create the ansible user for AWX to communicate to it and we have to use the key created in a single region. While we can add all the private keys to AWX that we have in each region, a template can't dynamically choose an SSH key to use on a machine based on, for example, what groups the instance falls into (e.g. us_west_2 vs us_west_1)

The only other option for us is to replicate templates and workflow templates that only differ by what region they serve so we can set the region-specific credential for that template. Now scale that horizontally by account and possibly by tier (dev, test, qa, prod) per account per region.

This feature request has been out there for 5 years and it's clearly something that lots of users are requesting :(

I have this case:
The goal is to ssh to a number of hosts using userX and ssh_priv_keyX. But in one task I want to clone a github repo using ansible.builtin.git module delegated to localhost (git@github.... which is ssh too) but using userY and ssh_priv_keyY.
Please note that localhost in this case is the execution environment and I don't want to set a permanent localhost inventory variable ansible_ssh_private_key_file.
I don't get how to do this with a custom credential type. Is it possible at all or I have to use the same user for host and github?

Only the "Machine" credential is unique per job. But you can pass another set of credential from another type, you don't need a custom credential. Then you can use the variable of this credential in your git command
…
On Fri, 23 Sep 2022, 04:41 Manolis Kartsonakis, @.> wrote: I have this case: The goal is to ssh to a number of hosts using userX and ssh_priv_keyX. But in one task I want to clone a github repo using ansible.builtin.git module delegated to localhost @. which is ssh too) but using userY and ssh_priv_keyY. Please note that localhost in this case is the execution environment and I don't want to set a permanent localhost inventory variable ansible_ssh_private_key_file. I don't get how to do this with a custom credential type. Is it possible at all or I have to use the same user for host and github? — Reply to this email directly, view it on GitHub <#286 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEWXDHLO5ZCZ2E7JASRSLVDV7S76HANCNFSM4D4M2XWA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

I agree but in my case, git push is done via a shell command since git module doesn't support push operations (and I don't want to mess with git_acp yet). So in other words I have 2 kinds of ssh (machine) connections in one playbook. For git module tasks, there is no way afaik to tell awx to use "GitHub Personal Access Token" either.

I cannot believe that the AWX team has let this extremely important issue fester for over SIX YEARS. What in the world are they working on? This is probably the most important issue in the whole project. Yet they have offered us zero realistic workarounds for scaling AWX beyond a handful of inventories.

A proper fix would allow you to associate machine credentials with an inventory at minimum, or ideally, individual hosts. Then, when you run a template, you just select the inventory, and you don't have to worry about selecting the CORRECT machine credentials for that inventory (an error prone process, tedious and time-consuming; terrible design).

An alternative would be to allow the playbook to IMPORT its own credentials from AWX based on credential name, or some unique identifier. That way, the inventory could include variables telling the tasks WHICH credentials to use, and life will be all good again.

Failure to provide either of these very reasonable features, again after SIX YEARS of knowing this is an urgent requirement, totally explains why Ansible Semaphore exists. However, it's even more buggy than AWX.

How anyone is paying for an Ansible Tower subscription with this basic functionality missing, boggles the mind.

There's a reason this has hung out so long. It is hard to implement and nuanced to hell. If you read Bill's original issue description that still holds true today. There are ways and means to do this including that posted by Joe from CyberArk which demonstrates per machine credentials.

We added prompt on launch to just about every runtime job parameter so you get more runtime flexibility. And with constructed inventory you can carve up multiple inventories in many variabled ways (so your handful of inventories comment is a little disrespectful to the great engineers that work on this project).

I see you are a designer and entrepreneur, so given this is an open source project you are free and we would welcome any help you'd like to bring to the table to get this done.

+1 @ffirg I have tried to solve this problem 100 different ways and this is not falling on deaf ears, it is just not as easy a problem to solve. Many people do this by calling external systems hence why we have not prioritized this in the past, but we are listening.

There's a reason this has hung out so long. It is hard to implement and nuanced to hell. If you read Bill's original issue description that still holds true today. There are ways and means to do this including that posted by Joe from CyberArk which demonstrates per machine credentials.

We added prompt on launch to just about every runtime job parameter so you get more runtime flexibility. And with constructed inventory you can carve up multiple inventories in many variabled ways (so your handful of inventories comment is a little disrespectful to the great engineers that work on this project).

I see you are a designer and entrepreneur, so given this is an open source project you are free and we would welcome any help you'd like to bring to the table to get this done.

First of all, I sincerely apologize for any perceived disrespect due to my poorly chosen phrasing and the implications of that.

One idea I could see that would be really useful would be just a table with two columns: one column would be the id of a machine credential, and the other column would be the id of a host. Then you could do the same thing for host groups, and finally entire inventories. That way, at least the problem of having to select an inventory, then having to select the machine credential(s) which correspond to that inventory, or a host group within that inventory, or individual hosts within that inventory, would be solved.

The other problem, which is the inability of Ansible tasks to selectively import credentials into variables, can easily be solved by reverting to Ansible Vault files which are dynamically loaded using ansible.builtin.include_vars, as long as you encrypt all of your Vault files with the same password, and set up a Vault type credential with that password and add it to every AWX Template. This is exactly how I ended up working around the issue of not being able to link credentials to inventories, host groups or individual hosts.

Back to the Vault solution I ended up using. If the Credentials could be somehow treated as Vault files and imported into tasks in the playbooks using ansible.builtin.include_vars, that could be pretty cool. Say, for example, you provide a way for a template to list out every type of credential which should be exported to Vault files. Then, in each Credential, you add a field where the user can enter a unique Vault filename for that credential (it must be unique). Then, when you run the Template, you export all Credentials of each of the credential types that Template says it needs, into a temporary directory where the Ansible tasks can use ansible.builtin.include_vars to get those credentials imported. Now, you have something.

I'm using AWX in Kubernetes via AWX Operator, so the various tiers of the AWX stack are running in different Pods/Containers. That said, the logic behind this type of feature would need to be spread across the stack in such a way that each Pod knows how to handle its portion of the workflow.

You guys have created an incredibly useful tool. I just think that with a few minor tweaks in this area, it's going to be so much easier to use at scale.

I am 100% behind Ansible and AWX. I love the concepts behind it. If there are any tasks related to this issue that you would like to assign to me, I would be happy to help out. I see that at least the web part of AWX is Django, and I'm currently employed by a Django shop. I'm not as good with Django as the other guys on my team, but I'm slowly coming up to speed. It might be more productive for me to just give you ideas and let someone else who is better at Django implement them. But I'm not averse to trying.

We have a similar problem of a mix of passwords for any given hosts in a technology type...

The hosts are running OpenWRT so we're able to copy the authorized_keys, trying to do so - using all possible passwords until we succeed, then all other tasks don't need that manual authentication but rather use the ssh key

Not perfect, a little hacky but it works...
Also the passwords dict should be stored in a vault

- name: Ensure reachability
  hosts: openwrt
  gather_facts: no

  roles:
     - openwrt

  vars:
    passwords:
      - 'password123'
      - 'password456'
      - 'password789'
 
  tasks:
    # This will fail if we don't have the authorized keys installed on the device
    - name: test connectivity with ansible ping
      ansible.builtin.ping:
      register: ping_test
      ignore_errors: true
      ignore_unreachable: true
    
    # This will only run if the ping_test above is unsuccessful 
    - name: copy authorized_keys (using first password)
      ansible.builtin.copy:
        src: "{{ playbook_dir }}/files/authorized_keys"
        dest: /etc/dropbear/authorized_keys"
      vars:
        ansible_user: "root"
        ansible_ssh_pass: "{{ passwords[0] }}"
      when: ping_test.unreachable is defined
      ignore_unreachable: true
      register: first_attempt

    # This will only run if the previous task runs and fails
    - name: copy authorized_keys (using second password)
      ansible.builtin.copy:
        src: "{{ playbook_dir }}/files/authorized_keys"
        dest: "/etc/dropbear/authorized_keys"
      vars:
        ansible_user: "root"
        ansible_ssh_pass: "{{ passwords[1] }}"
      when: first_attempt.unreachable is defined
      ignore_unreachable: true
      register: second_attempt

    # This will only run if the previous task runs and fails
    - name: copy authorized_keys (using third password)
      ansible.builtin.copy:
        src: "{{ playbook_dir }}/files/authorized_keys"
        dest: "/etc/dropbear/authorized_keys"
      vars:
        ansible_user: "root"
        ansible_ssh_pass: "{{ passwords[2] }}"
      when: second_attempt.unreachable is defined
      ignore_unreachable: true