MSAL Token Cache Multiple Token Problem - Enterprise Azure Account #918

siddallhotmail · 2022-07-12T07:33:41Z

SUMMARY

Whilst using an enterprise azure login (multiple subscriptions under a single tenant). The auth_sources property when set to use CLI fails with User 'xx' does not exist in MSAL token cache.

Using a standard azure subscription / 1-2-1 logon the azure modules work fine.

I suspect that this is because of the multiple tokens I obtain when logging in using the (now depreciated) function in azure_rm_common.py

self.log('Retrieving credentials from Azure CLI profile')
cli_credentials = self._get_azure_cli_credentials(subscription_id=params.get('subscription_id'))
return cli_credentials

Which in turn calls get_cli_profile() which is part of the azure-sdk-for-python

These functions are depreciated now ... and the documentation suggests moving to the Azure Identity client library for Python.

https://docs.microsoft.com/en-us/python/api/azure-common/azure.common.credentials?view=azure-python

I think the crux of the issue is that due to the multiple tokens received when using the AZ LOGIN command, we would need to be able to specify the subscription ID and tenant ID to correctly obtain the token from the cache.

ISSUE TYPE

Bug Report

COMPONENT NAME

azure_rm_common.py

ANSIBLE VERSION

ansible [core 2.13.1]
  config file = /home/g3.martin.siddall@hosting.civica.com/ccsautomation/05_provision_developer/ansible.cfg
  configured module search path = ['/home/g3.martin.siddall@hosting.civica.com/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/g3.martin.siddall@hosting.civica.com/ansible_stable/lib/python3.8/site-packages/ansible
  ansible collection location = /home/g3.martin.siddall@hosting.civica.com/ccsautomation/00_common/shared/resources/collections
  executable location = /home/g3.martin.siddall@hosting.civica.com/ansible_stable/bin/ansible
  python version = 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0]
  jinja version = 3.1.2
  libyaml = True

COLLECTION VERSION

azure.azcollection            1.12.0

CONFIGURATION

COLLECTIONS_PATHS(/home/g3.martin.siddall@hosting.civica.com/ccsautomation/05_provision_developer/ansible.cfg) = ['/home/g3.martin.siddall@hosting.civica.com/ccsautomation/>
DEFAULT_FORKS(/home/g3.martin.siddall@hosting.civica.com/ccsautomation/05_provision_developer/ansible.cfg) = 30
DEFAULT_HOST_LIST(/home/g3.martin.siddall@hosting.civica.com/ccsautomation/05_provision_developer/ansible.cfg) = ['/home/g3.martin.siddall@hosting.civica.com/ccsautomation/>
DISPLAY_SKIPPED_HOSTS(/home/g3.martin.siddall@hosting.civica.com/ccsautomation/05_provision_developer/ansible.cfg) = False
HOST_KEY_CHECKING(/home/g3.martin.siddall@hosting.civica.com/ccsautomation/05_provision_developer/ansible.cfg) = False
LOCALHOST_WARNING(/home/g3.martin.siddall@hosting.civica.com/ccsautomation/05_provision_developer/ansible.cfg) = False

OS / ENVIRONMENT

Ubuntu 20.04

STEPS TO REPRODUCE

You will require a AZ Login which has multiple tenants under the same subscription id.

Example of tokens obtained when logging in., you will note tenant id is the same ... but has multiple subscriptions.


  {
    "cloudName": "AzureCloud",
    "id": "xxxxxxx-8ec9-4bf7-8f4f-fba9ed4069ed",
    "isDefault": true,
    "name": "Visual Studio Professional",
    "state": "Enabled",
    "tenantId": "xxxxxxx-727e-413f-b5b8-cd0f27c02b6b",
    "user": {
      "name": "example.user@test.domain",
      "type": "user"
    }
  },
  {
    "cloudName": "AzureCloud",
    "id": "xxxxxxx-9d7d-4450-901a-e16f09f9fd1a",
    "isDefault": false,
    "name": "Visual Studio Professional",
    "state": "Enabled",
    "tenantId": "xxxxxxx-727e-413f-b5b8-cd0f27c02b6b",
    "user": {
      "name": "example.user@test.domain",
      "type": "user"
    }
  },
  {
    "cloudName": "AzureCloud",
    "id": "xxxxxxx-5cad-4521-99f6-3d7ad1d8a384",
    "isDefault": false,
    "name": "Visual Studio Professional",
    "state": "Enabled",
    "tenantId": "xxxxxxx-727e-413f-b5b8-cd0f27c02b6b",
    "user": {
      "name": "example.user@test.domain",
      "type": "user"
    }
  },
  {
    "cloudName": "AzureCloud",
    "id": "xxxxxxx-016a-43ee-8bd4-ce97f73afc27",
    "isDefault": false,
    "name": "Visual Studio Professional",
    "state": "Enabled",
    "tenantId": "xxxxxxx-727e-413f-b5b8-cd0f27c02b6b",
    "user": {
      "name": "example.user@test.domain",
      "type": "user"
    }
  }
]

Running the following module against the localhost.

azure_rm_resourcegroup_info:
      auth_source: cli
      subscription_id: xxxxx-5cad-4521-99f6-3d7ad1d8a384
      tenant: xxxxx-727e-413f-b5b8-cd0f27c02b6b

EXPECTED RESULTS

Azure CLI profile cannot be loaded - User 'xxxx' does not exist in MSAL token cache.

The text was updated successfully, but these errors were encountered:

Fred-sun · 2022-07-14T08:14:50Z

@martinpsiddall If you use 'az login', can you specify the subscription_id you want to use and retry? Thank you very much!

az login
az account set -s *************

Fred-sun · 2022-08-04T06:30:26Z

kindly ping!

siddallgoogle · 2022-08-04T11:48:44Z

Apologies Fred, been away on holiday.

Unfortunately the suggested work-around presents the same MSAL error.

siddallgoogle · 2022-08-31T15:18:21Z

Any further suggestions / debugging I can do ?

Fred-sun · 2022-09-01T00:29:49Z

@MartinSiddall After you log in, you need to specify which account subscription you use, such as "az account set-s sub_use_id". Thank you!

siddallgoogle · 2022-09-01T13:22:25Z

Hi Fred this was the same suggested az login commands you made on the 14th July.

As per my previous reply this has the same MSAL login error

Azure CLI profile cannot be loaded - User 'xxxx' does not exist in MSAL token cache. Run az login."

Fred-sun · 2022-09-01T15:05:25Z

"****" means the subscription ID you want to use. Thank you very much!

siddallgoogle · 2022-09-01T15:15:19Z

I have no idea what you're referring to the User 'xxxx' bit is me removing my email address.

I have tried the command you've suggested but they result in the same error.

14th July

az login
az account set -s *************

Does not work.

Today, the same command in your comment is basically the same suggestion as the 14th of July.

"az account set-s sub_use_id"

Again results in the MSAL error.

Thanks.

Fred-sun · 2022-09-01T15:21:12Z

I would like to know the ID after "az account set -s" that you specified to execute, and did you execute this command successfully?

siddallgoogle · 2022-09-01T15:24:19Z

The command worked fine, with the relevant subscription ID.

Again I've obfuscated the beginning of the ID from the first post as this is a public site.

az login
az account set -s xxxxxxx-9d7d-4450-901a-e16f09f9fd1a

As an example.

Thanks

Fred-sun · 2022-09-01T15:27:00Z

I will try again, Multiple accounts for one tenant.

siddallgoogle · 2022-09-01T15:34:10Z

Yep thats what we / the team will have.

Just as a FYI using an incorrect subscription id results in

The subscription of 'xxxx' doesn't exist in cloud 'AzureCloud'.

So the set -s works fine.

Additionally the az account show ... also shows the correct subscription ID ... active subscription.

Fred-sun · 2022-09-07T01:52:31Z

Yep thats what we / the team will have.

Just as a FYI using an incorrect subscription id results in

The subscription of 'xxxx' doesn't exist in cloud 'AzureCloud'.

So the set -s works fine.

Additionally the az account show ... also shows the correct subscription ID ... active subscription.

Do you mean that the problem has been solved?

siddallgoogle · 2022-09-07T07:37:20Z

No the problem still persists, just confirming that the az commands work. We still have the MSAL token cache error message.

Cyclonit · 2022-09-08T15:44:38Z

I have exactly the same issue. I would like to use the azurerm_inventory plugin. The plugin initializes just fine, but when executing the following command I get an error:

$ ansible-inventory -i dev.azure_rm.yml --graph
[WARNING]:  * Failed to parse /mnt/c/Projects/XXXXXXXX/ansible/dev.azure_rm.yml with script plugin: problem running /mnt/c/Projects/XXXXXXXX/ansible/dev.azure_rm.yml --list ([Errno 8] Exec format error: '/mnt/c/Projects/XXXXXXXX/ansible/dev.azure_rm.yml')
[WARNING]:  * Failed to parse /mnt/c/Projects/XXXXXXXX/ansible/dev.azure_rm.yml with auto plugin: Azure CLI profile cannot be loaded - User 'my.user@company.com' does not exist in MSAL token cache. Run `az login`.
[WARNING]:  * Failed to parse /mnt/c/Projects/XXXXXXXX/ansible/dev.azure_rm.yml with yaml plugin: Plugin configuration YAML file, not YAML inventory
[WARNING]:  * Failed to parse /mnt/c/Projects/XXXXXXXX/ansible/dev.azure_rm.yml with ini plugin: Invalid host pattern 'plugin:' supplied, ending in ':' is not allowed, this character is reserved to provide a port.
[WARNING]: Unable to parse /mnt/c/Projects/XXXXXXXX/ansible/dev.azure_rm.yml as an inventory source
[WARNING]: No inventory was parsed, only implicit localhost is available

I login the following way. But the error persists. Executing commands using the Azure CLI works without issue.

$ az login
$ az account set --subscription <SUBSCRIPTION_ID>

I did some searching around, and found another issue which I believe might be related to this one:
#688 Could not retrieve credential from local cache for service principal ***. Please run 'az login' for this service principal.

It suggests using different combinations of Azure CLI and the Python plugin azure-cli-core. Apparently there is a mismatch on how they store/access tokens in some versions. However, neither of the suggested combinations of versions worked for me. Additionaly the issue appears to be outdated, as the requirements for this collection reference fairly new versions.

siddallgoogle · 2022-09-23T14:43:31Z

Just for reference my pip / apt package versions

azure-cli-core==2.34.0
azure-cli-telemetry==1.0.6
azure-common==1.1.11
azure-containerregistry==1.0.0
azure-core==1.24.2
azure-graphrbac==0.61.1
azure-identity==1.7.0
azure-keyvault==1.0.0a1
azure-mgmt-advisor==1.0.1
azure-mgmt-apimanagement==0.2.0
azure-mgmt-applicationinsights==0.1.1
azure-mgmt-authorization==0.51.1
azure-mgmt-automation==1.0.0
azure-mgmt-batch==5.0.1
azure-mgmt-batchai==2.0.0
azure-mgmt-billing==0.2.0
azure-mgmt-cdn==3.0.0
azure-mgmt-cognitiveservices==3.0.0
azure-mgmt-commerce==1.0.1
azure-mgmt-compute==26.1.0
azure-mgmt-consumption==2.0.0
azure-mgmt-containerinstance==1.4.0
azure-mgmt-containerregistry==9.1.0
azure-mgmt-containerservice==9.1.0
azure-mgmt-core==1.3.0
azure-mgmt-cosmosdb==0.15.0
azure-mgmt-datafactory==2.0.0
azure-mgmt-datalake-analytics==0.6.0
azure-mgmt-datalake-nspkg==2.0.0
azure-mgmt-datalake-store==0.5.0
azure-mgmt-datamigration==1.0.0
azure-mgmt-devspaces==0.1.0
azure-mgmt-devtestlabs==3.0.0
azure-mgmt-dns==8.0.0
azure-mgmt-eventgrid==1.0.0
azure-mgmt-eventhub==2.0.0
azure-mgmt-hanaonazure==0.1.1
azure-mgmt-hdinsight==0.1.0
azure-mgmt-iotcentral==0.1.0
azure-mgmt-iothub==0.7.0
azure-mgmt-iothubprovisioningservices==0.2.0
azure-mgmt-keyvault==1.1.0
azure-mgmt-loganalytics==1.0.0
azure-mgmt-logic==3.0.0
azure-mgmt-machinelearningcompute==0.4.1
azure-mgmt-managedservices==1.0.0
azure-mgmt-managementgroups==0.2.0
azure-mgmt-managementpartner==0.1.1
azure-mgmt-maps==0.1.0
azure-mgmt-marketplaceordering==0.1.0
azure-mgmt-media==1.0.1
azure-mgmt-monitor==3.0.0
azure-mgmt-msi==0.2.0
azure-mgmt-network==19.1.0
azure-mgmt-notificationhubs==2.0.0
azure-mgmt-nspkg==2.0.0
azure-mgmt-policyinsights==0.1.0
azure-mgmt-powerbiembedded==2.0.0
azure-mgmt-privatedns==0.1.0
azure-mgmt-rdbms==1.9.0
azure-mgmt-recoveryservices==0.4.0
azure-mgmt-recoveryservicesbackup==0.6.0
azure-mgmt-redis==13.0.0
azure-mgmt-relay==0.1.0
azure-mgmt-reservations==0.2.1
azure-mgmt-resource==10.2.0
azure-mgmt-scheduler==2.0.0
azure-mgmt-search==8.0.0
azure-mgmt-servicebus==0.5.3
azure-mgmt-servicefabric==0.2.0
azure-mgmt-signalr==0.1.1
azure-mgmt-sql==3.0.1
azure-mgmt-storage==19.0.0
azure-mgmt-subscription==0.2.0
azure-mgmt-trafficmanager==0.50.0
azure-mgmt-web==6.1.0
azure-nspkg==2.0.0
azure-storage==0.36.0
azure-storage-blob==2.1.0
azure-storage-common==2.1.0

azure-cli/focal-updates,now 2.0.81+ds-4ubuntu0.2 all [installed]
python3-azure-cli-core/focal-updates,now 2.0.81+ds-4ubuntu0.2 all [installed,automatic]
python3-azure-cli-telemetry/focal-updates,now 2.0.81+ds-4ubuntu0.2 all [installed,automatic]
python3-azure-cli-testsdk/focal-updates 2.0.81+ds-4ubuntu0.2 all
python3-azure-cli/focal-updates,now 2.0.81+ds-4ubuntu0.2 all [installed,automatic]

siddallgoogle · 2022-09-23T15:07:54Z

So fiddling about with versions i've managed to get my login issue fixed.

This is a similar problem

#688

MS Article on registering repos ... removing of azure-cli apt package

https://learn.microsoft.com/en-us/cli/azure/install-azure-cli-linux?pivots=apt

Packages to obtain

pip install azure-cli-core==2.30
sudo apt-get install azure-cli=2.30.0-1~focal

Make sure that you az logout and then use az login --use-device-code

siddallgoogle · 2022-09-23T15:13:40Z

Also works with latest packages

PIP
azure-cli-core==2.34.0

APT
azure-cli/focal,now 2.40.0-1~focal

soerennielsen · 2024-05-23T12:50:10Z

The problem can be caused by having access to multiple tenants with different accounts and then az login will just pick one of them, and if it picks the wrong one you get the "does not exist in MSAL token cache. Run az login." error.

Solution is simple, specify the tenant at az login. You cannot specify a subscription there, so any attempted resolution above that talks about subscriptions is not going to work.

Fix:
az login --tenant yourtenant.onmicrosoft.com

Fred-sun added medium_priority Medium priority work in In trying to solve, or in working with contributors labels Jul 14, 2022

Fred-sun added the hold The problem holds for a particular reason label Aug 4, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MSAL Token Cache Multiple Token Problem - Enterprise Azure Account #918

MSAL Token Cache Multiple Token Problem - Enterprise Azure Account #918

siddallhotmail commented Jul 12, 2022 •

edited

Loading

Fred-sun commented Jul 14, 2022

Fred-sun commented Aug 4, 2022

siddallgoogle commented Aug 4, 2022

siddallgoogle commented Aug 31, 2022

Fred-sun commented Sep 1, 2022

siddallgoogle commented Sep 1, 2022 •

edited

Loading

Fred-sun commented Sep 1, 2022

siddallgoogle commented Sep 1, 2022 •

edited

Loading

Fred-sun commented Sep 1, 2022

siddallgoogle commented Sep 1, 2022

Fred-sun commented Sep 1, 2022

siddallgoogle commented Sep 1, 2022 •

edited

Loading

Fred-sun commented Sep 7, 2022

siddallgoogle commented Sep 7, 2022 •

edited

Loading

Cyclonit commented Sep 8, 2022 •

edited

Loading

siddallgoogle commented Sep 23, 2022 •

edited

Loading

siddallgoogle commented Sep 23, 2022

siddallgoogle commented Sep 23, 2022

soerennielsen commented May 23, 2024

MSAL Token Cache Multiple Token Problem - Enterprise Azure Account #918

MSAL Token Cache Multiple Token Problem - Enterprise Azure Account #918

Comments

siddallhotmail commented Jul 12, 2022 • edited Loading

SUMMARY

ISSUE TYPE

COMPONENT NAME

ANSIBLE VERSION

COLLECTION VERSION

CONFIGURATION

OS / ENVIRONMENT

STEPS TO REPRODUCE

EXPECTED RESULTS

Fred-sun commented Jul 14, 2022

Fred-sun commented Aug 4, 2022

siddallgoogle commented Aug 4, 2022

siddallgoogle commented Aug 31, 2022

Fred-sun commented Sep 1, 2022

siddallgoogle commented Sep 1, 2022 • edited Loading

Fred-sun commented Sep 1, 2022

siddallgoogle commented Sep 1, 2022 • edited Loading

Fred-sun commented Sep 1, 2022

siddallgoogle commented Sep 1, 2022

Fred-sun commented Sep 1, 2022

siddallgoogle commented Sep 1, 2022 • edited Loading

Fred-sun commented Sep 7, 2022

siddallgoogle commented Sep 7, 2022 • edited Loading

Cyclonit commented Sep 8, 2022 • edited Loading

siddallgoogle commented Sep 23, 2022 • edited Loading

siddallgoogle commented Sep 23, 2022

siddallgoogle commented Sep 23, 2022

soerennielsen commented May 23, 2024

siddallhotmail commented Jul 12, 2022 •

edited

Loading

siddallgoogle commented Sep 1, 2022 •

edited

Loading

siddallgoogle commented Sep 1, 2022 •

edited

Loading

siddallgoogle commented Sep 1, 2022 •

edited

Loading

siddallgoogle commented Sep 7, 2022 •

edited

Loading

Cyclonit commented Sep 8, 2022 •

edited

Loading

siddallgoogle commented Sep 23, 2022 •

edited

Loading