scim individual users not mapping correct groups when generating tokens

[thelangley]

What version of UAA are you running?

77.16.0

How are you deploying the UAA?

I am deploying the UAA

• using a bosh release I downloaded from bosh.io
• using cf-deployment v43.6.0

What did you do?

Using ops files, insert sections to create/import specific accounts in LDAP to give specific permissions when used.

Here's an extract from a uaa.yml

scim:
  users:
  - service_account1||service_account1|||cloud_controller.admin,scim.read,scim.write,doppler.firehose,routing.router_groups.read,routing.router_groups.write,routing.routes.read,routing.routes.write|ldap
  - service_account2||service_account2|||cloud_controller.admin_read_only,scim.read,doppler.firehose|ldap

What did you expect to see? What goal are you trying to achieve with the UAA?

On uaa 77.15.0 (the previous release), the correct groups are found when logging in.

[2024-09-30T06:19:13.093476Z] uaa - 11 [https-jsse-nio-8443-exec-4] - [redacted,redacted] .... INFO --- Audit: TokenIssuedEvent ('["openid","routing.router_groups.write","scim.read","cloud_controller.admin","uaa.user","routing.router_groups.read","cloud_controller.read","password.write","cloud_controller.write","doppler.firehose","scim.write"]'): principal=redacted, origin=[client=cf, user=service_account1], identityZoneId=[uaa]

This is the correct behaviour and results in service_account1 being able to see all the orgs/spaces, etc in CF as they have the cloud_controller.admin group assigned. Same goes for service_account2, it has read only access to all orgs/spaces. Logs not shared for service_account2.

What did you see instead?

On uaa 77.16.0, different groups are allocated, not the ones specified in the uaa.yml

[2024-09-30T06:13:13.470329Z] uaa - 12 [https-jsse-nio-8443-exec-4] - [redacted,redacted] .... INFO --- Audit: TokenIssuedEvent ('["openid","uaa.user","cloud_controller.read","password.write","cloud_controller.write"]'): principal=redacted, origin=[client=cf, user=service_account1], identityZoneId=[uaa]

This results in the user not being able to see all the orgs and spaces that we could see in previous versions. I assume because the cloud_controller.admin group is not being used in the token generation.

[strehle]

you create tokens via cf login ?

[thelangley]

yeah it's just doing cf login. those logs are from /var/vcap/sys/logs/uaa/uaa.log

[strehle]

ok, then we can check if #3033 is the causing PR for your change.

do a "cf login xxx --origin ldap" or set defaultIdentityProvider: ldap into uaa.yaml

Because I think you problem is the mix of uaa / ldap origin... the groups are now assigned to ldap origin but the token creation is done with origin uaa ( I guess ) and then it would explain the missing groups. / authorities in your token

[thelangley]

Using cf cli v8.7.10, I've done cf login xxx --origin ldap and the uaa logs still appear to say that the groups allocated are incorrect. I also cannot see the orgs/spaces that someone with cloud_controller.admin should be able to see.

[strehle]

are you familiar with uaac ?

If yes, please call: uaac group get cloud_controller.admin

[thelangley]

Apologies for lateness. I'm not well at the moment so working odd hours.

uaac group get cloud_controller.admin
  id: redacted
  meta
    version: 0
    created: 2024-08-27T07:37:07.256Z
    lastmodified: 2024-08-27T07:37:07.256Z
  members:
  -
    origin: ldap
    type: USER
    value: redacted-73fc8d54b811
  -
    origin: ldap
    type: USER
    value: redacted-5eeb9bafb311
  -
    origin: uaa
    type: USER
    value: redacted-559c70fb1e11
  -
    origin: uaa
    type: USER
    value: redacted-6926f2e3a411
  -
    origin: uaa
    type: USER
    value: redacted-704ef09b2111
  -
    origin: uaa
    type: USER
    value: redacted-7b2400599011
  -
    origin: uaa
    type: USER
    value: redacted-0ed5c4754011
  -
    origin: uaa
    type: USER
    value: redacted-3840d0e7b411
  schemas: urn:scim:schemas:core:1.0
  displayname: cloud_controller.admin
  zoneid: uaa

A little thing extra. One of the oddities with this feature is that if you add the user to an LDAP group, and you configure UAA to say that any members of LDAP group are a member of the cloud_controller_admin group, it honours this. This functionality is the same between uaa 77.15 and 77.16.

It only appears to be named LDAP accounts with specific permissions provisioned directly to the account which are not honoured.

[strehle]

Thanks. but your token is created with origin uaa ?

So I have to align with the others about a revert / partial revert of #3033

Are you able to consume a uaa-release without integrated into cf-deployment ? because then we can provide it to you and you can verify that it works again

[thelangley]

That'd be super thanks.

I'd need it to be available on bosh.io/releases or somewhere else then I can use an ops file to consume the new release. it'd need to be part of cf-deployment but I can merge it in to the manifest without anyone upstream changing cf-deployment.yml

I should be able to test pretty quick tho if that's helpful

[peterhaochen47]

but your token is created with origin uaa ?

Wouldn't their token be created with origin ldap, since they said: "I've done cf login xxx --origin ldap"?

But then, maybe cf cli doesn't actually use that --origin ldap information, because I think cf cli is just doing a UAA password grant, which does not take origin as a param so UAA is just treating origin=uaa and origin=ldap as the same pool of users (for password grant).

[peterhaochen47]

Ok here is my local reproduction.
UAA 3069 repro.txt

TLDR: The interesting difference is that in the latest UAA, the ldap user's groups* bootstrapped by UAA.yml get wiped away after a login (via password grant). I don't yet know the connection between this behavior and #3033.

*the ldap user's groups as returned by /Users endpoint (aka uaac user get).

[strehle]

Ok here is my local reproduction. UAA 3069 repro.txt

TLDR: The interesting difference is that in the latest UAA, the ldap user's groups* bootstrapped by UAA.yml get wiped away after a login (via password grant). I don't yet know the connection between this behavior and #3033.

*the ldap user's groups as returned by /Users endpoint (aka uaac user get).

Thanks a lot because this helped me to understand howto test and with it I created #3072 which now returns back the scopes (groups from scim setup).

I splitted the createUser into the case where user is created from uaa.yml and from shadowUser so that we dont revert everything from #3033 but if you bootstrap users withing uaa.yml it fixes the problem from here.

[peterhaochen47]

@thelangley What I would recommend to grant external LDAP users group is via external group mapping rather than pre-creating those shadow LDAP users (which I am not sure is a supported / intended feature). Would the former work for you?

[thelangley]

It isn't a core requirement to create shadow LDAP users. But it is a core requirement to have an LDAP user who is not a member of any LDAP groups to have a certain level of access. We do want to map these particular LDAP users to groups (like cloud_controller.admin) so they have permissions to do CF things. These singular accounts are for robots to only do CF things as part of pipeline tasks so don't need to be a part of any LDAP group.

Can you perform external group mapping but for instead of groups, map specific users in LDAP who are not in any LDAP groups?

We already use external group mappings for real people who are members of LDAP groups. Permissions continue to work fine for them between 77.15 and 77.16

[peterhaochen47]

@strehle I continued to investigate why #3033 has resulted in my local reproduction and my conclusion is that the behavior described in the local reproduction (previously-bootstrapped group memberships of an external user get wiped away after an external login by the said user) and this issue are the original intended design of UAA.

UAA has always intended to "refresh" the external user's group membership by clearing out the user's previous group membership upon an ExternalGroupAuthorizationEvent event. Before #3033, this intended behavior was not working because UAA is querying the external user's previous group membership with an origin filter; however, we now know that before #3033, the origin values in the group_membership table were all incorrect. Now #3033 fixes that and this intended behavior is working again.

And I like this stricter, more deterministic behavior: when you map LDAP group some-ldap-group to cloud_controller.admin, then ONLY the LDAP users in some-ldap-group at the time of login will get cloud_controller.admin.

[peterhaochen47]

@thelangley

Can you perform external group mapping but for instead of groups, map specific users in LDAP who are not in any LDAP groups?

Currently, I don't see how to do that.

The simplest way to accomplish your goal would be to add your "singular accounts...for robots to only do CF things as part of pipeline tasks" to those LDAP groups (that are mapped to CF scopes/groups) just like the human accounts, since these service accounts and human accounts need to have similar power in CF.

If you don't wanna add the service accounts to the existing LDAP groups (because maybe those groups are for human), you could add a new LDAP group just for service accounts. And then add an additional external group mapping to UAA to map that new LDAP group.

Another option is to not use LDAP for service accounts. You can bootstrap some UAA clients.

[thelangley]

Security want us to use LDAP accounts for service accounts so we'll need to continue to start using them. Service accounts and puny humans cannot co-exist in the same LDAP groups.

So it looks like we'll need to work around this new uaa functionality by creating new LDAP groups. I work in Enterprise so it'll take ages to get this done.

We've used this functionality for a good few years now and it's the first time that we've encountered this feature.

I'll kick off the process to create new LDAP groups, add the service accounts into these LDAP groups then map said groups to the uaa groups via extra external group mappings.

[strehle]

@thelangley if you have admin client from UAA, then you can do workaround and create the groups manually and assign them manually "uaa" origin, then UAA 77.16.0 behaves like 77.15.0

e.g. use cloud_controller.admin as example
you see with uaac group get cloud_controller.admin
entries with orogin LDAP.

Either do not setup with ldap in uaa.yml or remove them first,, with

uaac member delete cloud_controller.admin service_account1
uaac member delete cloud_controller.admin service_account2

If you omit group assignment in uaa.yml, then you can directly perform

uaac member add cloud_controller.admin service_account1
uaac member add cloud_controller.admin service_account2

Result of

uaac group get cloud_controller.admin

should then show only origins with UAA and then your cf login will retrieve your expected scopes, e.g. cloud_controller.admin

---

the example can be done for all your other groups / scope you have setup in your uaa.yml with scim.users section.

This possibility is not well documented, see
#2190

Finally. we have discussed this topic but we wont use #3072 to revert the boot strapping behavior. The behavior is treated as bug and we agreed to stay with the fix before.
Means: your final solution should be LDAP groups but as long as you need a workaround use the manual assignment