ADMIN REST API - searching and listing clients, users - discrepancies

• multiple method naming UserResource, ClientResource -> list(), findAll()
• User
  -- get(UUID) -> return UserResource
  -- search(name) -> return UserRepresentation

I concur with @stianst to have single search method allowing multiple input parameters and potentially multiple return options UserResource/UserRepresentation

Good points. I would like to share proposals and additional notes:

Current API is not really stateless: Support create-or-update and maybe object-versions

With PUT for create-or-update it should be irrelevant if the resource exists. If a tool want to make sure it is the one which creates the object, object version should be supported and sth. like PUT ...?revision=0 should fail with conflict response, if the resource exists

Clients need to understand semantics

Yes, defining a client with roles should be possible with one call. It is also important that a single client role can be addressed for assignments without the need to query anything about the client which defines the role.

Default values clash with the declarative approach: Attributes not provided should mean "use default" - not replace with default

Having defaults applied is necessary as it makes an API easier to learn and the product easier to use. The problem is that omitting a value on creation leads to that null value being replaced with defaults in the resource representation (GET returns something else than PUT or POSTed).
Solution: An attribute should just stay null and the defaults are applied internally. Setting it to null or omitting it on creation has the meaning "apply defaults, whatever they currently are". Whereas setting a value means "use exactly this value", even if default was changed by confirguration or by a Keycloak update.
Example from thread: When a new users required_actions are not provided on create, a GET should just return the user without this field. Defaults should still apply internally. Of cause it would be good if defaults can be resolved also via API. Example GET ... defaults/user could return
{..., required_actions ["TERMS_AND_CONDITIONS", "UPDATE_PASSWORD"], ...}

It is not possible to have declaratively configured resources immutable from other interfaces: Access control problem

If there is an easy to use model to make resources unmodifieable for admins working at admin UI, this could be solved. Fine grained permissions are hard to apply. Resource Ownership, assigning a resource to a edit and view security groups could help.
You would create security group "Operator controlled" and add it to the list of security groups who can edit the object. The operator user would be assigned to the security group. It's the only one who can update or delete all other can view (if they have the realm role for it).

Also PUT should set everything not included in JSON representation to null.
"Not set" vs. "Explicitly set" to null should mean the same. it should not make a difference in behavior if a setting is optional and not provided or if someone has reset it from a non-null value.
Example: These two requests should have the same result:

PUT /realms/myrealm
{ 
  displayName: null
}

PUT /realms/myrealm
{ 
}

Furthermore a setting which is represented and stored as null sometimes means that a default is used. It is important that this default value not just magically replaces the null value at internal storage. When null is set by am API call it must be returned as null, while a default can be applied at runtime.

@stianst I fully agree, it is very unexpected, and I was surprised that I cannot set null to 'admin url' of a client (had to use empty string) so now we have some discrepancy - newly created clients have this property = null, while existing ones are 'patched' with empty string.

Thank you we have the same problem with the pkce setting of a client. The pkce method was set to a "S256" and can't be changed back to null.
This case could also be prevented if there was an explicit option to disable pkce instead of requiring a null value that can't be set once it was changed!

Having empty strings or null values as defaults is generally problematic. When using terraform for instance, lots of variables for clients are set via map. Terraform goes into an infinite reconciliation loop when setting keys with empty values, and not setting these keys will not set the values to default.
Is it feasable to have some constant like "DEFAULT" set instead of empty strings to make management via terraform and crossplane simpler?

Follow up to this, in the ticket I linked the team has replied that what I'm trying to do is not possible with the current API.

My use case is, I'm writing an application where I would like to present a simple single/unified search bar, so the user can look up KC Users from a particular subset denoted by a custom attribute on their KC account. When querying all KC users (not just the subset), free-text search (search query param) works well for this, and I can set a max param to return only enough results to populate an autocomplete drop-down so the query completes reasonably quickly. But limiting the query to this subset by custom-attribute under the current API would mean either making multiple individual field queries (firstName / lastName / email) and de-duplicating the results on the client side, or making one big search query and picking out the subset on the client.

Both workarounds would perform poorly, and involve sending a bunch of unused data, possibly providing inconsistent results. Supporting search and q at the same time (or something analogous to this) would provide a much more elegant solution.

Are you referring to these API docs? It seems to cover your cases pretty well, though I agree that it would be great to include examples of how to actually make a request from the command line. I would also suggest links from the REST API page to any known/popular client libraries, like this one for Node applications.

Yes, I am referring to those docs. The docs you mention are the only docs I could find, other than external blog entries, and what those docs are, is effectively a dictionary, and not a manual. It feels like it's generated from the internal code structure of the API itself, and only covers every "leaf" endpoint and none of the intermediate ones. I'm sure if you do nothing but write REST calls all day, it makes some sense.

For example, I said there's no documented way to list all clients in a realm. Google AI (famously wrong a lot of the time) somehow figured it out:

But the only listing in the docs is:

GET /admin/realms/{realm}/clients/{client-uuid}

Get representation of the client

Which seems to be talking about a client (singular) and would seem to require you have the client UUID. It doesn't go into much detail about what "representation of the client" entails, either. Is this what I'm looking for? Only way to know is try it out.

I mentioned not being able to search for a client, as mentioned in this stack overflow answer

And that's done by putting some kind of matching query string in the URL with a question mark, while still actually making a POST request. Is this a hack? A feature? A standard SAML thing? Is it going to go away? The docs are silent on this method.

Finally, like I said: there's an api endpoint that covers creating a client. In my case, I was trying to create a SAML one.

That was the docs I found. The link to "ClientRepresentation" just gives you a bunch of fields (all optional). Nothing that tells you which values are valid for a given client type. Nothing anywhere to tell you what field determines what type of client.

At best, if I wanted to create a new SAML client, I could create one via the web UI, and blindly copy all the values, which feels like a bad idea for a security product.

And as mentioned, there was zero mention of how to actually interact with the API, how to get an initial auth token, which HTTP headers were required, which permissions are required.

I found this blog helpful for that, but this feels like something that should be on the main site.

I agree that those API docs are not very helpful to users who might not have a lot of experience using REST APIs, for a couple of reasons. They do appear to be automatically generated from code annotations, which can be a good thing -- they're much less likely to "drift" if the code changes and the author forgets to update the docs, for example -- but they're pretty hard to use since they're only loosely grouped/sorted, all on one big page, with no index of individual endpoints. The effort to build a comprehensive OpenAPI specification (previously "Swagger") could help with this, because there are a number of tools out there to build a better docs page out of a Swagger spec.

As an example, try searching in your browser for the exact phrase "Get clients belonging to the realm". You would probably be surprised to find exactly what you originally asked for:

I don't blame you for not finding it though, even if you know the endpoint would be called GET /admin/realms/{realm}/clients, there are 59 hits on the page for that string. If all you know is what you're trying to accomplish -- list all the clients -- search for individual keywords or short phrases gets you nowhere, so you're stuck search the entire site, which works out to... let me check... 298 printed pages! No one can fault you for failing to find the needle in that haystack.

So, in short:

• usage examples (curl commands)
• table of contents/index based on route hierarchy
• better search based on use case/keywords
• as you point out, every field of every structure should have a good description, ideally with meaningful data examples.

Okay, I missed it! In my defense "list all the clients" was well after things like:

GET /admin/realms/{realm}/clients/{client-uuid}/evaluate-scopes/scope-mappings/{roleContainerId}/granted

Get effective scope mapping of all roles of particular role container, which this client is defacto allowed to have in the accessToken issued for him.

Description

This contains scope mappings, which this client has directly, as well as scope mappings, which are granted to all client scopes, which are linked with this client.

Helpful note here: it would be useful if the menu pictured at the left here:

Were made dynamic, and actually listed the "use" of each of the endpoints.

But yes. A short "Getting started with the API" document would go a long way here. Especially considering most people will want this to do things like migratre from another system so will want to use it to mass-create users...and then never touch it again.

Agreed. This is my biggest issue. A GraphQL API would be very much appreciated for this sort of use case.

I would also agree, we also developed an application, where this is my biggest performance problem. It needs me second by about 50 groups and 150 users and this on starting the application. This is not really a good user experience. Also I think APIs should not be needed to be explained. New users of keycloak need easy examples. A good UX would help to grow the keycloak community faster.

We've created a topic on pagination: #41036