Additional Readings

• Keycloak Cluster Deployment with Vagrant, VMs, and JDBC-PING: A step-by-step guide on deploying and managing a cluster of three Keycloak instances using Vagrant.
• Keycloak Cluster Configuration with Docker Compose and JDBC-PING: Follow this guide to launch a cluster of three Keycloak Docker containers utilizing JDBC-PING.
• Keycloak Cluster Setup with Docker Compose and UDP: Learn how to set up a Keycloak cluster of three Docker containers using UDP for distributed caching.

Prerequisites

To follow this tutorial, ensure that you have Docker Desktop installed on your machine. Instructions for installation on Mac, Windows, and Linux can be found in the Docker documentation.

Understanding Distributed Caching in Keycloak

Simply put, Keycloak is an open-source tool for identity and access management (IAM) that provides functionalities for managing user identities, authentication, and authorization for various applications.

The current distributed cache within Keycloak is built on Infinispan, an open-source, distributed, in-memory key-value store that emphasizes scalability, availability, and fault tolerance.

When you initiate Keycloak in development mode using the "start-dev" command, distributed caching is entirely deactivated. Conversely, when starting Keycloak in production mode with the "start" command, caching is activated, enabling the discovery of all Keycloak instances within your network. By default, caching employs a UDP transport mechanism, which allows instances to be discovered via IP multicast transport.

Nevertheless, alternatives to UDP, such as JDBC-PING, exist, enabling distributed caching through a JDBC-compatible database for storing and sharing cached data among multiple instances. The subsequent sections will guide you through the configuration process.

Configuring JDBC-PING in Infinispan

As mentioned earlier, the distributed cache relies on Infinispan. Let’s delve into the configuration process.

Run the following command in your terminal. This command will launch a Docker container running the keycloak image in interactive mode, display the contents of the cache-ispn.xml file using the cat command, and subsequently remove the container after it halts.

docker run -it --rm --entrypoint=cat quay.io/keycloak/keycloak:22.0.1 opt/keycloak/conf/cache-ispn.xml

You should see XML content similar to the following:

<?xml version="1.0" encoding="UTF-8"?>

<!--

~ Copyright 2019 Red Hat, Inc. and/or its affiliates

~ and other contributors as indicated by the @author tags.

~

~ Licensed under the Apache License, Version 2.0 (the "License");

~ you may not use this file except in compliance with the License.

~ You may obtain a copy of the License at

~

~ http://www.apache.org/licenses/LICENSE-2.0

~

~ Unless required by applicable law or agreed to in writing, software

~ distributed under the License is distributed on an "AS IS" BASIS,

~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

~ See the License for the specific language governing permissions and

~ limitations under the License.

—>

<infinispan xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:schemaLocation="urn:infinispan:config:14.0 http://www.infinispan.org/schemas/infinispan-config-14.0.xsd"

xmlns="urn:infinispan:config:14.0">

<cache-container name="keycloak">

<transport lock-timeout="60000"/>

<local-cache name="realms" simple-cache="true">

<encoding>

<key media-type="application/x-java-object"/>

<value media-type="application/x-java-object"/>

</encoding>

<memory max-count="10000"/>

</local-cache>

...

</cache-container>

</infinispan>

To implement JDBC-PING, we need to alter the Infinispan configuration. First, create a file named cache-ispn-jdbc-ping.xml and copy the content from cache-ispn.xml into this new file.

Next, we will integrate the JDBC-PING configuration. In this instance, we will utilize a Postgres database. Add the bold lines within the <infinispan> element and adjust the <transport> element to indicate the postgres-jdbc-ping-tcp stack.

<infinispan xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:schemaLocation="urn:infinispan:config:14.0 http://www.infinispan.org/schemas/infinispan-config-14.0.xsd"

xmlns="urn:infinispan:config:14.0">

<jgroups>

<stack name="postgres-jdbc-ping-tcp" extends="tcp">

<TCP external_addr="${env.JGROUPS_DISCOVERY_EXTERNAL_IP:127.0.0.1}"/>

<JDBC_PING connection_driver="org.postgresql.Driver"

connection_username="${env.KC_DB_USERNAME}"

connection_password="${env.KC_DB_PASSWORD}"

connection_url="jdbc:postgresql://${env.KC_DB_URL_HOST}:${env.KC_DB_URL_PORT:5432}/${env.KC_DB_URL_DATABASE}"

initialize_sql="CREATE SCHEMA IF NOT EXISTS ${env.KC_DB_SCHEMA:public}; CREATE TABLE IF NOT EXISTS ${env.KC_DB_SCHEMA:public}.JGROUPSPING (own_addr varchar(200) NOT NULL, cluster_name varchar(200) NOT NULL, bind_addr varchar(200) NOT NULL, updated timestamp default current_timestamp, ping_data BYTEA, constraint PK_JGROUPSPING PRIMARY KEY (own_addr, cluster_name));"

insert_single_sql="INSERT INTO ${env.KC_DB_SCHEMA:public}.JGROUPSPING (own_addr, cluster_name, bind_addr, updated, ping_data) values (?, ?, '${env.JGROUPS_DISCOVERY_EXTERNAL_IP:127.0.0.1}', NOW(), ?);"

delete_single_sql="DELETE FROM ${env.KC_DB_SCHEMA:public}.JGROUPSPING WHERE own_addr=? AND cluster_name=?;"

select_all_pingdata_sql="SELECT ping_data, own_addr, cluster_name FROM ${env.KC_DB_SCHEMA:public}.JGROUPSPING WHERE cluster_name=?"

info_writer_sleep_time="500"

remove_all_data_on_view_change="true"

stack.combine="REPLACE"

stack.position="MPING"/>

</stack>

</jgroups>

<cache-container name="keycloak">

<transport lock-timeout="60000" stack="postgres-jdbc-ping-tcp"/>

...

</cache-container>

</infinispan>

Within this configuration, the <jgroups> tag is employed to set up JGroups, a library that facilitates reliable group communication between nodes in a distributed environment.

The <stack> tag specifies a stack named postgres-jdbc-ping-tcp that extends the tcp stack, which is a pre-defined JGroups stack enabling TCP-based communication between nodes.

Inside the <stack> tag, you will find two tags configuring JGroups behavior for this stack: - <TCP>: Configures the transport protocol for JGroups network communication and specifies the external IP address for the discovery process. - <JDBC_PING>: Configures JGroups to utilize a JDBC-based discovery protocol, defining necessary JDBC connection parameters and SQL statements for initializing the schema and ping data management.

Starting a Keycloak Cluster

Let’s create a Docker network named keycloak-net. In your terminal, execute the following command:

docker network create keycloak-net

Next, we will launch a Postgres Docker container. The docker run command will start a new Docker container in detached mode (-d) with the name postgres.

docker run -d --rm --name postgres

-e POSTGRES_DB=keycloak

-e POSTGRES_USER=keycloak

-e POSTGRES_PASSWORD=password

—network keycloak-net

postgres:15.3

Below are the commands to initiate the Keycloak instances:

Note: Ensure you are in the directory where the cache-ispn-jdbc-ping.xml file is saved before starting the two Keycloak instances.

To start the keycloak-1 Docker container, run:

docker run --rm --name keycloak-1 -p 8080:8080

-e KEYCLOAK_ADMIN=admin

-e KEYCLOAK_ADMIN_PASSWORD=admin

-e KC_DB=postgres

-e KC_DB_URL_HOST=postgres

-e KC_DB_URL_DATABASE=keycloak

-e KC_DB_USERNAME=keycloak

-e KC_DB_PASSWORD=password

-e JGROUPS_DISCOVERY_EXTERNAL_IP=keycloak-1

-e KC_CACHE_CONFIG_FILE=cache-ispn-jdbc-ping.xml

-v ${PWD}/cache-ispn-jdbc-ping.xml:/opt/keycloak/conf/cache-ispn-jdbc-ping.xml

—network keycloak-net

quay.io/keycloak/keycloak:22.0.1 start-dev

In another terminal, execute the command below to start the keycloak-2 Docker container:

docker run --rm --name keycloak-2 -p 8081:8080

-e KEYCLOAK_ADMIN=admin

-e KEYCLOAK_ADMIN_PASSWORD=admin

-e KC_DB=postgres

-e KC_DB_URL_HOST=postgres

-e KC_DB_URL_DATABASE=keycloak

-e KC_DB_USERNAME=keycloak

-e KC_DB_PASSWORD=password

-e JGROUPS_DISCOVERY_EXTERNAL_IP=keycloak-2

-e KC_CACHE_CONFIG_FILE=cache-ispn-jdbc-ping.xml

-v ${PWD}/cache-ispn-jdbc-ping.xml:/opt/keycloak/conf/cache-ispn-jdbc-ping.xml

—network keycloak-net

quay.io/keycloak/keycloak:22.0.1 start-dev

As seen in the commands above, ensure to provide the Docker container name for the JGROUPS_DISCOVERY_EXTERNAL_IP environment variable. We are also creating a volume between the host machine and the Docker containers so that the cache-ispn-jdbc-ping.xml file is accessible inside the container.

Checking the logs will reveal entries related to the Infinispan cluster, indicating a successful setup!

Verifying Distributed Caching via Keycloak UI

Open two different browsers, such as Chrome and Safari or Chrome and Incognito Chrome.

In one browser, navigate to http://localhost:8080/admin/ and in the other, go to http://localhost:8081/admin/.

Log in with admin as both the username and password.

After logging in, click on Sessions in the left menu. You should observe that admin has two active sessions.

Inspecting the JGROUPSPING Table

To view the JGROUPSPING table in Postgres where the ping data is stored, access the Postgres shell by entering its Docker container:

docker exec -it postgres psql -U keycloak -d keycloak

Once inside the Postgres shell, execute the following SQL command:

select * from JGROUPSPING;

You should see output resembling the following:

Shutting Down the Keycloak Cluster

To shut down the Keycloak cluster we established, go to the terminals where the Keycloak instances are running and press Ctrl+C. Then, execute the following commands to stop the Postgres Docker container and remove the keycloak-net network:

docker rm -fv postgres

docker network rm keycloak-net

Conclusion

That's a wrap! In this article, we successfully set up a Keycloak cluster utilizing JDBC-PING for distributed caching. While this example featured a Postgres database, Keycloak also supports other databases like MySQL, MariaDB, and Microsoft SQL Server.

I've been maintaining a Docker image named keycloak-clustered on Docker Hub, which offers out-of-the-box clustering for Keycloak instances using JDBC-PING for all supported databases. The source code is available in the keycloak-clustered GitHub repository.

Support and Engagement

If you found this article useful and would like to show your support, please consider the following actions: - Engage by clapping, highlighting, and replying to my story. I'm happy to answer any questions! - Share this story on Social Media. - Follow me on: Medium | LinkedIn | Twitter. - Subscribe to my newsletter to stay updated on my latest posts.