How to secure Azure credentials for OpenShift Lightspeed

Red Hat OpenShift Lightspeed is a generative AI-based virtual assistant integrated into the Red Hat OpenShift web console. Red Hat OpenShift Lightspeed answers questions related to OpenShift and its layered offerings, using an English natural-language interface. Using our extensive experience in OpenShift and mission-critical applications, Red Hat OpenShift Lightspeed assists with troubleshooting and investigating cluster resources.

Key benefits of Red Hat OpenShift Lightspeed include:

1. It is easy to use Red Hat OpenShift Lightspeed in your working environment in the OpenShift web console.
2. Get assistance with troubleshooting and investigating cluster resources in Red Hat OpenShift to increase productivity.
3. Generative AI that works with different large language models (LLMs), such as LLMs hosted in vLLM on Red Hat OpenShift AI, IBM WatsonX, Microsoft Azure OpenAI, and OpenAI.

This article will demonstrate how Red Hat OpenShift Lightspeed automates the process of connecting to Azure AI services using dynamic credentials. You will also learn about protecting main application secrets while enabling seamless LLM authentication for ephemeral setups.

The OpenShift Lightspeed demo environment

The Red Hat OpenShift Lightspeed demo environment we have created needs to connect to an LLM backend which can be running locally or on Red Hat OpenShift AI and vLLM or on cloud-based AI services. In this use case, we will focus on Microsoft Azure OpenAI Services. 

There is a resource group rg-ai and a resource llm-gpt4-lightspeed with an endpoint using Azure AI services. We need to create programmatically dynamic tokens to access these end points and delete them once the environment is deleted. This process repeats many times, depending on who orders the lab from the demo platform catalogs. This helps OpenShift Lightspeed securely connect to Azure AI services every time a new demo environment is launched.

Use case details

The demo environment needs to be live for some time and connected to Azure AI services (gpt-4o service). Once the person who orders the demo environment completes the demo, it's deleted to save resources. This should also delete the Azure credentials used in this environment so that it's not being used by anyone else. The reason to delete this is because the secrets created in OpenShift for OpenShift Lightspeed service are visible to the admin. Hence, the credentials need to be deleted upon deletion of each environment. 

Next time someone else can order or create the same demo (we have such demo environment creation automated), but they will need to use different Azure credentials since the previous demo environment was deleted and those credentials need to be deleted.

OpenShift Lightspeed AI solution design

The best way to provide a mechanism where the OpenShift Lightspeed service can authenticate many instances of the service against the Azure AI cognitive services is to use an Azure app registration that Red Hat Ansible Automation Platform uses for impersonation to authenticate against Azure and use the Azure app service principal to access the Azure AI cognitive services. We use the main app service principal (from the vault during the environment creation) to create a child app service principal for this specific environment only.

Figure 1 depicts the high-level architecture of the OpenShift Lightspeed AI solution.

The following steps describe how this process works:

1. Use the master app registration to create a temporary app registration for the new instance. 
2. Use the master app registration to add the new temporary app registration to an Azure Ad Group added to the AI cognitive service resource with the required Azure role-based access control (RBAC) permissions. 
3. Then you can use the temporary app registration to authenticate to Azure and use that bearer token for connecting to the AI service for the new instance.

2 Azure credentials

Azure credentials are used to authenticate and authorize access to Azure resources. The two main types of credentials are:

1. Azure app registration:
   • An app registration in Azure represents an application (e.g., a web app, mobile app, or API) that needs to interact with Azure services.
   • It provides a unique identity for the application, including a client ID and tenant ID.
2. Service principal (SP):
   • A service principal is an identity created for an application to access Azure resources. It is essentially a user account for non-human entities like apps or services.
   • It includes a client ID, client secret (or certificate), and tenant ID for authentication.

Steps to connect to Azure AI services

Complete the following steps to connect to Azure AI services:

1. Generate access token for the main app:
   • Requires the tenant ID, main app client ID, and its secret.
2. Create a child app registration:
   • Use the main app ID and its token to register the child app.
   • Capture the child app ID and its token for connecting to Azure AI services.
3. Create a service principal for the child app:
   • This enables the child app to authenticate and access resources.
4. Add the child app to an Azure AD group:
   • Ensure the group (e.g., "rg-ai") has access to Azure AI cognitive services.
5. Generate an access token for the child app:
   • This token will be used to connect to Azure AI services.
6. Connect to Azure AI cognitive services:
   • Use the child app ID, its token, and the tenant ID to establish the connection.

Automation with Ansible

An Ansible Playbook will automate the previous steps. This playbook also demonstrates how to integrate Azure credentials with the application, OpenShift Lightspeed. A separate script is also available to remove the child app registration and service principal when no longer needed.

The following is an Ansible Playbook that uses REST API services to create the necessary credentials (like app registration and service principal) and connect to Azure AI Cognitive Services. This playbook demonstrates a different approach compared to the one mentioned earlier. Instead of tying the steps to a specific application, it focuses on automating the process of creating and managing Azure credentials.

# Dynamic Name Update:
    # Purpose:
    # - Creates a "child" Azure AD app registration and service principal dynamically.
    # - Interacts with Azure AI Cognitive Services using this child app.
    # - Cleans up the resources (child app and service principal) after completing the interaction.
    # - Initially, the app is created with a placeholder name.
    # - After retrieving the appId, the last 6 characters are extracted using child_app_id[-6:].
    # - The app's display name is updated with this suffix.
    # Steps Added:
    # - Update Child App Name with App ID Suffix: Generates the name with the appId suffix.
    # - Update Child App Display Name: Updates the app's name in Azure using the Microsoft Graph API.
---
- name: Automate Azure AI Cognitive Services with Ansible
  hosts: localhost
  gather_facts: no
  vars:
    azure_tenant_id: "64dc69e4-d083-49fc-9569-ebece1dd1408"
    master_client_id: "0837ad92-f450-451e-a82a-8b622d6f417b"
    master_client_secret: "RGv8Q~Wc~2C4i~zIvoymDpXf~t21pMHr8H2dIary"
    azure_graph_scope: "https://graph.microsoft.com/.default"
    cognitive_services_scope: "https://cognitiveservices.azure.com/.default"
    azure_ad_group_id: "{{ group_id | default('') }}"  # Optional fallback if undefined.
  tasks:
    - name: Validate Required Variables
      fail:
        msg: >
          Required variables are missing:
          azure_tenant_id, master_client_id, master_client_secret, or azure_ad_group_id.
      when: azure_tenant_id == "" or master_client_id == "" or master_client_secret == "" or azure_ad_group_id == ""
    - name: Get Token for Master App
      uri:
        url: "https://login.microsoftonline.com/{{ azure_tenant_id }}/oauth2/v2.0/token"
        method: POST
        headers:
          Content-Type: "application/x-www-form-urlencoded"
        body:
          grant_type: client_credentials
          client_id: "{{ master_client_id }}"
          client_secret: "{{ master_client_secret }}"
          scope: "{{ azure_graph_scope }}"
        body_format: form-urlencoded
        return_content: yes
      register: master_token_response
      failed_when: master_token_response.status != 200
    - name: Set Master Access Token
      set_fact:
        master_access_token: "{{ master_token_response.json.access_token }}"
    - name: Recreate Child App Registration
      uri:
        url: "https://graph.microsoft.com/v1.0/applications"
        method: POST
        headers:
          Authorization: "Bearer {{ master_access_token }}"
          Content-Type: "application/json"
        body:
          displayName: "Temporary-Name"  # Placeholder name
          passwordCredentials:
            - displayName: "ChildAppRegistrationSecret"
        return_content: yes
      register: child_app_response
      failed_when: child_app_response.status != 201
    - name: Set Child App Variables
      set_fact:
        child_app_id: "{{ child_app_response.json.appId }}"
        child_client_secret: "{{ child_app_response.json.passwordCredentials[0].secretText }}"
    - name: Update Child App Name with App ID Suffix
      set_fact:
        child_app_display_name: "RHDP-llm-gpt4-lightspeed-Child-{{ child_app_id[-6:] }}"
    - name: Update Child App Display Name
      uri:
        url: "https://graph.microsoft.com/v1.0/applications/{{ child_app_id }}"
        method: PATCH
        headers:
          Authorization: "Bearer {{ master_access_token }}"
          Content-Type: "application/json"
        body:
          displayName: "{{ child_app_display_name }}"
        status_code: 204
    - name: Create Service Principal for Child App
      uri:
        url: "https://graph.microsoft.com/v1.0/servicePrincipals"
        method: POST
        headers:
          Authorization: "Bearer {{ master_access_token }}"
          Content-Type: "application/json"
        body:
          appId: "{{ child_app_id }}"
        return_content: yes
      register: service_principal_response
      failed_when: service_principal_response.status != 201
    - name: Set Service Principal Object ID
      set_fact:
        child_service_principal_object_id: "{{ service_principal_response.json.id }}"
    - name: Add Child App to Azure AD Group
      uri:
        url: "https://graph.microsoft.com/v1.0/groups/{{ azure_ad_group_id }}/members/$ref"
        method: POST
        headers:
          Authorization: "Bearer {{ master_access_token }}"
          Content-Type: "application/json"
        body:
          "@odata.id": "https://graph.microsoft.com/v1.0/directoryObjects/{{ child_service_principal_object_id }}"
        status_code: 204
      when: azure_ad_group_id != ""
    - name: Get Token for Child App
      uri:
        url: "https://login.microsoftonline.com/{{ azure_tenant_id }}/oauth2/v2.0/token"
        method: POST
        headers:
          Content-Type: "application/x-www-form-urlencoded"
        body:
          grant_type: client_credentials
          client_id: "{{ child_app_id }}"
          client_secret: "{{ child_client_secret }}"
          scope: "{{ cognitive_services_scope }}"
        body_format: form-urlencoded
        return_content: yes
      register: child_token_response
      failed_when: child_token_response.status != 200
    - name: Set Child Access Token
      set_fact:
        child_access_token: "{{ child_token_response.json.access_token }}"
    - name: Ask AI Service Using Child App
      uri:
        url: "https://llm-gpt4-lightspeed.openai.azure.com/openai/deployments/gpt-4/chat/completions?api-version=2023-06-01-preview"
        method: POST
        headers:
          Authorization: "Bearer {{ child_access_token }}"
          Content-Type: "application/json"
        body:
          messages:
            - role: system
              content: "You are a helpful assistant."
            - role: user
              content: "What is the largest city in Spain?"
          max_tokens: 100
        return_content: yes
      register: ai_response
    - name: Display AI Response
      debug:
        msg: "{{ ai_response.json }}"
    - name: Prompt User for Cleanup
      pause:
        prompt: "Would you like to clean up the child app registration and service principal? (yes/no)"
      register: user_response
    - name: Perform Cleanup if User Agrees
      when: user_response.user_input | lower == 'yes'
      block:
        - name: Delete Child App Registration
          uri:
            url: "https://graph.microsoft.com/v1.0/applications/{{ child_app_id }}"
            method: DELETE
            headers:
              Authorization: "Bearer {{ master_access_token }}"
            status_code: 204
        - name: Delete Child Service Principal
          uri:
            url: "https://graph.microsoft.com/v1.0/servicePrincipals/{{ child_service_principal_object_id }}"
            method: DELETE
            headers:
              Authorization: "Bearer {{ master_access_token }}"
            status_code: 204

Recap

This article explained how applications like Red Hat OpenShift Lightspeed can automate the process of connecting to Azure AI services using dynamic credentials. We provided a detailed workflow, implementation steps, and automation code for you to adapt and use in your own applications. Learn more about OpenShift Lightspeed.