Run a Managed AI Model with Confluent Cloud for Apache Flink

You can run real-time AI model inference in Confluent Cloud for Apache Flink® using managed AI models, without provisioning any external infrastructure. Managed models behave like other Flink SQL resources, such as tables and functions.

You create a model resource with an SQL statement and use it for inference in streaming queries, or you can reference certain models directly with no setup. You can use the SQL interface in Confluent Cloud Console or in the SQL shell.

Managed models run on the cloud provider that hosts your compute pool: Amazon Bedrock on AWS, Vertex AI on Google Cloud, and Azure OpenAI on Azure. The syntax is the same on every cloud. Only the model name differs. You don’t need an endpoint or API keys. To use a model that managed models don’t offer, or your own endpoint and API keys, see Run a Remote AI Model.

Choose the approach that fits your use case:

• Run a managed model directly

• Create and run a managed model resource

• Create and run a managed embedding model resource

Prerequisites

Before you use a managed AI model, make sure you have the following:

• Access to Confluent Cloud.

• Access to a Flink compute pool that runs on AWS, Google Cloud, or Azure, in a region that supports managed models. For the supported regions, see Confluent Cloud Regions and Availability by Cloud Provider.

• Enough permissions to create models. For more information, see RBAC for model inference.

Supported models

Your compute pool must be in a region that supports managed models. For the supported regions, see Confluent Cloud Regions and Availability by Cloud Provider. The following tabs list the model families for each cloud provider. To see the exact model names available in your region, run the SHOW MODELS statement.

AWS

Amazon Bedrock hosts managed models on AWS, including the following model families:

• Anthropic Claude Opus, Sonnet, and Haiku

• Amazon Nova

• OpenAI GPT-OSS Safeguard

• Amazon Titan Text Embeddings

In some regions, the available models vary.

Azure

Azure OpenAI hosts managed models on Azure, including the following model families:

• OpenAI GPT-5.4, GPT-5.4 mini, and GPT-5.4 nano

• OpenAI text-embedding-3-large and text-embedding-3-small

Every model is available in every supported region.

Google Cloud

Vertex AI hosts managed models on Google Cloud, including the following model families:

• Google Gemini Flash, Flash-Lite, and Pro

• Google Gemini Embedding

Every model is available in every supported region.

Set up your SQL workspace

You can use a Flink SQL workspace to invoke a managed model.

1. Log in to Confluent Cloud Console at https://confluent.cloud/login.

2. In the navigation menu, click SQL workspaces to open the SQL workspaces page.

3. Click Create workspace to open the New workspace page.

4. Click Create new workspace.

5. In the dialog, for Cloud, select your cloud provider.

6. For Region, select a supported region.

7. Click Create workspace.

Run a managed model directly

Reference a model by name directly in the AI_COMPLETE or AI_EMBEDDING functions, using the form system.<provider>.<model>. Use this approach for one-off or exploratory queries, when you don’t need to reuse a named model resource or configure model parameters.

In your SQL workspace, complete the following steps.

1. Run the following statement to see the model names available to you in your region.

   SHOW MODELS;
   

   You can call directly any model whose name starts with system.. Other entries represent model resources that someone has already created with the CREATE MODEL statement.

2. Run one of the following statements to call a model directly, with no CREATE MODEL statement required. Quote the whole model name as a single identifier.

   • To classify customer feedback with the AI_COMPLETE function:

     AWS

     SELECT t.prompt, r.`response`
     FROM (VALUES ('Classify the sentiment: fast shipping, damaged box.'))
       AS t(prompt),
     LATERAL TABLE(
       AI_COMPLETE(`system.anthropic.claude-opus-4-8`, t.prompt)
     ) AS r;
     

     Azure

     SELECT t.prompt, r.`response`
     FROM (VALUES ('Classify the sentiment: fast shipping, damaged box.'))
       AS t(prompt),
     LATERAL TABLE(
       AI_COMPLETE(`system.openai.gpt-5.4-mini`, t.prompt)
     ) AS r;
     

     Google Cloud

     SELECT t.prompt, r.`response`
     FROM (VALUES ('Classify the sentiment: fast shipping, damaged box.'))
       AS t(prompt),
     LATERAL TABLE(
       AI_COMPLETE(`system.google.gemini-3.5-flash`, t.prompt)
     ) AS r;
     

   • To generate an embedding with the AI_EMBEDDING function:

     AWS

     SELECT t.text, e.*
     FROM (VALUES ('Fast shipping, but the product arrived damaged.'))
       AS t(text),
     LATERAL TABLE(
       AI_EMBEDDING(`system.amazon.titan-embed-text-v2`, t.text)
     ) AS e;
     

     Azure

     SELECT t.text, e.*
     FROM (VALUES ('Fast shipping, but the product arrived damaged.'))
       AS t(text),
     LATERAL TABLE(
       AI_EMBEDDING(`system.openai.text-embedding-3-small`, t.text)
     ) AS e;
     

     Google Cloud

     SELECT t.text, e.*
     FROM (VALUES ('Fast shipping, but the product arrived damaged.'))
       AS t(text),
     LATERAL TABLE(
       AI_EMBEDDING(`system.google.gemini-embedding-2`, t.text)
     ) AS e;
     

   Note

   If you split the model name into separate parts, for example system, <provider>, and <model>, Flink looks for a catalog named system and the statement fails.

Create and run a managed model resource

A managed model resource is a named model that you create with the CREATE MODEL statement. You can reuse it across multiple queries and configure it with parameters. For example, you can set a system prompt that gives the model behavioral instructions. In your SQL workspace, complete the following steps to create and query a managed model resource that generates text.

1. Run the following statement to create the model.

   AWS

   CREATE MODEL `managed_model_llm`
   INPUT (prompt STRING)
   OUTPUT (response STRING)
   WITH (
     'provider' = 'confluent',
     'task' = 'text_generation',
     'confluent.model' = 'anthropic.claude-sonnet-4-6',
     'confluent.system_prompt' = 'Fact-check each statement in ten words or fewer.'
   );
   

   Azure

   CREATE MODEL `managed_model_llm`
   INPUT (prompt STRING)
   OUTPUT (response STRING)
   WITH (
     'provider' = 'confluent',
     'task' = 'text_generation',
     'confluent.model' = 'openai.gpt-5.4-mini',
     'confluent.system_prompt' = 'Fact-check each statement in ten words or fewer.'
   );
   

   Google Cloud

   CREATE MODEL `managed_model_llm`
   INPUT (prompt STRING)
   OUTPUT (response STRING)
   WITH (
     'provider' = 'confluent',
     'task' = 'text_generation',
     'confluent.model' = 'google.gemini-3.5-flash',
     'confluent.system_prompt' = 'Fact-check each statement in ten words or fewer.'
   );
   

   The confluent.system_prompt parameter gives the model behavioral instructions. To find a value for the confluent.model parameter, run the SHOW MODELS statement and use the name of a managed model without the system. prefix. For example, if the statement lists system.<provider>.<model>, use <provider>.<model>.

2. Run the following statement to create a table that contains prompts for the managed model. Choose a table name that’s unique to your environment, to avoid colliding with a topic or schema that already exists.

   CREATE TABLE managed_model_prompts (
     id BIGINT, prompt STRING
   );
   

3. Run the following statement to populate the managed_model_prompts table with example prompts.

   INSERT INTO managed_model_prompts
     VALUES
       (1, 'The mitochondria is the powerhouse of the cell'),
       (2, 'Goldfish have a three-second memory'),
       (3, 'Lightning never strikes the same place twice');
   

4. Run the following statement to call the AI_COMPLETE function with the example prompt data.

   SELECT id, prompt, response
   FROM managed_model_prompts, LATERAL TABLE(AI_COMPLETE('managed_model_llm', prompt));
   

   Your output should resemble:

   id	prompt	response
   1	The mitochondria is the powerhouse of the cell	True. Mitochondria produce ATP, the cell’s main energy currency.
   2	Goldfish have a three-second memory	False. Goldfish memory lasts months, not three seconds.
   3	Lightning never strikes the same place twice	False. Lightning frequently strikes the same place multiple times.

Create and run a managed embedding model resource

A managed embedding model resource is a named model that returns vector embeddings instead of free-text responses, useful for semantic search or retrieval-augmented generation (RAG). In your SQL workspace, complete the following steps. This example reuses the managed_model_prompts table created in Create and run a managed model resource.

1. Run the following statement to create the embedding model.

   AWS

   CREATE MODEL `managed_model_embedding`
   INPUT (text STRING)
   OUTPUT (embedding ARRAY<FLOAT>)
   WITH (
     'provider' = 'confluent',
     'task' = 'embedding',
     'confluent.model' = 'amazon.titan-embed-text-v2'
   );
   

   Azure

   CREATE MODEL `managed_model_embedding`
   INPUT (text STRING)
   OUTPUT (embedding ARRAY<FLOAT>)
   WITH (
     'provider' = 'confluent',
     'task' = 'embedding',
     'confluent.model' = 'openai.text-embedding-3-small'
   );
   

   Google Cloud

   CREATE MODEL `managed_model_embedding`
   INPUT (text STRING)
   OUTPUT (embedding ARRAY<FLOAT>)
   WITH (
     'provider' = 'confluent',
     'task' = 'embedding',
     'confluent.model' = 'google.gemini-embedding-2'
   );
   

   To find a value for the confluent.model parameter, run the SHOW MODELS statement and use the name of an embedding model without the system. prefix.

2. Run the following statement to call the AI_EMBEDDING function to generate embeddings for the example prompt data.

   SELECT id, prompt, embedding
   FROM managed_model_prompts, LATERAL TABLE(AI_EMBEDDING('managed_model_embedding', prompt));
   

   Your output should resemble:

   id	prompt	response
   1	The mitochondria is the powerhouse of the cell	-0.009094955,-0.0041054585,0.028798368,-0.022862062,0.027289227, …
   2	Goldfish have a three-second memory	-0.05532522,0.081224516,0.0600239,0.033652354,-0.021552816, …
   3	Lightning never strikes the same place twice	-0.045672793,0.0021575035,0.0040705274,-0.010357216,-0.006122007, …

Related content

• Create Embedding Action

• Run a Remote AI Model

• CREATE MODEL statement

Note

This website includes content developed at the Apache Software Foundation under the terms of the Apache License v2.