Build AI with Flink SQL and Confluent Cloud for Apache Flink¶
AI Model Inference with Confluent Cloud for Apache Flink® enables you to integrate large-language models (LLMs) and machine learning (ML) into your streaming data workflows seamlessly.
With Confluent’s AI Model Inference features, you get:
- Streaming Agents: Use streaming agents to build AI workflows that can invoke tools to interact with external systems, perform actions, or retrieve information as part of an AI workflow.
- Integration with Flink SQL: AI models are treated as first-class resources in Flink SQL, similar to tables and functions. This integration enables you to use SQL statements to create model resources and perform inference in streaming queries.
- Remote Model Support: AI with Flink supports remote AI models hosted on various platforms, like OpenAI, AWS Bedrock, AWS Sagemaker, Google Cloud’s Vertex AI, and Azure AI. This means you can leverage models hosted outside Confluent Cloud for real-time predictions and inference.
- AI/ML Functions: Confluent Cloud for Apache Flink provides built-in functions for your AI/ML workflows, like AI_COMPLETE, AI_EMBEDDING, and ML_DETECT_ANOMALIES.
- External Tables: Confluent Cloud for Apache Flink provides functions for searching over external tables, like KEY_SEARCH_AGG, TEXT_SEARCH_AGG, and VECTOR_SEARCH_AGG.
Key features¶
-
Registers a remote model in your Flink environment for real-time prediction and inference over streaming data.
Model inference in streaming queries
Once a model is registered, you can use it in your Flink SQL statements to run inference. Confluent Cloud for Apache Flink provides the following functions for building AI workflows with your registered models.
- AI_TOOL_INVOKE: Invoke model context protocol (MCP) tools and user-defined functions (UDFs) in your streaming agents.
- AI_COMPLETE: Generate text completions.
- AI_EMBEDDING: Create embeddings.
- ML_DETECT_ANOMALIES: Detect anomalies in your data.
- ML_EVALUATE: Evaluate the performance of an AI/ML model.
- ML_FORECAST: Forecast trends in your data.
- ML_PREDICT: Run a remote AI/ML model for tasks like predicting outcomes, generating text, and classification.
Supported platforms
- AWS Bedrock: Navigate to the AWS Bedrock site, select a foundation model, and use the provided model ID and endpoint for inference.
- AWS Sagemaker: Similar to AWS Bedrock, but using Sagemaker Studio.
- OpenAI: Use OpenAI’s API for model inference.
- Google Cloud Vertex AI: Integrate models hosted on Vertex AI.
- Azure AI Foundry: Use models hosted on Azure AI and Azure ML.
Security and management
- Customer secrets encryption: Secure storage and management of credentials required to access remote endpoints in reusable connection resources.
- Model Versioning: Basic model versioning capabilities to manage different versions of models.
RBAC for model inference¶
The following table shows the model actions that are available for different RBAC permissions.
| Role | CREATE MODEL | Invoke model for prediction | List/Describe Models | DROP MODEL | Grant permissions on models |
|---|---|---|---|---|---|
| CloudClusterAdmin | Yes [1] | Yes [1] | Yes [1] | Yes [1] | Yes [1] |
| EnvironmentAdmin | Yes | Yes | Yes | Yes | Yes |
| DeveloperManage | Yes | No | Yes | Yes | No |
| DeveloperRead | No | Yes | Yes | No | No |
| DeveloperWrite | Yes | Yes | Yes | No | No |
| ResourceOwner | Yes | Yes | Yes | Yes | Yes |
| OrganizationAdmin | Yes | Yes | Yes | Yes | Yes |
| [1] | (1, 2, 3, 4, 5) For own Kafka cluster. |