Build AI with Flink SQL and Confluent Cloud for Apache Flink¶
Note
The AI model inference feature is available for preview. A Preview feature is a Confluent Cloud component that is being introduced to gain early feedback from developers. Preview features can be used for evaluation and non-production testing purposes or to provide feedback to Confluent. The warranty, SLA, and Support Services provisions of your agreement with Confluent do not apply to Preview features. Confluent may discontinue providing preview releases of the Preview features at any time in Confluent’s sole discretion.
AI Model Inference with Confluent Cloud for Apache Flink® enables you to integrate AI/ML models into your streaming data workflows seamlessly.
With AI Model Inference, you get:
- Integration with Flink SQL: AI models are treated as first-class resources in Flink SQL, similar to tables and functions. This integration enables you to use SQL statements to create model resources and perform inference in streaming queries.
- Remote Model Support: AI with Flink supports remote AI models hosted on various platforms, like OpenAI, AWS Bedrock, AWS Sagemaker, Google Cloud’s Vertex AI, and Azure AI. This means you can leverage models hosted outside Confluent Cloud for real-time predictions and inference.
Key features¶
CREATE MODEL Statement
Registers a remote model in your Flink environment for real-time prediction and inference.
CREATE MODEL my_model WITH ('provider' = 'openai', 'model_id' = 'gpt-3')
Model inference in streaming queries
Once a model is registered, you can use it in your streaming queries to perform inference.
SELECT ML_PREDICT(my_model, input_column) FROM my_stream
Supported Platforms
- AWS Bedrock: Navigate to the AWS Bedrock site, select a foundation model, and use the provided model ID and endpoint for inference.
- AWS Sagemaker: Similar to AWS Bedrock, but using Sagemaker Studio.
- OpenAI: Use OpenAI’s API for model inference.
- Google Cloud Vertex AI: Integrate models hosted on Vertex AI.
- Azure AI: Use models hosted on Azure AI and Azure ML.
Security and management
- Customer secrets encryption: Secure storage and management of credentials required to access remote endpoints.
- Model Versioning: Basic model versioning capabilities to manage different versions of models.