Package io.confluent.flink.plugin

Class ConfluentMLExpressions

java.lang.Object
io.confluent.flink.plugin.ConfluentMLExpressions
@PublicEvolving public final class ConfluentMLExpressions extends Object
Entry point of the ML Function Expression DSL for Confluent Flink ML functions.

This class contains static methods for creating ML function expressions that can be used in Table API operations. Similar to Flink's Expressions class, this provides a convenient way to access ML functions in a type-safe manner. 


 import static io.confluent.flink.plugin.ConfluentMLExpressions.*;

Example usage: 


 table_env.select(mlOneHotEncoder($("category")).as("one_hot_encoded"));

  • Method Details

    • aiComplete

      public static ApiExpression aiComplete(String modelName, Object... arguments)
      Calls the AI_COMPLETE function to generate text completions using AI models.

      Example usage: 

      
 table_env.from("table-name").joinLateral(aiComplete("model-name",$("input"), lit(",")))
             .select("input", "completion");
      Parameters:
      modelName - the name of the AI model to use
      arguments - the input arguments for text completion
      Returns:
      an expression that calls the AI_COMPLETE function

    • aiEmbedding

      public static ApiExpression aiEmbedding(String modelName, Object... arguments)
      Calls the AI_EMBEDDING function to generate embeddings using AI models.

      Example usage: 

      
 table_env.from("table-name").joinLateral(aiEmbedding("model-name",$("input"), lit(",")))
             .select("input", "embedding");
      Parameters:
      modelName - the name of the embedding model to use
      arguments - the input arguments for embedding generation
      Returns:
      an expression that calls the AI_EMBEDDING function

    • aiRunAgent

      public static ApiExpression aiRunAgent(String agentName, Object... arguments)
      Calls the AI_RUN_AGENT function to execute an AI agent workflow.

      Example usage: 

      
 table_env.select(aiRunAgent("agent-name", $("input")).as("agent_response"));
      Parameters:
      agentName - the name of the agent to run
      arguments - the input arguments for the agent
      Returns:
      an expression that calls the AI_RUN_AGENT function

    • aiToolInvoke

      public static ApiExpression aiToolInvoke(String modelName, Object... arguments)
      Calls the AI_TOOL_INVOKE function to invoke AI tools.

      Example usage: 

      
 table_env.select(aiToolInvoke("model-name", $("input")).as("result"))
      Parameters:
      modelName - the name of the text_generation model to use
      arguments - the input arguments for the tool
      Returns:
      an expression that calls the AI_TOOL_INVOKE function

    • mlEvaluate

      public static ApiExpression mlEvaluate(String modelName, Object... arguments)
      Calls the ML_EVALUATE function to evaluate ML models.

      Example usage: 

      
 table_env.groupBy($("group_key"))
      .aggregate(mlEvaluate("model-name", $("features"), $("labels")).as("metrics"))
      Parameters:
      modelName - the name of the ML model to evaluate
      arguments - the input arguments for model evaluation
      Returns:
      an expression that calls the ML_EVALUATE function

    • mlEvaluateAll

      public static ApiExpression mlEvaluateAll(String modelName, Object... arguments)
      Calls the ML_EVALUATE_ALL function to evaluate all versions of ML models.

      Example usage: 

      
 table_env.groupBy($("group_key"))
      .aggregate(mlEvaluateAll("model-name", $("features"), $("labels")).as("all_metrics"))
      Parameters:
      modelName - the name of the ML model to evaluate all versions
      arguments - the input arguments for model evaluation
      Returns:
      an expression that calls the ML_EVALUATE_ALL function

    • mlDetectAnomalies

      public static ApiExpression mlDetectAnomalies(Object value, Object timestamp, Object... arguments)
      Calls the ML_DETECT_ANOMALIES function to detect anomalies in time series data.
      Parameters:
      value - the numeric value column
      timestamp - the timestamp column
      arguments - additional optional arguments
      Returns:
      an expression that calls the ML_DETECT_ANOMALIES function

    • mlForecast

      public static ApiExpression mlForecast(Object value, Object timestamp, Object... arguments)
      Calls the ML_FORECAST function to forecast time series data.
      Parameters:
      value - the numeric value column
      timestamp - the timestamp column
      arguments - additional optional arguments
      Returns:
      an expression that calls the ML_FORECAST function

    • mlBucketize

      public static ApiExpression mlBucketize(Object value, Object arraySplitPoints, Object... arguments)
      Calls the ML_BUCKETIZE function to bucketize continuous values into discrete buckets.

      Example usage: 

      
 table_env.select(mlBucketize($("age"), [5, 18, 65, 100]).as("age_bucket"))
      Parameters:
      value - the value to bucketize
      arguments - the bucket boundaries or configuration
      Returns:
      an expression that calls the ML_BUCKETIZE function

    • mlLabelEncoder

      public static ApiExpression mlLabelEncoder(Object value, Object categories, Object... arguments)
      Calls the ML_LABEL_ENCODER function to encode categorical labels as integers.

      Example usage: 

      
 table_env.select(mlLabelEncoder($("category")).as("encoded_category"))
      Parameters:
      value - the categorical value to encode
      arguments - additional encoding configuration
      Returns:
      an expression that calls the ML_LABEL_ENCODER function

    • mlOneHotEncoder

      public static ApiExpression mlOneHotEncoder(Object value, Object... arguments)
      Calls the ML_ONE_HOT_ENCODER function to perform one-hot encoding of categorical variables.

      Example usage: 

      
 table_env.select(mlOneHotEncoder($("category")).as("one_hot_encoded"))
      Parameters:
      value - the categorical value to one-hot encode
      arguments - additional encoding configuration
      Returns:
      an expression that calls the ML_ONE_HOT_ENCODER function

    • mlMinMaxScaler

      public static ApiExpression mlMinMaxScaler(Object value, Object... arguments)
      Calls the ML_MIN_MAX_SCALER function to scale features to a fixed range (typically [0, 1]).

      Example usage: 

      
 table_env.select(mlMinMaxScaler($("feature")).as("scaled_feature"))
      Parameters:
      value - the numeric value to scale
      arguments - additional scaling configuration (min, max values)
      Returns:
      an expression that calls the ML_MIN_MAX_SCALER function

    • mlMaxAbsScaler

      public static ApiExpression mlMaxAbsScaler(Object features, Object... arguments)
      Calls the ML_MAX_ABS_SCALER function to scale features by their maximum absolute value.

      Example usage: 

      
 table_env.select(mlMaxAbsScaler(array($("feature1"), $("feature2"))).as("scaled_features"))
      Parameters:
      features - the features to scale (typically an array)
      arguments - additional scaling configuration
      Returns:
      an expression that calls the ML_MAX_ABS_SCALER function

    • mlStandardScaler

      public static ApiExpression mlStandardScaler(Object value, Object... arguments)
      Calls the ML_STANDARD_SCALER function to standardize features (zero mean, unit variance).

      Example usage: 

      
 table_env.select(mlStandardScaler($("feature")).as("standardized_feature"))
      Parameters:
      value - the numeric value to standardize
      arguments - additional scaling configuration
      Returns:
      an expression that calls the ML_STANDARD_SCALER function

    • mlRobustScaler

      public static ApiExpression mlRobustScaler(Object value, Object... arguments)
      Calls the ML_ROBUST_SCALER function to scale features using robust statistics.

      Example usage: 

      
 table_env.select(mlRobustScaler($("feature")).as("robust_scaled_feature"))
      Parameters:
      value - the numeric value to scale
      arguments - additional scaling configuration
      Returns:
      an expression that calls the ML_ROBUST_SCALER function

    • mlNormalizer

      public static ApiExpression mlNormalizer(Object value, Object... arguments)
      Calls the ML_NORMALIZER function to normalize vectors to have unit norm.

      Example usage: 

      
 table_env.select(mlNormalizer($("vector_feature")).as("normalized_vector"))
      Parameters:
      value - the vector to normalize
      arguments - additional normalization configuration (e.g., norm type)
      Returns:
      an expression that calls the ML_NORMALIZER function

    • mlCharacterTextSplitter

      public static ApiExpression mlCharacterTextSplitter(Object text, Object... arguments)
      Calls the ML_CHARACTER_TEXT_SPLITTER function to split text by character count.

      Example usage: 

      
 table_env.select(mlCharacterTextSplitter($("text"), 100).as("text_chunks"))
      Parameters:
      text - the text to split
      arguments - splitting configuration (chunk size, overlap, etc.)
      Returns:
      an expression that calls the ML_CHARACTER_TEXT_SPLITTER function

    • mlRecursiveTextSplitter

      public static ApiExpression mlRecursiveTextSplitter(Object text, Object... arguments)
      Calls the ML_RECURSIVE_TEXT_SPLITTER function to split text recursively with hierarchy.

      Example usage: 

      
 table_env.select(mlRecursiveTextSplitter($("text"), 500, 50).as("text_chunks"))
      Parameters:
      text - the text to split
      arguments - splitting configuration (chunk size, overlap, separators, etc.)
      Returns:
      an expression that calls the ML_RECURSIVE_TEXT_SPLITTER function

    • mlFileFormatTextSplitter

      public static ApiExpression mlFileFormatTextSplitter(Object text, Object... arguments)
      Calls the ML_FILE_FORMAT_TEXT_SPLITTER function to split text based on file format.

      Example usage: 

      
 table_env.select(mlFileFormatTextSplitter($("document"), "markdown").as("text_chunks"))
      Parameters:
      text - the text/document to split
      arguments - splitting configuration (format type, chunk size, etc.)
      Returns:
      an expression that calls the ML_FILE_FORMAT_TEXT_SPLITTER function

    • mlNgrams

      public static ApiExpression mlNgrams(Object input, Object... arguments)
      Calls the ML_NGRAMS function to generate n-grams from text.

      Example usage: 

      
 table_env.select(mlNgrams($("text"), 2).as("bigrams"))  // Generate bigrams
      Parameters:
      input - the text to generate n-grams from
      arguments - n-gram configuration (n value, tokenizer options, etc.)
      Returns:
      an expression that calls the ML_NGRAMS function