Functions

Functions available in the query language.

abs

abs(expr)

Returns the absolute value of the input.

add

add(expr1, expr2)

Adds the two inputs.

and

and(expr1, expr2)

Logical and operation of two boolean columns.

automated_readability_index

automated_readability_index(text)

Returns the ARI (Automated Readability Index) which outputs a number that approximates the grade level needed to comprehend the text. For example if the ARI is 6.5, then the grade level to comprehend the text is 6th to 7th grade.

bleu

bleu(output, reference)

Computes the BLEU score between two columns.

character_count

character_count(text)

Returns the number of characters in a text column.

• Aliases

  • num_chars

coalesce

coalesce(expr)

Return the first expression that evaluates to a non-null value.

concat

concat(expr)

Concatenates multiple text columns into one.

contains

contains(text, text)

Returns true if the input string contains the substring.

divide

divide(expr1, expr2)

Divides the two inputs.

embed

embed(text)

Returns the embedding of a text column. Embedding model: all-mpnet-base-v2.

equal_to

equal_to(expr1, expr2)

Computes the element-wise equal to comparison of two columns.

• Aliases

  • eq

filter

filter(expr1, expr2)

Filters a column using another column as a mask.

flesch_kincaid_grade

flesch_kincaid_grade(text)

Returns the Flesch-Kincaid Grade of the given text. This is a grade formula in that a score of 9.3 means that a ninth grader would be able to read the document.

greater_than

greater_than(expr1, expr2)

Computes the element-wise greater than comparison of two columns. input1 > input2

• Aliases

  • gt

greater_than_or_equal_to

greater_than_or_equal_to(expr1, expr2)

Computes the element-wise greater than or equal to comparison of two columns. input1 >= input2

• Aliases

  • gte

is_valid_json

is_valid_json(text)

Returns true if the input string is valid json.

less_than

less_than(expr1, expr2)

Computes the element-wise less than comparison of two columns. input1 < input2

• Aliases

  • lt

less_than_or_equal_to

less_than_or_equal_to(expr1, expr2)

Computes the element-wise less than or equal to comparison of two columns. input1 <= input2

• Aliases

  • lte

levenshtein

levenshtein(output, reference)

Returns Damerau-Levenshtein distance between two strings.

list_contains

list_contains(list, value)

Returns True if the list contains the value.

list_extract

list_extract(list_expr, index_expr)

Extracts the item at the given index from a list.

list_has_duplicate

list_has_duplicate(expr)

Returns True if the list has duplicated items.

list_length

list_length(expr)

Returns the length of lists in a list column.

list_most_common

list_most_common(expr)

Most common item in list.

list_starts_with

list_starts_with(list, prefix)

Returns True if the list starts with the value.

list_zip

list_zip(expr)

Zips multiple lists into a list of structs.

llm_answer_groundedness

llm_answer_groundedness(model_name, prompt_version, answer, context)

Judge if the answer is adhering to the context

llm_answer_refusal

llm_answer_refusal(model_name, prompt_version, answer)

Judge if the answer is a refusal to answer the question

llm_answer_relevancy

llm_answer_relevancy(model_name, prompt_version, question, answer)

Judge if the answer is relevant to the question

• Aliases

  • rag_answer_relevancy

llm_classify

llm_classify(model_name, prompt, classes)

Classify text into custom categories using an LLM.

llm_context_relevancy

llm_context_relevancy(model_name, prompt_version, question, context)

LLM as Judge if the contexts are relevant to the question

llm_question_clarity

llm_question_clarity(model_name, prompt_version, question)

Judge if the question is clear

llm_score

llm_score(model_name, prompt)

Score text using an LLM.

llm_summarization

llm_summarization(model_name, prompt_version, input, output)

Summarize the input and output of a conversational system.

llm_text_frustration

llm_text_frustration(model_name, prompt_version, text)

Judge the frustration of text (default to input) on a scale of 1 to 5.

llm_text_sentiment

llm_text_sentiment(model_name, prompt_version, text)

Judge the sentiment of a text as positive, negative, or neutral.

• Aliases

  • text_sentiment

llm_text_similarity

llm_text_similarity(model_name, prompt_version, output, reference)

Judge the similarity of an output on a scale of 1 to 5, as compared to a target.

• Aliases

  • text_similarity

llm_text_toxicity

llm_text_toxicity(model_name, prompt_version, text)

Judge the toxicity of a text on a scale of 1 to 5.

map_extract

map_extract(map_expr, key_expr)

Extracts the value for a given key from a map, returning null if the key is not in the map.

multiply

multiply(expr1, expr2)

Multiplies the two inputs.

negate

negate(expr)

Returns the negation of the input.

not

not(expr)

Logical not operation of a boolean column.

not_equal_to

not_equal_to(expr1, expr2)

Computes the element-wise not equal to comparison of two columns.

• Aliases

  • neq

or

or(expr1, expr2)

Logical or operation of two boolean columns.

rouge1

rouge1(output, reference)

Returns the rouge1 score between two columns.

rouge2

rouge2(output, reference)

Returns the rouge2 score between two columns.

rougeL

rougeL(output, reference)

Returns the rougeL score between two columns.

rougeLsum

rougeLsum(output, reference)

Returns the rougeLsum score between two columns.

sentence_count

sentence_count(text)

Returns the number of sentences in a text column.

• Aliases

  • num_sentences

struct_extract

struct_extract(struct_expr, field_name)

Extracts a field from a struct expression.

subtract

subtract(expr1, expr2)

Subtracts the two inputs.

token_count

token_count(text)

Returns the number of tokens in a text column.

word_count

word_count(text)

Returns the number of words in a text column.

• Aliases

  • num_words