Functions

abs

Returns the absolute value of the input.

Syntax

abs(expr)

add

Adds the two inputs.

Syntax

add(expr1, expr2)

and

Logical and operation of two or more boolean columns.

Syntax

and(expr1, expr2)

automated_readability_index

Returns the ARI (Automated Readability Index) which outputs a number that approximates the grade level needed to comprehend the text. For example if the ARI is 6.5, then the grade level to comprehend the text is 6th to 7th grade.

Syntax

automated_readability_index(expr)

bleu

Computes the BLEU score between two columns.

Syntax

bleu(expr1, expr2)

character_count

Returns the number of characters in a text column.

Syntax

character_count(expr)

Aliases

• num_chars

divide

Divides the two inputs.

Syntax

divide(expr1, expr2)

equal_to

Computes the element-wise equal to comparison of two columns.

Syntax

equal_to(expr1, expr2)

Aliases

• eq

filter

Filters a column using another column as a mask.

Syntax

filter(expr1, expr2)

flesch_kincaid_grade

Returns the Flesch-Kincaid Grade of the given text. This is a grade formula in that a score of 9.3 means that a ninth grader would be able to read the document.

Syntax

flesch_kincaid_grade(expr)

greater_than

Computes the element-wise greater than comparison of two columns. input1 > input2

Syntax

greater_than(expr1, expr2)

Aliases

• gt

greater_than_or_equal_to

Computes the element-wise greater than or equal to comparison of two columns. input1 >= input2

Syntax

greater_than_or_equal_to(expr1, expr2)

Aliases

• gte

is_valid_json

Returns true if the input string is valid json.

Syntax

is_valid_json(expr)

less_than

Computes the element-wise less than comparison of two columns. input1 < input2

Syntax

less_than(expr1, expr2)

Aliases

• lt

less_than_or_equal_to

Computes the element-wise less than or equal to comparison of two columns. input1 <= input2

Syntax

less_than_or_equal_to(expr1, expr2)

Aliases

• lte

levenshtein

Returns Damerau-Levenshtein distance between two strings.

Syntax

levenshtein(expr1, expr2)

list_has_duplicate

Returns True if the list has duplicated items.

Syntax

list_has_duplicate(expr)

list_len

Returns the length of lists in a list column.

Syntax

list_len(expr)

list_most_common

Most common item in list.

Syntax

list_most_common(expr)

multiply

Multiplies the two inputs.

Syntax

multiply(expr1, expr2)

negate

Returns the negation of the input.

Syntax

negate(expr)

not

Logical not operation of a boolean column.

Syntax

not(expr)

not_equal_to

Computes the element-wise not equal to comparison of two columns.

Syntax

not_equal_to(expr1, expr2)

Aliases

• neq

or

Logical or operation of two or more boolean columns.

Syntax

or(expr1, expr2)

rouge1

Returns the rouge1 score between two columns.

Syntax

rouge1(expr1, expr2)

rouge2

Returns the rouge2 score between two columns.

Syntax

rouge2(expr1, expr2)

rougeL

Returns the rougeL score between two columns.

Syntax

rougeL(expr1, expr2)

rougeLsum

Returns the rougeLsum score between two columns.

Syntax

rougeLsum(expr1, expr2)

sentence_count

Returns the number of sentences in a text column.

Syntax

sentence_count(expr)

Aliases

• num_sentences

subtract

Subtracts the two inputs.

Syntax

subtract(expr1, expr2)

token_count

Returns the number of tokens in a text column.

Syntax

token_count(expr)

word_count

Returns the number of words in a text column.

Syntax

word_count(expr)

Aliases

• num_words