Logstash Mutate Filter Cheat Sheet: Everything You Need to Know

The mutate filter in Logstash is used to perform mutations on data fields. For example, to add a field, rename a field, or copy a field to another field.

This tutorial will show you all types of mutations you can perform with the mutate filter in Logstash.

What is the Mutate Filter in Logstash?

The mutate filter is a plugin of Logstash that helps you to perform some specified actions on the data. The actions, in processing order, include:

The basic structure of a Logstash pipeline config is as follows:

input {
	# input plugins
	stdin {} # read from the console
}

filter {
	# filter plugins
	# put the mutate filter here
}

output {
	# output plugins
	stdout {} # output to the console
}

So you can put the mutate filter in the filter section of the Logstash pipeline.

We will go deeper to each type of mutation in the next sections.

How to Use the Mutate Filter in Logstash?

The mutate filter can be used in the filter section of the Logstash pipeline, and specify the mutate actions to be performed.

filter {
	mutate {
		# put the mutate actions here
	}
}

1. coerce

The coerce action is used to set the default value of a field when it is null.

filter {
	mutate {
		coerce => {
			"field_name" => "default_value" # "[user][name]" => "Minh Vu"
		}
	}
}

The code above uses the coerce action to set the value of field_name to default_value, so that when the field_name is null, it will be set to default_value.

2. rename

The rename action is used to rename a field.

filter {
	mutate {
		rename => {
			"source_field" => "destination_field"
		}
	}
}

There are 2 points to remember when using the rename action:

• If the destination field already exists, it will be overwritten.
• If the source field does not exist, the destination field will not be created.

3. update

The update action is used to update the value of a field.

filter {
	mutate {
		update => {
			"field_name" => "new_value"
		}
	}
}

If the field does not exist, nothing will happen.

4. replace

The replace action is used to replace the value of a field and create the field if it does not exist.

filter {
	mutate {
		replace => {
			"field_name" => "new_value"
		}
	}
}

The replace actions is different from the update action in that it will create the field if it does not exist.

5. convert

The convert action is used to convert the value of a field to another data type, e.g. string to integer.

filter {
	mutate {
		convert => {
			"field_name" => "integer"
			"field_name_2" => "boolean"
		}
	}
}

If the field is an array, the conversion will be applied to all elements of the array.

Valid data types are:

• integer
• integer_eu
• float
• float_eu
• string
• boolean

You can visit the docs for more information about the data types.

6. gsub

The gsub action is used to replace a string with another string.

filter {
	mutate {
		gsub => [
			"field_name", "pattern", "replacement"
		]
	}
}

For example, to remove all dots from a string:

filter {
	mutate {
		gsub => [
			"field_name", "\.", ""
		]
	}
}

7. uppercase

The uppercase action is used to convert every character of a string or an array of strings field to uppercase.

filter {
	mutate {
		uppercase => [
			"field_name", # "wisecode blog" => "WISECODE BLOG"
			"array_field_name" # ["wisecode blog", "elastic"] => ["WISECODE BLOG", "ELASTIC"]
		]
	}
}

8. capitalize

The capitalize action is used to capitalize the first character of a string or an array of strings field.

filter {
	mutate {
		capitalize => [
			"field_name", # "wisecode blog" => "Wisecode blog"
			"array_field_name" # ["wisecode blog", "elastic"] => ["Wisecode blog", "Elastic"]
		]
	}
}

9. lowercase

The lowercase action is used to convert every character of a string or an array of strings field to lowercase.

filter {
	mutate {
		lowercase => [
			"field_name", # "WISECODE BLOG" => "wisecode blog"
			"array_field_name" # ["WISECODE BLOG", "ELASTIC"] => ["wisecode blog", "elastic"]
		]
	}
}

10. strip

The strip action is used to remove leading and trailing whitespaces from a string or an array of strings field.

filter {
	mutate {
		strip => [
			"field_name", # " wisecode blog " => "wisecode blog"
			"array_field_name" # [" wisecode blog ", " elastic "] => ["wisecode blog", "elastic"]
		]
	}
}

11. split

The split action is used to split a string field into an array of strings.

filter {
	mutate {
		split => {
			"field_name" => "separator"
		}
	}
}

For example, I want to split the message field by the comma character:

filter {
	mutate {
		split => {
			"message" => "," # "Hello, world" => ["Hello", " world"]
		}
	}
}

12. join

The join action is used to join an array of strings into a string field.

filter {
	mutate {
		join => {
			"field_name" => "separator"
		}
	}
}

The field being joined must be an array of strings. Otherwise, the join action will not work.

For example, I want to join the tags field by the comma character:

filter {
	mutate {
		join => {
			"tags" => "," # ["elastic", "logstash"] => "elastic,logstash"
		}
	}
}

13. merge

The merge action is used to merge two fields into one field.

filter {
	mutate {
		merge => {
			"destination_field" => "added_field"
		}
	}
}

Valid data types are:

• string + string = array of 2 strings ("elastic" + "logstash" => ["elastic", "logstash"])
• array of strings + string = array of strings (["elastic", "logstash"] + "tutorial" => ["elastic", "logstash", "tutorial"])

For example, I want to merge the tags field and the category field into the tags field:

filter {
	mutate {
		merge => {
			"tags" => "category" # ["elastic", "logstash"] + "tutorial" => ["elastic", "logstash", "tutorial"]
		}
	}
}

14. copy

The copy action is used to copy the value of a field to another field.

filter {
	mutate {
		copy => {
			"source_field" => "destination_field"
		}
	}
}

If the destination field already exists, it will be overwritten.

Conclusion

The mutate filter is a powerful plugin of Logstash that helps you to perform several actions on the data.

In this tutorial, we have learned all types of mutations you can perform with the mutate filter.

I hope you find this tutorial useful. Feel free to leave a comment below if you have any questions.