Skip to content

Instantly share code, notes, and snippets.

@paul-butcher

paul-butcher/gist:45f2f33dac3c59b28f704b60e56842b8

Last active February 8, 2024 16:37
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save paul-butcher/45f2f33dac3c59b28f704b60e56842b8 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save paul-butcher/45f2f33dac3c59b28f704b60e56842b8 to your computer and use it in GitHub Desktop.
Download ZIP
An Elasticsearch runtime field to remove some impertinent confounding content from a log entry
Raw
gistfile1.txt
if(!doc.containsKey("log.keyword") || doc['log.keyword'].empty){
emit("");
} else {
def log = doc['log.keyword'].value;
if (log != null) {
int closingBracketIndex = log.lastIndexOf(']');
if (closingBracketIndex > 0) {
emit(log.substring(closingBracketIndex+1));
return;
}
emit(log)
}
}
emit("");
@paul-butcher
Copy link
Author

paul-butcher commented Feb 8, 2024 •
edited
Loading

I was recently trying to find unique instances of a common error reported in some logs. However, these logs all contained some preamble about the process that raised the error. e.g.

[blah blah, my-process-instance-5] could not do the thing because 'ABCDEF'
[blah blah, my-process-instance-3] could not do the thing because 'EFGHIJ'
[blah blah, my-process-instance-12] could not do the thing because 'ABCDEF'

Creating a report on this was a bit of a challenge in Kibana, as all of these rows are unique, but WRT what I was interested in, there are two unique values here, and one of them appears twice.

To deal with this, I created a runtime field. It's slow and a bit of a palaver, but it works.

This Painless script strips off everything up to ']'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment