Field Formatting

[spalger]

Configurable field formats would allow us to specify transformations that should be applied to data before it is rendered and doesn't impact the data stored or indexed by elasticsearch.

options will include:

• anchor
• bytes
• ip
• per field date format
• configurable numeral js format
• anchor with a template
• currency
• image

[jescar50dx]

Additional options:

• Long Integer to IP address conversion

[jimmyjones2]

Maybe this is pushing it too far, but it would be great to be able to do IP/ASN/OUI lookups at display time.

Something simple like passing certain fields to an external REST API that does the transform would be very powerful.

[garyelephant]

Additional options:
Long Integer in Y-axis of a histogram such as 10000 to 10k, 100000 to 100k, 1000000 to 1million etc.

[spalger]

@jimmyjones2 I think you're right, implementing some sort of "load replacements via API endpoint" formatter does not sound like something something Kibana users would benefit from.

Maybe (BIG maybe) formatters could wrap values in angular markup, which would allow using a custom directive to load up values the way your API prefers. You'd have to write and support the directive and the formatter, but it could work just the way your want.

@garyelephant @jescar50dx these suggestions make sense to me. Thanks

[burtonator]

Right now if I index a field as HTML and it's stored as HTML but not indexed as HTML the output is rendered as 'source' not rendered as HTML.

For example, b, em, and a should all be rendered properly.

This should be easy to accomplish and for applications that wish to search HTML, viewing the HTML in Kibana would be nice.

This would require the input HTML to be safe as it could break Kibana. Another option could be to use an inline iframe for each entry and then use data: URIs for the iframe src.

[agirbal]

+1 on this, would be great to be able to load up a file like this to translate a field: http://gdeltproject.org/data/lookups/CAMEO.eventcodes.txt

[spalger]

@burtonator If we do provide a way to preview HTML via field formatting, I think that little expandable iframes (or a pop-up, or new tab) would probably be the best/safest bet.

@agirbal I dig the idea of a mapping file, but think it's probably not a good fit for all kibana users. That type of transformation is usually best made before storing the data in elasticsearch (via logstash for example). Maybe someday you will be able to do this with a custom field format though [dreamcode].

[burtonator]

Curious if this is going to make it into a future beta. We really need this. We want to launch a social media platform based on Kibana in January/Feb timeframe, If this does not make it in our plan B is to convert all the HTML to text.. which wouldn't be any fun.

[spalger]

@burtonator this definitely is not on the roadmap. In fact, I would guess that it is unlikely to ever be on the roadmap. Previewing html documents isn't something we consider a common use case, and our current focus is improving and optimizing common use cases.

We always consider pull requests :)

[burtonator]

OK.. fair enough. I'd be willing to hire a contractor at some point to build a patch and contribute it if this works out for us. Would be nice if there was a supported plugin API as this would also allow us to do this on our end.

[proofy]

transform from DateTimeFormat (http://joda-time.sourceforge.net/apidocs/org/joda/time/format/DateTimeFormat.html) to Moment (http://momentjs.com/docs/#/parsing/string-format/) date Strings. (e.g. yyyyMMdd -> YYYYMMDD)

[vikrim1]

+1

[noamgal]

What about this one?
#2235

In kibana 3 if a field contains some URL (string type, for example: http://www.geoiptool.com/en/?IP=70.197.5.18), it is recognized automatically by kibana as url and in table view it is blue colored and clickable, opens new tab in the browser.
This is not supported in kibana4, and will be very nice and helfull.

[MajorTal]

+1

[maniel]

also it would be nice to "compose" clickable urls, i sometimes have hostname and http arguments in different fields, also protocol (http/https) is another field

[spalger]

@maniel field concatenation will be available in scripted fields someday soon

[acheriat]

Hi Spalger,
Is there a way (or good news) to format a filed currency, in particular the €? Thanks

[blacktop]

How do I make the URL into a clickable link programmatically? Can I do it with an ES mapping?

This is an amazing feature. Thanks.

[spalger]

@cherah30 unfortunately numeral.js only supports using one currency symbol at a time. Since we don't currently have any sort of localization support, currency symbols other than $ are not yet available.

@blacktop there isn't any way to do this with raw es mappings yet. I like the idea, but I'm not sure how it might work. Want to investigate it and open up a ticket?

[mvfcva]

Some requests regarding field color were linked to this "field formatting" feature (#335 #1304)
I tried last snapshot and I noticed that field formatting feature does not allow to set a color on a given field
This would be very useful for highlighting log severity (red=critical, yellow=warning ...)
Is this planned in the future ?
Should we open a new feature request for this ?

[AlexClineBB]

4.1.0 was released, but field formatting (bytes -> MB) doesn't seem to be available. Is there documentation that I'm missing on how to implement this?

[simitter]

@AlexClineBB it took me some time to find this too,

You can click on Settings -> Indices -> Your index pattern and than in the list of field go to the field which stores bytes and click on the edit icon, there you can choose a format for the field

[AlexClineBB]

Thanks @Tschebel. Found it and working as needed!

[colmaengus]

Is there a plan to add duration formatting ? We have duration fields in milliseconds that would be much nicer in sec/min/hour etc.

[missnebun]

@Tschebel Thank you :P You save me a lot of time 👍

[deepakas]

+1 Is there a workaround to format aggregation of amount field to two decimal places or remove the decimal places.

[maniel]

@deepakas you can change format by editing Numeral.js format pattern field in place mentioned by @Tschebel, default is 0,0.[000]b, you can change it to 0,0.[00]b to achieve 2 decimal places

[deepakas]

Thanks Daniel.
On Jul 7, 2015 10:32 AM, "Daniel Kamiński" notifications@github.com wrote:

@deepakas https://github.com/deepakas you can change format by editing Numeral.js
format pattern field, default is 0,0.[000]b, you can change it to
0,0.[00]b to achieve 2 decimal places

—
Reply to this email directly or view it on GitHub
#1543 (comment).

[billw2012]

@colmaengus
Yeah we also store durations in ms, some formatting support would be greatly appreciated!

[serialdoom]

+1

[pjcard]

+1 we store durations in seconds, but need the ability to render them as minutes or (or ideally, and) hours.

This feature would work around the issue I mentioned here regarding the removal of groovy:
#3180 (comment)

edit: For anyone stumbling upon this, my solution in Kibana 4.1.1 is to set the format of the field to numeral and set the format as '00:00:00'. This works for seconds but not milliseconds, exposing the manipulate function of numeral.js could solve even that, I think.

[jnbala]

+1

[myGerritTest1]

It would be tremendous to be able to convert milliseconds/microseconds on the y-axis, any solution yet

there isnt any solution up there or is there any thing i missed regarding this issue.. plz help

[marcelolavor]

Hi spalger, I found this thread opened by you since 2014 to group up several fields format, but I couldn't see the way I can format them. Could you please provide me a link about it?

[kribor]

Hi spalger, I found this thread opened by you since 2014 to group up several fields format, but I couldn't see the way I can format them. Could you please provide me a link about it?

While not a complete guide. In index management -> pattern -> choose edit on a field you can do this. For example colorize by regex