Identify classes of key data that we can use to aggregate raw flow data in order to increase performance

[TheWitness]

Is your feature request related to a problem? Please describe.

It's clear, even from my little home install that data will pile up in the raw tables over time and that for most queries, the tool will become unusable if for example, you want to see the top traffic by source domain over a month, the queries may never complete, even if done in parallel. So, it's important to aggregate key data by day or by day and hour such that the user interface is usable.

In order to do that, we first must hear from users as to what is important to capture for aggregation and high level reporting.

We will use this ticket to start the discussion on that topic.

Describe the solution you'd like

We would like users of FlowView to describe what data is key outside of the totals of packet, bytes, and flows so that we can consolidate that data by hour or day into more analytically focused partitions.

Describe alternatives you've considered

Just parallelizing queries is not enough if this tool is to be more than a nice utility to use for your home network. So, we have to give this topic some thought. We will parallelize, it's just not enough.

[bmfmancini]

Source to destination traffic counts/ports/traffic types (sip,https) administrators can use this data to look for traffic trends and identify any use of an app such as bit torrent that should be banned on the network

Traffic to domains - to keep tracks of flows to domains usually tools will have a whois lookup for the domain when hovered over or clicked

IP Top talkers - Engineers can look at which client/server is hogging the LAN!

BGP Metrics - engineers can use this data to plan peering links i.e I have increase in traffic going to telco a we may need to upgrade our peering link they would look at (Destination ASN,peer,as path)

[arno-st]

source to destination, excluding some protocol (like I need to remove the video traffic, otherwise it will be in the top 50)

in a range of IP, source to destination traffic

traffic from inside to outside (inside base on ip range))