Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Select links based on message Priority & Reliability #1360

Open
wants to merge 36 commits into
base: main
Choose a base branch
from

Conversation

fuzzypixelz
Copy link
Member

The goal of this pull request is to enable users to choose the link on which a message is sent as a function of the message's Priority & Reliability.

Supersedes #1324 and #1356.

@fuzzypixelz fuzzypixelz added enhancement Existing things could work better new feature Something new is needed labels Sep 5, 2024
@fuzzypixelz fuzzypixelz self-assigned this Sep 5, 2024
@fuzzypixelz fuzzypixelz force-pushed the link-selection branch 2 times, most recently from f873bc9 to 91ee110 Compare September 6, 2024 10:23
@fuzzypixelz
Copy link
Member Author

fuzzypixelz commented Sep 6, 2024

Preliminary throughput measurements for 743c199 and db099c1

msg/s,scenario,commit,payload
743c1998,1 TCP link,2420878.4302636916,8
743c1998,1 TCP link,2916512.8935243557,8
743c1998,1 TCP link,2959452.48235027,8
743c1998,1 TCP link,2968195.429786387,8
743c1998,1 TCP link,2916831.054403508,8
743c1998,1 TCP link,2959617.236254376,8
743c1998,1 TCP link,2918744.034087194,8
743c1998,1 TCP link,2964756.3990116213,8
743c1998,1 TCP link,2921003.3342668856,8
743c1998,1 TCP link,2957219.9104648437,8
743c1998,2 TCP links,2415267.0576484874,8
743c1998,2 TCP links,2903860.833979539,8
743c1998,2 TCP links,2898339.7759079044,8
743c1998,2 TCP links,2911794.579181915,8
743c1998,2 TCP links,2888570.420790474,8
743c1998,2 TCP links,2935927.80882342,8
743c1998,2 TCP links,2917058.233116825,8
743c1998,2 TCP links,2937648.4412117647,8
743c1998,2 TCP links,2907668.0760812284,8
743c1998,2 TCP links,2523742.9955405965,8
743c1998,"1 TCP link, 1 UDP link",2495499.740542892,8
743c1998,"1 TCP link, 1 UDP link",2937859.885999873,8
743c1998,"1 TCP link, 1 UDP link",2903053.233983884,8
743c1998,"1 TCP link, 1 UDP link",2944454.167686429,8
743c1998,"1 TCP link, 1 UDP link",2902464.256007564,8
743c1998,"1 TCP link, 1 UDP link",2942362.675446834,8
743c1998,"1 TCP link, 1 UDP link",2909023.8851805944,8
743c1998,"1 TCP link, 1 UDP link",2946084.0400522486,8
743c1998,"1 TCP link, 1 UDP link",2912579.0561333206,8
743c1998,"1 TCP link, 1 UDP link",2949154.540799975,8
743c1998,8 TCP links,2477550.113220324,8
743c1998,8 TCP links,2975888.990293959,8
743c1998,8 TCP links,2945258.8592429277,8
743c1998,8 TCP links,2980142.3578242627,8
743c1998,8 TCP links,2941574.188721928,8
743c1998,8 TCP links,2975269.116810702,8
743c1998,8 TCP links,2934292.7923029508,8
743c1998,8 TCP links,2982046.825649921,8
743c1998,8 TCP links,2942303.719066016,8
743c1998,8 TCP links,2980885.0152221876,8
743c1998,"8 TCP links, 8 UDP links",2249783.0646679895,8
743c1998,"8 TCP links, 8 UDP links",3061132.530875807,8
743c1998,"8 TCP links, 8 UDP links",3002104.6254476705,8
743c1998,"8 TCP links, 8 UDP links",3051733.8776975535,8
743c1998,"8 TCP links, 8 UDP links",3017937.1699282117,8
743c1998,"8 TCP links, 8 UDP links",3064982.1892352495,8
743c1998,"8 TCP links, 8 UDP links",3016990.2408813275,8
743c1998,"8 TCP links, 8 UDP links",3052964.6316795084,8
743c1998,"8 TCP links, 8 UDP links",3021179.4346140637,8
743c1998,"8 TCP links, 8 UDP links",3053132.5047905175,8
743c1998,"1 TCP link, 1 Priority",2184683.246926926,8
743c1998,"1 TCP link, 1 Priority",2985861.2899028263,8
743c1998,"1 TCP link, 1 Priority",3009424.4641230707,8
743c1998,"1 TCP link, 1 Priority",2970458.640299292,8
743c1998,"1 TCP link, 1 Priority",2904404.0669673095,8
743c1998,"1 TCP link, 1 Priority",2962659.7621683395,8
743c1998,"1 TCP link, 1 Priority",2631563.711999562,8
743c1998,"1 TCP link, 1 Priority",2599232.9299731036,8
743c1998,"1 TCP link, 1 Priority",3041467.4277391443,8
743c1998,"1 TCP link, 1 Priority",3006621.2414303026,8
743c1998,"2 TCP links, 2 Priorities",2442950.3305934747,8
743c1998,"2 TCP links, 2 Priorities",2963596.6018690104,8
743c1998,"2 TCP links, 2 Priorities",2927426.863216448,8
743c1998,"2 TCP links, 2 Priorities",2966175.7487606425,8
743c1998,"2 TCP links, 2 Priorities",2927355.7353719594,8
743c1998,"2 TCP links, 2 Priorities",2969665.6964174905,8
743c1998,"2 TCP links, 2 Priorities",2929278.1734239887,8
743c1998,"2 TCP links, 2 Priorities",2954212.105279966,8
743c1998,"2 TCP links, 2 Priorities",2492763.2589828465,8
743c1998,"2 TCP links, 2 Priorities",2523113.294725495,8
743c1998,"2 TCP links, 2 Priorities, Reliable",2106216.897247219,8
743c1998,"2 TCP links, 2 Priorities, Reliable",2452054.909061743,8
743c1998,"2 TCP links, 2 Priorities, Reliable",2623314.8153540515,8
743c1998,"2 TCP links, 2 Priorities, Reliable",2921069.802088466,8
743c1998,"2 TCP links, 2 Priorities, Reliable",2915921.843030562,8
743c1998,"2 TCP links, 2 Priorities, Reliable",2936052.799389818,8
743c1998,"2 TCP links, 2 Priorities, Reliable",3003550.948108401,8
743c1998,"2 TCP links, 2 Priorities, Reliable",2484687.2452108525,8
743c1998,"2 TCP links, 2 Priorities, Reliable",2548291.0064745448,8
743c1998,"2 TCP links, 2 Priorities, Reliable",2540159.541324408,8
743c1998,"1 TCP link (Priority 0, Reliable), 1 UDP link (Priority 1, Reliable)",2373166.8680180092,8
743c1998,"1 TCP link (Priority 0, Reliable), 1 UDP link (Priority 1, Reliable)",2968584.891019093,8
743c1998,"1 TCP link (Priority 0, Reliable), 1 UDP link (Priority 1, Reliable)",2925742.1481419113,8
743c1998,"1 TCP link (Priority 0, Reliable), 1 UDP link (Priority 1, Reliable)",2943086.620366224,8
743c1998,"1 TCP link (Priority 0, Reliable), 1 UDP link (Priority 1, Reliable)",2923723.3898046603,8
743c1998,"1 TCP link (Priority 0, Reliable), 1 UDP link (Priority 1, Reliable)",2943046.776814901,8
743c1998,"1 TCP link (Priority 0, Reliable), 1 UDP link (Priority 1, Reliable)",2975116.866309405,8
743c1998,"1 TCP link (Priority 0, Reliable), 1 UDP link (Priority 1, Reliable)",2979338.8215535143,8
743c1998,"1 TCP link (Priority 0, Reliable), 1 UDP link (Priority 1, Reliable)",2941366.53477285,8
743c1998,"1 TCP link (Priority 0, Reliable), 1 UDP link (Priority 1, Reliable)",2982029.040549452,8
743c1998,"8 TCP links, 8 Priorities",2311929.5053853393,8
743c1998,"8 TCP links, 8 Priorities",2616562.7006004145,8
743c1998,"8 TCP links, 8 Priorities",2950983.80488284,8
743c1998,"8 TCP links, 8 Priorities",3013121.3903747345,8
743c1998,"8 TCP links, 8 Priorities",2953862.964507505,8
743c1998,"8 TCP links, 8 Priorities",2978653.7164334767,8
743c1998,"8 TCP links, 8 Priorities",3059562.9285854874,8
743c1998,"8 TCP links, 8 Priorities",3008716.401851985,8
743c1998,"8 TCP links, 8 Priorities",2999713.6773294993,8
743c1998,"8 TCP links, 8 Priorities",3033129.6313804192,8
743c1998,"8 TCP links (Reliable), 8 UDP links (BestEffort), 16 Priorities",2427120.8424633527,8
743c1998,"8 TCP links (Reliable), 8 UDP links (BestEffort), 16 Priorities",3050639.0524446648,8
743c1998,"8 TCP links (Reliable), 8 UDP links (BestEffort), 16 Priorities",3009720.645254009,8
743c1998,"8 TCP links (Reliable), 8 UDP links (BestEffort), 16 Priorities",3012194.6289798543,8
743c1998,"8 TCP links (Reliable), 8 UDP links (BestEffort), 16 Priorities",3038168.572540194,8
743c1998,"8 TCP links (Reliable), 8 UDP links (BestEffort), 16 Priorities",3039459.5403255126,8
743c1998,"8 TCP links (Reliable), 8 UDP links (BestEffort), 16 Priorities",3011037.318615865,8
743c1998,"8 TCP links (Reliable), 8 UDP links (BestEffort), 16 Priorities",2978936.7721861675,8
743c1998,"8 TCP links (Reliable), 8 UDP links (BestEffort), 16 Priorities",3040766.3996285885,8
743c1998,"8 TCP links (Reliable), 8 UDP links (BestEffort), 16 Priorities",2945506.1046644943,8
db099c1c,1 TCP link,2391342.307785938,8
db099c1c,1 TCP link,2887109.3196148677,8
db099c1c,1 TCP link,2920784.924730788,8
db099c1c,1 TCP link,2889788.3001776845,8
db099c1c,1 TCP link,2930418.988669623,8
db099c1c,1 TCP link,2887063.4756416003,8
db099c1c,1 TCP link,2919759.860262629,8
db099c1c,1 TCP link,2750577.167359913,8
db099c1c,1 TCP link,2927948.0015173797,8
db099c1c,1 TCP link,2897279.952456795,8
db099c1c,2 TCP links,2345891.083795816,8
db099c1c,2 TCP links,2849603.620136439,8
db099c1c,2 TCP links,2644363.680288921,8
db099c1c,2 TCP links,2841797.233277366,8
db099c1c,2 TCP links,2887246.0266514174,8
db099c1c,2 TCP links,2894654.184493274,8
db099c1c,2 TCP links,2868983.4798480165,8
db099c1c,2 TCP links,2904302.7593896687,8
db099c1c,2 TCP links,2867565.227789345,8
db099c1c,2 TCP links,2833516.269866243,8
db099c1c,"1 TCP link, 1 UDP link",2453235.083557923,8
db099c1c,"1 TCP link, 1 UDP link",2878429.998352675,8
db099c1c,"1 TCP link, 1 UDP link",2906022.90974785,8
db099c1c,"1 TCP link, 1 UDP link",2877611.637590943,8
db099c1c,"1 TCP link, 1 UDP link",2912122.7359556183,8
db099c1c,"1 TCP link, 1 UDP link",2407935.6505203103,8
db099c1c,"1 TCP link, 1 UDP link",2919938.896190683,8
db099c1c,"1 TCP link, 1 UDP link",2887384.414031753,8
db099c1c,"1 TCP link, 1 UDP link",2932427.3149504936,8
db099c1c,"1 TCP link, 1 UDP link",2892583.4507432957,8
db099c1c,8 TCP links,2406558.217687582,8
db099c1c,8 TCP links,2948124.2456302824,8
db099c1c,8 TCP links,2952783.2536322996,8
db099c1c,8 TCP links,2996050.456485738,8
db099c1c,8 TCP links,2959610.141210695,8
db099c1c,8 TCP links,2331496.61915504,8
db099c1c,8 TCP links,2959993.0594082745,8
db099c1c,8 TCP links,3003912.656313101,8
db099c1c,8 TCP links,2963792.5614319365,8
db099c1c,8 TCP links,3001449.4599732105,8
db099c1c,"8 TCP links, 8 UDP links",2686746.189696855,8
db099c1c,"8 TCP links, 8 UDP links",3001200.029831928,8
db099c1c,"8 TCP links, 8 UDP links",2957951.185759808,8
db099c1c,"8 TCP links, 8 UDP links",2986496.1990564223,8
db099c1c,"8 TCP links, 8 UDP links",2960656.3704117453,8
db099c1c,"8 TCP links, 8 UDP links",2968610.447478246,8
db099c1c,"8 TCP links, 8 UDP links",3007125.835736654,8
db099c1c,"8 TCP links, 8 UDP links",3001672.081432962,8
db099c1c,"8 TCP links, 8 UDP links",2959304.4734858745,8
db099c1c,"8 TCP links, 8 UDP links",3005087.161801856,8
db099c1c,"1 TCP link, 1 Priority",1892345.451295437,8
db099c1c,"1 TCP link, 1 Priority",2810242.558498782,8
db099c1c,"1 TCP link, 1 Priority",2876008.5694701737,8
db099c1c,"1 TCP link, 1 Priority",2903879.3853950896,8
db099c1c,"1 TCP link, 1 Priority",2884724.9486879553,8
db099c1c,"1 TCP link, 1 Priority",2924728.05805398,8
db099c1c,"1 TCP link, 1 Priority",2890482.424696212,8
db099c1c,"1 TCP link, 1 Priority",2926660.0645117825,8
db099c1c,"1 TCP link, 1 Priority",2895544.2265135604,8
db099c1c,"1 TCP link, 1 Priority",2913164.508846873,8
db099c1c,"2 TCP links, 2 Priorities",2383191.5596506414,8
db099c1c,"2 TCP links, 2 Priorities",2909376.810905229,8
db099c1c,"2 TCP links, 2 Priorities",2878879.134894517,8
db099c1c,"2 TCP links, 2 Priorities",2912445.8772901506,8
db099c1c,"2 TCP links, 2 Priorities",2880054.0989361946,8
db099c1c,"2 TCP links, 2 Priorities",2916048.537277977,8
db099c1c,"2 TCP links, 2 Priorities",2887919.747944675,8
db099c1c,"2 TCP links, 2 Priorities",2880955.103454954,8
db099c1c,"2 TCP links, 2 Priorities",2913959.0644699456,8
db099c1c,"2 TCP links, 2 Priorities",2879443.737541007,8
db099c1c,"2 TCP links, 2 Priorities, Reliable",2451287.283711892,8
db099c1c,"2 TCP links, 2 Priorities, Reliable",2905534.8725055656,8
db099c1c,"2 TCP links, 2 Priorities, Reliable",2876719.260478328,8
db099c1c,"2 TCP links, 2 Priorities, Reliable",2912394.1359293563,8
db099c1c,"2 TCP links, 2 Priorities, Reliable",2870166.030494366,8
db099c1c,"2 TCP links, 2 Priorities, Reliable",2911565.676390772,8
db099c1c,"2 TCP links, 2 Priorities, Reliable",2872132.104290563,8
db099c1c,"2 TCP links, 2 Priorities, Reliable",2912499.3170189103,8
db099c1c,"2 TCP links, 2 Priorities, Reliable",2851830.30468955,8
db099c1c,"2 TCP links, 2 Priorities, Reliable",2900369.887072068,8
db099c1c,"1 TCP link (Priority 0, Reliable), 1 UDP link (Priority 1, Reliable)",2236236.0176268173,8
db099c1c,"1 TCP link (Priority 0, Reliable), 1 UDP link (Priority 1, Reliable)",2905996.646131151,8
db099c1c,"1 TCP link (Priority 0, Reliable), 1 UDP link (Priority 1, Reliable)",2869668.5504127587,8
db099c1c,"1 TCP link (Priority 0, Reliable), 1 UDP link (Priority 1, Reliable)",2898855.6506422083,8
db099c1c,"1 TCP link (Priority 0, Reliable), 1 UDP link (Priority 1, Reliable)",2875615.7304347204,8
db099c1c,"1 TCP link (Priority 0, Reliable), 1 UDP link (Priority 1, Reliable)",2914190.9764349484,8
db099c1c,"1 TCP link (Priority 0, Reliable), 1 UDP link (Priority 1, Reliable)",2842726.1744012507,8
db099c1c,"1 TCP link (Priority 0, Reliable), 1 UDP link (Priority 1, Reliable)",2911939.569566411,8
db099c1c,"1 TCP link (Priority 0, Reliable), 1 UDP link (Priority 1, Reliable)",2867843.1897757947,8
db099c1c,"1 TCP link (Priority 0, Reliable), 1 UDP link (Priority 1, Reliable)",2416280.764479339,8
db099c1c,"8 TCP links, 8 Priorities",2189937.3866242045,8
db099c1c,"8 TCP links, 8 Priorities",2408995.4395066435,8
db099c1c,"8 TCP links, 8 Priorities",2906218.761781084,8
db099c1c,"8 TCP links, 8 Priorities",2582367.3170634205,8
db099c1c,"8 TCP links, 8 Priorities",2798843.193326707,8
db099c1c,"8 TCP links, 8 Priorities",2408101.4311587615,8
db099c1c,"8 TCP links, 8 Priorities",2870115.780470584,8
db099c1c,"8 TCP links, 8 Priorities",2899134.6691830773,8
db099c1c,"8 TCP links, 8 Priorities",2863430.9515123786,8
db099c1c,"8 TCP links, 8 Priorities",2907472.7049369933,8
db099c1c,"8 TCP links (Reliable), 8 UDP links (BestEffort), 16 Priorities",2498940.1994614084,8
db099c1c,"8 TCP links (Reliable), 8 UDP links (BestEffort), 16 Priorities",2988776.7845209097,8
db099c1c,"8 TCP links (Reliable), 8 UDP links (BestEffort), 16 Priorities",2953295.9284621943,8
db099c1c,"8 TCP links (Reliable), 8 UDP links (BestEffort), 16 Priorities",2983521.9486074024,8
db099c1c,"8 TCP links (Reliable), 8 UDP links (BestEffort), 16 Priorities",2976306.162376431,8
db099c1c,"8 TCP links (Reliable), 8 UDP links (BestEffort), 16 Priorities",2293663.5731796687,8
db099c1c,"8 TCP links (Reliable), 8 UDP links (BestEffort), 16 Priorities",2925497.267980494,8
db099c1c,"8 TCP links (Reliable), 8 UDP links (BestEffort), 16 Priorities",2991962.9293399486,8
db099c1c,"8 TCP links (Reliable), 8 UDP links (BestEffort), 16 Priorities",2972398.5449039056,8
db099c1c,"8 TCP links (Reliable), 8 UDP links (BestEffort), 16 Priorities",2973122.3201390323,8

DEFAULT_CONFIG.json5 Outdated Show resolved Hide resolved
commons/zenoh-protocol/src/core/mod.rs Show resolved Hide resolved
/// 3. QoS is enabled and priority range is available but reliability is unavailable
/// 4. QoS is enabled and reliability is available but priority range is unavailable
/// 5. QoS is enabled and both priority range and reliability are available
fn to_u64(&self) -> u64 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should not this be in zenoh-codec/zenoh-protocol cartes ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to @Mallets only StateAccept could have a codec implementation as it's serialized on the wire. Otherwise this function is used for creating the ZExtZ64 and ZExtUnit extensions.

Other establishment extensions also just put their codec implementations in the same file. Even if everything were to have a codec I would still keep the try_from_64/to_u64 functions because it's the same logic.

tracing::trace!("Scheduled: {:?}", $msg);
return pl.push_network_message($msg);
};
/// Returns the index of the best matching [`Reliability`]-[`PriorityRange`] pair.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 'smaller range' rule prevents us from returning as soon as we find a matching range. Is it worth it ?
Even with this rule we could return as soon as we find a full match with a range of 1 (cannot find a smaller range later).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry to disappoint but the search isn't even short-circuiting. It searches for three kinds of matches at the same time.

You could do search for link matching Priority and Reliability and early return on a result, but if you don't find any result you need to search by Reliability only. And if you don't find any result either you pick any link.

The search uses a fold to keep track of all three kinds of matches.

DEFAULT_CONFIG.json5 Outdated Show resolved Hide resolved
DEFAULT_CONFIG.json5 Outdated Show resolved Hide resolved
@@ -311,6 +326,10 @@ where
n_exts -= 1;
self.write(&mut *writer, (qos, n_exts != 0))?;
}
if let Some(qos_optimized) = ext_qos_optimized.as_ref() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as above: both ext_qos and ext_qos_optimized could be sent on the network.
If ext_qos_optimized then I don't see much the need of sending as well ext_qos.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See above.

///
/// ## Metadata
///
/// - **`priorities`**: a range bounded inclusively below and above (e.g. `2..=4` signifies
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd use some easier format, e.g.
priorities=1-4 would mean all priorities from 1 to 4 both included.
priorities=1 would mean only priority 1.

This is more efficient in terms of network overhead compared to 2..=4 where we need 3 characters ..= to express the same meaning of -.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand. Although - is a questionable choice since it looks like substraction in this context. I suggest : (what Python uses for slicing sequences).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well not so questionable in terms of network overhead. I'd even suggest having something like prio=1-4 to save a few bytes more during scouting and discovery.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Existing things could work better new feature Something new is needed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants