why skip cached FS when enabling P2P?

[fengwang-db]

What happened in your environment?

When enabling P2P proxy in overlaybd, I noticed there are many more (10x) remote calls and the range size is also much smaller. I believe the reason is in this line:

overlaybd/src/image_service.cpp

Line 450 in b5b704b

global_fs.remote_fs = global_fs.srcfs;

When p2p is enabled, the global_fs.remote_fs is set to the srcfs instead of cached_fs. I'm curious what's the rationale of the choice and would setting remote_fs to cached_fs still works for P2P?

What did you expect to happen?

I would prefer the number of remote calls stay the same when P2P is enabled.

How can we reproduce it?

Enable p2p proxy

What is the version of your Overlaybd?

0.6.17

What is your OS environment?

Ubuntu 20.04

Are you willing to submit PRs to fix it?

• Yes, I am willing to fix it.

[shuaichang]

@lihuiba do you think we can add an option in P2P config to enable overlaybd cache?

[shuaichang]

@fengwang-db made a diff and verified it works as expected a34915a, requests to P2P agent was significantly reduced.

[BigVan]

We assume that p2pproxy has its own cache by default. If enable the cache of overlaybd at the same time, it will make additional cache and disk read/write, which will not improve performance and may cause IO performance degradation in scenarios where disk performance is limited (HDD or essd PL0/1)

If you want to reduce the number of requests to P2P, this place may need an aligned-fs to align the LBA offset and length with cache refillunit

[BigVan]

Or I think we can add a 'forceEnable' option in the configuration to enable overlaybd's cache whatever