RFC: XCM Asset Transfer Program Builder

[acatangiu]

Following the enablement of arbitrary XCM execution on Kusama and Polkadot system chains, users/wallets/dapps can interact uniformly across multiple chains directly using XCM programs, without having to figure out chain-specific pallets or calls/extrinsics.

Improving Asset Transfer UX/DX

Cross-chain asset transfers are currently hard to generalize across the ecosystem. Each Parachain uses its own assets transfer pallet (usually xtokens or pallet-xcm) that exposes asset transfer extrinsics unique to that chain, and wallets/dapps need to integrate with all of them. These extrinsics then build opinionated XCM programs to do the asset transfers.

With the ability to execute arbitrary XCM programs, we can make use of this powerful layer of abstraction, and slowly transition to switching that around:

Wallets/dApps define their desired asset transfers in an XCM program and execute that using the "universal" pallet-xcm::execute() extrinsic. This has two major advantages:

• Arbitrary XCM program (vs extrinsic) allows very high flexibility in what it can do. Current xtokens and pallet-xcm transfer extrinsics are limited to basic use-cases, and trying to expose more flexible ones results in ugly calls like pallet-xcm::transfer_assets_using_type_and_then().
• Allows for high reusability across multiple chains. The same XCM program (save some relative location parameters) can be reused on other chains for the same purpose.

The disadvantage of running "raw" XCM programs is that they're hard to build, it's a low-level language with gotchas and corner-cases, and easy to get wrong. Calling an extrinsic to build and execute an XCM program for you is definitely simpler and more reliable.

But what if we could fix (or mitigate) that?

XCM Asset Transfer Tooling

For automated, safe, reliable building of a complex asset transfer XCM we will require multiple layers of "helper" tools/libraries each operating at a different level. A quick exercise of imagination produces the following top-down view:

1. wallet/dApp: handles the "business usecase": gets the user input and models "actions" based on it: some of these actions will be generic asset transfers like:

   • "(somehow) move AssetX from AccountA on ChainA to AccountB on ChainB".

2. some asset-transfer-library (ecosystem-stateful): has ecosystem-level knowledge/state, able to identify what cross-chain interactions are required to "move AssetX from ChainA to ChainB", depending on the actual assets and chains involved it can define things like:

   • reserve location of AssetX,
   • cross-chain path from ChainA to ChainB (is it direct, through a reserve chain, over a bridge, etc?), and the "hops" involved
   • type of transfer of each asset between each hop (teleport, reserve-deposit, reserve-withdraw)

   libraries like asset-transfer-api and its assets-registry (cc @IkerAlus)

3. some XCM builder tool (ecosystem-stateless): Easy to use helper tool/library to hide away as much XCM complexity and be able to take the "detailed definition" of an asset transfer (involved accounts, assets, chains, transfer-types) and build a sanity-checked, corner-case-proofed, reliable XCM program for it.

This ^, alongside tooling that exposes the DryRun runtime APIs, should allow dApps to confidently switch to using flexible, portable, interoperable XCM programs instead of opinionated, chain-specific asset transfer extrinsics.

XCM Asset Transfer Builder

This RFC proposes the following (Rust) builder pattern tool for addressing layer (3) above. Same or similar can be created in JS or other relevant languages.

Builder properties:

• sanity-checks any new "actions" (against the program built so far) handling gotchas and corner-cases,
• "tag" assets in a given context (one chain) and use same tag to reference same asset in all other contexts (other chains) without having to explicitly manage XCM Locations context-relative views (no need to worry about reanchoring all the time),
• hides away implementation details like how fees and assets might need separate transfers,
• allows "natural", in-order sequenced definition of actions across chains (vs the XCM program which will be a Matryoska doll that is built "backwards", from the destination to the source),
• enforces generated XCM correctness (assuming input/scenario-definition correctness).

Example 1 - HDX, USDT, GLMR from Hydration Network to Moonbeam (through AH)

fn example_hydra_to_ah_to_moonbeam() {
	// Initial context needs to start with `GlobalConsensus`
	let hydra_context = InteriorLocation::X2(GlobalConsensus(Polkadot), Parachain(HYDRA_PARAID));

	// used assets IDs as seen by the `hydra_context` chain
	let hdx_id = AssetId(Here.into());
	let usdt_id = AssetId(Location::new(
		1,
		X3(Parachain(ASSET_HUB_PARAID), PalletInstance(50), GeneralIndex(1984)),
	));
	let glmr_id = AssetId(Location::new(1, X1(Parachain(MOONBEAM_PARAID))));

	// Initialize the builder by defining the starting context and,
	let asset_transfer_builder = AssetTransferBuilder::using_context(hydra_context)
		// registering easy-to-use tags for the assets to transfer - caller can use these tags to
		// easily identify the assets without having to worry about reanchoring and contexts
		.define_asset("HDX", hdx_id)
		.define_asset("USDT", usdt_id)
		.define_asset("GLMR", glmr_id)
		// create the builder
		.create();

	let xcm = asset_transfer_builder
		// the starting context is Hydration Network
		// withdraw assets to transfer from origin account
		.withdraw("HDX", Definite(hdx_amount))
		.withdraw("USDT", Definite(usdt_amount))
		.withdraw("GLMR", Definite(glmr_amount))
		// set AssetHub as the destination for this leg of the transfer
		.set_next_hop(Location::new(1, X1(Parachain(ASSET_HUB_PARAID))))
		// teleport all HDX to Asset Hub
		.transfer("HDX", All, Teleport)
		// reserve-withdraw all USDT on Asset Hub
		.transfer("USDT", All, ReserveWithdraw)
		// reserve-withdraw all GLMR on Asset Hub
		.transfer("GLMR", All, ReserveWithdraw)
		// use USDT to pay for fees on Asset Hub (can define upper limit)
		.pay_remote_fess_with("USDT", Definite(max_usdt_to_use_for_fees))
		// "execute" current leg of the transfer, move to next hop (Asset Hub)
		.execute_hop()

		// from here on, context is Asset Hub
		// set Moonbeam as the destination for this (final) leg of the transfer
		.set_next_hop(Location::new(1, X1(Parachain(MOONBEAM_PARAID))))
		// reserve-deposit HDX to Moonbeam (note we don't need to worry about reanchoring in the new
		// context)
		.transfer("HDX", All, ReserveDeposit)
		// reserve-deposit USDT to Moonbeam (asset reanchoring done behind the scenes)
		.transfer("USDT", All, ReserveDeposit)
		// teleport GLMR to Moonbeam (asset reanchoring done behind the scenes)
		.transfer("GLMR", All, Teleport)
		// use GLMR to pay for fees on Moonbeam (no limit)
		.pay_remote_fess_with("GLMR", All)
		// "execute" current leg of the transfer, move to next hop (Moonbeam)
		.execute_hop()

		// from here on, context is Moonbeam
		// deposit all received assets to `beneficiary`
		.deposit_all(beneficiary)
		// build the asset transfer XCM Program!
		.finalize();

	// Profit!
	println!("Asset transfer XCM: {:?}", xcm);
}

Example 2 - KSM from Karura to Acala (over bridge, going through both Asset Hubs)

fn example_karura_to_acala() {
	// Initial context needs to start with `GlobalConsensus`
	let karura_context = InteriorLocation::X2(GlobalConsensus(Kusama), Parachain(KARURA_PARAID));

	// used assets IDs as seen by the `karura_context` chain
	let ksm_id = AssetId(Location::new(1, Here));

	// Initialize the builder by defining the starting context and,
	let asset_transfer_builder = AssetTransferBuilder::using_context(karura_context)
		// registering easy-to-use tags for the assets to transfer - caller can use these tags to
		// easily identify the assets without having to worry about reanchoring and contexts
		.define_asset("KSM", ksm_id)
		// create the builder
		.create();

	let xcm = asset_transfer_builder
		// the starting context is Karura
		// withdraw assets to transfer from origin account
		.withdraw("KSM", Definite(ksm_amount))
		// set Kusama AssetHub as the destination for this leg of the transfer
		.set_next_hop(Location::new(1, X1(Parachain(KUSAMA_ASSET_HUB_PARAID))))
		// reserve-withdraw all KSM on Asset Hub
		.transfer("KSM", All, ReserveWithdraw)
		// use KSM to pay for fees on Kusama Asset Hub (no limit)
		.pay_remote_fess_with("KSM", All)
		// "execute" current leg of the transfer, move to next hop (Kusama Asset Hub)
		.execute_hop()

		// from here on, context is Kusama Asset Hub
		// set Polkadot Asset Hub as the destination for this leg of the transfer
		.set_next_hop(Location::new(
			2,
			X2(GlobalConsensus(Polkadot), Parachain(POLKADOT_ASSET_HUB_PARAID)),
		))
		// reserve-deposit KSM to Polkadot Asset Hub (asset reanchoring done behind the scenes)
		.transfer("KSM", All, ReserveDeposit)
		// use KSM to pay for fees on Polkadot Asset Hub (no limit)
		.pay_remote_fess_with("KSM", All)
		// "execute" current leg of the transfer, move to next hop (Polkadot Asset Hub)
		.execute_hop()

		// from here on, context is Polkadot Asset Hub
		// set Acala as the destination for this leg of the transfer
		.set_next_hop(Location::new(1, X1(Parachain(ACALA_PARAID))))
		// reserve-deposit KSM to Acala (asset reanchoring done behind the scenes)
		.transfer("KSM", All, ReserveDeposit)
		// use KSM to pay for fees on Acala (no limit)
		.pay_remote_fess_with("KSM", All)
		// "execute" current leg of the transfer, move to next hop (Acala)
		.execute_hop()

		// from here on, context is Acala
		// deposit all received assets to `beneficiary`
		.deposit_all(beneficiary)
		// build the asset transfer XCM Program!
		.finalize();

	// Profit!
	println!("Asset transfer XCM: {:?}", xcm);
}

cc @xlc @franciscoaguirre

[Polkadot-Forum]

This issue has been mentioned on Polkadot Forum. There might be relevant details there:

https://forum.polkadot.network/t/rfc-xcm-asset-transfer-program-builder/8528/1

[Polkadot-Forum]

This issue has been mentioned on Polkadot Forum. There might be relevant details there:

https://forum.polkadot.network/t/rfc-xcm-asset-transfer-program-builder/8528/2

[franciscoaguirre]

I really like the API for this!
We already have the XCM builder pattern as well but each method maps 1:1 to instructions.
It would make a lot of sense to supercharge this with some utilities as you describe here.
The define_asset is a good example of something very easy to add. That along with the context allows for nice behind-the-scenes reanchoring.
A utility for better separating hops is also a good idea to add, since now building multiple hops requires you to start the process from scratch.

[Polkadot-Forum]

This issue has been mentioned on Polkadot Forum. There might be relevant details there:

https://forum.polkadot.network/t/rfc-xcm-asset-transfer-program-builder/8528/3

[acatangiu]

We could start simple: single transfer type per hop (for mixing multiple asset transfer types we need RFC 100, earliest: XCMv5).

Simple example: USDC from Bifrost to Polkadex (through AH)

fn example_bifrost_to_ah_to_polkadex() {
	// Initial context needs to start with `GlobalConsensus`
	let bifrost_context = InteriorLocation::X2(GlobalConsensus(Polkadot), Parachain(BIFROST_PARAID));

	// used assets IDs as seen by the `bifrost_context` chain
	let usdc_id = AssetId(Location::new(
		1,
		X3(Parachain(ASSET_HUB_PARAID), PalletInstance(50), GeneralIndex(1337)),
	));

	// Initialize the builder by defining the starting context and,
	let asset_transfer_builder = AssetTransferBuilder::using_context(bifrost_context)
		// registering easy-to-use tags for the assets to transfer - caller can use these tags to
		// easily identify the assets without having to worry about reanchoring and contexts
		.define_asset("USDC", usdc_id)
		// create the builder
		.create();

	let xcm = asset_transfer_builder
		// the starting context is Bifrost
		// withdraw assets to transfer from origin account
		.withdraw("USDC", Definite(usdc_amount))
		// set AssetHub as the destination for this leg of the transfer
		.set_next_hop(Location::new(1, X1(Parachain(ASSET_HUB_PARAID))))
		// reserve-withdraw all USDC on Asset Hub
		.transfer("USDC", All, ReserveWithdraw)
		// use USDC to pay for fees on Asset Hub (can define upper limit)
		.pay_remote_fees_with("USDC", Definite(max_usdc_to_use_for_fees))
		// "execute" current leg of the transfer, move to next hop (Asset Hub)
		.execute_hop()

		// from here on, context is Asset Hub
		// set Polkadex as the destination for this (final) leg of the transfer
		.set_next_hop(Location::new(1, X1(Parachain(POLKADEX_PARAID))))
		// reserve-deposit USDC to Polkadex (asset reanchoring done behind the scenes)
		.transfer("USDC", All, ReserveDeposit)
		// use USDC to pay for fees on Polkadex (no limit)
		.pay_remote_fees_with("USDC", All)
		// "execute" current leg of the transfer, move to next hop (Polkadex)
		.execute_hop()

		// from here on, context is Polkadex
		// deposit all received assets to `beneficiary`
		.deposit_all(beneficiary)
		// build the asset transfer XCM Program!
		.finalize();

	// Profit!
	println!("Asset transfer XCM: {:?}", xcm);
}

This would create the following XCM program:

Xcm([
	WithdrawAsset(({ parents: 1, interior: X3(Parachain(1000), PalletInstance(50), GeneralIndex(1337)) }, 100).into()),
	InitiateReserveWithdraw {
		assets: Wild,
		reserve: { parents: 1, interior: X1(Parachain(1000) },
		xcm: Xcm([
			BuyExecution {
				fees: ({ parents: 0, interior: X2(PalletInstance(50), GeneralIndex(1337)) }, 1).into(),
			},
			DepositReserveAsset {
				assets: Wild,
				dest: { parents: 1, interior: X1(Parachain(2040) },
				xcm: Xcm([
					BuyExecution {
						fees: ({ parents: 1, interior: X3(Parachain(1000), PalletInstance(50), GeneralIndex(1337)) }, 1).into(),
					},
					DepositAsset {
						assets: Wild,
						beneficiary: ({ parents: 0, interior: X1(AccountId32(0x2cb783d5c0ddcccd2608c83d43ee6fc19320408c24764c2f8ac164b27beaee37)) }),
					},
				]),
			},
		])
	}
])

This XCM can be executed on Bifrost with pallet_xcm::execute() to enact the transfer.

Encoded call here

[acatangiu]

Bridge example - KSM from Kusama Treasury to Interlay (over bridge, going through both Asset Hubs)

Showcasing how any Polkadot parachain can get funding from Kusama Treasury (same works for Kusama parachains and Polkadot Treasury):

fn KSM_from_Kusama_to_Interlay() {
	// Initial context needs to start with `GlobalConsensus`
	let kusama_context = InteriorLocation::X1(GlobalConsensus(Kusama));

	// used assets IDs as seen by the `kusama_context` chain
	let ksm_id = AssetId(Location::new(0, Here));

	// Initialize the builder by defining the starting context and,
	let asset_transfer_builder = AssetTransferBuilder::using_context(kusama_context)
		// registering easy-to-use tags for the assets to transfer - caller can use these tags to
		// easily identify the assets without having to worry about reanchoring and contexts
		.define_asset("KSM", ksm_id)
		// create the builder
		.create();

	let xcm = asset_transfer_builder
		// the starting context is Kusama
		// withdraw assets to transfer from origin account
		.withdraw("KSM", Definite(ksm_amount))
		// set Kusama AssetHub as the destination for this leg of the transfer
		.set_next_hop(Location::new(1, X1(Parachain(KUSAMA_ASSET_HUB_PARAID))))
		// teleport all KSM on Asset Hub
		.transfer("KSM", All, Teleport)
		// use KSM to pay for fees on Kusama Asset Hub (no limit)
		.pay_remote_fees_with("KSM", All)
		// "execute" current leg of the transfer, move to next hop (Kusama Asset Hub)
		.execute_hop()

		// from here on, context is Kusama Asset Hub
		// set Polkadot Asset Hub as the destination for this leg of the transfer
		.set_next_hop(Location::new(
			2,
			X2(GlobalConsensus(Polkadot), Parachain(POLKADOT_ASSET_HUB_PARAID)),
		))
		// reserve-deposit KSM to Polkadot Asset Hub (asset reanchoring done behind the scenes)
		.transfer("KSM", All, ReserveDeposit)
		// use KSM to pay for fees on Polkadot Asset Hub (no limit)
		.pay_remote_fees_with("KSM", All)
		// "execute" current leg of the transfer, move to next hop (Polkadot Asset Hub)
		.execute_hop()

		// from here on, context is Polkadot Asset Hub
		// set Interlay as the destination for this leg of the transfer
		.set_next_hop(Location::new(1, X1(Parachain(INTERLAY_PARAID))))
		// reserve-deposit KSM to Interlay (asset reanchoring done behind the scenes)
		.transfer("KSM", All, ReserveDeposit)
		// use KSM to pay for fees on Interlay (no limit)
		.pay_remote_fees_with("KSM", All)
		// "execute" current leg of the transfer, move to next hop (Interlay)
		.execute_hop()

		// from here on, context is Interlay
		// deposit all received assets to `beneficiary`
		.deposit_all(beneficiary)
		// build the asset transfer XCM Program!
		.finalize();

	// Profit!
	println!("Asset transfer XCM: {:?}", xcm);
}

Would create the following XCM program:

// executes on Kusama Relay
Xcm([
	WithdrawAsset(({ parents: 0, interior: Here }, 100).into()),
	InitiateTeleport {
		assets: Wild,
		dest: { parents: 0, interior: X1(Parachain(1000) },
		// executes on Kusama Asset Hub
		xcm: Xcm([
			BuyExecution {
				fees: ({ parents: 1, interior: Here }, 1).into(),
			},
			DepositReserveAsset {
				assets: Wild,
				dest: { parents: 2, interior: X2(GlobalConsensus(Polkadot), Parachain(1000) },
				// executes on Polkadot Asset Hub
				xcm: Xcm([
					BuyExecution {
						fees: ({ parents: 2, interior: X1(GlobalConsensus(Kusama)) }, 1).into(),
					},
					DepositReserveAsset {
						assets: Wild,
						dest: { parents: 1, interior: X1(Parachain(2032) },
						// executes on Interlay (Polkadot parachain)
						xcm: Xcm([
							BuyExecution {
								fees: ({ parents: 2, interior: X1(GlobalConsensus(Kusama)) }, 1).into(),
							},
							DepositAsset {
								assets: Wild,
								beneficiary: ({ parents: 0, interior: X1(AccountId32(0x2cb783d5c0ddcccd2608c83d43ee6fc19320408c24764c2f8ac164b27beaee37)) }),
							},
						]),
					},
				]),
			},
		])
	}
])

[Polkadot-Forum]

This issue has been mentioned on Polkadot Forum. There might be relevant details there:

https://forum.polkadot.network/t/polkadot-technical-summit-xcm/9384/1