Salable Quantity Calculation and Mechanism of Reservations
- General Infrastructure and Reservation Entity
- Description of Order Placement operation Step by Step
- Description of Order Processing
- More sophisticated Computation of Salable Quantity
- Reservation API
- Tasks on GitHub
Reservation - is an entity which is used to keep calculation of Salable product quantity consistent, and prevent overselling. It is created as an "inventory request" when an order is placed and exists until the time when the order would be processed and corresponding source deduction (deduction of specific SourceItems) happen, along with that the initial Reservation should be compensated. Introducing Reservation entity we could be sure that merchant would not sell more products than he has in stock even if latency between order placement and order processing (deduction of specific SourceItems) is high. Reservations are append-only operations and help us to prevent blocking operations and race conditions at the time of checkout.
There are 3 physical sources: Source A, Source B, Source C and There is Product with SKU-1 which is stored on each of these sources in next quantity:
- SourceItem A — 20
- SourceItem B — 25
- SourceItem C — 10
There is the only sales channel (Default Website). For this sales channel, we create a virtual aggregated stock - Stock A and assign all existing physical sources to it. Thus, StockItem A for SKU-1 has Quantity 20+25+10 = 55
When a customer comes to the website, the system detects Stock which should be used (in our case Website -> Stock A), in the scope of this Stock the system calculates Salable Quantities for each product by this formula:
StockItem Qty (qty from Stock Item index) + All Reservations created for given SKU on the given StockId
In our case for SKU-1 and Stock A
Let's assume customer comes to Default Website and wants to buy product SKU-1 in the amount of 30 items.
Magento needs to decide whether we can sell (do we have enough products to sell in stock), Quantity of SKU-1 on StockItem A is 55, plus an aggregated quantity of all the reservations for product SKU-1 on Stock A. In our case there are no reservations created, so the number is 0, 55 - 0 > 30, so we can proceed to checkout and place an order.
At the time of order placement, the system is agnostic to the fact from which physical sources the order would be fulfilled and the qty of SKU-1 would be deducted afterwards, that's why we don't use SourceItem interfaces during this process (order placement). Also, we can't deduct Qty of StockItem A, because it's read-only interface and represents index value. Thus, we create a Reservation for SKU-1 on Stock A in the amount of (-30) items. Reservation creation is append-only operation, so there are no checks and blocking operations (locks) needed.
Amount of SKU-1 on physical sources:
- SourceItem A — 20
- SourceItem B — 25
- SourceItem C — 10
The quantity of SKU-1 on StockItem A — 55 (has not changed) Reservation for SKU-1 on Stock A created in the amount of (-30) items.
While we didn't process first order yet, because of high latency, another customer comes to the website and wants to order SKU-1 in amount of 10 items.
Magento starts to follow the steps mentioned above. Magento needs to decide whether we can sell (do we have enough products to sell in stock), Quantity of SKU-1 on StockItem A is 55, plus an aggregated quantity of all the reservations for product SKU-1 on Stock A 55 + (-30) = 25 > 10 Thus, we make a decision that we can proceed to checkout.
We do not bind a Reservation to Order placed (by order id, or order item ids), because Order is not the only business event which could emit reservations. For example, one of the features out of MSI product backlog is to introduce Shopping Cart reservations where fixed time reservation (say, reservation for 15 minutes) would be created as soon as customer will add a product into his shopping cart to guarantee that particular product is reserved for the customer for particular amount of time, thus he/she can continue shopping and no need immediately proceed to checkout to make sure that he can get desirable product. But along with that MSI provides an ability to log Business event which produced given reservation. This data could be found in Reservation Metadata, and looks like:
mysql> select * from inventory_reservation;
+----------------+----------+------------------------+-----------+----------------------------------------------------------------------------+
| reservation_id | stock_id | sku | quantity | metadata |
+----------------+----------+------------------------+-----------+----------------------------------------------------------------------------+
| 21 | 2 | configurable -red | -13.0000 | {"event_type":"order_placed","object_type":"order","object_id":"8"} |
| 22 | 2 | configurable -red | 13.0000 | {"event_type":"creditmemo_created","object_type":"order","object_id":"8"} |
| 23 | 2 | testSimpleProduct2 | -10.0000 | {"event_type":"order_placed","object_type":"order","object_id":"9"} |
| 24 | 2 | testSimpleProduct2 | 5.0000 | {"event_type":"shipment_created","object_type":"order","object_id":"9"} |
| 25 | 2 | testSimpleProduct2 | 5.0000 | {"event_type":"shipment_created","object_type":"order","object_id":"9"} |
| 29 | 2 | testSimpleProduct2 | -15.0000 | {"event_type":"order_placed","object_type":"order","object_id":"11"} |
| 30 | 2 | testSimpleProduct2 | 5.0000 | {"event_type":"shipment_created","object_type":"order","object_id":"11"} |
| 31 | 2 | testSimpleProduct2 | 5.0000 | {"event_type":"creditmemo_created","object_type":"order","object_id":"11"} |
| 32 | 2 | testSimpleProduct2 | 5.0000 | {"event_type":"creditmemo_created","object_type":"order","object_id":"11"} |
| 33 | 1 | testSimpleProduct | -10.0000 | {"event_type":"order_placed","object_type":"order","object_id":"12"} |
| 34 | 1 | testSimpleProduct | 10.0000 | {"event_type":"shipment_created","object_type":"order","object_id":"12"} |
| 35 | 1 | testSimpleProduct | -10.0000 | {"event_type":"order_placed","object_type":"order","object_id":"13"} |
| 36 | 1 | testSimpleProduct | 10.0000 | {"event_type":"order_canceled","object_type":"order","object_id":"13"} |
We consider Reservation as append-only operation. Like a log of events (in Event Sourcing terms). Our stock calculation for product(SKU) is next: get StockItem Quantity (which represents aggregated amount among all the physical sources for the current Scope/SalesChannel) for particular SKU plus all created reservations for this SKU made in the same Scope/SalesChannel. So, let’s imagine that Customer A bought 30 items of some product - system creates a reservation for this sale.
ReservationID - 1 , StockId - 1, SKU - SKU-1, Qty - (-30)
if the order is canceled the system just creates another Reservation
ReservationID - 2 , StockId - 1, SKU - SKU-1, Qty - (+30)
So, new Inventory infrastructure doesn't remove or modify already created reservations, just append another one, which makes quantity correction (we call these kinds of reservations - compensational ones).
So, the second reservation will compensate the 1st one Like (-30) + 30 = 0
As from the calculation perspective it would be easier to have both negative (<0) and positive (>0) Qty values in Reservations. Like when we placed an order -> we created Reservation with Qty -30, when we processed the Order and deducted SourceItems -> we created a reservation with Qty +30 That will provide an efficient way of how we can get Sum of Grouped Reservations. For example, executing this query:
select
SUM(r.qty) as total_reservation_qty
from
Reservations as r
where
stockId = {%id%} and sku = {%sku%}pretty simple query, and we've given total reservation qty.
As it was mentioned above, just initial reservation (when the order is placed) has negative quantity value, all further reservations created while processing given order suppose to compensate the initial one, and finally when the order will come to the finite state (complete|canceled) - the sum of all created reservations while working with it should be ZERO.
There are some tricky cases of computation for complex order lifecycles, especially if the partial invoice involved, because Magento currently does not provide a possibility to track particular item in the scope of ordered quantity. That's why system should make an assumption what merchant wants to accomplish making some operation with an order. Sometimes this is especially tricky taking into account that magento allows to ship non invoiced products.
Here is the main assumption MSI does. Let's consider next example:
- Order Placed for SKU-1 in Qty = 10 (Reservation for SKU-1 in Qty -10 created)
- Partial Invoice created for Qty = 7
- Partial Shipment created for Qty = 3 (Source Selection Algorithm suggests which source should be deducted and merchant creates shipment compensating Qty +3 with reservation and deducting SourceItem Qty on phisical source)
- Credit Memo created for Qty = 5
???
At this point the system should make an assumption and decide whether already Shipped products should be refunded and send back to merchant or system will preferably refund Invoiced, but not shipped products.
As Magento currently does not provide this possibility, the inventory system should do it by itself. Thus, MSI tries to refund Invoiced, but not shipped items first and if there are not enough such items, MSI would refund already shipped items then.
Based on the above the system will handle Credit Memo created for Qty = 5 next way:
- Current amount of invoiced but not shipped items is 7 - 3 = 4, so system will compensate them first, creating compensational reservation in Qty = +4
- Then the system should refund and return to stock one of 3 items which were already shipped, but in this case no need to create Compensational Reservation, as this Qty has been already compensated when Merchant shipped this item. So, in this case just SourceItem quantity is increased on +1 directly.
After step 4. the system will have next reservations created (-10 +3 +4) and the SourceItem quantity deducted by 2, and there are still 3 products awaiting to be handled in the scope of this Order.
/**
* Copyright © Magento, Inc. All rights reserved.
* See COPYING.txt for license details.
*/
declare(strict_types=1);
namespace Magento\InventoryReservationsApi\Model;
/**
* The entity responsible for reservations, created to keep inventory amount (product quantity) up-to-date.
* It is created to have a state between order creation and inventory deduction (deduction of specific SourceItems).
*
* Reservations are designed to be immutable entities.
*
* @api
*/
interface ReservationInterface
{
/**
* Constants for keys of data array. Identical to the name of the getter in snake case
*/
const RESERVATION_ID = 'reservation_id';
const STOCK_ID = 'stock_id';
const SKU = 'sku';
const QUANTITY = 'quantity';
const METADATA = 'metadata';
/**
* Get Reservation Id
*
* @return int|null
*/
public function getReservationId(): ?int;
/**
* Get Stock Id
*
* @return int
*/
public function getStockId(): int;
/**
* Get Product SKU
*
* @return string
*/
public function getSku(): string;
/**
* Get Product Qty
*
* This value can be positive (>0) or negative (<0) depending on the Reservation semantic.
*
* For example, when an Order is placed, a Reservation with negative quantity is appended.
* When that Order is processed and the SourceItems related to ordered products are updated, a Reservation with
* positive quantity is appended to neglect the first one.
*
* @return float
*/
public function getQuantity(): float;
/**
* Get Reservation Metadata
*
* Metadata is used to store serialized data that encapsulates the semantic of a Reservation.
*
* @return string|null
*/
public function getMetadata(): ?string;
}We no need to expose Reservation API for WebAPI (REST and SOAP), because we can consider Reservations as SPI, which created as a side-effect of some particular business operation (like order placement, or return).
Currently, in Magento 2 WebAPI imposes some restrictions for entity interfaces (existence getter and setter methods).
Thus, if we would not expose Reservation entity for WebAPI (REST, SOAP) -> we could use any interface we want (don't have mandatory setter methods).
And because we agreed that Reservations are append-only immutable entities we could eliminate all the setter methods.
So, we will end-up with ReservationInterface consisting of just getter methods.
And we need to introduce ReservationBuilderInterface which will allow the possibility to set data into the reservation when we need to create one.
After that, we could build Reservation entity.
$reservations[] = $this->reservationBuilder
->setSku($item->getSku())
->setQuantity((float)$item->getQuantity())
->setStockId($stockId)
->setMetadata($this->serializer->serialize($this->salesEventToArrayConverter->execute($salesEvent)))
->build();
$reservationAppend->execute($reservations);Doing so, we could ensure immutability on the level of Reservation interface.
Append Reservation Service - is an internal service (Service Provider Interface, SPI) used at a time when a business event which leads to reservation creation happened (for example, Order Placement/Cancelled/Shipped/Refunded). At this time, we create a bunch of Reservations, each one responsible for particular SKU and add these reservations for being processing. Responsibility of the service is to guarantee that client doesn't use ReservationAppend service to update already created reservations. Because Reservations are append-only entities. For example, if we will use Database generated IDs, we could check the ReservationId which is passed in the scope of ReservationInterface is nullified.
interface AppendReservationsInterface
{
/**
* Append reservations
*
* @param ReservationInterface[] $reservations
* @return void
* @throws \Magento\Framework\Exception\InputException
* @throws \Magento\Framework\Exception\CouldNotSaveException
*/
public function execute(array $reservations): void;
}AppendReservationsInterface service should NOT be used directly in the business logic which emits sales business event. A more high-level service should be used instead:
namespace Magento\InventorySalesApi\Api;
/**
* This service is responsible for creating reservations upon a sale event.
*
* @api
*/
interface PlaceReservationsForSalesEventInterface
{
/**
* @param \Magento\InventorySalesApi\Api\Data\ItemToSellInterface[] $items
* @param \Magento\InventorySalesApi\Api\Data\SalesChannelInterface $salesChannel
* @param \Magento\InventorySalesApi\Api\Data\SalesEventInterface $salesEvent
* @return void
*
* @throws \Magento\Framework\Exception\LocalizedException
* @throws \Magento\Framework\Exception\InputException
* @throws \Magento\Framework\Exception\CouldNotSaveException
*/
public function execute(
array $items,
\Magento\InventorySalesApi\Api\Data\SalesChannelInterface $salesChannel,
\Magento\InventorySalesApi\Api\Data\SalesEventInterface $salesEvent
): void;
}The main difference in comparison to previos CatalogInventory implementation is that Quantity is not a static data and should be always retrieved as a result of dedicated Service call. That's why we don't have anymore "StockItem" data interface which is part of Product entity. There are a bunch of dynamic services introduced instead of it.
To get Salable Product Quantity for specified Stock there is GetProductSalableQtyInterface service.
declare(strict_types=1);
namespace Magento\InventorySalesApi\Api;
/**
* Service which returns Quantity of products available to be sold by Product SKU and Stock Id.
* This service calculates the salable qty taking into account existing reservations for
* given sku and stock id and subtracting min qty (a.k.a. "Out-of-Stock Threshold")
*
* @api
*/
interface GetProductSalableQtyInterface
{
/**
* Get Product Quantity for given SKU and Stock
*
* @param string $sku
* @param int $stockId
* @return float
* @throws \Magento\Framework\Exception\InputException
* @throws \Magento\Framework\Exception\LocalizedException
*/
public function execute(string $sku, int $stockId): float;Another service added to check whether product is in Stock - IsProductSalableInterface
declare(strict_types=1);
namespace Magento\InventorySalesApi\Api;
/**
* Service which detects whether Product is salable for a given Stock (stock data + reservations)
*
* @api
*/
interface IsProductSalableInterface
{
/**
* Get is product in salable for given SKU in a given Stock
*
* @param string $sku
* @param int $stockId
* @return bool
*/
public function execute(string $sku, int $stockId): bool;
}And the last but not least service to check whether we have enough salable quantity to fulfill Order or Place product into the Shopping Cart - IsProductSalableForRequestedQtyInterface
declare(strict_types=1);
namespace Magento\InventorySalesApi\Api;
/**
* Service which detects whether a certain Qty of Product is salable for a given Stock (stock data + reservations)
*
* @api
*/
interface IsProductSalableForRequestedQtyInterface
{
/**
* Get is product salable for given SKU in a given Stock for a certain Qty
*
* @param string $sku
* @param int $stockId
* @param float $requestedQty
* @return \Magento\InventorySalesApi\Api\Data\ProductSalableResultInterface
* @throws \Magento\Framework\Exception\LocalizedException
*/
public function execute(
string $sku,
int $stockId,
float $requestedQty
): \Magento\InventorySalesApi\Api\Data\ProductSalableResultInterface;
}As reservations are append-only operations it was decided not to modify the status of created Reservation, but add another reservation which neglects already existing Reservation (like in the example above -30 +30 = 0). From the inventory point of view we don't bind Reservation to Order or other business operation, that's why we don't introduce Reservation Statuses (and apply State Machine design pattern for changing the reservation from one state to another one). All we need to do is to create another reservation. That's all.
Order Placed for SKU-1 in Qty 30 => Created Reservation for SKU-1 with Qty (-30)
Canceled above order => Created Reservation for SKU-1 with Qty = (+30)
or
Completed above order => Created Reservation for SKU-1 with Qty = (+30)
Idea is to clear reservation table (if needed) to prevent overloading, finding Complete chains of reservations. When we have a chain of reservations, the sum of which is equal to O (Zero), like -30 and +30. These reservations don't affect the final quantity, thus could be deleted.
Launching a script periodically we could find such pairs and remove them from the table not affecting the calculation.
select
GROUP_CONCAT(reservation_id)
from
inventory_reservation as r
group by
stock_id, sku
having
SUM(quantity) = 0After executing this query we will get all the reservations to be deleted.
It doesn't matter how fast (read how slow) is above query because it's executed for service purposes only to remove unneeded reservations.
GitHub Issues related to quantity Calculation labeled with "Wrong QTY Calculation" label.
Thus, you can find all the tickets related to this topic following the Label above.