Remove deals from the index if they fail to seal on time or are slashed

[brendalee]

SP reported seeing this error when retrieving:
ERROR retrieval impl/provider.go:379 Retrieval query: GetAsk: GetRetrievalPricingInput: failed to fetch storage deal state: 1 error occurred:
* deal 8835657 not found - deal may not have completed sealing before deal proposal start epoch, or deal may have been slashed

From Dirk:
The reason for the GetRetrievalPricingInput: failed to fetch error is because
the client attempts to retrieve deal 8835657
in order to work out the price of the deal, boost queries the chain for the deal's Piece Size and Verified Deal status
the deal is not on chain (presumably because it was not sealed in time)

TODO:
To fix this we should add code to boost that will remove deals from the index if they fail to seal on time or are slashed.

[willscott]

I wonder if removing the advertisement is the fix?

if the SP has the data, just because it hasn't proved it on-chain / hasn't started the official deal term doesn't mean the SP shouldn't be able to make money from serving retrievals of that data.

[brendalee]

Interesting - so you're saying that they might have the unsealed data around somewhere, but something just went wrong with sealing / on-chain proofs?

That is an interesting use case, I don't think I have enough info on how common this is. Would you propose in this case, we query the piece directory directly to figure out whether or not the SP has the CID stored somewhere?

[willscott]

a client can man agree on a deal with the SP and when you commit to it on chain you say e.g. "this will start in 3 days" - to give time for data transfer and then sealing.

The data transfer will happen much faster than 3 days - so there may well be a deal with that data, but it might not have started yet (or likewise it may have finished but the SP might not have deleted it.)
If we find a deal for that CID, even if not active, that should have the pricing for that data

[jacobheun]

Proposal currently being discussed in #892 should resolve this once implemented.