Deleting a mapping should not delete data

[shivangshah]

Hi,

I am just deleting the mapping so that I can recreate the mapping with different core types for some fields (especially when "updating a mapping type" doesn't really do much good on changing core types at all). And even if we don't care about the past data that does not mean it should just delete the data. At the very least this should be more of a configurable option rather than an internal clean up in elastic search. I do not want to go about reindexing all my data just because I changed a core type. I just want to keep the past data as document storage and somehow mark them as "old data". But if I delete and recreate the mapping, I don't even have a choice to do that because the corresponding documents will be deleted!

[clintongormley]

HI @shivangshah

The ability to delete a mapping has been removed in master (see #8877). If you need to change the mapping, you need to reindex.

[shivangshah]

@clintongormley but how do we update the mapping if there is no "delete" ? I understand the reindexing part. Right now if you don't delete and recreate you cannot update the core mapping of a field. How can you do it in master now?

[clintongormley]

@shivangshah you can't. you have to reindex. changing a mapping means that your existing indexed data will now be incorrect. the only way to correct that data is to reindex it all.

[shivangshah]

@clintongormley So basically you can't delete, you can't update and you are stuck with whatever is created (by first document indexed or manually put mappings) for a given field mapping?

[clintongormley]

correct

[shivangshah]

got it .. and this will be release in v2.0 I am guessing ?

[clintongormley]

Yes, but even with delete-mapping today, it is trappy. you can still have vestiges of old fields in old segments, even though you have deleted the mapping. So the same advice still applied: you want to change a field? reindex your data

[shivangshah]

reindexing the data is not a problem. The problem is with reindexing the data in a whole different index by putting new mappings to it. That's just not a good solution. My feedback here would be let the clients update the mappings and just warn them to reindex the data .. I'd rather get all the data from the same index and reindex in the same one rather than going through the route of creating a new index, reindexing all the data there, update ALL the aliases that pointed to the old index and than drop the old index. The process is extremely cumbersome especially you know that your mappings are going to change a whole lot. I described a part of our usecase here: #8870

I still believe that instead of throttling so much at the mapping level, make it more flexible for clients but make them aware of the risks (reindexing and stuff). You will have to reindex regardless when you update the mapping. The question is which is the easiest route for a client .. And my vote goes for flexible implementation + understanding of risks