Chunk Problem: Missing Tag

[CodyScheer]

My world is losing chunks when upgrading from 1.12 to 1.13. I figured it was because of chunks being corrupted during the conversion process. I ran this tool and expected to find corrupted chunks, but none were found. Instead, the vast majority of my chunks were flagged as "Missing Tag". I'm not seeing any reference to this chunk issue at this page: https://github.com/Fenixin/Minecraft-Region-Fixer/wiki/Chunk-and-region-problems

Here are my scan results:

Welcome to Region Fixer!
(version: 0.2.2)

############################################################
##############  Scanning world: <world>  ##############
############################################################

World info:
There are 635 region files, 2927 player files and 711 data files in the world directory.

-------------------- Checking level.dat --------------------
'level.dat' is readable

---------------- Scanning UUID player files ----------------
2927 of 2927|####################################################|Time: 0:00:05

------------- Scanning old format player files -------------
Info: No files to scan.

---------- Scanning structures and map data files ----------
711 of 711|######################################################|Time: 0:01:10

------------------ Scanning region files -------------------
635 of 635|######################################################|Time: 0:05:46


############################################################
############## Scan results for: <world> ##############
############################################################

miss tag prob 433306
WHAT THE ACTUAL FFFF

Unreadable player files:
No problems found.

Unreadable data files:
No problems found.

Chunk problems:
---------------------------------------------------------------------------------
| Problem | Corrupted  Wrong l.  Entities  Shared o.  Missing tag  Total chunks |
---------------------------------------------------------------------------------
| Counts  |     0         0         0          0        433306        505778    |
---------------------------------------------------------------------------------

Region problems:
No problems found.

The scan output incudes:

miss tag prob 433306
WHAT THE ACTUAL FFFF

I'd say this is probably a bad sign?

[Fenixin]

Hello!

Thanks for your reporting this.

miss tag prob 433306
WHAT THE ACTUAL FFFF

That some very proffesional debugging code I left there by accident, I think it's no longer there, sorry for that. Git bash, windows and python do some really strange things sometimes.

The "Missing Tag" is a completely new issue that has popped up recently. The problem is that those chunk have no "Entities" tag . I don't really know if this is a feature of a problem. but looking at https://minecraft.gamepedia.com/Chunk_format I get the idea that the "Entities" tag is mandatory and this should be treated as a problem.

Right now regionfixer can only detect these chunks, if you want to remove them I can make a quick change to test if that helps.

[CodyScheer]

Not having looked into the "Entities" tag (will do that next), would it be at all possible to recreate the missing tags? I'm hoping to come out on the other side of this without having to remove chunks. That being said, I think it would be a good addition to your tool if it is optional.

[CodyScheer]

After digging into my NBT structure, I noticed that both the Entities and Sections tags are missing for the chunks/regions not showing in my converted world. The lack of the Sections tag must be the culprit, as I believe that is where the actual block data is stored? Of course, there will be no way to rebuild those, I will have to discover what is causing these Sections (and Entities) tags to disappear when the world is converted.

[CodyScheer]

It turns out that the Entities and Sections tags are left out when the 1.12 to 1.13 conversion process decides not to convert a given chunk. In my case this was due to the lightPopulated tag being set to false in many of my region files. It appears this is due to opening the world in a 3rd party tool (Spigot included). I was directed to the following Mojang bug report:
https://bugs.mojang.com/browse/MC-133855

This post explains that if the world has ever been opened in Spigot, it will need to be converted in Spigot. I had tried to do this initially on my production server but that had resulted in a broken world a number of times. After that I pulled all of my worlds to my local machine to try and convert with the Vanilla client. According to the above bug report, the vanilla client will not convert chunks with the lightPopulated tag set to false. When I learned this, I spun up a local Spigot server and tried the conversion there as opposed to my production server. The conversion took about 30 minutes but all seems to be well.

What does this mean for you, Fenixin? You may want to update your tool to state that missing tags could be the result of a botched conversion of world formats. You can include as much of the above case as you would like, but I'm not sure if the symptom (missing tags in NBT structure) can always be linked to the same cause (lightPopulated tag set to false which breaks conversion).

[Fenixin]

Thanks very much for sharing your research.

I want want to add the option to add missing mandatory tags to regionfixer. And I will update the wiki with this information, maybe even adding some kind of warning in the tool. But to do both things I need some free time.

It seems that you have fixed your world (without my help) and that you don't need anything else, I will live this open so I remember to add an option to recreate the tags.

[Fenixin]

I'm going to close this issue, Entities, POIs and world format have changed a lot. feel free to open a new one if needed.