Saved Data Schema Migration

[Cranktrain]

Hi Joel, I have a hard time imagining that this hasn't been discussed before, so please forgive me if so! The forum search didn't turn up anything.

My game features a fairly complex simulation, with lots of non-trivial data (often nested) getting saved, I think I'm at about 70 custom types so far. As I'm in the midst of development, I'm often changing the schema of my data and what this leads to is of course a breaking of compatibility with my old save files. I don't believe the answer is just to do more ES2.Exists checks, as I'm not just moving from one save Key/id to another, but changing the layout/Type of the data saved.

The nightmare scenario I'm envisioning is this:

• I release my game on Steam.
• Oops! There's a bug. I fix it, but in doing so unwittingly break how old games are loaded or saved in some instance.
• I brick my player's progress and they have to start their games again.

If I merely save the 'version' of save file layouts I'm working on, and then spread this sort of code throughout my scripts:

Code: Select all

if (saveVersion == 1) {
    load in this way
} else if (saveVersion == 2) {
    load in this other way
} if (saveVersion == 3) {
    load in yet another way
}

...I think I'll lose my mind. The range/nestedness of data and the way it's often so strongly coupled in my game doesn't make this option seem good for my sanity.

In my view; when loading some file, a pre-load check for the version of save should be made, and then the file itself ought to be processed through a series (version 1 -> version 2 -> etc -> most up-to-date version) of migration routines to 'patch' the file itself. In another life I was a backend web developer, and database scheme migration is its own ordeal. Quick-and-easy fixes tend to go badly, and often there's significant tooling in order to do everything required to minimise a data-loss catastrophy, so I know I'm not opening up a small topic of discussion.

What are your thoughts on this, Joel? Any helpful patterns, or things to keep in mind that I should ponder?

[Joel]

Hi there,

The easiest solution would be to use Easy Save 3, as changes to an ES3Type do not invalidate the save data.

Because in Easy Save 2 ES2Types are designed to represent the lowest level of serialisation, they're written sequentially. Unfortunately for this reason it's necessary to manually manage your changes using version numbers within the ES2Type.

All the best,
Joel