Page 1 of 1

Background Thread Error

Posted: Mon May 18, 2020 5:37 am
by Byte2020
Hi, I am running into NullReferenceException when calling Load from a background thread some of the time. (full error log below) Some other times, it runs OK.

I setup ES3 to load from background thread using System.Thread. I am using Version: 3.3.1f2 • May 13, 2020

The simplified code

Code: Select all

     
       // Main Thread
        public LoadFromDisk()
        {
            ES3Settings settings = new ES3Settings();
            Thread thread = new Thread(() => LoadAndUpdateValues(settings));
            thread.Start();
        }

        // Background Thread
        protected void LoadAndUpdateValues(ES3Settings settings)
        {
            string path = GetPathString;
            Foo foo = ES3.Load<Foo>(path, path, settings);

            // Update Values.
        }
I tried some other ways as well, like using the two below with the same inconsistency results.

Code: Select all

ES3.Load<Foo>(path, settings);
//or
reader.Read<Foo>(key)
Error log

Code: Select all

NullReferenceException: Object reference not set to an instance of an object
System.Collections.Generic.Dictionary`2[TKey,TValue].TryInsert (TKey key, TValue value, System.Collections.Generic.InsertionBehavior behavior) (at <437ba245d8404784b9fbab9b439ac908>:0)
System.Collections.Generic.Dictionary`2[TKey,TValue].set_Item (TKey key, TValue value) (at <437ba245d8404784b9fbab9b439ac908>:0)
ES3Internal.ES3TypeMgr.Add (System.Type type, ES3Types.ES3Type es3Type) (at Assets/Plugins/Easy Save 3/Scripts/Types/ES3TypeMgr.cs:47)
ES3Types.ES3Type..ctor (System.Type type) (at Assets/Plugins/Easy Save 3/Scripts/Types/ES3Type.cs:29)
ES3Types.ES3Type_ES3Prefab..ctor () (at Assets/Plugins/Easy Save 3/Scripts/ES3Prefab.cs:139)
System.Reflection.MonoCMethod.InternalInvoke (System.Object obj, System.Object[] parameters) (at <437ba245d8404784b9fbab9b439ac908>:0)
Rethrow as TargetInvocationException: Exception has been thrown by the target of an invocation.
System.Reflection.MonoCMethod.InternalInvoke (System.Object obj, System.Object[] parameters) (at <437ba245d8404784b9fbab9b439ac908>:0)
System.RuntimeType.CreateInstanceMono (System.Boolean nonPublic) (at <437ba245d8404784b9fbab9b439ac908>:0)
System.RuntimeType.CreateInstanceSlow (System.Boolean publicOnly, System.Boolean skipCheckThis, System.Boolean fillCache, System.Threading.StackCrawlMark& stackMark) (at <437ba245d8404784b9fbab9b439ac908>:0)
System.RuntimeType.CreateInstanceDefaultCtor (System.Boolean publicOnly, System.Boolean skipCheckThis, System.Boolean fillCache, System.Threading.StackCrawlMark& stackMark) (at <437ba245d8404784b9fbab9b439ac908>:0)
System.Activator.CreateInstance (System.Type type, System.Boolean nonPublic) (at <437ba245d8404784b9fbab9b439ac908>:0)
System.Activator.CreateInstance (System.Type type) (at <437ba245d8404784b9fbab9b439ac908>:0)
ES3Internal.ES3Reflection.GetInstances[T] () (at Assets/Plugins/Easy Save 3/Scripts/ES3Reflection.cs:312)
ES3Internal.ES3TypeMgr.Init () (at Assets/Plugins/Easy Save 3/Scripts/Types/ES3TypeMgr.cs:127)
ES3Internal.ES3TypeMgr.GetOrCreateES3Type (System.Type type, System.Boolean throwException) (at Assets/Plugins/Easy Save 3/Scripts/Types/ES3TypeMgr.cs:18)
ES3Reader.Read[T] (System.String key) (at Assets/Plugins/Easy Save 3/Scripts/Readers/ES3Reader.cs:181)
ES3.Load[T] (System.String key, ES3Settings settings) (at Assets/Plugins/Easy Save 3/Scripts/ES3.cs:374)
ES3.Load[T] (System.String key, System.String filePath, ES3Settings settings) (at Assets/Plugins/Easy Save 3/Scripts/ES3.cs:358)
//...LoadAndUpdateValues

Re: Background Thread Error

Posted: Mon May 18, 2020 7:22 am
by Joel
Hi there,

If you're running in a separate thread to improve performance, I would actually recommend storing to cache as described here. This eliminates the overhead of random access, and doesn't require any

If you absolutely need to run in a separate thread, it's first worth mentioning that much of Unity's API is not thread safe, so you should ensure that the type of data you're loading is itself thread safe.

If using it in another thread, you should also ensure that Easy Save is only used in one thread at a time. The error you're receiving occurs when two threads try to modify the ES3Types Dictionary at the same time.

It would also be worth called ES3.Init() synchronously in the main thread before calling any Easy Save methods asynchronously.

All the best,
Joel

Re: Background Thread Error

Posted: Mon May 18, 2020 3:27 pm
by Byte2020
Thank Joel,

It is indeed the race condition accessing ES3Types, introducing a lock works reliably.
I will look into the cache options.

Best,
Byte

Re: Background Thread Error

Posted: Wed Jan 20, 2021 8:33 pm
by Byte2020
Hi Joel,

I just got around to circle back to this. I went the route of using cache as you mentioned. However, I cannot seem to figure out how to store/retrieve the saved files.

I followed https://docs.moodkie.com/easy-save-3/es ... rformance/

Code: Select all

ES3.StoreCachedFile();
ES3.CacheFile("MyFile.es3");
I cannot seem to find the right way to specify the path where it is stored? Also how to retrieve it from specific folder?

I tried

Code: Select all

          
            ES3.StoreCachedFile(new ES3Settings(FolderPath() + "/test.es3")); 
With error FileNotFoundException: The file [redacted]/test.es3 could not be stored because it could not be found in the cache.

Could you point me to the right api?

Thanks!
Byte

Re: Background Thread Error

Posted: Wed Jan 20, 2021 8:47 pm
by Joel
Hi there,

The first parameter of the ES3.StoreCachedFile method is a filePath, which can be an absolute path, or a relative path (which will be relative to Application.persistentDataPath).

Regarding your error, please could you show me the code you're using to store data in the cache (or put a file into the cache)?

All the best,
Joel

Re: Background Thread Error

Posted: Wed Jan 20, 2021 9:50 pm
by Byte2020
Thanks Joel,

I simplified my code here. I am sure I misunderstood some fundamentals about caching system.

Code: Select all

void Store(string key, string value)
{
  ES3.Save(key, value, new ES3Settings(ES3.Location.Cache));
}

void StoreCache()
{
  ES3.StoreCachedFile(new ES3Settings(Application.streamingAssetsPath + "/Storage.es3"));
}

Start() 
{
  Store("k","v");
  StoreCache();
}

Re: Background Thread Error

Posted: Wed Jan 20, 2021 10:00 pm
by Joel
Hi there,

Your ES3.Save call is saving to the default file, but your ES3.StoreCachedFile is storing to a file in streamingAssetsPath called Storage.es3. You should use the same parameters for both.

All the best,
Joel