Dynamically generated type is not restored from persistence region after migration to Apache Ignite 2.11.0

Question

Recently we've migrated our project from Apache Ignite 2.10.0 to version 2.11.0. We use Apache Ignite to store dynamically generated .NET objects (objects with type generated by Reflection.Emit). For some objects, we use the persistence data region to be able to load data after Apache Ignite restart. After migration we faced the following issue: objects with dynamically generated types could not be loaded after server restart. The issue was not reproduced in Apache Ignite 2.10.0. Option "peerAssemblyLoadingMode" is set to "CurrentAppDomain".

Here is a simple example of the issue (actions are described in comments): https://github.com/olegbevz/ApacheIgniteDynamicTypeInPersistenceRegion

Here is an exception message:

    No matching type found for object [typeId=601048920, typeName=DynamicType1]. This usually indicates that assembly with a specified type is not loaded on a node. When using Apache.Ignite.exe, make sure to load assemblies with -assembly parameter. Alternatively, set IgniteConfiguration.PeerAssemblyLoadingMode to CurrentAppDomain.

   at Apache.Ignite.Core.Impl.Binary.BinaryReader.ReadFullObject[T](Int32 pos, Type typeOverride)
   at Apache.Ignite.Core.Impl.Binary.BinaryReader.TryDeserialize[T](T& res, Type typeOverride)
   at Apache.Ignite.Core.Impl.Binary.BinaryReader.Deserialize[T](Type typeOverride)
   at Apache.Ignite.Core.Impl.Binary.BinaryReader.ReadBinaryObject[T](Boolean doDetach)
   at Apache.Ignite.Core.Impl.Binary.BinaryReader.TryDeserialize[T](T& res, Type typeOverride)
   at Apache.Ignite.Core.Impl.Binary.BinaryReader.Deserialize[T](Type typeOverride)
   at Apache.Ignite.Core.Impl.Binary.BinaryReader.ReadObject[T]()
   at Apache.Ignite.Linq.Impl.CacheFieldsQueryExecutor.<GetResultSelector>b__e[T](IBinaryRawReader reader, Int32 count)
   at Apache.Ignite.Core.Impl.Client.Cache.CacheClient`2.<>c__DisplayClass8d`1.<GetFieldsCursorNoColumnNames>b__8c(BinaryReader r)
   at Apache.Ignite.Core.Impl.Cache.Query.QueryCursorBase`1.ConvertGetBatch(IBinaryStream stream)
   at Apache.Ignite.Core.Impl.Cache.Query.QueryCursorBase`1..ctor(Marshaller marsh, Boolean keepBinary, Func`2 readFunc, IBinaryStream initialBatchStream)
   at Apache.Ignite.Core.Impl.Client.Cache.Query.ClientQueryCursorBase`1..ctor(ClientSocket socket, Int64 cursorId, Boolean keepBinary, IBinaryStream initialBatchStream, ClientOp getPageOp, Func`2 readFunc)
   at Apache.Ignite.Core.Impl.Client.Cache.CacheClient`2.GetFieldsCursorNoColumnNames[T](ClientResponseContext ctx, Func`3 readerFunc)
   at Apache.Ignite.Core.Impl.Client.Cache.CacheClient`2.<>c__DisplayClass2f`1.<Query>b__2e(ClientResponseContext ctx)
   at Apache.Ignite.Core.Impl.Client.ClientSocket.DecodeResponse[T](BinaryHeapStream stream, Func`2 readFunc, Func`3 errorFunc)
   at Apache.Ignite.Core.Impl.Client.ClientSocket.DoOutInOp[T](ClientOp opId, Action`1 writeAction, Func`2 readFunc, Func`3 errorFunc)
   at Apache.Ignite.Core.Impl.Client.ClientFailoverSocket.DoOutInOp[T](ClientOp opId, Action`1 writeAction, Func`2 readFunc, Func`3 errorFunc)
   at Apache.Ignite.Core.Impl.Client.Cache.CacheClient`2.DoOutInOp[T](ClientOp opId, Action`1 writeAction, Func`2 readFunc)
   at Apache.Ignite.Core.Impl.Client.Cache.CacheClient`2.Query[T](SqlFieldsQuery sqlFieldsQuery, Func`3 readerFunc)
   at Apache.Ignite.Linq.Impl.CacheFieldsQueryExecutor.ExecuteCollection[T](QueryModel queryModel)
   at Remotion.Linq.Clauses.StreamedData.StreamedSequenceInfo.ExecuteCollectionQueryModel[T](QueryModel queryModel, IQueryExecutor executor)
   at Remotion.Linq.Clauses.StreamedData.StreamedSequenceInfo.ExecuteQueryModel(QueryModel queryModel, IQueryExecutor executor)
   at Remotion.Linq.QueryModel.Execute(IQueryExecutor executor)
   at Apache.Ignite.Linq.Impl.CacheFieldsQueryProvider.Execute(Expression expression)
   at Apache.Ignite.Linq.Impl.CacheFieldsQueryProvider.Execute[TResult](Expression expression)
   at Remotion.Linq.QueryableBase`1.GetEnumerator()
   at System.Collections.Generic.LargeArrayBuilder`1.AddRange(IEnumerable`1 items)
   at System.Collections.Generic.EnumerableHelpers.ToArray[T](IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)

I'll investigate and get back to you. – Pavel Tupitsyn Nov 18 '21 at 14:36 — Pavel Tupitsyn, Nov 18 '21 at 14:36

score 1 · Answer 1 · answered Nov 18 '21 at 17:45

1

This is a regression, I've explicitly added if (asm.IsDynamic) return null; as part of unrelated change thinking that we don't need them.

Ticket filed, you can expect the fix in 2.12: https://issues.apache.org/jira/browse/IGNITE-15954

A workaround is to register the dynamic type explicitly before using it:

    protected IEnumerable GetAll<TValue>()
    {
        _igniteClient.GetBinary().GetBinaryType(typeof(TValue));

        return _igniteClient.GetCache<int, TValue>(_cacheConfiguration.Name)
            .AsCacheQueryable()
            .Select(x => x.Value)
            .ToArray();
    }

answered Nov 18 '21 at 17:45

Pavel Tupitsyn

8,393
3
22
44

Pavel, thank you, workaround works. BTW is there any way to reach you directly? to not create questions on stackoverflow each time – Oleg Bevz Nov 22 '21 at 10:50
1

@OlegBevz when we discuss things on StackOverflow, other people can find that later and solve similar issues :) If we discuss something in private, there is no such benefit. Don't hesitate to create more questions! Also I welcome you to join our Telegram chat for a more informal setting https://t.me/RU_Ignite – Pavel Tupitsyn Nov 22 '21 at 14:54

Dynamically generated type is not restored from persistence region after migration to Apache Ignite 2.11.0

1 Answers1