I'm using two great libraries BPlusTree and Protobuf-net to store/retrieve a large number of items to/from disk. I'm allowed to modify any of the serialized items ... everything works perfect till this point. At first modification the speed drops to 1/3, at second modification it drops to 1/4 and so on as shown on following plot:
Each line represents a different run on same data; the important point is that at all tests speed degrades when an item from the collection is modified. The modified item is a list of a class; till first degradation (i.e., 22th group - roughly 22*200,000th item) this list contains only one instance. After that items one by one are updated to have more two objects of the class, till 42th group (roughly 42*200,000th item) when items start having 3 instances each, and so on so forth.
My items are from class 'B' which implements like:
public class B<C, M>
{
internal B(char tau, M metadata)
{
_lambda = new List<Lambda<C, M>>();
_lambda.Add(new Lambda<C, M>(tau: tau, atI: metadata));
}
[ProtoMember(1)]
internal int omega { private set; get; }
[ProtoMember(2)]
private List<Lambda<C, M>> _lambda { set; get; }
internal ReadOnlyCollection<Lambda<C, M>> lambda { get { return _lambda.AsReadOnly(); } }
internal B<C, M> Update(char tau, M metadata)
{
B<C, M> newB= new B<C, M>();
newB._lambda = new List<Lambda<C, M>>(this._lambda);
newB._lambda.Add(new Lambda<C, M>(tau: tau, atI: metadata));
return newB;
}
}
public class Lambda<C, M>
{
internal Lambda(char tau, M atI)
{
this.tau = tau;
this.atI = atI;
}
[ProtoMember(1)]
internal char tau { private set; get; }
[ProtoMember(2)]
internal M atI { private set; get; }
}
And I define my (de)serializer as following:
public B<C, M> ReadFrom(System.IO.Stream stream)
{
return Serializer.DeserializeWithLengthPrefix<B<C, M>>(stream, PrefixStyle.Fixed32);
}
public void WriteTo(B<C, M> value, System.IO.Stream stream)
{
Serializer.SerializeWithLengthPrefix<B<C, M>>(stream, value, PrefixStyle.Fixed32);
}
How can I say _lambda<...> size is the cause of speed drop ? Please check the following plot for clarification. As you notice, the moment _lambda<...> size is changed I start having speed penalties.
Any suggestion what is going wrong ?
PS: There are thousands of lines doing the work, but narrowing down the code seems like the problem is raised by 'ReadFrom' and 'WriteTo' functions. Hence I'm putting only these lines here.
