Serialization issue with protobuf and WCF

Question

I am using protobuf to to serialize dataset and send it from WCF service to winfom client.

I am facing two issues here...

DataSet is coming from DB and it contains a column of type system.Object which protobuf is not able to handle and throwing below exception...

Cannot serialize data column of type 'System.Object'. Only the following column types are supported: Boolean, Byte, Byte[], Char, Char[], DateTime, Decimal, Double, Guid, Int16, Int32, Int64, Single, String."}

Code for serialization of Dataset

using (var ms = new MemoryStream())
                    {
                        if (ReportData.Tables.Count > 0)
                        {
                            DataSerializer.Serialize(ms, ReportData);
                           }
                    }

How can I force protobuf to to serialize system.Object data type column. I tried to convert this system.Object type column to string then serialize... in this case everything works fine but it is time consuming process as I need to clone the table and import all the rows (my dataset contains millions of rows).

 DataTable dtCloned = ReportData.Tables[0].Clone();
                    foreach (DataColumn column in dtCloned.Columns)
                    {
                        if (column.DataType == typeof(System.Object))
                            objectColumns.Add(column);
                    }
                    if (objectColumns.Count > 0)
                    {
                        foreach (DataColumn column in objectColumns)
                        {
                            column.DataType = typeof(System.String);
                        }
                    }
                    try
                    {
                        foreach (DataRow row in ReportData.Tables[0].Rows)
                        {
                            dtCloned.ImportRow(row);
                        }
                    }
                    catch (Exception ex)
                    {

                    }

This is the sample code i wrote for the same. DB type for this System.Object is SQL_Variant and we cannot change this as this is legecy SP and the column contains different type of data (decimal, int varchar).

Is ther any better way to achieve the same.

When this serialization failed and throw error... on client side I am getting communication Exception which contains no clue what went wrong.The only thing it says is...

"The underlying secure session has faulted before the reliable session fully completed. The reliable session was faulted."

I have tried setting below thing in WCF service side but no use...

 <serviceDebug includeExceptionDetailInFaults="true" />

also tried with passing FaultException... no luck

How can I pass the correct exception and info from WCF service to my client.

Thanks in advance.

Can I check: what is DataSerializer here? Protobuf-net has never shipped with DataTable serialization in the core library, so: what is that, exactly? Is that the demo custom serialization code? — Marc Gravell, Jan 19 '18 at 07:40
We are using Assembly: protobuf-net-data, Version=2.0.6.640, Culture=neutral, PublicKeyToken=3a92fdc8f425ccde for the same. It ships with protobuf. — Abhash786, Jan 19 '18 at 09:32
"It ships with protobuf" - no, it doesn't. I'm the author of protobuf-net; protobuf-net-data is a completely separate and independent tool by another developer that *uses* protobuf-net, but which has no affiliation or association with protobuf-net. Compare the "Owners" field [here](https://www.nuget.org/packages/protobuf-net-data/) and [here](https://www.nuget.org/packages/protobuf-net/) — Marc Gravell, Jan 19 '18 at 10:27
yes... protobuf-net-data is separate dll... which support DataSet/DataTable serialization. Does protoBuf has any way to serialize dataTable which contains System.Object type — Abhash786, Jan 19 '18 at 12:02
not really; I expanded on this in my answer and the comment on my answer; it just doesn't like `object` (for good reasons) very much. It can be forced, but a: I don't recommend it, and b: I doubt you'd be able to combine that with the DataTable support added via this other library — Marc Gravell, Jan 19 '18 at 12:05

score 1 · Accepted Answer · answered Jan 19 '18 at 07:48

1

Ultimately, protobuf-net has some pretty fundamental objections to object. It also doesn't ship with pre-built support for DataTable. Frankly I'd challenge the suitability of using DataTable as an exchange type, preferring a well-typed DTO.

If you must use DataTable for some reason, I wonder whether your best bet is to serialize that separately making sure to use the binary format (see RemotingFormat) and writing to a MemoryStream, then fetch out the bytes (ToArray in this case) and just hand WCF the byte[]. Reverse at the other end.

answered Jan 19 '18 at 07:48

Marc Gravell

1,026,079
266
2,566
2,900

Yes... I tried that as serializing DataTable separately with Binary serializer and that worked fine. wanted to check with Protobuf can handle object data. Also... passing the correct exception message to client... any suggestion... – Abhash786 Jan 19 '18 at 09:31
@Abhash786 if you use `BinaryFormatter`: make sure to set the `RemotingFormat` on the data table. As for the exception message: no idea. And on protobuf-net / object: there *is* some basic support via "dynamic types" (nothing to do with the `dynamic` keyword - just: burning some type meta into the object), but it is a crude hack that isn't very efficient and can be brittle. I would like to support the "Any" feature that has been added to the google library, but I haven't found the time to investigate it yet. That would *also* be inefficient, but would at least be compliant with the new spec :) – Marc Gravell Jan 19 '18 at 10:12
Hi i have the same requirement where I use WCF using dataset from legacy system and need to move the services to gRPC still using datasets, did you find a way to do this ? Appreciate it – VAAA Nov 01 '20 at 15:28
@VAAA are you using protobuf-net.Grpc? I'd be happy to show you how to swap out the marshaller for specific types, so you can use the same approach from this answer – Marc Gravell Nov 01 '20 at 19:37
Thanks a lot @MarcGravell I would really appreicate it because this is breaking my head since 1 week. How can you show me that? Thanks a lot – VAAA Nov 01 '20 at 19:39
@MarcGravell you can look what I tried right now https://stackoverflow.com/questions/64636497/c-sharp-grpc-service-returning-untyped-dataset-serialization-problem Appreciate it – VAAA Nov 01 '20 at 20:16
@VAAA based on that question, you're using vanilla gRPC, not protobuf-net.Grpc (which works a lot more like WCF, code first interfaces etc). That means I can't really help you with the marshaller swap, since that is protobuf-net.Grpc specific. I'll try to comment on the overall theme, though. But: ultimately, working with DataTable etc just makes this actively hostile :/ – Marc Gravell Nov 01 '20 at 20:20
@MarcGravell I just did that sample based on some tutorials but I can use protobuf-net for sure if this allow me to do what I need to. Appreciate any guide. – VAAA Nov 01 '20 at 20:22

Serialization issue with protobuf and WCF

1 Answers1