Bug 552891

Summary: Mono can't deserialize Nullable objects created by .NET (BinaryFormatter)
Product: [Mono] Mono: Class Libraries Reporter: Jorge Matias <jorge.matias>
Component: CORLIBAssignee: Lluis Sanchez <lluis>
Status: RESOLVED DUPLICATE QA Contact: Mono Bugs <mono-bugs>
Severity: Major    
Priority: P5 - None CC: forgotten_oDRaEXi7Ku, mika.aalto
Version: 2.4.x   
Target Milestone: ---   
Hardware: x86   
OS: All   
Whiteboard:
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---
Attachments: Quick & dirty fix that deserializes MS .NET Nullable objects
modification of Jorge's NUnit test

Description Jorge Matias 2009-11-05 16:02:11 UTC
User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.27 Safari/532.0

Mono can't deserialize classes that include any kind of Nullable object when they have been serialized with .NET and they have been assigned a value which is not null.

Sample code:

using System;
using System.Collections.Generic;
using System.Text;
using System.IO;
using System.Runtime.Serialization.Formatters.Binary;

using NUnit.Framework;

namespace TestSerialization
{
    [Serializable]
    class SimpleClass
    {
        string id;
        DateTime? nullableTimeStamp;
        int? nullableInt;

        public string Id
        {
            get { return id; }
            set { id = value; }
        }

        public DateTime? NullableTimeStamp
        {
            get { return nullableTimeStamp; }
            set { nullableTimeStamp = value; }
        }

        public int? NullableInt
        {
            get { return nullableInt; }
            set { nullableInt = value; }
        }

        public override bool Equals(object obj)
        {
            try
            {
                SimpleClass c = (SimpleClass)obj;
                return id.Equals(c.id) 
                    && nullableTimeStamp.Equals(c.nullableTimeStamp)
                    && nullableInt.Equals(c.nullableInt);
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex);
                return false;
            }
        }
    }

    [TestFixture]
    public class NullableSerializationTest
    {
        SimpleClass c;
        
        public NullableSerializationTest()
        {
            c = new SimpleClass();
            c.Id = "123456";
            c.NullableTimeStamp = null;
            c.NullableInt = null;
        }

        void SerializeObject(Object o, string file)
        {
            Stream stream = File.Create(file);
            BinaryFormatter oFormatter = new BinaryFormatter();
            oFormatter.Serialize(stream, o);
            stream.Close();
        }

        Object DeserializeObject(string file)
        {
            Stream stream = File.OpenRead(file);
            BinaryFormatter oFormatter = new BinaryFormatter();
            Object o = oFormatter.Deserialize(stream);
            stream.Close();
            return o;
        }

        [Test]
        public void TestSerialization()
        {
            SerializeObject(c, "simpleobj.bin");
        }

        [Test]
        public void TestDeserialization()
        {
            SimpleClass c2 = (SimpleClass) DeserializeObject("simpleobj.bin");
            Assert.AreEqual(c, c2);
        }
    }
}


Reproducible: Always

Steps to Reproduce:
1. Change the following lines:
 a) Replace 
      c.NullableTimeStamp = null; 
    by 
      c.NullableTimeStamp = DateTime.Parse("1/1/2009");
or
 b) Replace
      c.NullableInt = null;
    by
      c.NullableInt = 10;

2. Run TestSerialization from .NET 2.0 or 3.5 (I tested both).

3. Run TestDeserialization from Mono 2.4 (tested on Windows, OpenSUSE and Mac OS X).

Actual Results:  
1.a (after changing the Nullable DateTime value):

TestCase 'TestSerialization.NullableSerializationTest.TestDeserialization'
failed: System.ArgumentOutOfRangeException : Value 4159925407799315720 is outside the valid range [0,3155378975999999999].
Parameter name: ticks
	at System.DateTime..ctor (Int64 ticks) [0x00000]
	at System.DateTime..ctor (Int64 ticks, DateTimeKind kind) [0x00000]
	at System.DateTime.FromBinary (Int64 dateData) [0x00000]
	at System.Runtime.Serialization.Formatters.Binary.ObjectReader.ReadPrimitiveTypeValue (System.IO.BinaryReader reader, System.Type type) [0x00000]

1.b (after changing the Nullable int value):

TestCase 'TestSerialization.NullableSerializationTest.TestDeserialization'
failed: System.Runtime.Serialization.SerializationException : Unexpected binary element: 0
	at System.Runtime.Serialization.Formatters.Binary.ObjectReader.ReadObject (BinaryElement element, System.IO.BinaryReader reader, System.Int64& objectId, System.Object& value, System.Runtime.Serialization.SerializationInfo& info) [0x00000]
	at System.Runtime.Serialization.Formatters.Binary.ObjectReader.ReadNextObject (System.IO.BinaryReader reader) [0x00000]

Expected Results:  
The object is deserialized correctly and the test is succesfully run for every case.

The exceptions don't happen if:
- Both objects are set to null.
- The Nullable part of the object definitions is removed, that is, the objects are defined as DateTime or int or whatever.

BTW, do you guys know how could a workaround be developed for this problem with a custom SurrogateSelector? Thanks!
Comment 1 Andres Aragoneses 2009-11-06 16:37:57 UTC
Jorge: have you tried binary serialization of a different type? I mean, a nullable that doesn't surround DateTime but other struct or primitive type. If that case works, then this bug is a dupe of bug 321869, bug 325067 or bug 360429.
Comment 2 Jorge Matias 2009-11-06 18:15:01 UTC
Andres, the test case 1b is about an int deserialization: 

 b) Replace
      c.NullableInt = null;
    by
      c.NullableInt = 10;

Do you need me to send more tests with other primitive types?

Regards.

PS. I guess the issues regarding DateTime serialization were fixed in 2.0 (at least for 32 bit machines).
Comment 3 Jorge Matias 2009-11-09 23:07:35 UTC
Created attachment 326371 [details]
Quick & dirty fix that deserializes MS .NET Nullable objects

The following patch is able to deserialize Nullable objects created by MS .NET, but should be considered a _hack_ since I'm far of being an expert on the matter.

It seems MS .NET runtime does some tricky things (sort of optimization?) with Nullable objects when they are not null, which doesn't allow Mono to deserialize them.

For example, if a Nullable<int> is set to null, the serialized binary contains the following string:

System.Nullable`1[[System.Int32, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]]

But if it's not null, the serialized binary contains just the System.Int32, with the only difference that is detected by the Mono BinaryFormatter as a RuntimeType instead a PrimitiveType. This _may_ be related with autoboxing; after the change ReadValue detects the element as a BoxedPrimitiveTypeValue and procedes to deserialize the object correctly.

HTH.
Comment 4 Andres Aragoneses 2009-11-10 01:40:43 UTC
(In reply to comment #3)
> Created an attachment (id=326371) [details]
> Quick & dirty fix that deserializes MS .NET Nullable objects
> 
> The following patch is able to deserialize Nullable objects created by MS .NET,

Nice.


> but should be considered a _hack_ since I'm far of being an expert on the
> matter.

The best way to find out if you're on the right track is: first, check that the unit tests don't break with this change (no regressions). Second, if they don't, the patch will be most likely accepted if it includes new unit tests as well that pass with the modification and that don't pass without it.

Thanks!
Comment 5 Forgotten User oDRaEXi7Ku 2010-09-13 12:01:53 UTC
*** Bug 567522 has been marked as a duplicate of this bug. ***
Comment 6 Forgotten User oDRaEXi7Ku 2010-09-13 12:04:51 UTC
This is not the mono runtime, but corlib.
Comment 7 Miguel de Icaza 2010-09-13 17:24:39 UTC
Lluis, would you mind reviewing this patch and approving it if it is OK?

We should also get the nice test in this sample incorporated.
Comment 8 Gonzalo Paniagua Javier 2010-10-21 07:16:04 UTC
I fixed bug #646556 which fixes this issue.

*** This bug has been marked as a duplicate of bug 646556 ***
Comment 9 Eric Slosser 2010-10-21 15:52:24 UTC
Created attachment 396319 [details]
modification of Jorge's NUnit test

Includes two of each data type member,  int and DateTime, with one =null and other =<something>.

Removes need to tweak code and rebuild.
Comment 10 Eric Slosser 2010-10-21 15:55:38 UTC
Comment on attachment 396319 [details]
modification of Jorge's NUnit test

I used attachment 396319 [details] to confirm that this bug is fixed.

Mono can now deserialize regardless of whether .NET (3.5) or Mono (2.6.4 or 2.8) did the serialization.