[Boost-users] [serialization] Object reference tracking on instances bug?

[Robert P]

In my serialize function I am exporting a class which is a template wrapper
around std:vector.

template <typename E>
class MyArray
{
public:
vector<E> array;
...

template <typename Archive>
void MyArray::serialize(Archive & ar, const unsigned int version)
{
ar & BOOST_SERIALIZATION_NVP(array);
}

}

When I add unique class instance variables to this array and then serialize
it to XML...
...
MyArray<Color> colorsArray;

Color a;
a.name = "one";
colorsArray.add(a);

Color b;
b.name = "two";
colorsArray.add(b);
...


template <typename Archive>
void MyColorsObject::serialize(Archive & ar, const unsigned int version)
{
ar & BOOST_SERIALIZATION_NVP(colorsArray);
}

Sometimes the XML comes out using object_id_reference instead of the unique
values, and this then gets improperly unserialized with incorrect values,
i.e. both colors will have "one" as the name.

I was able to disable object tracking and it works now, but I don't think
it should be doing object reference tracking on instance variables. Is it
because of the template wrapper around vector?

[Raoul Wols]

Can you give an example of the expected XML output and the actual XML output?

What is MyColorsObject?

Can you provide a complete minimal example?

--Raoul

In my serialize function I am exporting a class which is a template wrapper around std:vector.
template <typename E>
class MyArray
{
vector<E> array;
...
template <typename Archive>
void MyArray::serialize(Archive & ar, const unsigned int version)
{
ar & BOOST_SERIALIZATION_NVP(array);
}
}
When I add unique class instance variables to this array and then serialize it to XML...
...
MyArray<Color> colorsArray;
Color a;
a.name = "one";
colorsArray.add(a);
Color b;
b.name = "two";
colorsArray.add(b);
...
template <typename Archive>
void MyColorsObject::serialize(Archive & ar, const unsigned int version)
{
ar & BOOST_SERIALIZATION_NVP(colorsArray);
}
Sometimes the XML comes out using object_id_reference instead of the unique values, and this then gets improperly unserialized with incorrect values, i.e. both colors will have "one" as the name.
I was able to disable object tracking and it works now, but I don't think it should be doing object reference tracking on instance variables. Is it because of the template wrapper around vector?
_______________________________________________
Boost-users mailing list
http://lists.boost.org/mailman/listinfo.cgi/boost-users

[Robert P]

The code that it's actually contained in is pretty complex, I'll try and
put together a minimal example that produces the behavior if I need to. It
doesn't seem to do object reference tracking on the initial
serialization/deserialization, but after I have deserialized the class from
XML and then serialized it again it does.

It does serialize the first reference to the object correctly, but later
objects just use the object_id_reference if they were similar to the first
one (even if they contain different data!). So the first color is
serialized correctly.

The faulty behavior is in that the objects are 1) not references or
pointers, but local instance variables so should not be tracked, 2) contain
unique data and should not be referenced to other objects even if they were
pointers, and 3) do not unserialize correctly, instead resulting in objects
with default values.

Does boost keep a global table of reference tracking values that does not
get reset between serializations? That would possibly explain the problem.

MyColorsObject is an object that contains an array of colors, trying to
give a minimal example.

Expected XML output is something like this:

<colorsArray>
<v>
<count>2</count>
<item_version>1</item_version>
<item id="_1">
<name>cyan</name>
<r>0</r>
<g>255</g>
<b>255</b>
<a>255</a>
</item>
<item id="_2">
<name>magenta</name>
<r>255</r>
<g>0</g>
<b>127</b>
<a>255</a>
</item>
</v>
</colorsArray>

Actual XML output is something like

<colorsArray>
<v>
<count>2</count>
<item_version>1</item_version>
<item>
<object_id_reference = "_1">
</item>
<item>
<object_id_reference = "_2">
</item>
</v>
</colorsArray>

and looking at the data after deserializing the objects look like:

colorsArray.at(0).name == "";
colorsArray.at(0).r == 0;
colorsArray.at(0).g == 0;
colorsArray.at(0).b == 0;

as they are not being initialized "properly" from the serialized references.

Message: 3

Date: Fri, 3 Feb 2017 19:46:26 +0100
Subject: Re: [Boost-users] [serialization] Object reference tracking
on instances bug?
Content-Type: text/plain; charset="us-ascii"
Can you give an example of the expected XML output and the actual XML
output?
What is MyColorsObject?
Can you provide a complete minimal example?
--Raoul

Post by Robert P
In my serialize function I am exporting a class which is a template

wrapper around std:vector.

Post by Robert P
template <typename E>
class MyArray
{
vector<E> array;
...
template <typename Archive>
void MyArray::serialize(Archive & ar, const unsigned int version)
{
ar & BOOST_SERIALIZATION_NVP(array);
}
}
When I add unique class instance variables to this array and then

serialize it to XML...

Post by Robert P
...
MyArray<Color> colorsArray;
Color a;
a.name = "one";
colorsArray.add(a);
Color b;
b.name = "two";
colorsArray.add(b);
...
template <typename Archive>
void MyColorsObject::serialize(Archive & ar, const unsigned int version)
{
ar & BOOST_SERIALIZATION_NVP(colorsArray);
}
Sometimes the XML comes out using object_id_reference instead of the

unique values, and this then gets improperly unserialized with incorrect
values, i.e. both colors will have "one" as the name.

Post by Robert P
I was able to disable object tracking and it works now, but I don't

think it should be doing object reference tracking on instance variables.
Is it because of the template wrapper around vector?

Post by Robert P
_______________________________________________
Boost-users mailing list
http://lists.boost.org/mailman/listinfo.cgi/boost-users

-------------- next part --------------
HTML attachment scrubbed and removed

Post by Robert P
In my serialize function I am exporting a class which is a template
wrapper around std:vector.
template <typename E>
class MyArray
{
vector<E> array;
...
template <typename Archive>
void MyArray::serialize(Archive & ar, const unsigned int version)
{
ar & BOOST_SERIALIZATION_NVP(array);
}
}
When I add unique class instance variables to this array and then
serialize it to XML...
...
MyArray<Color> colorsArray;
Color a;
a.name = "one";
colorsArray.add(a);
Color b;
b.name = "two";
colorsArray.add(b);
...
template <typename Archive>
void MyColorsObject::serialize(Archive & ar, const unsigned int version)
{
ar & BOOST_SERIALIZATION_NVP(colorsArray);
}
Sometimes the XML comes out using object_id_reference instead of the
unique values, and this then gets improperly unserialized with incorrect
values, i.e. both colors will have "one" as the name.
I was able to disable object tracking and it works now, but I don't think
it should be doing object reference tracking on instance variables. Is it
because of the template wrapper around vector?

[r***@primef.actor]

Post by Robert P
It does serialize the first reference to the object correctly, but
later objects just use the object_id_reference if they were similar
to
the first one (even if they contain different data!).  So the first
color is serialized correctly.

The saving machinery kicks in at

boost/archive/detail/oserializer.hpp

on line 510. That is where it selects, at compile time, wether the
object to serialize is a pointer, enum, "array", or regular non-pointer
type. In your case, I'm guessing it should always select the regular
non-pointer overload. You can add a breakpoint there and start your
debugger to see if it does the correct thing. Can you try that?

Regards,

Raoul

[Robert Ramey]

Post by Robert P
The faulty behavior is in that the objects are 1) not references or
pointers, but local instance variables so should not be tracked, 2)
contain unique data and should not be referenced to other objects even
if they were pointers, and 3) do not unserialize correctly, instead
resulting in objects with default values.
Does boost keep a global table of reference tracking values that does
not get reset between serializations? That would possibly explain the
problem.

The serialization library does not track object contents, only their
addresses.

Post by Robert P
I was able to disable object tracking and it works now,

This is the correct solution

Post by Robert P
don't think it should be doing object reference tracking on
instance variables.

I think if you spent more time considering this this, you'd change your
opinion.

Is it because of the template wrapper

Post by Robert P
around vector?

Not relevant

The question is really about what it means to serialize something on the
stack. Something like

MyT my_object;

...

// serialize/deserialize my object

Is going to be fine.

But something like

...
for(...)
MyT my_object;
...
// serialize/deserialize my_object

or worse
MyT::serialize(ar, version) const {
x = f(*this)
ar & x;
}

is going to be asking for trouble as we're reusing the same address. As
you've discovered. the library provides a work around for this. But
when confronted with the need to use it, it should be seen and an
opportunity to step back and ask oneself: "Is this really what I want to
do? Why am I saving the state of something that is changing?"

Robert Ramey