boost::serialization throws random exceptions

[Adrian Grigore]

Hi,

I'm experiencing occasional problems with boost::serialization when
trying to write then read two simple classes, to a file. I am using
boost 1.33.1

Most of the time everything's working fine, but sometimes the serialized
data seems to be corrupt and boost::archive throws an exception when I
try to load it back into the program. The error occurs about about 1 in
10 times I try this, and seems to depend on the exact values of the
variables being saved.

Here's a call stack of the thrown application:

WxNonogram.exe!_CxxThrowException(void * pExceptionObject=0x00129680,
const _s__ThrowInfo * pThrowInfo=0x00ff4954) + 0x50 bytes C++
WxNonogram.exe!boost::throw_exception<boost::archive::archive_exception>(const
boost::archive::archive_exception & e={...}) + 0x40 bytes C++

WxNonogram.exe!boost::archive::basic_binary_iprimitive<boost::archive::binary_iarchive,std::basic_istream<char,std::char_traits<char>

::load_binary(void * address=0x0012d05c, unsigned int count=4) +

0xaf bytes C++

WxNonogram.exe!boost::archive::basic_binary_iprimitive<boost::archive::binary_iarchive,std::basic_istream<char,std::char_traits<char>

::load<int>(int & t=-858993460) + 0x31 bytes C++

WxNonogram.exe!boost::archive::load_access::load_primitive<boost::archive::binary_iarchive,int>(boost::archive::binary_iarchive
& ar={...}, int & t=-858993460) + 0x2d bytes C++

WxNonogram.exe!boost::archive::detail::load_non_pointer_type<boost::archive::binary_iarchive,int>::load_primitive::invoke(boost::archive::binary_iarchive
& ar={...}, int & t=-858993460) + 0x2b bytes C++

WxNonogram.exe!boost::archive::detail::load_non_pointer_type<boost::archive::binary_iarchive,int>::invoke(boost::archive::binary_iarchive
& ar={...}, int & t=-858993460) + 0x2b bytes C++

WxNonogram.exe!boost::archive::load<boost::archive::binary_iarchive,int>(boost::archive::binary_iarchive
& ar={...}, int & t=-858993460) + 0x2b bytes C++

WxNonogram.exe!boost::archive::basic_binary_iarchive<boost::archive::binary_iarchive>::load_override<int>(int
& t=-858993460, int __formal=0) + 0x38 bytes C++

WxNonogram.exe!boost::archive::binary_iarchive_impl<boost::archive::binary_iarchive>::load_override<int>(int
& t=-858993460, int __formal=0) + 0x31 bytes C++

WxNonogram.exe!boost::archive::detail::interface_iarchive<boost::archive::binary_iarchive>::operator>><int>(int
& t=-858993460) + 0x38 bytes C++

WxNonogram.exe!boost::archive::detail::interface_iarchive<boost::archive::binary_iarchive>::operator&<int>(int
& t=-858993460) + 0x39 bytes C++

WxNonogram.exe!NoNo::TGameProgress::TProfileState::serialize<boost::archive::binary_iarchive>(boost::archive::binary_iarchive
& ar={...}, const unsigned int Version=4) + 0x7a bytes C++


Please help! I'm at a loss on how to proceed.

Thanks for your time,

Adrian

[Robert Ramey]

Here are a couple of ideas:

a) catch the archive exceptions and display an error message which
describes what kind of exception has been thrown.

b) Set your debugger to trap when the exception is thrown
and look at the source code at that spot. This should contain
some extra info about why the library is unable to proceed.

c) Just a while guess - thrown from load binary might suggest
either the file isn't being opened binary, or the size of data
being read isn't the same as what's being saved.

d) Sometimes this can occur when the save/load serialization
is "out of sync". that is the save and load are not code
symetrically. This might occur due to a program change
without the use of versioning. It is sometimes helpful to
save/load with xml_archive just to double check. The
serialization library archives for xml actually check the
xml tags so that if something is out of sync it is detected
immediatly.

Robert Ramey

Post by Adrian Grigore
Hi,
I'm experiencing occasional problems with boost::serialization when
trying to write then read two simple classes, to a file. I am using
boost 1.33.1
Most of the time everything's working fine, but sometimes the
serialized data seems to be corrupt and boost::archive throws an
exception when I try to load it back into the program. The error
occurs about about 1 in 10 times I try this, and seems to depend on
the exact values of the variables being saved.
WxNonogram.exe!_CxxThrowException(void * pExceptionObject=0x00129680,
const _s__ThrowInfo * pThrowInfo=0x00ff4954) + 0x50 bytes C++
WxNonogram.exe!boost::throw_exception<boost::archive::archive_exception>(const
boost::archive::archive_exception & e={...}) + 0x40 bytes C++
WxNonogram.exe!boost::archive::basic_binary_iprimitive<boost::archive::binary_iarchive,std::basic_istream<char,std::char_traits<char>

load_binary(void * address=0x0012d05c, unsigned int count=4) +
0xaf bytes C++

WxNonogram.exe!boost::archive::basic_binary_iprimitive<boost::archive::binary_iarchive,std::basic_istream<char,std::char_traits<char>

load<int>(int & t=-858993460) + 0x31 bytes C++

WxNonogram.exe!boost::archive::load_access::load_primitive<boost::archive::binary_iarchive,int>(boost::archive::binary_iarchive
& ar={...}, int & t=-858993460) + 0x2d bytes C++
WxNonogram.exe!boost::archive::detail::load_non_pointer_type<boost::archive::binary_iarchive,int>::load_primitive::invoke(boost::archive::binary_iarchive
& ar={...}, int & t=-858993460) + 0x2b bytes C++
WxNonogram.exe!boost::archive::detail::load_non_pointer_type<boost::archive::binary_iarchive,int>::invoke(boost::archive::binary_iarchive
& ar={...}, int & t=-858993460) + 0x2b bytes C++
WxNonogram.exe!boost::archive::load<boost::archive::binary_iarchive,int>(boost::archive::binary_iarchive
& ar={...}, int & t=-858993460) + 0x2b bytes C++
WxNonogram.exe!boost::archive::basic_binary_iarchive<boost::archive::binary_iarchive>::load_override<int>(int
& t=-858993460, int __formal=0) + 0x38 bytes C++
WxNonogram.exe!boost::archive::binary_iarchive_impl<boost::archive::binary_iarchive>::load_override<int>(int
& t=-858993460, int __formal=0) + 0x31 bytes C++
WxNonogram.exe!boost::archive::detail::interface_iarchive<boost::archive::binary_iarchive>::operator>><int>(int
& t=-858993460) + 0x38 bytes C++
WxNonogram.exe!boost::archive::detail::interface_iarchive<boost::archive::binary_iarchive>::operator&<int>(int
& t=-858993460) + 0x39 bytes C++
WxNonogram.exe!NoNo::TGameProgress::TProfileState::serialize<boost::archive::binary_iarchive>(boost::archive::binary_iarchive
& ar={...}, const unsigned int Version=4) + 0x7a bytes C++
Please help! I'm at a loss on how to proceed.
Thanks for your time,
Adrian

[Adrian Grigore]

Robert,

Thanks for your swift reply.

Post by Robert Ramey
a) catch the archive exceptions and display an error message which
describes what kind of exception has been thrown.

I did something similar, but it is still a poor workaround. I save data
incrementally and rotate savefile names to make sure the player does not
lose all data, but just the changes since the last increment. Still,
losing all game progress since the last savefile increment is a major
turnoff for most players.

Post by Robert Ramey
b) Set your debugger to trap when the exception is thrown
and look at the source code at that spot. This should contain
some extra info about why the library is unable to proceed.

Usually this is indeed helpful, but in this case the boost source code
was too cryptic for me to understand the problem.

Post by Robert Ramey
c) Just a while guess - thrown from load binary might suggest
either the file isn't being opened binary, or the size of data
being read isn't the same as what's being saved.

I doubt that since there is only one save and one load routine in my
application. If I opened the file as text when saving and as binary when
loading this would always fail. But in my case everything is working
fine most of the time.

Post by Robert Ramey
d) Sometimes this can occur when the save/load serialization
is "out of sync". that is the save and load are not code
symetrically. This might occur due to a program change
without the use of versioning. It is sometimes helpful to
save/load with xml_archive just to double check. The
serialization library archives for xml actually check the
xml tags so that if something is out of sync it is detected
immediatly.

This happens with the same executable, and I have not changed the format
of the data being saved (i.e. amount and order of variables) so "out of
sync" problems seem very unlikely here.

Thanks,

Adrian Grigore

[Adrian Grigore]

In the meantime I was able to provoke the exception in a more
deterministic manner. Merely incrementing the value of an unsigned int
and saving the player profile leads to the exception I mentioned:

for (int i=0;i<100;i++)
{
//this is just a regular size_t variable
CURRENT_PROFILE.NextLevel=i;
g_GameProgress.SaveToFile();
//this always trows an exception when i==26
g_GameProgress.LoadFromFile();
}

The problem does not seem to be related to side-effects in previous file
save / reload attempts. If I merely do this, it leads to the same exception:

//this is just a regular size_t variable
CURRENT_PROFILE.NextLevel=26;
g_GameProgress.SaveToFile();
//same exception here!
g_GameProgress.LoadFromFile();


the exception is thrown in line 114 basic_binar_iprimitive:

if(is.fail())
boost::throw_exception(archive_exception(archive_exception::stream_error));

Thanks for your help,

Adrian Grigore

[Nat Goodspeed]

-----Original Message-----
Sent: Saturday, February 03, 2007 9:11 AM
Subject: Re: [Boost-users] boost::serialization throws random

exceptions

Post by Robert Ramey
a) catch the archive exceptions and display an error message which
describes what kind of exception has been thrown.

I did something similar, but it is still a poor workaround. I save

data

incrementally and rotate savefile names to make sure the player does

not

lose all data, but just the changes since the last increment. Still,
losing all game progress since the last savefile increment is a major
turnoff for most players.

[Nat] I think he was suggesting that you do that as a diagnostic measure
to help discover and solve the problem you're encountering. I don't
think he meant for you merely to put in the catch code and release the
program.

Post by Robert Ramey
c) Just a while guess - thrown from load binary might suggest
either the file isn't being opened binary, or the size of data
being read isn't the same as what's being saved.

I doubt that since there is only one save and one load routine in my
application. If I opened the file as text when saving and as binary

when

loading this would always fail. But in my case everything is working
fine most of the time.

[Nat] But you COULD be having troubles even if you open the file as text
both for save and for load. In fact, the magic value 26 is a strong hint
that this may be the trouble. A byte containing binary 26 may, in text
mode, be interpreted as the character ctrl-Z -- which was the
traditional MS-DOS EOF marker.

Please try Robert's suggestion: ensure that you're opening the file
using the binary flag both for saving and for loading the file.

[Adrian Grigore]

Nate,

Post by Nat Goodspeed
[Nat] But you COULD be having troubles even if you open the file as text
both for save and for load. In fact, the magic value 26 is a strong hint
that this may be the trouble. A byte containing binary 26 may, in text
mode, be interpreted as the character ctrl-Z -- which was the
traditional MS-DOS EOF marker.

Thanks very much for pointing this out again. I Misunderstood Robert's
suggestion and just checked that I open the archives in binary mode, not
the file itself. Once I fixed that everything was running smoothly.

Robert, Nate: Thanks very much for your help! :-)

Adrian