Editable XML Serialization

[Anne van Rossum]

Dear list members,

I guess it's one of the hardest search terms around but I would like to know
if there is any editable XML serialization provided by Boost or other C++
libraries. And I don't mean the kind of thing that requires you to go
jumping around through trees, with iterators etc. I don't want to see any
iterator!

I really iike the XML serialization method that boost::serialization
provides:

//! Give boost serialization libraries access to private fields
friend class boost::serialization::access;

//! Tell which private fields to serialize
template<class Archive>
void serialize(Archive & ar, const unsigned int version) {
ar & boost::serialization::make_nvp("Phase", phase);
ar & boost::serialization::make_nvp("Period", period);
ar & boost::serialization::make_nvp("DutyCycle", dutycycle);
}

Just adding this to a file, and then:

xml_out << boost::serialization::make_nvp("ApplicationConfig",
config);

Or:
xml_in >> boost::serialization::make_nvp("ApplicationConfig",
config);

It's awesome! I don't need to care about descending trees, etc. However, the
default serialization class does not allow for different orders of the XML
tags, etc. It doesn't use the "ApplicationConfig" string, but just assumes
that XML tag should be there if this read operation. If you change anything
in the XML file, it is likely you will get a segfault, and you won't know
what is causing the problem exactly.

It would be so great if there is something like this in which the XML files
can actually be edited! I am almost sure that there should be something like
that, but I can't find it.

Thanks in advance,

Anne

[Robert Ramey]

Post by Anne van Rossum
Dear list members,
I guess it's one of the hardest search terms around but I would like
to know if there is any editable XML serialization provided by Boost
or other C++ libraries. And I don't mean the kind of thing that
requires you to go jumping around through trees, with iterators etc.
I don't want to see any iterator!
I really iike the XML serialization method that boost::serialization
//! Give boost serialization libraries access to private fields
friend class boost::serialization::access;
//! Tell which private fields to serialize
template<class Archive>
void serialize(Archive & ar, const unsigned int version) {
ar & boost::serialization::make_nvp("Phase", phase);
ar & boost::serialization::make_nvp("Period", period);
ar & boost::serialization::make_nvp("DutyCycle", dutycycle);
}
xml_out <<
boost::serialization::make_nvp("ApplicationConfig", config);
xml_in >> boost::serialization::make_nvp("ApplicationConfig",
config);
It's awesome! I don't need to care about descending trees, etc.
However, the default serialization class does not allow for different
orders of the XML tags, etc. It doesn't use the "ApplicationConfig"
string, but just assumes that XML tag should be there if this read
operation. If you change anything in the XML file, it is likely you
will get a segfault, and you won't know what is causing the problem
exactly.
It would be so great if there is something like this in which the XML
files can actually be edited! I am almost sure that there should be
something like that, but I can't find it.

A rich subject which comes up from time to time. Here are a few
miscelleaneous observations.

a) Most of the time what people want when they ask this question is

xml_in >> my data structure for some arbitrarily defined structure. Of
course after you become familiar with the library, it's easy to see that
this is not possible. It's annoying to me how often this is stated as
a failing of the library.

b) It is possble to so some limited editing of xml (or other
archives) but this would be an ad hoc procedure subject to
errors. you might change a value here and there but the minute
you change something tracked or add a value to a collection
or whatever, you're not going to be able to keep things consistent.
I guess you're familiar with this.

I've thought about this alot. Here are some ideas that I thought
about.

a) create an xml_archive along with an xml schema which would
be friendly with known xml editors. I thought about this alot an
concluded that it would be too much like training an ant to train
a flea. I don't think it would be possible to make a bunch of
rules that an xml editor could follow and guarentee that that
the resulting C++ data object would be correct. Even it were
possible, one would likely have to add a bunch of "helper"
information to get it right. It would be hard to use, easy to misuse,
and a maintainence (and support) nightmare.

So what I want to do is to make the following:

edit_oarchive<widgets_library>
edit_iarchive<widgets_library>

The usage would be the following:

edit_archive<mfc_widgets> ea
...
ea << my_data

// the data is rendered using the supplied widgets library. A
// widgets library would exist for QT, MFC, WTL, and whatever.
//
// This archive class would map types to standard things like
// int would be mapped to a labeled text box
// list of ints would be mapped to list of integers
// enum would be mapped to a drop down list
// ...

// the user would edit the widgets - like a dialog box and
// hit OK. Then

ea >> my_data

which syncronizes my_data with the widgets.

We actually made a working proof of concept a few years
ago. I made the mfc version and someone else - forgot the name - sorry
- made one for qt (I think). I concluded that it would require
too much work to get it ready for prime time and that there
were more pressing issues. There still are but maybe
some day.

The main think I liked about this is that

a) it could be guarenteed to work - at least as much
as the current library is.
b) it would be maintainable and supportable.
c) wouldn't be coupled with some external tool
d) wouldn't be coupled with xml.

Maybe someday someone will hire me to do this. It's not
not trivial task. But I'm convinced that it is actually
doable. Lately, there has been talk on the list about
a GUI library. (Yes, it's about that time again). This task
is much smaller and potentially much more immediately
useful.

Robert Ramey

[Anne van Rossum]

Post by Robert Ramey

Post by Anne van Rossum
Dear list members,
I guess it's one of the hardest search terms around but I would like
to know if there is any editable XML serialization provided by Boost
or other C++ libraries. And I don't mean the kind of thing that
requires you to go jumping around through trees, with iterators etc.
I don't want to see any iterator!
I really iike the XML serialization method that boost::serialization

A rich subject which comes up from time to time. Here are a few
miscelleaneous observations.
a) Most of the time what people want when they ask this question is
xml_in >> my data structure for some arbitrarily defined structure. Of
course after you become familiar with the library, it's easy to see that
this is not possible. It's annoying to me how often this is stated as
a failing of the library.

What I want "at the C++ side" so to say, is just the same as with
boost::serialization. That supports already "custom" data structures, because in
the serialize(Archive & ar,...) routine it is possible to define what fields to
write/read. I am not talking about marshalling / unmarshalling to entire
objects. Serializing arbitrary data structures does not make sense to me.

However, I am not familiar with the library, so perhaps I misunderstood you.
Forgive me, I just encountered it today.

Post by Robert Ramey
b) It is possble to so some limited editing of xml (or other
archives) but this would be an ad hoc procedure subject to
errors. you might change a value here and there but the minute
you change something tracked or add a value to a collection
or whatever, you're not going to be able to keep things consistent.
I guess you're familiar with this.

Yes.

Post by Robert Ramey
I've thought about this alot. Here are some ideas that I thought
about.
a) create an xml_archive along with an xml schema which would
be friendly with known xml editors. I thought about this alot an
concluded that it would be too much like training an ant to train
a flea. I don't think it would be possible to make a bunch of
rules that an xml editor could follow and guarentee that that
the resulting C++ data object would be correct. Even it were
possible, one would likely have to add a bunch of "helper"
information to get it right. It would be hard to use, easy to misuse,
and a maintainence (and support) nightmare.
edit_oarchive<widgets_library>
edit_iarchive<widgets_library>
edit_archive<mfc_widgets> ea
...
ea << my_data

That's a solution. I would then need to distribute a separate binary to
manipulate the XML files. My idea was to "misuse" boost::serialization for
writing XML configuration files without the need to manually build DOM-like
structures. For just changing configuration parameters, a graphical solution
might be a bit cumbersome. It's a nice idea nevertheless!

Post by Robert Ramey
Robert Ramey

Anne

[Robert Ramey]

Post by Anne van Rossum

Post by Robert Ramey
xml_in >> my data structure for some arbitrarily defined structure.
Of course after you become familiar with the library, it's easy to
see that this is not possible. It's annoying to me how often this
is stated as a failing of the library.

What I want "at the C++ side" so to say, is just the same as with
boost::serialization. That supports already "custom" data structures,
because in the serialize(Archive & ar,...) routine it is possible to
define what fields to write/read. I am not talking about marshalling
/ unmarshalling to entire objects. Serializing arbitrary data
structures does not make sense to me.

What I'm referring to is the the word xml_archive almost always
conjures up in many persons imagination something which would
take an arbitrary xml file and permit processng of it. Serialization
can't do this even in concept since it maps the xml to the underly
C++ data structures. This becomes obvious once you use it. But
often people take it up thinking that it's something that it isn't.

Post by Anne van Rossum
However, I am not familiar with the library, so perhaps I
misunderstood you. Forgive me, I just encountered it today.

lol - well it seems that you've actually used it - which makes
you a lot more familiar to you than it is to a many of those who
offer criticism.

Post by Anne van Rossum

Post by Robert Ramey
b) It is possble to so some limited editing of xml (or other
archives) but this would be an ad hoc procedure subject to
errors. you might change a value here and there but the minute
you change something tracked or add a value to a collection
or whatever, you're not going to be able to keep things consistent.
I guess you're familiar with this.

Yes.

...

Post by Anne van Rossum

Post by Robert Ramey
edit_archive<mfc_widgets> ea
...
ea << my_data

That's a solution. I would then need to distribute a separate binary
to manipulate the XML files.

yes - but that binary would be generated "for free" without
having to code anything.

Post by Anne van Rossum
My idea was to "misuse"
boost::serialization for writing XML configuration files without the
need to manually build DOM-like structures.

Indeed - any many have traveled this path. And, as I understand it
some success in specific cases. This has been done by tweaking
the xml_archive class or deriving from the current one. So for
those cases it's an option. Of course, personally I'm more interested in a
more general solution.

Post by Anne van Rossum
For just changing
configuration parameters, a graphical solution might be a bit
cumbersome. It's a nice idea nevertheless!

actually this would be a series of classes

edit_archive<qtwidgets>,
...
edit_archive<textwidgets> // no GUI necessary ! - just a text editor

This last would likely be something generated with qi/karma which
is a whole 'nother playpen.

For simple configuratoin files, there are other options as well.
boost property tree comes to mind. Also one might make archive
which traps problematic cases like pointers so it would be usable
to make only "simple editable" xml. etc,

Robert Ramey

[Anne van Rossum]

Post by Robert Ramey
Indeed - any many have traveled this path. And, as I understand it
some success in specific cases. This has been done by tweaking
the xml_archive class or deriving from the current one. So for
those cases it's an option. Of course, personally I'm more interested in a
more general solution.

Yes, of course, and it's already a nicer than most ways in which XML
configuration is written or read, so I will definitely continue to use it.
If you would be able to point me to some of those success cases, I would be
obliged.

Thanks for all the work, Robert!

Anne

[michi7x7]

Hi,

Post by Anne van Rossum
Dear list members,
I guess it's one of the hardest search terms around but I would like
to know if there is any editable XML serialization provided by Boost
or other C++ libraries. And I don't mean the kind of thing that
requires you to go jumping around through trees, with iterators etc. I
don't want to see any iterator!

I thought of something like this too. Actually, a specific XML-Format
could easily be implemented using boost.spirit:

qi::rule<iterator_t, data_t> = lit("<xml>") >> lit("<data1>") >> double_

Post by Anne van Rossum

lit("</data1>") >> lit("<data2>") >> int_ >> lit("</data2>") >>

lit("</xml>");

This way, you could define your own XML-based-grammar which then gets
parsed into structs. Thats enough for most applications IMO.
But this syntax is ugly, by writing own spirit-parsers, the syntax might
look like this:

qi::rule<iterator_t, data_t> = node("xml") [ node("data1") [double_] >>
node("data2") [double_] ];


Spirit will also complain if the format is unexpected, and throw an
exception containing an iterator pointing to the faulty position.

Is this enough for you? (You might want to take a look at spirit, to
understand what's going on here)

Post by Anne van Rossum
It would be so great if there is something like this in which the XML
files can actually be edited! I am almost sure that there should be
something like that, but I can't find it.

You can edit the file by using a karma-rule that works exactly like the
qi::rule, but is used for output instead. Spirit is very fast, probably
faster than editing the XML and saving it using another xml-library.


Regards,

michi7x7