Introduction
IoBind is a highly flexible library for serializing objects
to/from string. It uses meta-programming and policies to create
reader and writer for complex objects at compile time.
In other words, IoBind is a code factory that produces
customized readers and writers: you give the action to take and the objects to
process and it will generate the corresponding code.
The latest versions of IoBind are available at SourceForge.
Outline
- Quick examples
- Converting a string to a
string
- Escaping string to XML
- Combining policies
- Reading back data
- Compiling and instalation
- The encode method
- Policies
- operator +
- String manipulation
- Base64
- Zip
- XML escaping, Latin1 escaping
- sequence container
- pair structure
- associative containers
- History
- References
Quick examples
Let's start with some quick examples to see what IoBind is about.
Converting a string to a ... string
Let see a first snippet:
string s("<xml/>");
s = encode( s, to_string_p );
cerr<<s<<endl;
-- output
<xml/>
This seems to be useless since I'm converting a string to string. Anyway,
let's see what's in the second line:
encode is a template function that applies a conversion policy
(see below) to s,
to_string_p is a conversion policy that takes a value
and converts it to a string ( using boost::format library ).
Escaping string to XML
Now, suppose that you need to store this string into an xml document. You
need replace escape characters (<, >, etc..) by
<, >, etc:
s="escape me:&\"'<>"
s=encode(s, escape_to_xml_p );
cerr<<s<<endl;
-- output
escape me:&"'<>
What has happened here: the string has been converted to an escaped string
using the escape_to_xml_p conversion policy.
Hence using predefined policies you can easily transform a string:
- base64,
- encryption,
- zipping,
- etc...
Combining policies
Having a set of basic policies is good but creating new policies by
combining them really makes things interresting (this is where
meta-programming takes place).
Suppose that you have a vector of string, you
want to convert it to string and to escape this string to xml:
vector<string> v_string;
v_string.push_back("a string");
v_string.push_back("<string/>");
s = encode(
v_string.begin(),
v_string.end(),
escape_to_xml_p * sequence_to_string_p
);
cerr<<s<<endl;
-- output
a string,<string/>
In this call, sequence_to_string_p is a policy that converts an
iterator range to a string. The operator + combines the two
policies (similar to function composition) and creates a new conversion
policy:
a * b applied to a value v is equivalent to a( b( v )
)
Note that you can also specify a policy to be applied to each value of the
container. This, combined with a pair policy, can be used to serializes
associatives containers.
Reading back data
All the "to" policies have their "from" counter part to read data. For
example, we can easily read a vector<int> from a
string:
vector<int> v_int;
string str("0,1,2,3,4");
v_int = encode(str,
sequence_from_string_p
% v_int
<< from_string()
);
where
sequence_from_string_p is a generic conversion policy that read
a sequence of strings. This conversion policy is a template class that depends
on two template paramters:
Container, the type of container to fill,
Policy, the conversion policy to apply to each element,
- the operator
% builds a new sequence_from_string
conversion policy that adds values to a container of same type as
v_int. Concretely, this operator replaces the
Container template paramter with the type of v_int,
- the operator
<< is similar to % but it works
on the policy.
The above is rather confusing, let's go deeper to see how policies are
built step by steps:
sequence_from_string_p
% v_int
<< from_string()
sequence_from_string_p = A: this is a pre-defined
sequence conversion policy: it reads a sequence of strings separated by commas,
- Container =
vector<string>
- Policy =
from_string<string>
A % v_int = B: the container has been replaced by
v_int type:
- Container =
vector<int>
- Policy =
from_string<string>
B << from_string() : the conversion
poliy applied to the elements reads a int from a string:
- Container =
vector<int>
- Policy =
from_string<int>
So with this little statement, we have build a new conversion
policy that is a complex mixture of A and
B.
Compiling and installation
You need Boost (see [1]) and a good compiler (don't think VC6 will manage to
compile). IoBind heavily uses Boost: it uses
type_traits, mpl, call_traits,
spirit, (regex optional) and format
libraries.
The headers can be included by
#include <iobind/iobind.hpp>
Note also that all the IoBind classes are in the iobind
namespace.
The encode method
encode is a template method that applies a conversion policy to
an object. It's return type depends on the policy return type. It comes with two
overloads:
template<
typename Value,
typename Policy
>
typename Policy::return_type
encode(
Value const& value_,
Policy const& policy_
);
template<
typename Iterator,
typename Policy
>
typename Policy::return_type
encode(
Iterator begin_,
Iterator end_,
Policy const& policy_
);
This is the front end of IoBind to the user. Example of use of
this method were given in the secion above.
Conversion policies
The individual steering behaviors [...] are components of a larger
structure, like notes of a melody or words of a story. Craig Reynolds,
Steering Behaviors for Autonomous Characters, GDC99.
Conversion policies are the constitutive pieces of IoBind. You
can combine them to create complex serializers.
operator *
Takes two policies and combines them. If a and b
are two policies, then
a*b( value) = a( b( value ) ).
String manipulation
This is a basic string conversion policy not very useful used
alone, but it become quite handy when combined with others.
Example:
int i;
string str=encode(i, to_string() );
i=encode(str, from_string<std::string>() );
Note that almost all policies have predifined instances: the to_string
policy is instanciated as to_string_p .
Base64
Converts streams to the base64 scheme.
struct to_base64;
converts a stream to base64
notation,
struct from_base64;
converts back a stream from base64
notation,
Example:
string str="test";
str=encode( str, to_base64_p);
str=encode( str, from_base64_p);
These policies are base on the base64 iostream converter from Konstant
Pilipchuk:
XML and Latin1 escaping
This policy takes care of transforming a string to XML or
Latin1 conformant string. It replaces reserved characters <,>,... by
<, >, etc...
The usage is straight forward and similar to to_base64,
from_base64.
Sequence container
This policy handles sequence containers such as vector,
list, etc (as you will see later, it can also be used for
associative containers).
template<
typename Policy
>
struct sequence_to_string;converts a sequence to a
string. This policy has the following constructor: sequence_to_string(
policy_const_reference item_policy_,
string_param_type begin_ = "",
string_param_type delimiter_ = ",",
string_param_type end_ = ""
)
where item_<CODE>policy_ is the policy applied
to the sequence elements, and the other parameters are used to separated the
data. In fact, the core of the writer is: output
<<m_begin
..
<<m_policy.encode(value)<<m_delimiter,
..
<<m_end;
template<
typename Container,
typename Policy
>
struct sequence_from_string;reads a sequence from a
string and fills a container. This policy has the following constructor: sequence_from_string(
policy_const_reference item_policy_,
string_param_type begin_ = "",
string_param_type delimiter_ = ",",
string_param_type end_ = ""
)
where the parameters have similar behavior as above. The
item_policy_ is used to transform the string before adding it to
the container.
These policies support other operators that take care of policy
or container change:
<<, changes the policy,
%, changes the container type (only for
pair_from_string).
Example converting elements of a vector<float> to base64 and back:
vector<float> v_f;
for (i=1;i<5;++i)
v_f.push_back( 1/static_cast<FLOAT>(i) );
str=encode(
v_f.begin(),
v_f.end(),
sequence_to_string_p << to_base64()
);
cerr<<"\tv (to_string, base64): "<<str<<endl;
cerr<<"\tv is cleared..."<<endl;
v_f.clear();
v_f=encode(
str,
sequence_from_string_b( v_f, from_string<float>() * from_base64() )
);
cerr<<"\tv (from_string from base64): "<<encode(
v_f.begin(),
v_f.end(),
sequence_to_string_p
)
<<endl;
-- output
v (to_string, base64): MQA=,MC41AA==,MC4zMzMzMzMA,MC4yNQA=
v is cleared...
v (from_string from base64): 1,0.5,0.333333,0.25
Pair
This policy handles the famous std::pair structure.
template<
typename FirstPolicy,
typename SecondPolicy
>
class pair_to_string;
converts a pair to a string. This class has the following
constructor: pair_to_string(
first_policy_const_reference first_policy_,
second_policy_const_reference second_policy_,
string_param_type begin_ = "(",
string_param_type delimiter_ = ":",
string_param_type end_ = ")"
)
where first/second_policy_ are the policies applied
respectively to the first and second members of
pair, and the other parameters are used to separated the data. In
fact, the core of the writer is: output
<<m_begin
<<m_first_policy.encode(value.first)
<<m_delimiter,
<<m_second_policy.encode(value.second),
<<m_end;
template<
typename Pair,
typename FirstPolicy,
typename SecondPolicy
>
class pair_from_string;
reads a pair from a string. This class has the following
constructor: pair_from_string(
first_policy_const_reference first_policy_,
second_policy_const_reference second_policy_,
string_param_type begin_ = "(",
string_param_type delimiter_ = ":",
string_param_type end_ = ")"
)
where first/second_policy_ are the policies applied
respectively to the first and second members of
pair, and the other parameters are used to separated the data. In
fact, the core of the writer is: pair.first=m_frist_policy.encode(first_string);
pair.second=m_second_policy.encode(second_string);
These policies support new operators that take care of policy, pair type
change:
<<, changes the first policy,
>>, changes the second policy,
%, changes the pair type (only for
pair_from_string).
Example:
pair<int,string> p_fs(1,"second");
str=encode( p_fs, pair_to_string_p);
cerr<<"\tpair (1,second): "<<str<<endl;
cerr<<"\treseting pair"<<endl;
p_fs.first=0;
p_fs.second="";
p_fs=encode(
str,
pair_from_string_p
% p_fs
<< from_string<int>()
>> from_string<std::string>()
);
cerr<<"\tpair (from string):"<<encode( p_fs, pair_to_string_p)<<endl;
-- output
pair (1,second): (1:second)
reseting pair
pair (from string):(1:second)
Associative containers
Associative containers such as map,set, etc... are
just a combination of a sequence container and a pair (speaking about
serialization). Hence, using sequence_to/from_string and
pair_to/from_string, you can easily build serializers for them, witout
redifining new classes (the compiler will build them for you):
map<float,string> m_fs;
const char* numbers[] = {"one", "two", "three", "four", "five"};
for (i=1;i<5;++i)
m_fs[static_cast<float>(i)]=numbers[i];
str=encode(
m_fs.begin(),
m_fs.end(),
sequence_to_string_p << pair_to_string_p
);
cerr<<"\tm (to_string): "<<str<<endl;
cerr<<"\tm is cleared..."<<endl;
m_fs.clear();
m_fs=encode(
str,
sequence_from_string_p % m_fs
<< (
pair_from_string_p
% pair<float,std::string>()
<< from_string<float>()
>> from_string<std::string>()
)
);
cerr<<"\tm (from_string): "<<encode(
m_fs.begin(),
m_fs.end(),
sequence_to_string_p << pair_to_string_p
)
<<endl;
-- output
-- associative container:
combination of sequence_to/from_string and pair
m (to_string): (1:two),(2:three),(3:four),(4:five)
m is cleared...
m (from_string): (1:two),(2:three),(3:four),(4:five)
Zip
The zip policy uses the famous zlib C library (see
[2]). It compresses a buffer to another buffer. This policy, combined with
base64 can be used to store byte buffers in XML files:
vector<unsigned char> buffer;
string s;
s = encode( buffer, to_base64 * to_zip_p );
Others
There is room for a lot of other policies:
- encryption, decryption,
- url encoding, decoding,
- ...
History
- 06-30-2003 - Added zip, latin1, case, crc, hex policies.
- 05-04-2003 - Initial release.
Reference
