程序笔记   发布时间:2022-07-19  发布网站:大佬教程  code.js-code.com
大佬教程收集整理的这篇文章主要介绍了C++实现二进制序列化/反序列化大佬教程大佬觉得挺不错的,现在分享给大家,也给大家做个参考。

背景实现代码测试代码
toc

背景

涉及对象传输或存储时,均需要序列化的参与,将对象转为连续的二进制数据后才能进行传输或存储,需要还原对象时,通过反序列化逆向处理二进制数据遍能得到原对象

  • 这里的对象是一种广泛的概念,往大了说就是一段有意义的内存

实现

  • 实现过程中主要使用模板应对各种类型,特化+宏应对基础数据类型,逐层处理应对STL嵌套情况
  • C++语言本身不带反射,自定义结构需继承接口类serializable,并实现相应接口
  • 实现时使用了std::move()进行构造与赋值,自定义结构最好实现自己的移动构造函数与移动赋值函数
  • 获取序列化buffer时,先计算序列化目标在序列化后的总字节数,一次性申请足够内存
  • 获取序列化buffer时,返回的buffer之前藏了序列化目标的总字节数,藏东西这招跟redis学的
  • 藏的目标总字节数目的用于,反序列化前简单判断待反序列化buffer是否有效
  • 未支持指针类型,传递错误类型的指针(强转目标类型与实际类型不符合)会导致内存错误,序列化过程中没法判断类型是否正确
  • 未支持C风格字符串,C风格字符串以0结尾,非二进制安全
  • 需要序列化一段连续内存时,构造std::string后传入
  • 反序列化时,调用者需自行保证,待反序列化buffer的原始类型与目标类型一致
  • 使用方式 申请序列化buffer->序列化->三方库压缩->传输/落盘->接收/读取->三方库解压->简单检查->反序列化->释放buffer
  • 序列化后的数据格式如下
    • 先写入对象序列化后总长度,后面才接着写入对象序列化数据
      • 对象是基础类型则直接写入数据
      • 对象是STL先写入元素个数,再写入每个元素序列化数据
        • 如果元素是std::pairl类型,则先写入序列化Key,再写入序列化Value
      • 对象是自定义结构,则需用户定义数据格式

总体结构:

对象序列化后总长度 对象序列化数据

对象是基础数据:

对象序列化后总长度 对象序列化数据(charintshortlong.........)

对象是STL:

对象序列化后总长度 元素个数 元素1序列化数据 元素2序列化数据 元素3序列化数据 ..... 元素n序列化数据

对象是STL之Map:

对象序列化后总长度 元素个数 Pair 1之Key序列化数据 Pair 1之Value序列化数据 ..... Pair n之Key序列化数据 Pair n之Value序列化数据

代码

作为一个Demo写的,写得不对象,见笑了

#ifndef _serialization_H_
#define _serialization_H_

#include <map>
#include <set>
#include <list>
#include <String>
#include <vector>

//存放长度信息类型
using LEN = unsigned;

//基础数据长度计算
#define BASIC_data_length(BasicTypE)                                        
template<>                                                                    
unsigned Length(const BasicType& elem){                                        
    return sizeof(elem);                                                    
}                                        

//申请用于序列化的buffer         总格式    |数据总长度|数据|
#define MAKE_BUFFER(uLen){                                                    
    char *pBuffer = new char[uLen + sizeof(LEN)];                            
    if(nullptr == pBuffer){                                                    
        return nullptr;                                                        
    }                                                                        
    *reinterpret_cast<LEN *const>(pBuffer) = uLen;                            
    return pBuffer + sizeof(LEN);                                            
}

//基础数据序列化buffer获取
#define BASIC_DATA_GET_BUFFER(BasicTypE)                                    
template<>                                                                    
char *const GetBuffer(const BasicType& elem){                                
    LEN uLen = sizeof(elem);                                                
    MAKE_BUFFER(uLen);                                                        
}

//基础数据序列化主体
#define BASIC_DATA_seriaLIZE(BasicTypE)                                        
template<>                                                                    
unsigned serialize(const BasicType& elem, char *const pBuffer){                
    ::memcpy(pBuffer, &elem, sizeof(BasicTypE));                            
    return sizeof(BasicTypE);                                                
}

//基础数据反序列化主体
#define BASIC_DATA_UNseriaLIZE(BasicTypE)                                    
template<>                                                                    
unsigned Unserialize(const char *const pBuffer, BasicType& elem){            
    ::memcpy(&elem, pBuffer, sizeof(BasicTypE));                            
    return sizeof(BasicTypE);                                                
}

//自定义结构序列化基类
class serializable{
public:
    virtual unsigned Length() const = 0;
    virtual unsigned serialize(char *const pBuffer) const = 0;
    virtual unsigned Unserialize(const char *const pBuffer) = 0;
    virtual ~serializable(){};
};

/*
    前项声明部分,STL嵌套
*/
template<typename Elem>
unsigned Length(const std::list<Elem>& list);
template<typename Key, typENAME value>
unsigned Length(const std::map<Key, Value>& map);
template<typename Elem>
unsigned Length(const std::vector<Elem>& vec);
template<typename Elem>
unsigned Length(const std::set<Elem>& set);

template<typename Elem>
char *const GetBuffer(const std::list<Elem>& list);
template<typename Key, typENAME value>
char *const GetBuffer(const std::map<Key, Value>& map);
template<typename Elem>
char *const GetBuffer(const std::vector<Elem>& vec);
template<typename Elem>
char *const GetBuffer(const std::set<Elem>& set);

template<typename Elem>
unsigned serialize(const std::list<Elem>& list, char *const pBuffer);
template<typename Key, typENAME value>
unsigned serialize(const std::map<Key, Value>& map, char *const pBuffer);
template<typename Elem>
unsigned serialize(const std::vector<Elem>& vec, char *const pBuffer);
template<typename Elem>
unsigned serialize(const std::set<Elem>& set, char *const pBuffer);

template<typename Elem>
unsigned Unserialize(const char* pBuffer, std::list<Elem>& list);
template<typename Key, typENAME value>
unsigned Unserialize(const char* pBuffer, std::map<Key, Value>& map);
template<typename Elem>
unsigned Unserialize(const char* pBuffer, std::vector<Elem>& vec);
template<typename Elem>
unsigned Unserialize(const char* pBuffer, std::set<Elem>& set);
/*
    长度计算部分
    得到数据长度
*/
template<typename Elem>
unsigned Length(const Elem& elem){
    return elem.Length();
}

BASIC_data_length(char)
BASIC_data_length(short)
BASIC_data_length(int)
BASIC_data_length(float)
BASIC_data_length(long)
BASIC_data_length(doublE)
BASIC_data_length(long long)
BASIC_data_length(unsigned char)
BASIC_data_length(unsigned short)
BASIC_data_length(unsigned int)
BASIC_data_length(unsigned long)
BASIC_data_length(unsigned long long)

template<>
unsigned Length(const std::string& str){
    return sizeof(LEN) + static_cast<unsigned>(str.size());                //容器对象 格式 |元素个数|数据|
}

template<typename Elem>
unsigned Length(const std::list<Elem>& list){
    LEN uLen = 0;
    for(const auto& elem : list){
        uLen += Length(elem);
    }
    return sizeof(LEN) + uLen;
}

template<typename Key, typENAME value>
unsigned Length(const std::map<Key, Value>& map){
    LEN uLen = 0;
    for(const auto& elem : map){
        uLen += Length(elem.first);
        uLen += Length(elem.second);
    }
    return sizeof(LEN) + uLen;
}

template<typename Elem>
unsigned Length(const std::vector<Elem>& veC){
    LEN uLen = 0;
    for(const auto& elem : veC){
        uLen += Length(elem);
    }
    return sizeof(LEN) + uLen;
}

template<typename Elem>
unsigned Length(const std::set<Elem>& set){
    LEN uLen = 0;
    for(const auto& elem : set){
        uLen += Length(elem);
    }
    return sizeof(LEN) + uLen;
}

/*
    buffer申请、释放部分
    一次性申请足够内存
*/
template<typename Elem>
char *const GetBuffer(const Elem& elem){
    LEN uLen = elem.Length();
    MAKE_BUFFER(uLen);
}

BASIC_DATA_GET_BUFFER(char)
BASIC_DATA_GET_BUFFER(short)
BASIC_DATA_GET_BUFFER(int)
BASIC_DATA_GET_BUFFER(float)
BASIC_DATA_GET_BUFFER(long)
BASIC_DATA_GET_BUFFER(doublE)
BASIC_DATA_GET_BUFFER(long long)
BASIC_DATA_GET_BUFFER(unsigned char)
BASIC_DATA_GET_BUFFER(unsigned short)
BASIC_DATA_GET_BUFFER(unsigned int)
BASIC_DATA_GET_BUFFER(unsigned long)
BASIC_DATA_GET_BUFFER(unsigned long long)

template<>
char *const GetBuffer(const std::string& str){
    LEN uLen = Length(str);
    MAKE_BUFFER(uLen);
}

template<typename Elem>
char *const GetBuffer(const std::list<Elem>& list){
    LEN uLen = Length(list);
    MAKE_BUFFER(uLen);
}

template<typename Key, typENAME value>
char *const GetBuffer(const std::map<Key, Value>& map){
    LEN uLen = Length(map);
    MAKE_BUFFER(uLen);
}

template<typename Elem>
char *const GetBuffer(const std::vector<Elem>& veC){
    LEN uLen = Length(vec);
    MAKE_BUFFER(uLen);
}

template<typename Elem>
char *const GetBuffer(const std::set<Elem>& set){
    LEN uLen = Length(set);
    MAKE_BUFFER(uLen);
}

static void ReleaseBuffer(char *const pBuffer){
    delete[] (pBuffer - sizeof(LEN));
}

/*
    序列化部分
*/
template<typename Elem>
unsigned serialize(const Elem& elem, char *const pBuffer){
    return elem.serialize(pBuffer);
}

BASIC_DATA_seriaLIZE(char)
BASIC_DATA_seriaLIZE(short)
BASIC_DATA_seriaLIZE(int)
BASIC_DATA_seriaLIZE(float)
BASIC_DATA_seriaLIZE(long)
BASIC_DATA_seriaLIZE(doublE)
BASIC_DATA_seriaLIZE(long long)
BASIC_DATA_seriaLIZE(unsigned char)
BASIC_DATA_seriaLIZE(unsigned short)
BASIC_DATA_seriaLIZE(unsigned int)
BASIC_DATA_seriaLIZE(unsigned long)
BASIC_DATA_seriaLIZE(unsigned long long)

template<>
unsigned serialize(const std::string& str, char* const pBuffer){
    *reinterpret_cast<LEN *const>(pBuffer) = static_cast<LEN>(str.size());            //元素个数
    ::memcpy(pBuffer + sizeof(LEN), str.data(), str.size());                        //数据
    return static_cast<unsigned>(str.size()) + sizeof(LEN);
}

template<typename Elem>
unsigned serialize(const std::list<Elem>& list, char *const pBuffer){
    unsigned uPos = sizeof(LEN);
    *reinterpret_cast<LEN *const>(pBuffer) = static_cast<LEN>(list.size());            //元素个数
    for(const auto& elem : list){
        uPos += serialize(elem, pBuffer + uPos);
    }
    return uPos;
}

template<typename Key, typENAME value>
unsigned serialize(const std::map<Key, Value>& map, char *const pBuffer){
    unsigned uPos = sizeof(LEN);
    *reinterpret_cast<LEN *const>(pBuffer) = static_cast<LEN>(map.size());            //元素个数
    for(const auto& elem : map){
        uPos += serialize(elem.first, pBuffer + uPos);
        uPos += serialize(elem.second, pBuffer + uPos);
    }
    return uPos;
}

template<typename Elem>
unsigned serialize(const std::vector<Elem>& vec, char *const pBuffer){
    unsigned uPos = sizeof(LEN);
    *reinterpret_cast<LEN *const>(pBuffer) = static_cast<LEN>(vec.size());            //元素个数
    for(const auto& elem : veC){
        uPos += serialize(elem, pBuffer + uPos);
    }
    return uPos;
}

template<typename Elem>
unsigned serialize(const std::set<Elem>& set, char *const pBuffer){
    unsigned uPos = sizeof(LEN);
    *reinterpret_cast<LEN *const>(pBuffer) = static_cast<LEN>(set.size());            //元素个数
    for(const auto& elem : set){
        uPos += serialize(elem, pBuffer + uPos);
    }
    return uPos;
}

/*
    检查反序列化之前的buffer
    BufferLen指包含数据总长度在内的长度,即从硬盘上读取并解压缩后的整个buffer长度
*/
int checkLength(const char *const pBuffer, int BufferLen){
    if(*reinterpret_cast<const LEN *const>(pBuffer - sizeof(LEN)) + sizeof(LEN) != BufferLen){
        return 0;
    }
    return 1;
}

/*
    反序列化部分
*/
template<typename Elem>
unsigned Unserialize(const char* pBuffer, Elem& elem){
    return elem.Unserialize(pBuffer);
}

BASIC_DATA_UNseriaLIZE(char)
BASIC_DATA_UNseriaLIZE(short)
BASIC_DATA_UNseriaLIZE(int)
BASIC_DATA_UNseriaLIZE(float)
BASIC_DATA_UNseriaLIZE(long)
BASIC_DATA_UNseriaLIZE(doublE)
BASIC_DATA_UNseriaLIZE(long long)
BASIC_DATA_UNseriaLIZE(unsigned char)
BASIC_DATA_UNseriaLIZE(unsigned short)
BASIC_DATA_UNseriaLIZE(unsigned int)
BASIC_DATA_UNseriaLIZE(unsigned long)
BASIC_DATA_UNseriaLIZE(unsigned long long)

template<>
unsigned Unserialize(const char* pBuffer, std::string& str){
    LEN uLen = *reinterpret_cast<const LEN *>(pBuffer);
    std::string strTemp(pBuffer + sizeof(LEN), uLen);
    str = std::move(strTemp);
    return uLen + sizeof(LEN);
}

template<typename Elem>
unsigned Unserialize(const char* pBuffer, std::list<Elem>& list){
    LEN uLen = *reinterpret_cast<const LEN *>(pBuffer);
    unsigned uPos = sizeof(LEN);
    for(LEN i = 0; i < uLen; i++){
        Elem elem;
        uPos += Unserialize(pBuffer + uPos, elem);
        list.push_BACk(std::move(elem));
    }
    return uPos;
}

template<typename Key, typENAME value>
unsigned Unserialize(const char* pBuffer, std::map<Key, Value>& map){
    LEN uLen = *reinterpret_cast<const LEN *>(pBuffer);
    unsigned uPos = sizeof(LEN);
    for(LEN i = 0; i < uLen; i++){
        Key key;
        Value val;
        uPos += Unserialize(pBuffer + uPos, key);
        uPos += Unserialize(pBuffer + uPos, val);
        map.insert(std::make_pair(std::move(key), std::move(val)));
    }
    return uPos;
}

template<typename Elem>
unsigned Unserialize(const char* pBuffer, std::vector<Elem>& veC){
    LEN uLen = *reinterpret_cast<const LEN *>(pBuffer);
    vec.resize(uLen);
    unsigned uPos = sizeof(LEN);
    for(LEN i = 0; i < uLen; i++){
        Elem elem;
        uPos += Unserialize(pBuffer + uPos, elem);
        vec[i] = std::move(elem);
    }
    return uPos;
}

template<typename Elem>
unsigned Unserialize(const char* pBuffer, std::set<Elem>& set){
    LEN uLen = *reinterpret_cast<const LEN *>(pBuffer);
    unsigned uPos = sizeof(LEN);
    for(LEN i = 0; i < uLen; i++){
        Elem elem;
        uPos += Unserialize(pBuffer + uPos, elem);
        set.emplace(std::move(elem));
    }
    return uPos;
}

#endif // !_serialization_H_

测试代码

#include <cstdlib>
#include <Cassert>
#include <iostream>

#include "serialization.h"

template<typename T>
void Test(const T& a){
    std::cout << Length(a) << std::endl;

    auto pBuffer = GetBuffer(a);

    auto len = serialize(a, pBuffer);
    std::cout << len << std::endl;

    assert(1 == checkLength(pBuffer, Length(a) + sizeof(LEN)));

    T b = T();
    len = Unserialize(pBuffer, b);
    std::cout << len << std::endl;

    ReleaseBuffer(pBuffer);
    assert(a == b);
}

class myTest : public serializable{
public:
    myTest() : m_i(0), m_c(0), m_d(0){}

    myTest(int i, char c, double d, const std::string& str, const std::list<int>& list,
        const std::map<int, double>& map, const std::vector<int>& vec, const std::set<int>& set)
        :m_i(i), m_c(C), m_d(d), m_str(str), m_list(list), m_map(map), m_vec(veC), m_set(set){}

    myTest(int i, char c, double d, std::string&& str, std::list<int>&& list,
        std::map<int, double>&& map, std::vector<int>&& vec, std::set<int>&& set)        //针对列表初始化
        :m_i(i), m_c(C), m_d(d), m_str(std::move(str)), m_list(std::move(list)), m_map(std::move(map)), m_vec(std::move(veC)), m_set(std::move(set)){}

    bool operator==(const myTest& rhs) const{
        return m_i == rhs.m_i && m_c == rhs.m_c && m_d == rhs.m_d && m_str == rhs.m_str && m_list == rhs.m_list && m_map == rhs.m_map && m_vec == rhs.m_vec && m_set == rhs.m_set;
    }

    virtual unsigned Length() const override{
        return ::Length(m_i) + ::Length(m_C) + ::Length(m_d) + ::Length(m_str) + ::Length(m_list) + ::Length(m_map) + ::Length(m_veC) + ::Length(m_set);
    }
    virtual unsigned serialize(char *const pBuffer) const override{
        unsigned uPos = 0;
        uPos += ::serialize(m_i, pBuffer + uPos);
        uPos += ::serialize(m_c, pBuffer + uPos);
        uPos += ::serialize(m_d, pBuffer + uPos);
        uPos += ::serialize(m_str, pBuffer + uPos);
        uPos += ::serialize(m_list, pBuffer + uPos);
        uPos += ::serialize(m_map, pBuffer + uPos);
        uPos += ::serialize(m_vec, pBuffer + uPos);
        uPos += ::serialize(m_set, pBuffer + uPos);
        return  uPos;
    }
    virtual unsigned Unserialize(const char *const pBuffer)override{
        unsigned uPos = 0;
        uPos += ::Unserialize(pBuffer + uPos, m_i);
        uPos += ::Unserialize(pBuffer + uPos, m_c);
        uPos += ::Unserialize(pBuffer + uPos, m_d);
        uPos += ::Unserialize(pBuffer + uPos, m_str);
        uPos += ::Unserialize(pBuffer + uPos, m_list);
        uPos += ::Unserialize(pBuffer + uPos, m_map);
        uPos += ::Unserialize(pBuffer + uPos, m_vec);
        uPos += ::Unserialize(pBuffer + uPos, m_set);
        return uPos;
    }

private:
    int m_i;
    char m_c;
    double m_d;
    std::string m_str;
    std::list<int> m_list;
    std::map<int, double> m_map;
    std::vector<int> m_vec;
    std::set<int> m_set;
};



int main(int argc, char** argv){
    //基础类型
    Test<char>('a');
    Test<short>(-1);
    Test<int>(-1);
    Test<float>(-1.0);
    Test<long>(-1);
    Test<double>(-1.0);
    Test<long long>(-1);
    Test<unsigned char>(1);
    Test<unsigned short>(1);
    Test<unsigned int>(1);
    Test<unsigned long>(1);
    Test<unsigned long long>(1);

    //STL
    Test<std::string>("Test!!!");
    Test<std::list<int>>({1, 2, 3, 4, 5, 6});
    Test<std::map<int, double>>({{1, 1.0}, {2, 2.0}, {3, 3.0}});
    Test<std::vector<int>>({1, 2, 3, 4, 5, 6});
    Test<std::set<int>>({1, 2, 3, 4, 5, 6});

    //嵌套STL
    Test<std::set<std::string>>({"test1", "test2", "test3"});
    Test<std::list<std::set<std::string>>>({{"test1", "test2", "test3"}, {"test1", "test2", "test3"}, {"test1", "test2", "test3"}});
    Test<std::map<int, std::list<std::set<std::string>>>>({{1, {{"test1", "test2", "test3"}, {"test1", "test2", "test3"}, {"test1", "test2", "test3"}}},
        {2, {{"test1", "test2", "test3"}, {"test1", "test2", "test3"}, {"test1", "test2", "test3"}}},
        {3, {{"test1", "test2", "test3"}, {"test1", "test2", "test3"}, {"test1", "test2", "test3"}}}});
    Test<std::vector<std::map<int, std::list<std::set<std::string>>>>>({{{1, {{"test1", "test2", "test3"}, {"test1", "test2", "test3"}, {"test1", "test2", "test3"}}},
        {2, {{"test1", "test2", "test3"}, {"test1", "test2", "test3"}, {"test1", "test2", "test3"}}},
        {3, {{"test1", "test2", "test3"}, {"test1", "test2", "test3"}, {"test1", "test2", "test3"}}}},
        {{1, {{"test1", "test2", "test3"}, {"test1", "test2", "test3"}, {"test1", "test2", "test3"}}},
        {2, {{"test1", "test2", "test3"}, {"test1", "test2", "test3"}, {"test1", "test2", "test3"}}},
        {3, {{"test1", "test2", "test3"}, {"test1", "test2", "test3"}, {"test1", "test2", "test3"}}}},
        {{1, {{"test1", "test2", "test3"}, {"test1", "test2", "test3"}, {"test1", "test2", "test3"}}},
        {2, {{"test1", "test2", "test3"}, {"test1", "test2", "test3"}, {"test1", "test2", "test3"}}},
        {3, {{"test1", "test2", "test3"}, {"test1", "test2", "test3"}, {"test1", "test2", "test3"}}}}});

    //自定义结构
    myTest test(1, 'a', 1.0, "Test!!!", {1, 2, 3, 4, 5, 6}, {{1, 1.0}, {2, 2.0}, {3, 3.0}}, {1, 2, 3, 4, 5, 6}, {1, 2, 3, 4, 5, 6});
    Test<myTest>(test);

    system("pause");
    return 0;
}
来自为知笔记(Wiz)

大佬总结

以上是大佬教程为你收集整理的C++实现二进制序列化/反序列化全部内容,希望文章能够帮你解决C++实现二进制序列化/反序列化所遇到的程序开发问题。

如果觉得大佬教程网站内容还不错,欢迎将大佬教程推荐给程序员好友。

本图文内容来源于网友网络收集整理提供,作为学习参考使用,版权属于原作者。
如您有任何意见或建议可联系处理。小编QQ:384754419,请注明来意。