Is there a something in boost to facilitate binary buffer transformations into typed data structures/classes?

Graham Reitz

29 May 2008 29 May '08

6:19 p.m.

I didn't see anything, but I may have missed something. Problem: Binary data is stored in a char buffer from a proprietary device: char* buffer = new char[6]; Data in buffer: [byte 1][byte 2][byte 3][byte 4][byte 5][byte 6] [---16 bit uint--][------------32 bit uint-------------] ...would like to convert to this in a C++'ish style using a boost facility if it exists: (or consider directing me somewhere else if it doesn't) struct // or class { boost::uint16_t first; boost::uint32_t second; } data; I would like to avoid a C-style way of doing this. This seems like something that would have been solved many times. thanks, graham

Show replies by date

Scott McMurray

29 May 29 May

6:37 p.m.

New subject: Is there a something in boost to facilitate binary buffer transformations into typed data structures/classes?

On Thu, May 29, 2008 at 2:19 PM, Graham Reitz wrote:

...

...would like to convert to this in a C++'ish style using a boost facility if it exists: (or consider directing me somewhere else if it doesn't)

There isn't at the moment. However, Beman Dawes has recently updated his proposed Boost.Endian library (http://tinyurl.com/4bswsh), and there is a discussion about it over on the dev list (http://lists.boost.org/Archives/boost/2008/05/137820.php and followups). It would allow you to say (something like) struct // or class { boost::endian::uint_t<16, boost::endian::big_endian> first; boost::endian::uint_t<32, boost::endian::big_endian> second; } data; Which you could then std::copy your data into (through char*s), and use the implicit conversions to get native integral types out. HTH, ~ Scott

Graham Reitz

8:11 p.m.

New subject: Is there a something in boost to facilitate binary buffer transformations into typed data structures/classes?

This is very nice. Thanks Beman, Scott, and other contributors to this! I am coding it up right now. This will really, really help. Do you mean std::copy in <algorithm>? graham On Thu, May 29, 2008 at 1:37 PM, Scott McMurray wrote:

...

On Thu, May 29, 2008 at 2:19 PM, Graham Reitz wrote:

...
...would like to convert to this in a C++'ish style using a boost facility if it exists: (or consider directing me somewhere else if it doesn't)

There isn't at the moment.

However, Beman Dawes has recently updated his proposed Boost.Endian library (http://tinyurl.com/4bswsh), and there is a discussion about it over on the dev list (http://lists.boost.org/Archives/boost/2008/05/137820.php and followups).

It would allow you to say (something like)

struct // or class { boost::endian::uint_t<16, boost::endian::big_endian> first; boost::endian::uint_t<32, boost::endian::big_endian> second; } data;

Which you could then std::copy your data into (through char*s), and use the implicit conversions to get native integral types out.

HTH, ~ Scott _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

Graham Reitz

9:28 p.m.

New subject: Is there a something in boost to facilitate binary buffer transformations into typed data structures/classes?

I am having a little trouble getting this to work. Here's an example I am working on: #include struct data { boost::integer::little8_t first; boost::integer::little8_t second; boost::integer::little16_t third; }; int main(int argc, char *argv[]) { std::vector<unsigned char> binary_buffer; binary_buffer.push_back(0x03); binary_buffer.push_back(0x06); binary_buffer.push_back(0x00); binary_buffer.push_back(0x09); data_record dr; std::copy(binary_buffer.begin(), binary_buffer.end(), /* need an iterator to &dr? */); return 0; } Am I doing this correctly? thanks, graham On Thu, May 29, 2008 at 1:37 PM, Scott McMurray wrote:

...

On Thu, May 29, 2008 at 2:19 PM, Graham Reitz wrote:

...
...would like to convert to this in a C++'ish style using a boost facility if it exists: (or consider directing me somewhere else if it doesn't)

There isn't at the moment.

However, Beman Dawes has recently updated his proposed Boost.Endian library (http://tinyurl.com/4bswsh), and there is a discussion about it over on the dev list (http://lists.boost.org/Archives/boost/2008/05/137820.php and followups).

It would allow you to say (something like)

struct // or class { boost::endian::uint_t<16, boost::endian::big_endian> first; boost::endian::uint_t<32, boost::endian::big_endian> second; } data;

Which you could then std::copy your data into (through char*s), and use the implicit conversions to get native integral types out.

HTH, ~ Scott _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

Scott McMurray

30 May 30 May

1:58 a.m.

New subject: Is there a something in boost to facilitate binary buffer transformations into typed data structures/classes?

On Thu, May 29, 2008 at 5:28 PM, Graham Reitz wrote:

...

data_record dr;

std::copy(binary_buffer.begin(), binary_buffer.end(), /* need an iterator to &dr? */);

std::copy( binary_buffer.begin(), binary_buffer.end(), reinterpret_cast(&dr) );

Roman Perepelitsa

11:17 a.m.

New subject: [Boost-users] Is there a something in boost to facilitate binary buffer transformations into typed data structures/classes?

Scott McMurray writes:

...

std::copy( binary_buffer.begin(), binary_buffer.end(), reinterpret_cast(&dr) );

Or: memcpy(&dr, &buffer[0], buffer.size()); Or use union to avoid making a copy: union { data d; unsigned char bytes[4]; } u; bytes[0] = 3; bytes[1] = 6; bytes[2] = 0; bytes[3] = 9; // use d.first, d.second, d.third HTH, Roman Perepelitsa.

Scott McMurray

1:24 p.m.

New subject: Is there a something in boost to facilitate binary buffer transformations into typed data structures/classes?

On Fri, May 30, 2008 at 7:17 AM, Roman Perepelitsa wrote:

...

Scott McMurray writes:

...
std::copy( binary_buffer.begin(), binary_buffer.end(), reinterpret_cast(&dr) );

Or: memcpy(&dr, &buffer[0], buffer.size());

Though a good implementation will have implemented std::copy for bytes with memcpy.

...

Or use union to avoid making a copy:

union { data d; unsigned char bytes[4]; } u;

bytes[0] = 3; bytes[1] = 6; bytes[2] = 0; bytes[3] = 9;

// use d.first, d.second, d.third

If your compiler allows that as an extension. (GCC does, dunno about others.) It's officially not allowed, last I checked.

Brian Davis

27 Jul 27 Jul

7:51 p.m.

New subject: Is there a something in boost to facilitate binary buffer transformations into typed data structures/classes?

Does this handle byte packing and bitwise operations such as bitfields do? Is byte ordering 1 for free here as desired? #pragma push (pack ,1) struct { boost::uint32_t rawSensorValue1 : 12; boost::uint32_t errorFlag1: 1; boost::uint32_t rawSensorValue2 : 12; boost::uint32_t errorFlag2: 1; boost::uint32_t unused : 6; }; #pragma pop (pack) Brian

Scott McMurray

28 Jul 28 Jul

2:30 a.m.

New subject: Is there a something in boost to facilitate binary buffer transformations into typed data structures/classes?

On Sun, Jul 27, 2008 at 15:51, Brian Davis wrote:

...

Does this handle byte packing and bitwise operations such as bitfields do? Is byte ordering 1 for free here as desired?

Bitfields are not byte-order-independant, and that pragma is not portable (obviously). I wonder about the possibility of a template version of erlang's bitsyntax? ( http://www.erlang.se/workshop/2002/Gustafsson.pdf )

6121

Age (days ago)

6181

Last active (days ago)

List overview

Download

8 comments

4 participants

participants (4)

Brian Davis
Graham Reitz
Roman Perepelitsa
Scott McMurray