Introduction

This page explains the fundamental concepts behind buffer sequences and why they exist. After reading, you will understand when to use buffer sequences instead of raw pointers or standard containers.

Code snippets throughout the documentation are written as if the following declarations are in effect: 
#include <boost/buffers.hpp>
using namespace boost::buffers;

Why Buffer Sequences?

Many algorithms operate on raw bytes rather than typed data structures:

  • Encryption and decryption

  • Compression and decompression

  • JSON parsing and serialization

  • Network protocols such as HTTP and WebSocket

These algorithms are often composed. A network application may need to remove framing from a WebSocket stream, decompress the payloads, and parse the result as JSON. Buffer sequences provide vocabulary types that enable such composition without unnecessary copying.

Contiguous Buffers

The fundamental representation of unstructured bytes is a contiguous buffer: a pointer to memory plus a byte count. The library provides const_buffer for read-only data and mutable_buffer for writable data:

// const_buffer: read-only view of bytes
void const* data();
std::size_t size();

// mutable_buffer: writable view of bytes
void* data();
std::size_t size();

These types are lightweight handles. They do not own the memory they reference. The caller must ensure the underlying storage outlives any buffer objects that reference it.

Creating Buffers

Buffers can be constructed from pointers and sizes:

char storage[100];
mutable_buffer mb(storage, sizeof(storage));

char const* message = "hello";
const_buffer cb(message, 5);

A mutable_buffer converts implicitly to const_buffer, since writable memory can always be read:

const_buffer cb = mb;  // OK: mutable converts to const

The Scatter/Gather Problem

A single contiguous buffer is insufficient for many use cases. Consider an algorithm that adds framing bytes around caller-provided data. Without scatter/gather support, it must allocate new memory and copy everything:

// Without scatter/gather: expensive copy required
std::vector<char> framed;
framed.insert(framed.end(), header.begin(), header.end());
framed.insert(framed.end(), payload.begin(), payload.end());
framed.insert(framed.end(), trailer.begin(), trailer.end());
send(socket, framed.data(), framed.size());

Operating systems provide scatter/gather I/O (vectored I/O) to avoid this. Linux uses iovec, and Windows uses WSABUF. These accept a list of buffer pointers, sending or receiving data across multiple memory regions in a single system call.

Buffer Sequences

Rather than choosing a single container type like std::vector<const_buffer>, the library defines concepts that any suitable type can model:

A buffer sequence has these properties:

  • Cheap to copy. The sequence itself is a lightweight handle. Copying the sequence does not copy the underlying bytes.

  • Stable references. A copy of a buffer sequence refers to the same memory regions as the original.

  • Bidirectional iteration. Elements are accessed through iterators whose value type converts to const_buffer or mutable_buffer.

  • Single buffers are sequences. The types const_buffer and mutable_buffer are themselves valid buffer sequences of length one.

Standard Containers as Sequences

Any bidirectional range of buffers models the buffer sequence concepts:

std::vector<const_buffer> v;
v.push_back(const_buffer(header, header_size));
v.push_back(const_buffer(payload, payload_size));
// v is a valid ConstBufferSequence

std::array<mutable_buffer, 2> arr = {
    mutable_buffer(buf1, len1),
    mutable_buffer(buf2, len2)
};
// arr is a valid MutableBufferSequence

However, standard containers have limitations:

  • std::list is expensive to copy

  • std::vector may allocate when resized

  • std::array has fixed size determined at compile time

The library provides specialized types that address these limitations.

Buffer Pairs

Buffer sequences of exactly two elements occur frequently. A circular buffer’s readable region wraps around the end of its storage, appearing as two separate contiguous regions. The types const_buffer_pair and mutable_buffer_pair represent such two-element sequences efficiently:

mutable_buffer_pair mbp;
mbp[0] = mutable_buffer(region1, size1);
mbp[1] = mutable_buffer(region2, size2);

// Iterate like any buffer sequence
for(auto it = begin(mbp); it != end(mbp); ++it)
{
    mutable_buffer b = *it;
    std::cout << "Region: " << b.size() << " bytes\n";
}

Buffer pairs are returned by circular_buffer methods, discussed in Dynamic Buffers.

Iterating Buffer Sequences

To iterate a buffer sequence, use begin and end from this library. These handle both ranges and single-buffer types uniformly:

template<class ConstBufferSequence>
void print_sizes(ConstBufferSequence const& bs)
{
    auto const last = buffers::end(bs);
    for(auto it = buffers::begin(bs); it != last; ++it)
    {
        const_buffer b(*it);
        std::cout << b.size() << " bytes\n";
    }
}

// Works with ranges
std::vector<const_buffer> v = /* ... */;
print_sizes(v);

// Also works with single buffers
const_buffer cb(data, len);
print_sizes(cb);  // Prints one size
Buffer data is untyped. Casting the void* pointer to other types such as char* is the caller’s responsibility. Ensure such casts respect alignment requirements and do not violate type safety.

When to Use Buffer Sequences

Use buffer sequences when:

  • Composing I/O operations. Pass data between protocol layers without copying.

  • Interfacing with system calls. Map directly to iovec or WSABUF structures.

  • Writing generic algorithms. Accept any buffer representation through templates.

Use raw pointers or std::span when:

  • The data is always contiguous. No need for scatter/gather abstraction.

  • Performance is critical. Avoid iterator indirection in tight loops.

Use owning containers like std::vector<char> when:

  • The buffer must outlive the current scope. Sequences only reference memory.

  • Dynamic sizing with ownership. Standard containers manage their own storage.

Edit this Page