How to ensure that we read all lines from boost::child process

I think there's no proper way unless you use asynchronous methods.

Perhaps you can simple get a future to a vector and use string_views into that if you somehow really need it line-by-line.

std::future<std::vector<char> > output, error;

boost::asio::io_service svc;
bp::child c(bp::search_path("nm"), file, bp::std_out > output, bp::std_err > error, svc);
svc.run();

To read exactly like you did before you can use an istream on top of the vector:

#include <boost/process.hpp>
#include <boost/iostreams/device/array.hpp>
#include <boost/iostreams/stream_buffer.hpp>
#include <iostream>

namespace bp = boost::process;
namespace bio = boost::iostreams;
std::string const file = "./a.out";

int main() {
    std::future<std::vector<char> > output, error;

    boost::asio::io_service svc;
    bp::child c(bp::search_path("nm"), file, bp::std_out > output, bp::std_err > error, svc);
    svc.run();

    //then later
    {
        auto raw = output.get();
        std::vector<std::string> data;
        std::string line;
        bio::stream_buffer<bio::array_source> sb(raw.data(), raw.size());
        std::istream is(&sb);

        while (std::getline(is, line) && !line.empty())
            data.push_back(line);

        std::cout << data.at(rand()%data.size()) << "\n";
    }

}

I had the same issue... The best way to deal with this is by using async i/o.

Unfortunately, the boost documentation @ http://www.boost.org/doc/libs/master/doc/html/boost_process/extend.html#boost_process.extend.async was wrong... It makes it all look simple, but doesn't show that the buffers must be sized beforehand, and glosses over many details.

Here's my function, which send or receives one buffer in one go (no interaction like question/response0, I use stderr for error checking, since that's what was needed for my app, but you could catch the app's exit code by calling 'c.exit_code();`.

using tstring=basic_string<TCHAR>;

void Run(
    const tstring& exeName;
    const tstring& args,
    const std::string& input,
    std::string& output,
    std::string& error
)
{
    using namespace boost;

    asio::io_service ios;

    std::vector<char> vOut(128 << 10);
    auto outBuffer{ asio::buffer(vOut) };
    process::async_pipe pipeOut(ios);

    std::function<void(const system::error_code & ec, std::size_t n)> onStdOut;
    onStdOut = [&](const system::error_code & ec, size_t n)
    {
        output.reserve(output.size() + n);
        output.insert(output.end(), vOut.begin(), vOut.begin() + n);
        if (!ec)
        {
            asio::async_read(pipeOut, outBuffer, onStdOut);
        }
    };

    std::vector<char> vErr(128 << 10);
    auto errBuffer{ asio::buffer(vErr) };
    process::async_pipe pipeErr(ios);
    std::function<void(const system::error_code & ec, std::size_t n)> onStdErr;
    onStdErr = [&](const system::error_code & ec, size_t n)
    {
        error.reserve(error.size() + n);
        error.insert(error.end(), vErr.begin(), vErr.begin() + n);
        if (!ec)
        {
            asio::async_read(pipeErr, errBuffer, onStdErr);
        }
    };

    auto inBuffer{ asio::buffer(input) };
    process::async_pipe pipeIn(ios);

    process::child c(
        exeName + _T(" ") + args, 
        process::std_out > pipeOut, 
        process::std_err > pipeErr, 
        process::std_in < pipeIn
    );


    asio::async_write(pipeIn, inBuffer, 
        [&](const system::error_code & ec, std::size_t n) 
        {
            pipeIn.async_close();
        });

    asio::async_read(pipeOut, outBuffer, onStdOut);
    asio::async_read(pipeErr, errBuffer, onStdErr);

    ios.run();
    c.wait();
}

The app I execute is a decoder/encoder, I send an entire file to process, and receive the results simultaneously this way. No file size limit.

IMPORTANT

There is a bug fix in boost 1.64, only affects Windows, apparently

in file boost\process\detail\windows\async_pipe.hpp: ref: https://github.com/klemens-morgenstern/boost-process/issues/90

line 79:

    ~async_pipe()
    {
//fix
        //if (_sink .native()  != ::boost::detail::winapi::INVALID_HANDLE_VALUE_)
        //    ::boost::detail::winapi::CloseHandle(_sink.native());
        //if (_source.native() != ::boost::detail::winapi::INVALID_HANDLE_VALUE_)
        //    ::boost::detail::winapi::CloseHandle(_source.native());
        boost::system::error_code ec;
        close(ec);
//fix
    }