...one of the most highly
regarded and expertly designed C++ library projects in the
world. — Herb Sutter and Andrei
To set realistic expectations and prevent a litany of duplicate review statements, these notes address the most common questions and comments about Beast and other HTTP libraries that have gone through formal review.
It is not the intention of the library to provide turn-key solutions for specific HTTP or WebSocket use-cases. Instead, it is a sensible protocol layering on top of Boost.Asio which retains the Boost.Asio memory management style and asynchronous model.
Beast has a functional HTTP server in the example directory. The server supports both HTTP and WebSocket using synchronous and asynchronous shared or dedicated ports. In addition, the server supports encrypted TLS connections if OpenSSL is available, on dedicated ports. And the server comes with a "multi-port", a flexible single port which supports both encrypted and unencrypted connections, both HTTP and WebSocket, all on the same port. The server is not part of Beast's public interfaces, as that functionality is outside the scope of the library. The author feels that attempting to broaden the scope of the library will reduce its appeal for standardization.
"I just want to download a resource using HTTP" is a common cry from users and reviewers. Such functionality is beyond the scope of Beast. Building a full featured HTTP client is a difficult task and large enough to deserve its own library. There are many things to deal with such as the various message body encodings, complex parsing of headers, difficult header semantics such as Range and Cache-Control, redirection, Expect:100-continue, connection retrying, domain name resolution, TLS, and much, much more. It is the author's position that Boost first needs a common set of nouns and verbs for manipulating HTTP at the protocol level; Beast provides that language.
Many reviewers feel that HTTP/2 support is an essential feature of a HTTP library. The authors agree that HTTP/2 is important but also feel that the most sensible implementation is one that does not re-use the same network reading and writing interface for 2 as that for 1.0 and 1.1.
The Beast HTTP message model was designed with the new protocol in mind and should be evaluated in that context. There are plans to add HTTP/2 in the future, but there is no rush to do so. Users can work with HTTP/1 now; we should not deny them that functionality today to wait for a newer protocol tomorrow. It is the author's position that there is sufficient value in Beast's HTTP/1-only implementation that the lack of HTTP/2 should not be a barrier to acceptance.
The Beast HTTP message model is suitable for HTTP/2 and can be re-used. The IETF HTTP Working Group adopted message compatibility with HTTP/1.x as an explicit goal. A parser can simply emit full headers after decoding the compressed HTTP/2 headers. The stream ID is not logically part of the message but rather message metadata and should be communicated out-of-band (see below). HTTP/2 sessions begin with a traditional HTTP/1.1 Upgrade similar in fashion to the WebSocket upgrade. An HTTP/2 implementation can use existing Beast.HTTP primitives to perform this handshake.
Beast uses more than Boost.Asio, it depends on various other parts of Boost. The standalone Asio is currently farther ahead than the Boost version. Keeping Beast maintained against both versions of Asio is beyond the resources of the author at the present time. Compatibility with non-Boost libraries should not be an acceptance criteria. Beast is currently designed to be a part of Boost: nothing more, nothing less. Looking at the bigger picture, it is the author's goal to propose this library for standardization. A logical track for achieving this is as follows:
The energy invested in Beast went into the design of the interfaces, not performance. That said, the most sensitive parts of Beast have been optimized or designed with optimization in mind. The slow parts of WebSocket processing have been optimized, and the HTTP parser design is lifted from another extremely popular project which has performance as a design goal (see https://github.com/h2o/picohttpparser).
"Aim first for clarity and correctness; optimization should be only a secondary concern in most Boost libraries."
As the library matures it will undergo optimization passes; benchmarks will logically accompany this process. There is a small benchmarking program included in the tests which compares the performance of Beast's parser to the NodeJS reference parser, as well as some benchmarks which compare the performance of various Beast dynamic buffer implementations against Asio's.
The name "Boost.Http" or "Boost.WebSocket" would mislead users into believing they could perform an HTTP request on a URL or put up a WebSocket client or server in a couple of lines of code. Where would the core utilities go? Very likely it would step on the owner of Boost.Asio's toes to put things in the boost/asio directory; at the very least, it would create unrequested, additional work for the foreign repository.
"Beast" is sufficiently vague as to not suggest any particular functionality, while acting as a memorable umbrella term for a family of low level containers and algorithms. People in the know or with a need for low-level network protocol operations will have no trouble finding it, and the chances of luring a novice into a bad experience are greatly reduced. There is precedent for proper names: "Hana", "Fusion", "Phoenix", and "Spirit" come to mind. Is "Beast" really any worse than say, "mp11" for example? Beast also already has a growing body of users and attention from the open source community, the name Beast comes up in reddit posts and StackOverflow as the answer to questions about which HTTP or WebSocket library to use.
The server-framework example demonstrates how to implement a server that supports TLS using certificates. There are also websocket and HTTP client examples which use TLS. Furthermore, management of certificates is beyond the scope of the public interfaces of the library. Asio already provides documentation, interfaces, and examples for performing these tasks - Beast does not intend to reinvent them or to redundantly provide this information.
We presume this means a facility to match expressions against the URI in HTTP requests, and dispatch them to calling code. The authors feel that this is a responsibility of higher level code. Beast does not try to offer a web server. That said, the server-framework example has a concept of request routing called a Service. Two services are provided, one for serving files and the other for handling WebSocket upgrade requests.
Cookies, or managing these types of HTTP headers in general, is the responsibility of higher levels. Beast just tries to get complete messages to and from the calling code. It deals in the HTTP headers just enough to process the message body and leaves the rest to callers. However, for forms and file uploads the symmetric interface of the message class allows HTTP requests to include arbitrary body types including those needed to upload a file or fill out a form.
Beast works with the Stream concept, so it automatically works with
boost::asio::ssl::stream that you have already set
up through Asio.
The design goal for the library is to not try to invent a web server. We feel that there is a strong need for a basic implementation that models the HTTP message and provides functions to send and receive them over Asio. Such an implementation should serve as a building block upon which higher abstractions such as the aforementioned HTTP service or cgi-gateway can be built.
There are several HTTP servers in the example directory which deliver files, as well as some tested and compiled code snippets which can be used as a starting point for interfacing with other processes.
Deciding on whether to send the "Expect: 100-continue" header or how to handle it on the server side is the caller's responsibility; Beast provides the functionality to send or inspect the header before sending or reading the body.
Beast has already been on public servers receiving traffic and handling hundreds of millions of dollars' worth of financial transactions daily. The servers run rippled, open source software (repository) implementing the Ripple Consensus Protocol, technology provided by Ripple.
Furthermore, the repository has grown significantly in popularity in 2017. There are many users, and some of them participate directly in the repository by reporting issues, performing testing, and in some cases submitting pull requests with code contributions.
Beast WebSocket supports the permessage-deflate extension described in draft-ietf-hybi-permessage-compression-00. The library comes with a header-only, C++11 port of ZLib's "deflate" codec used in the implementation of the permessage-deflate extension.
websocket::stream wraps the socket or stream
that you provide (for example, a
boost::asio::ssl::stream). You establish your TLS connection
using the interface on
like shown in all of the Asio examples, then construct your
websocket::stream around it. It works perfectly
fine; Beast comes with an
wrapper in the example directory which allows the SSL stream to be
moved, overcoming an Asio limitation.
The WebSocket implementation does
provide support for shutting down the TLS connection through the use
of the ADL compile-time virtual functions
async_teardown. These will
properly close the connection as per rfc6455 and overloads are available
for TLS streams. Callers may provide their own overloads of these functions
for user-defined next layer types.